[Codegen][LegalizeIntegerTypes] New legalization strategy for scalar shifts: shift through stack

[lebedev.ri]

Summary

https://reviews.llvm.org/D140493 is going to teach SROA how to promote allocas
that have variably-indexed loads. That does bring up questions of cost model,
since that requires creating wide shifts.

Indeed, our legalization for them is not optimal.
We either split it into parts, or lower it into a libcall.
But if the shift amount is by a multiple of CHAR_BIT,
we can also legalize it throught stack.

The basic idea is very simple:

1. Get a stack slot 2x the width of the shift type
2. store the value we are shifting into one half of the slot
3. pad the other half of the slot. for logical shifts, with zero, for arithmetic shift with signbit
4. index into the slot (starting from the base half into which we spilled, either upwards or downwards)
5. load
6. split loaded integer

This works for both little-endian and big-endian machines:
https://alive2.llvm.org/ce/z/YNVwd5

And better yet, if the original shift amount was not a multiple of CHAR_BIT,
we can just shift by that remainder afterwards: https://alive2.llvm.org/ce/z/pz5G-K

I think, if we are going perform shift->shift-by-parts expansion more than once,
we should instead go through stack, which is what this patch does.

[arsenm]

Can I convince you to repeat this for globalisel

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

187 ↗

(On Diff #485161)

Unused?

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4150

Unused variable in release builds

4150

unsigned

4158

Can these be unsigned? I'm not sure why getScalarSizeInBits returns uint64_t. The SizeInBits methods in Datalayout and Type return unsigned.

4195

Can this be SRA of Shiftee by VTBits-1?

4212

Can this be clipped by masking with AND?

[lebedev.ri]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4218

This is a bug, it should be sext, patch incoming.

[lebedev.ri]

@craig.topper thank you for taking a look!
Addressing all review notes.

Correct alive2 link: https://alive2.llvm.org/ce/z/x9gu7H

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4195

Right, i was not sure what should we emit here.
SETCC to MVT::i1 is a bit questionable this late, so let's indeed go with ashr.

4212

Right. I was just looking into that. We just need to call clampDynamicVectorIndex().

[lebedev.ri]

Now with a RISCV test, too.

[lebedev.ri]

Do conjure MachinePointerInfo.

[craig.topper]

In D140638#4015836, @lebedev.ri wrote:

Do conjure MachinePointerInfo.

Isnt pointerinfo for stack automatically inferred?

[lebedev.ri]

Looks like i'm overengineering this.
The logic is even simpler: https://alive2.llvm.org/ce/z/YNVwd5
Instead of manually picking what goes into which half of stack slot,
just extend+position the value that is being shifted,
and just spill it. That allows the slot to be always filled in-order.

In D140638#4015850, @craig.topper wrote:

In D140638#4015836, @lebedev.ri wrote:

Do conjure MachinePointerInfo.

Isnt pointerinfo for stack automatically inferred?

Not for variable indexes into stack slot.

[lebedev.ri]

Aha, and i'm missing the point.
This *nicely* generalizes to non-CHAR_BIT-multiple shift amounts: https://alive2.llvm.org/ce/z/pz5G-K

[lebedev.ri]

(The patch without context in test diff's is already 7.5Mb, phab does not take patches that are larger than 8Mb.)

[lebedev.ri]

Actually, code changes don't need full context, and that significantly reduces patch size,
allowing for full context in test changes. Should have thought of that first.

[lebedev.ri]

ping

[lebedev.ri]

ping

[efriedma]

A few header changes seem to be missing? I can't find a definition of ShiftLegalizationStrategy.

It might be worth implementing a strategy that avoids unaligned loads (by splitting the shift amount by the native register width instead of CHAR_BIT). On targets that don't have native unaligned loads, they're pretty expensive. Even on targets that do have unaligned loads, an aligned load can reduce the cost of the store forwarding stall. (But on targets with fast unaligned loads, they're probably worth using if the shift amount is known to be a multiple of CHAR_BIT.)

[lebedev.ri]

Rebased.

In D140638#4024426, @efriedma wrote:

A few header changes seem to be missing? I can't find a definition of ShiftLegalizationStrategy.

It might be worth implementing a strategy that avoids unaligned loads (by splitting the shift amount by the native register width instead of CHAR_BIT). On targets that don't have native unaligned loads, they're pretty expensive. Even on targets that do have unaligned loads, an aligned load can reduce the cost of the store forwarding stall. (But on targets with fast unaligned loads, they're probably worth using if the shift amount is known to be a multiple of CHAR_BIT.)

@efriedma thank you for taking a look!
Indeed, the fact that loads are unaligned has occurred to me.
But, this patch is already much bigger than i already planned.
Honestly, initially i only looked to handle lshr by multiple of CHAR_BIT,
and *everything* ontop of that just kinda kept being added.

I may be interested in looking into further follow-up improvements,
as long as the original patch (in general) does not result in really dragged our review,
but i really don't feel it's reasonable to do even more stuff here.
WDYT?

[efriedma]

I won't insist on implementing everything in one patch. I was going to say we need a bailout for targets that don't have unaligned loads... but looking at llvm/test/CodeGen/RISCV/shifts.ll, I guess the previous codegen is terrible enough that it's an improvement even if we expand unaligned loads to byte loads. So I guess that's fine.

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4160

MachinePointerInfo::getUnknownStack instead of MachinePointerInfo()? Or actually, DAGTypeLegalizer::SplitVecRes_INSERT_SUBVECTOR does the following:

auto FrameIndex = cast<FrameIndexSDNode>(StackPtr.getNode())->getIndex();
auto PtrInfo = MachinePointerInfo::getFixedStack(MF, FrameIndex);

[lebedev.ri]

In D140638#4024588, @efriedma wrote:

I won't insist on implementing everything in one patch. I was going to say we need a bailout for targets that don't have unaligned loads... but looking at llvm/test/CodeGen/RISCV/shifts.ll, I guess the previous codegen is terrible enough that it's an improvement even if we expand unaligned loads to byte loads. So I guess that's fine.

That was precisely my point. It's already not great, and this does not seem to regress it.

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4160

As per @craig.topper, and looking at -debug,
we get right MachinePointerInfo for stack-based
memory operations automatically. Not doing that manually
frees us from having to track alignment, too.

[efriedma]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4160

I guess optimizations kick in, sure, so you don't need to compute the exact pointer info.

But please at least use getUnknownStack(); a default-constructed MachinePointerInfo implicitly points to address-space zero.

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4160

Isn't there a check for null pointer info in getLoad/getStore to infer the pointer info? Will that still trigger with getUnknownStack()?

[lebedev.ri]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4160

Yup: https://github.com/llvm/llvm-project/blob/62fc5f16405a7d39e62044bc461752f3f31bdca0/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp#L8007-L8010

[efriedma]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4160

Having an interface that says "the address-space of the store is based on the MachinePointerInfo()... unless the address refers to a frame index, in which case we ignore the MachinePointerInfo() you passed" is not at all intuitive. The fact that it works that way looks like a historical accident. Please don't abuse this behavior; if you want a more convenient interface for a store to a frame index, please add a new interface.

[lebedev.ri]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4160

FWIW, originally i did exactly what DAGTypeLegalizer::SplitVecRes_INSERT_SUBVECTOR() did,
but @craig.topper noted that it happens automatically.

[lebedev.ri]

Undo changes that were previously requested.

[lebedev.ri]

ping

[lebedev.ri]

@efriedma @craig.topper does anyone have any further thoughts here? Is this waiting on me? If not, i'd like to get this going and look into follow-ups.

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4134

Is this ByteVecVT only used to make clampDynamicVectorIndex work? It won't cause vector instructions to be generated from scalar code will it?

4164

I think the alignment is incorrect on this store.

4176

Is this always going to create an AND? Trying to decide if bringing "Vector" into this made this more confusing.

4190

Nit: Use curly braces for consistency with else

4212

Does this shift end up in ExpandShiftWithKnownAmountBit because of the AND?

[lebedev.ri]

@craig.topper thank you for taking a look!
Addressing all review notes.

[lebedev.ri]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4134

Yes, only for clampDynamicVectorIndex().

4164

Bingo!

4212

That's the goal, yes.

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

9352 ↗

(On Diff #489087)

We can drop this now right?

[lebedev.ri]

Drop clampDynamicVectorIndex() changes.

[craig.topper]

LGTM

[lebedev.ri]

In D140638#4053485, @craig.topper wrote:

LGTM

Thank you for the review!

llvm/include/llvm/CodeGen/TargetLowering.h

public:
   }
 
   /// Return the cost of the 'representative' register class for the specified
   /// value type.
   virtual uint8_t getRepRegClassCostFor(MVT VT) const {
     return RepRegClassCostForVT[VT.SimpleTy];
   }
 
-  /// Return true if SHIFT instructions should be expanded to SHIFT_PARTS
-  /// instructions, and false if a library call is preferred (e.g for code-size
-  /// reasons).
-  virtual bool shouldExpandShift(SelectionDAG &DAG, SDNode *N) const {
-    return true;
+  /// Return the preferred strategy to legalize tihs SHIFT instruction, with
+  /// \p ExpansionFactor being the recursion depth - how many expansion needed.
+  enum class ShiftLegalizationStrategy {
+    ExpandToParts,
+    ExpandThroughStack,
+    LowerToLibcall
+  };
+  virtual ShiftLegalizationStrategy
+  preferredShiftLegalizationStrategy(SelectionDAG &DAG, SDNode *N,
+                                     unsigned ExpansionFactor) const {
+    if (ExpansionFactor == 1)
+      return ShiftLegalizationStrategy::ExpandToParts;
+    return ShiftLegalizationStrategy::ExpandThroughStack;
   }
 
   /// Return true if the target has native support for the specified value type.
   /// This means that it has a register that directly holds it without
   /// promotions or expansions.
   bool isTypeLegal(EVT VT) const {
     assert(!VT.isSimple() ||
            (unsigned)VT.getSimpleVT().SimpleTy < std::size(RegClassForVT));

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

void DAGTypeLegalizer::ExpandIntRes_SDIV(SDNode *N,
 
   RTLIB::Libcall LC = RTLIB::UNKNOWN_LIBCALL;
   if (VT == MVT::i16)
     LC = RTLIB::SDIV_I16;
   else if (VT == MVT::i32)
     LC = RTLIB::SDIV_I32;
   else if (VT == MVT::i64)
     LC = RTLIB::SDIV_I64;
   else if (VT == MVT::i128)
     LC = RTLIB::SDIV_I128;
   assert(LC != RTLIB::UNKNOWN_LIBCALL && "Unsupported SDIV!");
 
   TargetLowering::MakeLibCallOptions CallOptions;
   CallOptions.setSExt(true);
   SplitInteger(TLI.makeLibCall(DAG, LC, VT, Ops, CallOptions, dl).first, Lo, Hi);
 }
 
+void DAGTypeLegalizer::ExpandIntRes_ShiftThroughStack(SDNode *N, SDValue &Lo,
+                                                      SDValue &Hi) {
+  SDLoc dl(N);
+  SDValue Shiftee = N->getOperand(0);
+  EVT VT = Shiftee.getValueType();
+  SDValue ShAmt = N->getOperand(1);
+  EVT ShAmtVT = ShAmt.getValueType();
+
+  // This legalization is optimal when the shift is by a multiple of byte width,
+  //   %x * 8 <-> %x << 3   so 3 low bits should be be known zero.
+  bool ShiftByByteMultiple =
+      DAG.computeKnownBits(ShAmt).countMinTrailingZeros() >= 3;
+
+  // If we can't do it as one step, we'll have two uses of shift amount,
+  // and thus must freeze it.
+  if (!ShiftByByteMultiple)
+    ShAmt = DAG.getFreeze(ShAmt);
+
+  unsigned VTBitWidth = VT.getScalarSizeInBits();
+  assert(VTBitWidth % 8 == 0 && "Shifting a not byte multiple value?");
+  unsigned VTByteWidth = VTBitWidth / 8;
+  assert(isPowerOf2_32(VTByteWidth) &&
+         "Shiftee type size is not a power of two!");
+  unsigned StackSlotByteWidth = 2 * VTByteWidth;
+  unsigned StackSlotBitWidth = 8 * StackSlotByteWidth;
+  EVT StackSlotVT = EVT::getIntegerVT(*DAG.getContext(), StackSlotBitWidth);
+
+  // Get a temporary stack slot 2x the width of our VT.
+  // FIXME: reuse stack slots?
+  // FIXME: should we be more picky about alignment?
+  Align StackSlotAlignment(1);
+  SDValue StackPtr = DAG.CreateStackTemporary(
+      TypeSize::getFixed(StackSlotByteWidth), StackSlotAlignment);
+  EVT PtrTy = StackPtr.getValueType();
+  SDValue Ch = DAG.getEntryNode();
+
+  MachinePointerInfo StackPtrInfo = MachinePointerInfo::getFixedStack(
+      DAG.getMachineFunction(),
+      cast<FrameIndexSDNode>(StackPtr.getNode())->getIndex());
+
+  // Extend the value, that is being shifted, to the entire stack slot's width.
+  SDValue Init;
+  if (N->getOpcode() != ISD::SHL) {
+    unsigned WideningOpc =
+        N->getOpcode() == ISD::SRA ? ISD::SIGN_EXTEND : ISD::ZERO_EXTEND;
+    Init = DAG.getNode(WideningOpc, dl, StackSlotVT, Shiftee);
+  } else {
+    // For left-shifts, pad the Shiftee's LSB with zeros to twice it's width.
+    SDValue AllZeros = DAG.getConstant(0, dl, VT);
+    Init = DAG.getNode(ISD::BUILD_PAIR, dl, StackSlotVT, AllZeros, Shiftee);
+  }
+  // And spill it into the stack slot.
+  Ch = DAG.getStore(Ch, dl, Init, StackPtr, StackPtrInfo, StackSlotAlignment);
+
+  // Now, compute the full-byte offset into stack slot from where we can load.
+  // We have shift amount, which is in bits, but in multiples of byte.
+  // So just divide by CHAR_BIT.
+  SDNodeFlags Flags;
+  if (ShiftByByteMultiple)
+    Flags.setExact(true);
+  SDValue ByteOffset = DAG.getNode(ISD::SRL, dl, ShAmtVT, ShAmt,
+                                   DAG.getConstant(3, dl, ShAmtVT), Flags);
+  // And clamp it, because OOB load is an immediate UB,
+  // while shift overflow would have *just* been poison.
+  ByteOffset = DAG.getNode(ISD::AND, dl, ShAmtVT, ByteOffset,
+                           DAG.getConstant(VTByteWidth - 1, dl, ShAmtVT));
+  // We have exactly two strategies on indexing into stack slot here:
+  // 1. upwards starting from the beginning of the slot
+  // 2. downwards starting from the middle of the slot
+  // On little-endian machine, we pick 1. for right shifts and 2. for left-shift
+  // and vice versa on big-endian machine.
+  bool WillIndexUpwards = N->getOpcode() != ISD::SHL;
+  if (DAG.getDataLayout().isBigEndian())
+    WillIndexUpwards = !WillIndexUpwards;
+
+  SDValue AdjStackPtr;
+  if (WillIndexUpwards) {
+    AdjStackPtr = StackPtr;
+  } else {
+    AdjStackPtr = DAG.getMemBasePlusOffset(
+        StackPtr, DAG.getConstant(VTByteWidth, dl, PtrTy), dl);
+    ByteOffset = DAG.getNegative(ByteOffset, dl, ShAmtVT);
+  }
+
+  // Get the pointer somewhere into the stack slot from which we need to load.
+  ByteOffset = DAG.getSExtOrTrunc(ByteOffset, dl, PtrTy);
+  AdjStackPtr = DAG.getMemBasePlusOffset(AdjStackPtr, ByteOffset, dl);
+
+  // And load it! While the load is not legal, legalizing it is obvious.
+  SDValue Res = DAG.getLoad(
+      VT, dl, Ch, AdjStackPtr,
+      MachinePointerInfo::getUnknownStack(DAG.getMachineFunction()), Align(1));
+  // We've performed the shift by a CHAR_BIT * [_ShAmt / CHAR_BIT_]
+
+  // If we may still have a less-than-CHAR_BIT to shift by, do so now.
+  if (!ShiftByByteMultiple) {
+    SDValue ShAmtRem = DAG.getNode(ISD::AND, dl, ShAmtVT, ShAmt,
+                                   DAG.getConstant(7, dl, ShAmtVT));
+    Res = DAG.getNode(N->getOpcode(), dl, VT, Res, ShAmtRem);
+  }
+
+  // Finally, split the computed value.
+  SplitInteger(Res, Lo, Hi);
+}
+
 void DAGTypeLegalizer::ExpandIntRes_Shift(SDNode *N,
                                           SDValue &Lo, SDValue &Hi) {
   EVT VT = N->getValueType(0);
   SDLoc dl(N);
 
   // If we can emit an efficient shift operation, do so now.  Check to see if
   // the RHS is a constant.
   if (ConstantSDNode *CN = dyn_cast<ConstantSDNode>(N->getOperand(1)))
void DAGTypeLegalizer::ExpandIntRes_Shift(SDNode *N,
   // will custom expand it. Don't lower this to SHL_PARTS when we optimise for
   // size, but create a libcall instead.
   EVT NVT = TLI.getTypeToTransformTo(*DAG.getContext(), VT);
   TargetLowering::LegalizeAction Action = TLI.getOperationAction(PartsOpc, NVT);
   const bool LegalOrCustom =
     (Action == TargetLowering::Legal && TLI.isTypeLegal(NVT)) ||
     Action == TargetLowering::Custom;
 
-  if (LegalOrCustom && TLI.shouldExpandShift(DAG, N)) {
+  unsigned ExpansionFactor = 1;
+  // That VT->NVT expansion is one step. But will we re-expand NVT?
+  for (EVT TmpVT = NVT;;) {
+    EVT NewTMPVT = TLI.getTypeToTransformTo(*DAG.getContext(), TmpVT);
+    if (NewTMPVT == TmpVT)
+      break;
+    TmpVT = NewTMPVT;
+    ++ExpansionFactor;
+  }
+
+  TargetLowering::ShiftLegalizationStrategy S =
+      TLI.preferredShiftLegalizationStrategy(DAG, N, ExpansionFactor);
+
+  if (S == TargetLowering::ShiftLegalizationStrategy::ExpandThroughStack)
+    return ExpandIntRes_ShiftThroughStack(N, Lo, Hi);
+
+  if (LegalOrCustom &&
+      S != TargetLowering::ShiftLegalizationStrategy::LowerToLibcall) {
     // Expand the subcomponents.
     SDValue LHSL, LHSH;
     GetExpandedInteger(N->getOperand(0), LHSL, LHSH);
     EVT VT = LHSL.getValueType();
 
     // If the shift amount operand is coming from a vector legalization it may
     // have an illegal type.  Fix that first by casting the operand, otherwise
     // the new SHL_PARTS operation would need further legalization.

llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h

private:
   void ExpandIntRes_BITREVERSE        (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_BSWAP             (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_PARITY            (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_MUL               (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_SDIV              (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_SREM              (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_UDIV              (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_UREM              (SDNode *N, SDValue &Lo, SDValue &Hi);
+  void ExpandIntRes_ShiftThroughStack (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_Shift             (SDNode *N, SDValue &Lo, SDValue &Hi);
 
   void ExpandIntRes_MINMAX            (SDNode *N, SDValue &Lo, SDValue &Hi);
 
   void ExpandIntRes_SADDSUBO          (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_UADDSUBO          (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_XMULO             (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_ADDSUBSAT         (SDNode *N, SDValue &Lo, SDValue &Hi);

llvm/lib/Target/AArch64/AArch64ISelLowering.h

bool hasAndNot(SDValue Y) const override {
     return !TS.isScalable() && TS.getFixedValue() >= 64; // vector 'bic'
   }
 
   bool shouldProduceAndByConstByHoistingConstFromShiftsLHSOfAnd(
       SDValue X, ConstantSDNode *XC, ConstantSDNode *CC, SDValue Y,
       unsigned OldShiftOpcode, unsigned NewShiftOpcode,
       SelectionDAG &DAG) const override;
 
-  bool shouldExpandShift(SelectionDAG &DAG, SDNode *N) const override;
+  ShiftLegalizationStrategy
+  preferredShiftLegalizationStrategy(SelectionDAG &DAG, SDNode *N,
+                                     unsigned ExpansionFactor) const override;
 
   bool shouldTransformSignedTruncationCheck(EVT XVT,
                                             unsigned KeptBits) const override {
     // For vectors, we don't have a preference..
     if (XVT.isVector())
       return false;
 
     auto VTIsOk = [](EVT VT) -> bool {

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

bool AArch64TargetLowering::
   // Does baseline recommend not to perform the fold by default?
   if (!TargetLowering::shouldProduceAndByConstByHoistingConstFromShiftsLHSOfAnd(
           X, XC, CC, Y, OldShiftOpcode, NewShiftOpcode, DAG))
     return false;
   // Else, if this is a vector shift, prefer 'shl'.
   return X.getValueType().isScalarInteger() || NewShiftOpcode == ISD::SHL;
 }
 
-bool AArch64TargetLowering::shouldExpandShift(SelectionDAG &DAG,
-                                              SDNode *N) const {
+TargetLowering::ShiftLegalizationStrategy
+AArch64TargetLowering::preferredShiftLegalizationStrategy(
+    SelectionDAG &DAG, SDNode *N, unsigned int ExpansionFactor) const {
   if (DAG.getMachineFunction().getFunction().hasMinSize() &&
       !Subtarget->isTargetWindows() && !Subtarget->isTargetDarwin())
-    return false;
-  return true;
+    return ShiftLegalizationStrategy::LowerToLibcall;
+  return TargetLowering::preferredShiftLegalizationStrategy(DAG, N,
+                                                            ExpansionFactor);
 }
 
 void AArch64TargetLowering::initializeSplitCSR(MachineBasicBlock *Entry) const {
   // Update IsSplitCSR in AArch64unctionInfo.
   AArch64FunctionInfo *AFI = Entry->getParent()->getInfo<AArch64FunctionInfo>();
   AFI->setIsSplitCSR(true);
 }
 

llvm/lib/Target/ARM/ARMISelLowering.h

public:
     bool supportSwiftError() const override {
       return true;
     }
 
     bool hasStandaloneRem(EVT VT) const override {
       return HasStandaloneRem;
     }
 
-    bool shouldExpandShift(SelectionDAG &DAG, SDNode *N) const override;
+    ShiftLegalizationStrategy
+    preferredShiftLegalizationStrategy(SelectionDAG &DAG, SDNode *N,
+                                       unsigned ExpansionFactor) const override;
 
     CCAssignFn *CCAssignFnForCall(CallingConv::ID CC, bool isVarArg) const;
     CCAssignFn *CCAssignFnForReturn(CallingConv::ID CC, bool isVarArg) const;
 
     /// Returns true if \p VecTy is a legal interleaved access type. This
     /// function checks the vector element type and the overall width of the
     /// vector.
     bool isLegalInterleavedAccessType(unsigned Factor, FixedVectorType *VecTy,

llvm/lib/Target/ARM/ARMISelLowering.cpp

bool ARMTargetLowering::isMaskAndCmp0FoldingBeneficial(
   ConstantInt *Mask = dyn_cast<ConstantInt>(AndI.getOperand(1));
   if (!Mask || Mask->getValue().getBitWidth() > 32u)
     return false;
   auto MaskVal = unsigned(Mask->getValue().getZExtValue());
   return (Subtarget->isThumb2() ? ARM_AM::getT2SOImmVal(MaskVal)
                                 : ARM_AM::getSOImmVal(MaskVal)) != -1;
 }
 
-bool ARMTargetLowering::shouldExpandShift(SelectionDAG &DAG, SDNode *N) const {
-  return !Subtarget->hasMinSize() || Subtarget->isTargetWindows();
+TargetLowering::ShiftLegalizationStrategy
+ARMTargetLowering::preferredShiftLegalizationStrategy(
+    SelectionDAG &DAG, SDNode *N, unsigned ExpansionFactor) const {
+  if (Subtarget->hasMinSize() && !Subtarget->isTargetWindows())
+    return ShiftLegalizationStrategy::LowerToLibcall;
+  return TargetLowering::preferredShiftLegalizationStrategy(DAG, N,
+                                                            ExpansionFactor);
 }
 
 Value *ARMTargetLowering::emitLoadLinked(IRBuilderBase &Builder, Type *ValueTy,
                                          Value *Addr,
                                          AtomicOrdering Ord) const {
   Module *M = Builder.GetInsertBlock()->getParent()->getParent();
   bool IsAcquire = isAcquireOrStronger(Ord);
 

llvm/lib/Target/AVR/AVRISelLowering.h

public:
   Register getRegisterByName(const char *RegName, LLT VT,
                              const MachineFunction &MF) const override;
 
   bool shouldSplitFunctionArgumentsAsLittleEndian(
       const DataLayout &DL) const override {
     return false;
   }
 
+  ShiftLegalizationStrategy
+  preferredShiftLegalizationStrategy(SelectionDAG &DAG, SDNode *N,
+                                     unsigned ExpansionFactor) const override {
+    return ShiftLegalizationStrategy::LowerToLibcall;
+  }
+
 private:
   SDValue getAVRCmp(SDValue LHS, SDValue RHS, ISD::CondCode CC, SDValue &AVRcc,
                     SelectionDAG &DAG, SDLoc dl) const;
   SDValue getAVRCmp(SDValue LHS, SDValue RHS, SelectionDAG &DAG,
                     SDLoc dl) const;
   SDValue LowerShifts(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerDivRem(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerGlobalAddress(SDValue Op, SelectionDAG &DAG) const;

llvm/lib/Target/RISCV/RISCVISelLowering.h

public:
   ISD::NodeType getExtendForAtomicOps() const override {
     return ISD::SIGN_EXTEND;
   }
 
   ISD::NodeType getExtendForAtomicCmpSwapArg() const override {
     return ISD::SIGN_EXTEND;
   }
 
-  bool shouldExpandShift(SelectionDAG &DAG, SDNode *N) const override {
+  TargetLowering::ShiftLegalizationStrategy
+  preferredShiftLegalizationStrategy(SelectionDAG &DAG, SDNode *N,
+                                     unsigned ExpansionFactor) const override {
     if (DAG.getMachineFunction().getFunction().hasMinSize())
-      return false;
-    return true;
+      return ShiftLegalizationStrategy::LowerToLibcall;
+    return TargetLowering::preferredShiftLegalizationStrategy(DAG, N,
+                                                              ExpansionFactor);
   }
+
   bool isDesirableToCommuteWithShift(const SDNode *N,
                                      CombineLevel Level) const override;
 
   /// If a physical register, this returns the register that receives the
   /// exception address on entry to an EH pad.
   Register
   getExceptionPointerRegister(const Constant *PersonalityFn) const override;
 

llvm/lib/Target/X86/X86ISelLowering.h

shouldTransformSignedTruncationCheck(EVT XVT,
       };
 
       // We are ok with KeptBitsVT being byte/word/dword, what MOVS supports.
       // XVT will be larger than KeptBitsVT.
       MVT KeptBitsVT = MVT::getIntegerVT(KeptBits);
       return VTIsOk(XVT) && VTIsOk(KeptBitsVT);
     }
 
-    bool shouldExpandShift(SelectionDAG &DAG, SDNode *N) const override;
+    ShiftLegalizationStrategy
+    preferredShiftLegalizationStrategy(SelectionDAG &DAG, SDNode *N,
+                                       unsigned ExpansionFactor) const override;
 
     bool shouldSplatInsEltVarIndex(EVT VT) const override;
 
     bool shouldConvertFpToSat(unsigned Op, EVT FPVT, EVT VT) const override {
       // Converting to sat variants holds little benefit on X86 as we will just
       // need to saturate the value back using fp arithmatic.
       return Op != ISD::FP_TO_UINT_SAT && isOperationLegalOrCustom(Op, VT);
     }

llvm/lib/Target/X86/X86ISelLowering.cpp

bool X86TargetLowering::shouldFoldMaskToVariableShiftPair(SDValue Y) const {
 
   // 64-bit shifts on 32-bit targets produce really bad bloated code.
   if (VT == MVT::i64 && !Subtarget.is64Bit())
     return false;
 
   return true;
 }
 
-bool X86TargetLowering::shouldExpandShift(SelectionDAG &DAG,
-                                          SDNode *N) const {
+TargetLowering::ShiftLegalizationStrategy
+X86TargetLowering::preferredShiftLegalizationStrategy(
+    SelectionDAG &DAG, SDNode *N, unsigned ExpansionFactor) const {
   if (DAG.getMachineFunction().getFunction().hasMinSize() &&
       !Subtarget.isOSWindows())
-    return false;
-  return true;
+    return ShiftLegalizationStrategy::LowerToLibcall;
+  return TargetLowering::preferredShiftLegalizationStrategy(DAG, N,
+                                                            ExpansionFactor);
 }
 
 bool X86TargetLowering::shouldSplatInsEltVarIndex(EVT VT) const {
   // Any legal vector type can be splatted more efficiently than
   // loading/spilling from memory.
   return isTypeLegal(VT);
 }
 

llvm/test/CodeGen/AArch64/wide-scalar-shift-by-byte-multiple-legalization.ll

; ALL-NEXT:    ret
   %res = ashr i128 %src, %bitOff
   store i128 %res, ptr %dst, align 1
   ret void
 }
 
 define void @lshr_32bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
 ; ALL-LABEL: lshr_32bytes:
 ; ALL:       // %bb.0:
+; ALL-NEXT:    sub sp, sp, #64
 ; ALL-NEXT:    ldr x9, [x1]
-; ALL-NEXT:    mov w8, #128
-; ALL-NEXT:    ldp x11, x10, [x0, #8]
-; ALL-NEXT:    lsl x9, x9, #3
-; ALL-NEXT:    ldr x12, [x0]
-; ALL-NEXT:    sub x8, x8, x9
-; ALL-NEXT:    ldr x13, [x0, #24]
-; ALL-NEXT:    and x17, x8, #0x38
-; ALL-NEXT:    mvn w0, w8
-; ALL-NEXT:    lsr x14, x10, #1
-; ALL-NEXT:    and x15, x9, #0x38
-; ALL-NEXT:    mvn w16, w9
-; ALL-NEXT:    tst x8, #0x40
-; ALL-NEXT:    lsl x3, x13, x17
-; ALL-NEXT:    lsr x14, x14, x0
-; ALL-NEXT:    lsl x17, x10, x17
-; ALL-NEXT:    orr x14, x3, x14
-; ALL-NEXT:    lsl x18, x13, #1
-; ALL-NEXT:    csel x0, xzr, x17, ne
-; ALL-NEXT:    csel x14, x17, x14, ne
-; ALL-NEXT:    lsl x17, x11, #1
-; ALL-NEXT:    lsr x8, x10, x15
-; ALL-NEXT:    lsl x1, x18, x16
-; ALL-NEXT:    lsr x3, x12, x15
-; ALL-NEXT:    lsl x16, x17, x16
-; ALL-NEXT:    orr x8, x1, x8
-; ALL-NEXT:    lsr x1, x13, x15
-; ALL-NEXT:    tst x9, #0x40
-; ALL-NEXT:    orr x16, x16, x3
-; ALL-NEXT:    lsr x15, x11, x15
-; ALL-NEXT:    csel x8, x1, x8, ne
-; ALL-NEXT:    csel x16, x15, x16, ne
-; ALL-NEXT:    csel x15, xzr, x15, ne
-; ALL-NEXT:    csel x17, xzr, x1, ne
-; ALL-NEXT:    subs x1, x9, #128
-; ALL-NEXT:    and x3, x1, #0x38
-; ALL-NEXT:    mvn w4, w1
-; ALL-NEXT:    csel x17, x17, xzr, lo
-; ALL-NEXT:    tst x1, #0x40
-; ALL-NEXT:    orr x16, x16, x0
-; ALL-NEXT:    orr x14, x15, x14
-; ALL-NEXT:    lsr x10, x10, x3
-; ALL-NEXT:    lsl x18, x18, x4
-; ALL-NEXT:    orr x10, x18, x10
-; ALL-NEXT:    lsr x13, x13, x3
-; ALL-NEXT:    csel x10, x13, x10, ne
-; ALL-NEXT:    csel x13, xzr, x13, ne
-; ALL-NEXT:    cmp x9, #128
-; ALL-NEXT:    csel x10, x16, x10, lo
-; ALL-NEXT:    csel x8, x8, xzr, lo
-; ALL-NEXT:    csel x13, x14, x13, lo
-; ALL-NEXT:    cmp x9, #0
-; ALL-NEXT:    csel x9, x12, x10, eq
-; ALL-NEXT:    csel x10, x11, x13, eq
-; ALL-NEXT:    stp x8, x17, [x2, #16]
-; ALL-NEXT:    stp x9, x10, [x2]
+; ALL-NEXT:    mov x8, sp
+; ALL-NEXT:    ldp x10, x11, [x0, #16]
+; ALL-NEXT:    movi v0.2d, #0000000000000000
+; ALL-NEXT:    ldr q1, [x0]
+; ALL-NEXT:    and x9, x9, #0x1f
+; ALL-NEXT:    add x8, x8, x9
+; ALL-NEXT:    stp q0, q0, [sp, #32]
+; ALL-NEXT:    stp x10, x11, [sp, #16]
+; ALL-NEXT:    str q1, [sp]
+; ALL-NEXT:    ldp x10, x9, [x8, #16]
+; ALL-NEXT:    ldr q0, [x8]
+; ALL-NEXT:    stp x10, x9, [x2, #16]
+; ALL-NEXT:    str q0, [x2]
+; ALL-NEXT:    add sp, sp, #64
 ; ALL-NEXT:    ret
   %src = load i256, ptr %src.ptr, align 1
   %byteOff = load i256, ptr %byteOff.ptr, align 1
   %bitOff = shl i256 %byteOff, 3
   %res = lshr i256 %src, %bitOff
   store i256 %res, ptr %dst, align 1
   ret void
 }
 define void @shl_32bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
 ; ALL-LABEL: shl_32bytes:
 ; ALL:       // %bb.0:
+; ALL-NEXT:    sub sp, sp, #64
 ; ALL-NEXT:    ldr x9, [x1]
-; ALL-NEXT:    mov w8, #128
-; ALL-NEXT:    ldp x10, x11, [x0, #8]
-; ALL-NEXT:    lsl x9, x9, #3
-; ALL-NEXT:    ldr x12, [x0, #24]
+; ALL-NEXT:    mov x8, sp
+; ALL-NEXT:    ldp x10, x11, [x0, #16]
+; ALL-NEXT:    movi v0.2d, #0000000000000000
+; ALL-NEXT:    add x8, x8, #32
+; ALL-NEXT:    ldr q1, [x0]
+; ALL-NEXT:    and x9, x9, #0x1f
 ; ALL-NEXT:    sub x8, x8, x9
-; ALL-NEXT:    ldr x13, [x0]
-; ALL-NEXT:    and x17, x8, #0x38
-; ALL-NEXT:    mvn w0, w8
-; ALL-NEXT:    lsl x14, x10, #1
-; ALL-NEXT:    and x15, x9, #0x38
-; ALL-NEXT:    mvn w16, w9
-; ALL-NEXT:    tst x8, #0x40
-; ALL-NEXT:    lsr x3, x13, x17
-; ALL-NEXT:    lsl x14, x14, x0
-; ALL-NEXT:    lsr x17, x10, x17
-; ALL-NEXT:    orr x14, x14, x3
-; ALL-NEXT:    lsr x18, x13, #1
-; ALL-NEXT:    csel x0, xzr, x17, ne
-; ALL-NEXT:    csel x14, x17, x14, ne
-; ALL-NEXT:    lsr x17, x11, #1
-; ALL-NEXT:    lsl x8, x10, x15
-; ALL-NEXT:    lsr x1, x18, x16
-; ALL-NEXT:    lsl x3, x12, x15
-; ALL-NEXT:    lsr x16, x17, x16
-; ALL-NEXT:    orr x8, x8, x1
-; ALL-NEXT:    lsl x1, x13, x15
-; ALL-NEXT:    tst x9, #0x40
-; ALL-NEXT:    orr x16, x3, x16
-; ALL-NEXT:    lsl x15, x11, x15
-; ALL-NEXT:    csel x8, x1, x8, ne
-; ALL-NEXT:    csel x16, x15, x16, ne
-; ALL-NEXT:    csel x15, xzr, x15, ne
-; ALL-NEXT:    csel x17, xzr, x1, ne
-; ALL-NEXT:    subs x1, x9, #128
-; ALL-NEXT:    and x3, x1, #0x38
-; ALL-NEXT:    mvn w4, w1
-; ALL-NEXT:    csel x17, x17, xzr, lo
-; ALL-NEXT:    tst x1, #0x40
-; ALL-NEXT:    orr x16, x16, x0
-; ALL-NEXT:    orr x14, x15, x14
-; ALL-NEXT:    lsl x10, x10, x3
-; ALL-NEXT:    lsr x18, x18, x4
-; ALL-NEXT:    orr x10, x10, x18
-; ALL-NEXT:    lsl x13, x13, x3
-; ALL-NEXT:    csel x10, x13, x10, ne
-; ALL-NEXT:    csel x13, xzr, x13, ne
-; ALL-NEXT:    cmp x9, #128
-; ALL-NEXT:    csel x10, x16, x10, lo
-; ALL-NEXT:    csel x8, x8, xzr, lo
-; ALL-NEXT:    csel x13, x14, x13, lo
-; ALL-NEXT:    cmp x9, #0
-; ALL-NEXT:    csel x9, x12, x10, eq
-; ALL-NEXT:    csel x10, x11, x13, eq
-; ALL-NEXT:    stp x17, x8, [x2]
-; ALL-NEXT:    stp x10, x9, [x2, #16]
+; ALL-NEXT:    stp q0, q0, [sp]
+; ALL-NEXT:    stp x10, x11, [sp, #48]
+; ALL-NEXT:    str q1, [sp, #32]
+; ALL-NEXT:    ldp x9, x10, [x8, #16]
+; ALL-NEXT:    ldr q0, [x8]
+; ALL-NEXT:    stp x9, x10, [x2, #16]
+; ALL-NEXT:    str q0, [x2]
+; ALL-NEXT:    add sp, sp, #64
 ; ALL-NEXT:    ret
   %src = load i256, ptr %src.ptr, align 1
   %byteOff = load i256, ptr %byteOff.ptr, align 1
   %bitOff = shl i256 %byteOff, 3
   %res = shl i256 %src, %bitOff
   store i256 %res, ptr %dst, align 1
   ret void
 }
 define void @ashr_32bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
 ; ALL-LABEL: ashr_32bytes:
 ; ALL:       // %bb.0:
+; ALL-NEXT:    sub sp, sp, #64
+; ALL-NEXT:    ldp x11, x10, [x0, #16]
+; ALL-NEXT:    mov x8, sp
 ; ALL-NEXT:    ldr x9, [x1]
-; ALL-NEXT:    mov w8, #128
-; ALL-NEXT:    ldp x11, x10, [x0, #8]
-; ALL-NEXT:    lsl x9, x9, #3
-; ALL-NEXT:    ldr x12, [x0]
-; ALL-NEXT:    sub x8, x8, x9
-; ALL-NEXT:    ldr x13, [x0, #24]
-; ALL-NEXT:    and x18, x8, #0x38
-; ALL-NEXT:    mvn w0, w8
-; ALL-NEXT:    lsr x14, x10, #1
-; ALL-NEXT:    and x15, x9, #0x38
-; ALL-NEXT:    mvn w16, w9
-; ALL-NEXT:    lsl x17, x13, #1
-; ALL-NEXT:    lsl x4, x13, x18
-; ALL-NEXT:    lsr x14, x14, x0
-; ALL-NEXT:    tst x8, #0x40
-; ALL-NEXT:    lsl x18, x10, x18
-; ALL-NEXT:    orr x14, x4, x14
-; ALL-NEXT:    lsr x8, x10, x15
-; ALL-NEXT:    lsl x1, x17, x16
-; ALL-NEXT:    csel x0, xzr, x18, ne
-; ALL-NEXT:    csel x14, x18, x14, ne
-; ALL-NEXT:    lsl x18, x11, #1
-; ALL-NEXT:    orr x8, x1, x8
-; ALL-NEXT:    lsr x1, x12, x15
-; ALL-NEXT:    lsl x16, x18, x16
-; ALL-NEXT:    asr x3, x13, x15
-; ALL-NEXT:    tst x9, #0x40
-; ALL-NEXT:    orr x16, x16, x1
-; ALL-NEXT:    lsr x15, x11, x15
-; ALL-NEXT:    asr x18, x13, #63
-; ALL-NEXT:    csel x8, x3, x8, ne
-; ALL-NEXT:    csel x16, x15, x16, ne
-; ALL-NEXT:    csel x15, xzr, x15, ne
-; ALL-NEXT:    csel x1, x18, x3, ne
-; ALL-NEXT:    subs x3, x9, #128
-; ALL-NEXT:    orr x16, x16, x0
-; ALL-NEXT:    and x4, x3, #0x38
-; ALL-NEXT:    mvn w5, w3
-; ALL-NEXT:    orr x14, x15, x14
-; ALL-NEXT:    lsr x10, x10, x4
-; ALL-NEXT:    lsl x17, x17, x5
-; ALL-NEXT:    orr x10, x17, x10
-; ALL-NEXT:    csel x17, x1, x18, lo
-; ALL-NEXT:    asr x13, x13, x4
-; ALL-NEXT:    tst x3, #0x40
-; ALL-NEXT:    csel x10, x13, x10, ne
-; ALL-NEXT:    csel x13, x18, x13, ne
-; ALL-NEXT:    cmp x9, #128
-; ALL-NEXT:    csel x10, x16, x10, lo
-; ALL-NEXT:    csel x8, x8, x18, lo
-; ALL-NEXT:    csel x13, x14, x13, lo
-; ALL-NEXT:    cmp x9, #0
-; ALL-NEXT:    csel x9, x12, x10, eq
-; ALL-NEXT:    csel x10, x11, x13, eq
-; ALL-NEXT:    stp x8, x17, [x2, #16]
-; ALL-NEXT:    stp x9, x10, [x2]
+; ALL-NEXT:    ldr q0, [x0]
+; ALL-NEXT:    asr x12, x10, #63
+; ALL-NEXT:    and x9, x9, #0x1f
+; ALL-NEXT:    add x8, x8, x9
+; ALL-NEXT:    stp x11, x10, [sp, #16]
+; ALL-NEXT:    str q0, [sp]
+; ALL-NEXT:    stp x12, x12, [sp, #48]
+; ALL-NEXT:    stp x12, x12, [sp, #32]
+; ALL-NEXT:    ldp x10, x9, [x8, #16]
+; ALL-NEXT:    ldr q0, [x8]
+; ALL-NEXT:    stp x10, x9, [x2, #16]
+; ALL-NEXT:    str q0, [x2]
+; ALL-NEXT:    add sp, sp, #64
 ; ALL-NEXT:    ret
   %src = load i256, ptr %src.ptr, align 1
   %byteOff = load i256, ptr %byteOff.ptr, align 1
   %bitOff = shl i256 %byteOff, 3
   %res = ashr i256 %src, %bitOff
   store i256 %res, ptr %dst, align 1
   ret void
 }

llvm/test/CodeGen/AArch64/wide-scalar-shift-legalization.ll

; ALL-NEXT:    ret
   %res = ashr i128 %src, %bitOff
   store i128 %res, ptr %dst, align 1
   ret void
 }
 
 define void @lshr_32bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
 ; ALL-LABEL: lshr_32bytes:
 ; ALL:       // %bb.0:
+; ALL-NEXT:    sub sp, sp, #64
 ; ALL-NEXT:    ldr x9, [x1]
-; ALL-NEXT:    mov w8, #128
+; ALL-NEXT:    mov x8, sp
 ; ALL-NEXT:    ldp x10, x11, [x0, #16]
-; ALL-NEXT:    sub x8, x8, x9
-; ALL-NEXT:    mvn w16, w9
-; ALL-NEXT:    ldp x13, x12, [x0]
-; ALL-NEXT:    mvn w0, w8
-; ALL-NEXT:    lsr x14, x10, #1
-; ALL-NEXT:    lsl x1, x11, x8
-; ALL-NEXT:    tst x8, #0x40
-; ALL-NEXT:    lsl x8, x10, x8
-; ALL-NEXT:    lsl x17, x11, #1
-; ALL-NEXT:    lsr x14, x14, x0
-; ALL-NEXT:    csel x0, xzr, x8, ne
-; ALL-NEXT:    orr x14, x1, x14
-; ALL-NEXT:    lsr x15, x10, x9
-; ALL-NEXT:    csel x8, x8, x14, ne
-; ALL-NEXT:    lsl x14, x12, #1
-; ALL-NEXT:    lsl x3, x17, x16
-; ALL-NEXT:    lsr x1, x13, x9
-; ALL-NEXT:    lsl x14, x14, x16
-; ALL-NEXT:    lsr x18, x11, x9
-; ALL-NEXT:    orr x15, x3, x15
-; ALL-NEXT:    tst x9, #0x40
-; ALL-NEXT:    orr x14, x14, x1
-; ALL-NEXT:    lsr x16, x12, x9
-; ALL-NEXT:    csel x15, x18, x15, ne
-; ALL-NEXT:    csel x14, x16, x14, ne
-; ALL-NEXT:    csel x16, xzr, x16, ne
-; ALL-NEXT:    csel x18, xzr, x18, ne
-; ALL-NEXT:    subs x1, x9, #128
-; ALL-NEXT:    orr x14, x14, x0
-; ALL-NEXT:    mvn w3, w1
-; ALL-NEXT:    orr x8, x16, x8
-; ALL-NEXT:    lsr x10, x10, x1
-; ALL-NEXT:    lsr x11, x11, x1
-; ALL-NEXT:    lsl x17, x17, x3
-; ALL-NEXT:    orr x10, x17, x10
-; ALL-NEXT:    csel x17, x18, xzr, lo
-; ALL-NEXT:    tst x1, #0x40
-; ALL-NEXT:    csel x10, x11, x10, ne
-; ALL-NEXT:    csel x11, xzr, x11, ne
-; ALL-NEXT:    cmp x9, #128
-; ALL-NEXT:    csel x10, x14, x10, lo
-; ALL-NEXT:    csel x14, x15, xzr, lo
-; ALL-NEXT:    csel x8, x8, x11, lo
-; ALL-NEXT:    cmp x9, #0
-; ALL-NEXT:    csel x9, x13, x10, eq
-; ALL-NEXT:    csel x8, x12, x8, eq
-; ALL-NEXT:    stp x14, x17, [x2, #16]
-; ALL-NEXT:    stp x9, x8, [x2]
+; ALL-NEXT:    movi v0.2d, #0000000000000000
+; ALL-NEXT:    ldr q1, [x0]
+; ALL-NEXT:    ubfx x12, x9, #3, #5
+; ALL-NEXT:    add x8, x8, x12
+; ALL-NEXT:    and x9, x9, #0x7
+; ALL-NEXT:    stp q0, q0, [sp, #32]
+; ALL-NEXT:    stp x10, x11, [sp, #16]
+; ALL-NEXT:    eor x11, x9, #0x3f
+; ALL-NEXT:    str q1, [sp]
+; ALL-NEXT:    ldp x10, x13, [x8, #8]
+; ALL-NEXT:    ldr x12, [x8, #24]
+; ALL-NEXT:    ldr x8, [x8]
+; ALL-NEXT:    lsl x14, x10, #1
+; ALL-NEXT:    lsr x10, x10, x9
+; ALL-NEXT:    lsl x15, x12, #1
+; ALL-NEXT:    lsl x14, x14, x11
+; ALL-NEXT:    lsl x11, x15, x11
+; ALL-NEXT:    mvn w15, w9
+; ALL-NEXT:    lsr x8, x8, x9
+; ALL-NEXT:    lsr x12, x12, x9
+; ALL-NEXT:    lsr x9, x13, x9
+; ALL-NEXT:    orr x8, x8, x14
+; ALL-NEXT:    orr x9, x9, x11
+; ALL-NEXT:    lsl x11, x13, #1
+; ALL-NEXT:    lsl x11, x11, x15
+; ALL-NEXT:    orr x10, x10, x11
+; ALL-NEXT:    stp x9, x12, [x2, #16]
+; ALL-NEXT:    stp x8, x10, [x2]
+; ALL-NEXT:    add sp, sp, #64
 ; ALL-NEXT:    ret
   %src = load i256, ptr %src.ptr, align 1
   %bitOff = load i256, ptr %bitOff.ptr, align 1
   %res = lshr i256 %src, %bitOff
   store i256 %res, ptr %dst, align 1
   ret void
 }
 define void @shl_32bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
 ; ALL-LABEL: shl_32bytes:
 ; ALL:       // %bb.0:
+; ALL-NEXT:    sub sp, sp, #64
 ; ALL-NEXT:    ldr x9, [x1]
-; ALL-NEXT:    mov w8, #128
-; ALL-NEXT:    ldp x11, x10, [x0]
-; ALL-NEXT:    sub x8, x8, x9
-; ALL-NEXT:    mvn w16, w9
-; ALL-NEXT:    ldp x12, x13, [x0, #16]
-; ALL-NEXT:    mvn w0, w8
-; ALL-NEXT:    tst x8, #0x40
-; ALL-NEXT:    lsl x14, x10, #1
-; ALL-NEXT:    lsr x1, x11, x8
-; ALL-NEXT:    lsr x8, x10, x8
-; ALL-NEXT:    lsr x17, x11, #1
-; ALL-NEXT:    lsl x14, x14, x0
-; ALL-NEXT:    csel x0, xzr, x8, ne
-; ALL-NEXT:    orr x14, x14, x1
-; ALL-NEXT:    lsl x15, x10, x9
-; ALL-NEXT:    csel x8, x8, x14, ne
-; ALL-NEXT:    lsr x14, x12, #1
-; ALL-NEXT:    lsr x3, x17, x16
-; ALL-NEXT:    lsl x1, x13, x9
-; ALL-NEXT:    lsr x14, x14, x16
-; ALL-NEXT:    lsl x18, x11, x9
-; ALL-NEXT:    orr x15, x15, x3
-; ALL-NEXT:    tst x9, #0x40
-; ALL-NEXT:    orr x14, x1, x14
-; ALL-NEXT:    lsl x16, x12, x9
-; ALL-NEXT:    csel x15, x18, x15, ne
-; ALL-NEXT:    csel x14, x16, x14, ne
-; ALL-NEXT:    csel x16, xzr, x16, ne
-; ALL-NEXT:    csel x18, xzr, x18, ne
-; ALL-NEXT:    subs x1, x9, #128
-; ALL-NEXT:    orr x14, x14, x0
-; ALL-NEXT:    mvn w3, w1
-; ALL-NEXT:    orr x8, x16, x8
-; ALL-NEXT:    lsl x10, x10, x1
-; ALL-NEXT:    lsl x11, x11, x1
-; ALL-NEXT:    lsr x17, x17, x3
-; ALL-NEXT:    orr x10, x10, x17
-; ALL-NEXT:    csel x17, x18, xzr, lo
-; ALL-NEXT:    tst x1, #0x40
-; ALL-NEXT:    csel x10, x11, x10, ne
-; ALL-NEXT:    csel x11, xzr, x11, ne
-; ALL-NEXT:    cmp x9, #128
-; ALL-NEXT:    csel x10, x14, x10, lo
-; ALL-NEXT:    csel x14, x15, xzr, lo
-; ALL-NEXT:    csel x8, x8, x11, lo
-; ALL-NEXT:    cmp x9, #0
-; ALL-NEXT:    csel x9, x13, x10, eq
-; ALL-NEXT:    csel x8, x12, x8, eq
-; ALL-NEXT:    stp x17, x14, [x2]
-; ALL-NEXT:    stp x8, x9, [x2, #16]
+; ALL-NEXT:    mov x8, sp
+; ALL-NEXT:    ldp x10, x11, [x0, #16]
+; ALL-NEXT:    movi v0.2d, #0000000000000000
+; ALL-NEXT:    add x8, x8, #32
+; ALL-NEXT:    ldr q1, [x0]
+; ALL-NEXT:    ubfx x12, x9, #3, #5
+; ALL-NEXT:    sub x8, x8, x12
+; ALL-NEXT:    and x9, x9, #0x7
+; ALL-NEXT:    mvn w12, w9
+; ALL-NEXT:    eor x14, x9, #0x3f
+; ALL-NEXT:    stp q0, q0, [sp]
+; ALL-NEXT:    stp x10, x11, [sp, #48]
+; ALL-NEXT:    str q1, [sp, #32]
+; ALL-NEXT:    ldp x11, x10, [x8, #8]
+; ALL-NEXT:    ldr x13, [x8]
+; ALL-NEXT:    ldr x8, [x8, #24]
+; ALL-NEXT:    lsr x15, x11, #1
+; ALL-NEXT:    lsl x11, x11, x9
+; ALL-NEXT:    lsr x16, x10, #1
+; ALL-NEXT:    lsr x12, x15, x12
+; ALL-NEXT:    lsr x15, x13, #1
+; ALL-NEXT:    lsr x16, x16, x14
+; ALL-NEXT:    lsr x14, x15, x14
+; ALL-NEXT:    lsl x13, x13, x9
+; ALL-NEXT:    lsl x8, x8, x9
+; ALL-NEXT:    lsl x9, x10, x9
+; ALL-NEXT:    orr x11, x11, x14
+; ALL-NEXT:    orr x8, x8, x16
+; ALL-NEXT:    orr x9, x9, x12
+; ALL-NEXT:    stp x13, x11, [x2]
+; ALL-NEXT:    stp x9, x8, [x2, #16]
+; ALL-NEXT:    add sp, sp, #64
 ; ALL-NEXT:    ret
   %src = load i256, ptr %src.ptr, align 1
   %bitOff = load i256, ptr %bitOff.ptr, align 1
   %res = shl i256 %src, %bitOff
   store i256 %res, ptr %dst, align 1
   ret void
 }
 define void @ashr_32bytes(ptr %src.ptr, ptr %bitOff.ptr, ptr %dst) nounwind {
 ; ALL-LABEL: ashr_32bytes:
 ; ALL:       // %bb.0:
+; ALL-NEXT:    sub sp, sp, #64
+; ALL-NEXT:    ldp x11, x10, [x0, #16]
+; ALL-NEXT:    mov x8, sp
 ; ALL-NEXT:    ldr x9, [x1]
-; ALL-NEXT:    mov w8, #128
-; ALL-NEXT:    ldp x11, x10, [x0, #8]
-; ALL-NEXT:    sub x8, x8, x9
-; ALL-NEXT:    ldr x13, [x0, #24]
-; ALL-NEXT:    mvn w18, w8
-; ALL-NEXT:    ldr x12, [x0]
-; ALL-NEXT:    mvn w16, w9
-; ALL-NEXT:    tst x8, #0x40
-; ALL-NEXT:    lsr x14, x10, #1
-; ALL-NEXT:    lsl x1, x13, x8
-; ALL-NEXT:    lsr x14, x14, x18
-; ALL-NEXT:    lsl x8, x10, x8
-; ALL-NEXT:    orr x14, x1, x14
-; ALL-NEXT:    lsl x17, x13, #1
-; ALL-NEXT:    csel x18, xzr, x8, ne
-; ALL-NEXT:    csel x8, x8, x14, ne
-; ALL-NEXT:    lsl x14, x11, #1
-; ALL-NEXT:    lsr x15, x10, x9
-; ALL-NEXT:    lsl x3, x17, x16
-; ALL-NEXT:    lsr x1, x12, x9
-; ALL-NEXT:    lsl x14, x14, x16
-; ALL-NEXT:    asr x0, x13, x9
-; ALL-NEXT:    orr x15, x3, x15
-; ALL-NEXT:    tst x9, #0x40
-; ALL-NEXT:    orr x14, x14, x1
-; ALL-NEXT:    lsr x16, x11, x9
-; ALL-NEXT:    asr x1, x13, #63
-; ALL-NEXT:    csel x15, x0, x15, ne
-; ALL-NEXT:    csel x14, x16, x14, ne
-; ALL-NEXT:    csel x16, xzr, x16, ne
-; ALL-NEXT:    csel x0, x1, x0, ne
-; ALL-NEXT:    subs x3, x9, #128
-; ALL-NEXT:    mvn w4, w3
-; ALL-NEXT:    orr x14, x14, x18
-; ALL-NEXT:    orr x8, x16, x8
-; ALL-NEXT:    lsr x10, x10, x3
-; ALL-NEXT:    asr x13, x13, x3
-; ALL-NEXT:    lsl x17, x17, x4
-; ALL-NEXT:    orr x10, x17, x10
-; ALL-NEXT:    csel x17, x0, x1, lo
-; ALL-NEXT:    tst x3, #0x40
-; ALL-NEXT:    csel x10, x13, x10, ne
-; ALL-NEXT:    csel x13, x1, x13, ne
-; ALL-NEXT:    cmp x9, #128
-; ALL-NEXT:    csel x10, x14, x10, lo
-; ALL-NEXT:    csel x14, x15, x1, lo
-; ALL-NEXT:    csel x8, x8, x13, lo
-; ALL-NEXT:    cmp x9, #0
-; ALL-NEXT:    csel x9, x12, x10, eq
-; ALL-NEXT:    csel x8, x11, x8, eq
-; ALL-NEXT:    stp x14, x17, [x2, #16]
-; ALL-NEXT:    stp x9, x8, [x2]
+; ALL-NEXT:    ldr q0, [x0]
+; ALL-NEXT:    asr x12, x10, #63
+; ALL-NEXT:    stp x11, x10, [sp, #16]
+; ALL-NEXT:    ubfx x10, x9, #3, #5
+; ALL-NEXT:    str q0, [sp]
+; ALL-NEXT:    add x8, x8, x10
+; ALL-NEXT:    and x9, x9, #0x7
+; ALL-NEXT:    stp x12, x12, [sp, #48]
+; ALL-NEXT:    eor x14, x9, #0x3f
+; ALL-NEXT:    stp x12, x12, [sp, #32]
+; ALL-NEXT:    mvn w12, w9
+; ALL-NEXT:    ldp x10, x11, [x8, #8]
+; ALL-NEXT:    ldr x13, [x8, #24]
+; ALL-NEXT:    ldr x8, [x8]
+; ALL-NEXT:    lsl x16, x10, #1
+; ALL-NEXT:    lsl x15, x11, #1
+; ALL-NEXT:    lsl x16, x16, x14
+; ALL-NEXT:    lsl x12, x15, x12
+; ALL-NEXT:    lsl x15, x13, #1
+; ALL-NEXT:    lsl x14, x15, x14
+; ALL-NEXT:    lsr x11, x11, x9
+; ALL-NEXT:    asr x13, x13, x9
+; ALL-NEXT:    lsr x8, x8, x9
+; ALL-NEXT:    lsr x9, x10, x9
+; ALL-NEXT:    orr x11, x11, x14
+; ALL-NEXT:    orr x8, x8, x16
+; ALL-NEXT:    orr x9, x9, x12
+; ALL-NEXT:    stp x11, x13, [x2, #16]
+; ALL-NEXT:    stp x8, x9, [x2]
+; ALL-NEXT:    add sp, sp, #64
 ; ALL-NEXT:    ret
   %src = load i256, ptr %src.ptr, align 1
   %bitOff = load i256, ptr %bitOff.ptr, align 1
   %res = ashr i256 %src, %bitOff
   store i256 %res, ptr %dst, align 1
   ret void
 }

llvm/test/CodeGen/Mips/llvm-ir/ashr.ll

 entry:
   %r = ashr i64 %a, %b
   ret i64 %r
 }
 
 define signext i128 @ashr_i128(i128 signext %a, i128 signext %b) {
 ; MIPS-LABEL: ashr_i128:
 ; MIPS:       # %bb.0: # %entry
-; MIPS-NEXT:    lw $2, 28($sp)
-; MIPS-NEXT:    addiu $1, $zero, 64
-; MIPS-NEXT:    subu $9, $1, $2
-; MIPS-NEXT:    sllv $10, $5, $9
-; MIPS-NEXT:    andi $13, $9, 32
-; MIPS-NEXT:    andi $3, $2, 32
-; MIPS-NEXT:    addiu $11, $zero, 0
-; MIPS-NEXT:    bnez $13, $BB5_2
-; MIPS-NEXT:    addiu $12, $zero, 0
-; MIPS-NEXT:  # %bb.1: # %entry
-; MIPS-NEXT:    move $12, $10
-; MIPS-NEXT:  $BB5_2: # %entry
-; MIPS-NEXT:    not $8, $2
-; MIPS-NEXT:    bnez $3, $BB5_5
-; MIPS-NEXT:    srlv $14, $6, $2
-; MIPS-NEXT:  # %bb.3: # %entry
-; MIPS-NEXT:    sll $1, $6, 1
-; MIPS-NEXT:    srlv $11, $7, $2
-; MIPS-NEXT:    sllv $1, $1, $8
-; MIPS-NEXT:    or $15, $1, $11
-; MIPS-NEXT:    bnez $13, $BB5_7
-; MIPS-NEXT:    move $11, $14
-; MIPS-NEXT:  # %bb.4: # %entry
-; MIPS-NEXT:    b $BB5_6
-; MIPS-NEXT:    nop
-; MIPS-NEXT:  $BB5_5:
-; MIPS-NEXT:    bnez $13, $BB5_7
-; MIPS-NEXT:    move $15, $14
-; MIPS-NEXT:  $BB5_6: # %entry
-; MIPS-NEXT:    sllv $1, $4, $9
-; MIPS-NEXT:    not $9, $9
-; MIPS-NEXT:    srl $10, $5, 1
-; MIPS-NEXT:    srlv $9, $10, $9
-; MIPS-NEXT:    or $10, $1, $9
-; MIPS-NEXT:  $BB5_7: # %entry
-; MIPS-NEXT:    addiu $24, $2, -64
-; MIPS-NEXT:    sll $13, $4, 1
-; MIPS-NEXT:    srav $14, $4, $24
-; MIPS-NEXT:    andi $1, $24, 32
-; MIPS-NEXT:    bnez $1, $BB5_10
-; MIPS-NEXT:    sra $9, $4, 31
-; MIPS-NEXT:  # %bb.8: # %entry
-; MIPS-NEXT:    srlv $1, $5, $24
-; MIPS-NEXT:    not $24, $24
-; MIPS-NEXT:    sllv $24, $13, $24
-; MIPS-NEXT:    or $25, $24, $1
-; MIPS-NEXT:    move $24, $14
-; MIPS-NEXT:    sltiu $14, $2, 64
-; MIPS-NEXT:    beqz $14, $BB5_12
-; MIPS-NEXT:    nop
-; MIPS-NEXT:  # %bb.9: # %entry
-; MIPS-NEXT:    b $BB5_11
-; MIPS-NEXT:    nop
-; MIPS-NEXT:  $BB5_10:
-; MIPS-NEXT:    move $25, $14
-; MIPS-NEXT:    sltiu $14, $2, 64
-; MIPS-NEXT:    beqz $14, $BB5_12
-; MIPS-NEXT:    move $24, $9
-; MIPS-NEXT:  $BB5_11:
-; MIPS-NEXT:    or $25, $15, $12
-; MIPS-NEXT:  $BB5_12: # %entry
-; MIPS-NEXT:    sltiu $12, $2, 1
-; MIPS-NEXT:    beqz $12, $BB5_18
-; MIPS-NEXT:    nop
-; MIPS-NEXT:  # %bb.13: # %entry
-; MIPS-NEXT:    bnez $14, $BB5_19
-; MIPS-NEXT:    nop
-; MIPS-NEXT:  $BB5_14: # %entry
-; MIPS-NEXT:    beqz $12, $BB5_20
-; MIPS-NEXT:    nop
-; MIPS-NEXT:  $BB5_15: # %entry
-; MIPS-NEXT:    bnez $3, $BB5_21
-; MIPS-NEXT:    srav $4, $4, $2
-; MIPS-NEXT:  $BB5_16: # %entry
-; MIPS-NEXT:    srlv $1, $5, $2
-; MIPS-NEXT:    sllv $2, $13, $8
-; MIPS-NEXT:    or $3, $2, $1
-; MIPS-NEXT:    bnez $14, $BB5_23
-; MIPS-NEXT:    move $2, $4
-; MIPS-NEXT:  # %bb.17: # %entry
-; MIPS-NEXT:    b $BB5_22
-; MIPS-NEXT:    nop
-; MIPS-NEXT:  $BB5_18: # %entry
-; MIPS-NEXT:    beqz $14, $BB5_14
-; MIPS-NEXT:    move $7, $25
-; MIPS-NEXT:  $BB5_19:
-; MIPS-NEXT:    bnez $12, $BB5_15
-; MIPS-NEXT:    or $24, $11, $10
-; MIPS-NEXT:  $BB5_20: # %entry
-; MIPS-NEXT:    move $6, $24
-; MIPS-NEXT:    beqz $3, $BB5_16
-; MIPS-NEXT:    srav $4, $4, $2
-; MIPS-NEXT:  $BB5_21:
-; MIPS-NEXT:    move $2, $9
-; MIPS-NEXT:    bnez $14, $BB5_23
-; MIPS-NEXT:    move $3, $4
-; MIPS-NEXT:  $BB5_22: # %entry
-; MIPS-NEXT:    move $2, $9
-; MIPS-NEXT:  $BB5_23: # %entry
-; MIPS-NEXT:    bnez $14, $BB5_25
-; MIPS-NEXT:    nop
-; MIPS-NEXT:  # %bb.24: # %entry
-; MIPS-NEXT:    move $3, $9
-; MIPS-NEXT:  $BB5_25: # %entry
-; MIPS-NEXT:    move $4, $6
+; MIPS-NEXT:    addiu $sp, $sp, -32
+; MIPS-NEXT:    .cfi_def_cfa_offset 32
+; MIPS-NEXT:    swl $7, 28($sp)
+; MIPS-NEXT:    swl $6, 24($sp)
+; MIPS-NEXT:    sra $1, $4, 31
+; MIPS-NEXT:    swl $5, 20($sp)
+; MIPS-NEXT:    swl $4, 16($sp)
+; MIPS-NEXT:    swl $1, 12($sp)
+; MIPS-NEXT:    swl $1, 8($sp)
+; MIPS-NEXT:    swl $1, 4($sp)
+; MIPS-NEXT:    swl $1, 0($sp)
+; MIPS-NEXT:    addiu $2, $sp, 0
+; MIPS-NEXT:    swr $7, 31($sp)
+; MIPS-NEXT:    swr $6, 27($sp)
+; MIPS-NEXT:    swr $5, 23($sp)
+; MIPS-NEXT:    swr $4, 19($sp)
+; MIPS-NEXT:    swr $1, 15($sp)
+; MIPS-NEXT:    swr $1, 11($sp)
+; MIPS-NEXT:    swr $1, 7($sp)
+; MIPS-NEXT:    swr $1, 3($sp)
+; MIPS-NEXT:    addiu $1, $2, 16
+; MIPS-NEXT:    lw $2, 60($sp)
+; MIPS-NEXT:    srl $3, $2, 3
+; MIPS-NEXT:    andi $3, $3, 15
+; MIPS-NEXT:    subu $1, $1, $3
+; MIPS-NEXT:    lwl $3, 4($1)
+; MIPS-NEXT:    lwr $3, 7($1)
+; MIPS-NEXT:    sll $4, $3, 1
+; MIPS-NEXT:    lwl $5, 8($1)
+; MIPS-NEXT:    lwr $5, 11($1)
+; MIPS-NEXT:    andi $2, $2, 7
+; MIPS-NEXT:    not $6, $2
+; MIPS-NEXT:    andi $6, $6, 31
+; MIPS-NEXT:    srlv $7, $5, $2
+; MIPS-NEXT:    sllv $4, $4, $6
+; MIPS-NEXT:    srlv $3, $3, $2
+; MIPS-NEXT:    lwl $6, 0($1)
+; MIPS-NEXT:    lwr $6, 3($1)
+; MIPS-NEXT:    sll $8, $6, 1
+; MIPS-NEXT:    xori $9, $2, 31
+; MIPS-NEXT:    sllv $8, $8, $9
+; MIPS-NEXT:    or $3, $3, $8
+; MIPS-NEXT:    or $4, $7, $4
+; MIPS-NEXT:    lwl $7, 12($1)
+; MIPS-NEXT:    lwr $7, 15($1)
+; MIPS-NEXT:    srlv $1, $7, $2
+; MIPS-NEXT:    sll $5, $5, 1
+; MIPS-NEXT:    sllv $5, $5, $9
+; MIPS-NEXT:    or $5, $1, $5
+; MIPS-NEXT:    srav $2, $6, $2
 ; MIPS-NEXT:    jr $ra
-; MIPS-NEXT:    move $5, $7
+; MIPS-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS32-LABEL: ashr_i128:
 ; MIPS32:       # %bb.0: # %entry
-; MIPS32-NEXT:    lw $9, 28($sp)
-; MIPS32-NEXT:    srlv $1, $7, $9
-; MIPS32-NEXT:    not $2, $9
-; MIPS32-NEXT:    sll $3, $6, 1
-; MIPS32-NEXT:    sllv $3, $3, $2
-; MIPS32-NEXT:    addiu $8, $zero, 64
-; MIPS32-NEXT:    or $1, $3, $1
-; MIPS32-NEXT:    srlv $10, $6, $9
-; MIPS32-NEXT:    subu $3, $8, $9
-; MIPS32-NEXT:    sllv $11, $5, $3
-; MIPS32-NEXT:    andi $12, $3, 32
-; MIPS32-NEXT:    andi $13, $9, 32
-; MIPS32-NEXT:    move $8, $11
-; MIPS32-NEXT:    movn $8, $zero, $12
-; MIPS32-NEXT:    movn $1, $10, $13
-; MIPS32-NEXT:    addiu $14, $9, -64
-; MIPS32-NEXT:    srlv $15, $5, $14
-; MIPS32-NEXT:    sll $24, $4, 1
-; MIPS32-NEXT:    not $25, $14
-; MIPS32-NEXT:    sllv $25, $24, $25
-; MIPS32-NEXT:    or $gp, $1, $8
-; MIPS32-NEXT:    or $1, $25, $15
-; MIPS32-NEXT:    srav $8, $4, $14
-; MIPS32-NEXT:    andi $14, $14, 32
-; MIPS32-NEXT:    movn $1, $8, $14
-; MIPS32-NEXT:    sllv $15, $4, $3
-; MIPS32-NEXT:    not $3, $3
-; MIPS32-NEXT:    srl $25, $5, 1
-; MIPS32-NEXT:    srlv $3, $25, $3
-; MIPS32-NEXT:    sltiu $25, $9, 64
-; MIPS32-NEXT:    movn $1, $gp, $25
-; MIPS32-NEXT:    or $15, $15, $3
-; MIPS32-NEXT:    srlv $3, $5, $9
-; MIPS32-NEXT:    sllv $2, $24, $2
-; MIPS32-NEXT:    or $5, $2, $3
-; MIPS32-NEXT:    srav $24, $4, $9
-; MIPS32-NEXT:    movn $5, $24, $13
-; MIPS32-NEXT:    sra $2, $4, 31
-; MIPS32-NEXT:    movz $1, $7, $9
-; MIPS32-NEXT:    move $3, $2
-; MIPS32-NEXT:    movn $3, $5, $25
-; MIPS32-NEXT:    movn $15, $11, $12
-; MIPS32-NEXT:    movn $10, $zero, $13
-; MIPS32-NEXT:    or $4, $10, $15
-; MIPS32-NEXT:    movn $8, $2, $14
-; MIPS32-NEXT:    movn $8, $4, $25
-; MIPS32-NEXT:    movz $8, $6, $9
-; MIPS32-NEXT:    movn $24, $2, $13
-; MIPS32-NEXT:    movn $2, $24, $25
-; MIPS32-NEXT:    move $4, $8
+; MIPS32-NEXT:    addiu $sp, $sp, -32
+; MIPS32-NEXT:    .cfi_def_cfa_offset 32
+; MIPS32-NEXT:    swl $7, 28($sp)
+; MIPS32-NEXT:    swl $6, 24($sp)
+; MIPS32-NEXT:    sra $1, $4, 31
+; MIPS32-NEXT:    swl $5, 20($sp)
+; MIPS32-NEXT:    swl $4, 16($sp)
+; MIPS32-NEXT:    swl $1, 12($sp)
+; MIPS32-NEXT:    swl $1, 8($sp)
+; MIPS32-NEXT:    swl $1, 4($sp)
+; MIPS32-NEXT:    swl $1, 0($sp)
+; MIPS32-NEXT:    addiu $2, $sp, 0
+; MIPS32-NEXT:    swr $7, 31($sp)
+; MIPS32-NEXT:    swr $6, 27($sp)
+; MIPS32-NEXT:    swr $5, 23($sp)
+; MIPS32-NEXT:    swr $4, 19($sp)
+; MIPS32-NEXT:    swr $1, 15($sp)
+; MIPS32-NEXT:    swr $1, 11($sp)
+; MIPS32-NEXT:    swr $1, 7($sp)
+; MIPS32-NEXT:    swr $1, 3($sp)
+; MIPS32-NEXT:    addiu $1, $2, 16
+; MIPS32-NEXT:    lw $2, 60($sp)
+; MIPS32-NEXT:    srl $3, $2, 3
+; MIPS32-NEXT:    andi $3, $3, 15
+; MIPS32-NEXT:    subu $1, $1, $3
+; MIPS32-NEXT:    lwl $3, 4($1)
+; MIPS32-NEXT:    lwr $3, 7($1)
+; MIPS32-NEXT:    sll $4, $3, 1
+; MIPS32-NEXT:    lwl $5, 8($1)
+; MIPS32-NEXT:    lwr $5, 11($1)
+; MIPS32-NEXT:    andi $2, $2, 7
+; MIPS32-NEXT:    not $6, $2
+; MIPS32-NEXT:    andi $6, $6, 31
+; MIPS32-NEXT:    srlv $7, $5, $2
+; MIPS32-NEXT:    sllv $4, $4, $6
+; MIPS32-NEXT:    srlv $3, $3, $2
+; MIPS32-NEXT:    lwl $6, 0($1)
+; MIPS32-NEXT:    lwr $6, 3($1)
+; MIPS32-NEXT:    sll $8, $6, 1
+; MIPS32-NEXT:    xori $9, $2, 31
+; MIPS32-NEXT:    sllv $8, $8, $9
+; MIPS32-NEXT:    or $3, $3, $8
+; MIPS32-NEXT:    or $4, $7, $4
+; MIPS32-NEXT:    lwl $7, 12($1)
+; MIPS32-NEXT:    lwr $7, 15($1)
+; MIPS32-NEXT:    srlv $1, $7, $2
+; MIPS32-NEXT:    sll $5, $5, 1
+; MIPS32-NEXT:    sllv $5, $5, $9
+; MIPS32-NEXT:    or $5, $1, $5
+; MIPS32-NEXT:    srav $2, $6, $2
 ; MIPS32-NEXT:    jr $ra
-; MIPS32-NEXT:    move $5, $1
+; MIPS32-NEXT:    addiu $sp, $sp, 32
 ;
 ; 32R2-LABEL: ashr_i128:
 ; 32R2:       # %bb.0: # %entry
-; 32R2-NEXT:    lw $9, 28($sp)
-; 32R2-NEXT:    srlv $1, $7, $9
-; 32R2-NEXT:    not $2, $9
-; 32R2-NEXT:    sll $3, $6, 1
-; 32R2-NEXT:    sllv $3, $3, $2
-; 32R2-NEXT:    addiu $8, $zero, 64
-; 32R2-NEXT:    or $1, $3, $1
-; 32R2-NEXT:    srlv $10, $6, $9
-; 32R2-NEXT:    subu $3, $8, $9
-; 32R2-NEXT:    sllv $11, $5, $3
-; 32R2-NEXT:    andi $12, $3, 32
-; 32R2-NEXT:    andi $13, $9, 32
-; 32R2-NEXT:    move $8, $11
-; 32R2-NEXT:    movn $8, $zero, $12
-; 32R2-NEXT:    movn $1, $10, $13
-; 32R2-NEXT:    addiu $14, $9, -64
-; 32R2-NEXT:    srlv $15, $5, $14
-; 32R2-NEXT:    sll $24, $4, 1
-; 32R2-NEXT:    not $25, $14
-; 32R2-NEXT:    sllv $25, $24, $25
-; 32R2-NEXT:    or $gp, $1, $8
-; 32R2-NEXT:    or $1, $25, $15
-; 32R2-NEXT:    srav $8, $4, $14
-; 32R2-NEXT:    andi $14, $14, 32
-; 32R2-NEXT:    movn $1, $8, $14
-; 32R2-NEXT:    sllv $15, $4, $3
-; 32R2-NEXT:    not $3, $3
-; 32R2-NEXT:    srl $25, $5, 1
-; 32R2-NEXT:    srlv $3, $25, $3
-; 32R2-NEXT:    sltiu $25, $9, 64
-; 32R2-NEXT:    movn $1, $gp, $25
-; 32R2-NEXT:    or $15, $15, $3
-; 32R2-NEXT:    srlv $3, $5, $9
-; 32R2-NEXT:    sllv $2, $24, $2
-; 32R2-NEXT:    or $5, $2, $3
-; 32R2-NEXT:    srav $24, $4, $9
-; 32R2-NEXT:    movn $5, $24, $13
-; 32R2-NEXT:    sra $2, $4, 31
-; 32R2-NEXT:    movz $1, $7, $9
-; 32R2-NEXT:    move $3, $2
-; 32R2-NEXT:    movn $3, $5, $25
-; 32R2-NEXT:    movn $15, $11, $12
-; 32R2-NEXT:    movn $10, $zero, $13
-; 32R2-NEXT:    or $4, $10, $15
-; 32R2-NEXT:    movn $8, $2, $14
-; 32R2-NEXT:    movn $8, $4, $25
-; 32R2-NEXT:    movz $8, $6, $9
-; 32R2-NEXT:    movn $24, $2, $13
-; 32R2-NEXT:    movn $2, $24, $25
-; 32R2-NEXT:    move $4, $8
+; 32R2-NEXT:    addiu $sp, $sp, -32
+; 32R2-NEXT:    .cfi_def_cfa_offset 32
+; 32R2-NEXT:    swl $7, 28($sp)
+; 32R2-NEXT:    swl $6, 24($sp)
+; 32R2-NEXT:    swl $5, 20($sp)
+; 32R2-NEXT:    sra $1, $4, 31
+; 32R2-NEXT:    swl $4, 16($sp)
+; 32R2-NEXT:    swl $1, 12($sp)
+; 32R2-NEXT:    swl $1, 8($sp)
+; 32R2-NEXT:    swl $1, 4($sp)
+; 32R2-NEXT:    swl $1, 0($sp)
+; 32R2-NEXT:    swr $7, 31($sp)
+; 32R2-NEXT:    swr $6, 27($sp)
+; 32R2-NEXT:    swr $5, 23($sp)
+; 32R2-NEXT:    swr $4, 19($sp)
+; 32R2-NEXT:    swr $1, 15($sp)
+; 32R2-NEXT:    swr $1, 11($sp)
+; 32R2-NEXT:    swr $1, 7($sp)
+; 32R2-NEXT:    swr $1, 3($sp)
+; 32R2-NEXT:    addiu $1, $sp, 0
+; 32R2-NEXT:    addiu $1, $1, 16
+; 32R2-NEXT:    lw $2, 60($sp)
+; 32R2-NEXT:    ext $3, $2, 3, 4
+; 32R2-NEXT:    subu $1, $1, $3
+; 32R2-NEXT:    lwl $3, 4($1)
+; 32R2-NEXT:    lwr $3, 7($1)
+; 32R2-NEXT:    sll $4, $3, 1
+; 32R2-NEXT:    lwl $5, 8($1)
+; 32R2-NEXT:    lwr $5, 11($1)
+; 32R2-NEXT:    andi $2, $2, 7
+; 32R2-NEXT:    not $6, $2
+; 32R2-NEXT:    andi $6, $6, 31
+; 32R2-NEXT:    srlv $7, $5, $2
+; 32R2-NEXT:    sllv $4, $4, $6
+; 32R2-NEXT:    srlv $3, $3, $2
+; 32R2-NEXT:    lwl $6, 0($1)
+; 32R2-NEXT:    lwr $6, 3($1)
+; 32R2-NEXT:    sll $8, $6, 1
+; 32R2-NEXT:    xori $9, $2, 31
+; 32R2-NEXT:    sllv $8, $8, $9
+; 32R2-NEXT:    or $3, $3, $8
+; 32R2-NEXT:    or $4, $7, $4
+; 32R2-NEXT:    lwl $7, 12($1)
+; 32R2-NEXT:    lwr $7, 15($1)
+; 32R2-NEXT:    srlv $1, $7, $2
+; 32R2-NEXT:    sll $5, $5, 1
+; 32R2-NEXT:    sllv $5, $5, $9
+; 32R2-NEXT:    or $5, $1, $5
+; 32R2-NEXT:    srav $2, $6, $2
 ; 32R2-NEXT:    jr $ra
-; 32R2-NEXT:    move $5, $1
+; 32R2-NEXT:    addiu $sp, $sp, 32
 ;
 ; 32R6-LABEL: ashr_i128:
 ; 32R6:       # %bb.0: # %entry
-; 32R6-NEXT:    lw $3, 28($sp)
-; 32R6-NEXT:    addiu $1, $zero, 64
+; 32R6-NEXT:    addiu $sp, $sp, -32
+; 32R6-NEXT:    .cfi_def_cfa_offset 32
+; 32R6-NEXT:    sra $1, $4, 31
+; 32R6-NEXT:    sw $7, 28($sp)
+; 32R6-NEXT:    sw $6, 24($sp)
+; 32R6-NEXT:    sw $5, 20($sp)
+; 32R6-NEXT:    sw $4, 16($sp)
+; 32R6-NEXT:    sw $1, 12($sp)
+; 32R6-NEXT:    sw $1, 8($sp)
+; 32R6-NEXT:    sw $1, 4($sp)
+; 32R6-NEXT:    sw $1, 0($sp)
+; 32R6-NEXT:    addiu $1, $sp, 0
+; 32R6-NEXT:    addiu $1, $1, 16
+; 32R6-NEXT:    lw $2, 60($sp)
+; 32R6-NEXT:    ext $3, $2, 3, 4
 ; 32R6-NEXT:    subu $1, $1, $3
-; 32R6-NEXT:    sllv $2, $5, $1
-; 32R6-NEXT:    andi $8, $1, 32
-; 32R6-NEXT:    selnez $9, $2, $8
-; 32R6-NEXT:    sllv $10, $4, $1
-; 32R6-NEXT:    not $1, $1
-; 32R6-NEXT:    srl $11, $5, 1
-; 32R6-NEXT:    srlv $1, $11, $1
-; 32R6-NEXT:    or $1, $10, $1
-; 32R6-NEXT:    seleqz $1, $1, $8
-; 32R6-NEXT:    or $1, $9, $1
-; 32R6-NEXT:    srlv $9, $7, $3
-; 32R6-NEXT:    not $10, $3
-; 32R6-NEXT:    sll $11, $6, 1
-; 32R6-NEXT:    sllv $11, $11, $10
-; 32R6-NEXT:    or $9, $11, $9
-; 32R6-NEXT:    andi $11, $3, 32
-; 32R6-NEXT:    seleqz $9, $9, $11
-; 32R6-NEXT:    srlv $12, $6, $3
-; 32R6-NEXT:    selnez $13, $12, $11
-; 32R6-NEXT:    seleqz $12, $12, $11
-; 32R6-NEXT:    or $1, $12, $1
-; 32R6-NEXT:    seleqz $2, $2, $8
-; 32R6-NEXT:    or $8, $13, $9
-; 32R6-NEXT:    addiu $9, $3, -64
-; 32R6-NEXT:    srlv $12, $5, $9
-; 32R6-NEXT:    sll $13, $4, 1
-; 32R6-NEXT:    not $14, $9
-; 32R6-NEXT:    sllv $14, $13, $14
-; 32R6-NEXT:    sltiu $15, $3, 64
-; 32R6-NEXT:    or $2, $8, $2
-; 32R6-NEXT:    selnez $1, $1, $15
-; 32R6-NEXT:    or $8, $14, $12
-; 32R6-NEXT:    srav $12, $4, $9
-; 32R6-NEXT:    andi $9, $9, 32
-; 32R6-NEXT:    seleqz $14, $12, $9
-; 32R6-NEXT:    sra $24, $4, 31
-; 32R6-NEXT:    selnez $25, $24, $9
-; 32R6-NEXT:    seleqz $8, $8, $9
-; 32R6-NEXT:    or $14, $25, $14
-; 32R6-NEXT:    seleqz $14, $14, $15
-; 32R6-NEXT:    selnez $9, $12, $9
-; 32R6-NEXT:    seleqz $12, $24, $15
-; 32R6-NEXT:    or $1, $1, $14
-; 32R6-NEXT:    selnez $14, $1, $3
-; 32R6-NEXT:    selnez $1, $2, $15
-; 32R6-NEXT:    or $2, $9, $8
-; 32R6-NEXT:    srav $8, $4, $3
-; 32R6-NEXT:    seleqz $4, $8, $11
-; 32R6-NEXT:    selnez $9, $24, $11
-; 32R6-NEXT:    or $4, $9, $4
-; 32R6-NEXT:    selnez $9, $4, $15
-; 32R6-NEXT:    seleqz $2, $2, $15
-; 32R6-NEXT:    seleqz $4, $6, $3
-; 32R6-NEXT:    seleqz $6, $7, $3
-; 32R6-NEXT:    or $1, $1, $2
-; 32R6-NEXT:    selnez $1, $1, $3
-; 32R6-NEXT:    or $1, $6, $1
-; 32R6-NEXT:    or $4, $4, $14
-; 32R6-NEXT:    or $2, $9, $12
-; 32R6-NEXT:    srlv $3, $5, $3
-; 32R6-NEXT:    sllv $5, $13, $10
-; 32R6-NEXT:    or $3, $5, $3
-; 32R6-NEXT:    seleqz $3, $3, $11
-; 32R6-NEXT:    selnez $5, $8, $11
-; 32R6-NEXT:    or $3, $5, $3
-; 32R6-NEXT:    selnez $3, $3, $15
-; 32R6-NEXT:    or $3, $3, $12
+; 32R6-NEXT:    lw $3, 4($1)
+; 32R6-NEXT:    sll $4, $3, 1
+; 32R6-NEXT:    lw $5, 8($1)
+; 32R6-NEXT:    andi $2, $2, 7
+; 32R6-NEXT:    not $6, $2
+; 32R6-NEXT:    andi $6, $6, 31
+; 32R6-NEXT:    srlv $7, $5, $2
+; 32R6-NEXT:    sllv $4, $4, $6
+; 32R6-NEXT:    srlv $3, $3, $2
+; 32R6-NEXT:    lw $6, 0($1)
+; 32R6-NEXT:    sll $8, $6, 1
+; 32R6-NEXT:    xori $9, $2, 31
+; 32R6-NEXT:    sllv $8, $8, $9
+; 32R6-NEXT:    or $3, $3, $8
+; 32R6-NEXT:    or $4, $7, $4
+; 32R6-NEXT:    lw $1, 12($1)
+; 32R6-NEXT:    srlv $1, $1, $2
+; 32R6-NEXT:    sll $5, $5, 1
+; 32R6-NEXT:    sllv $5, $5, $9
+; 32R6-NEXT:    or $5, $1, $5
+; 32R6-NEXT:    srav $2, $6, $2
 ; 32R6-NEXT:    jr $ra
-; 32R6-NEXT:    move $5, $1
+; 32R6-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS3-LABEL: ashr_i128:
 ; MIPS3:       # %bb.0: # %entry
 ; MIPS3-NEXT:    sll $2, $7, 0
 ; MIPS3-NEXT:    andi $1, $2, 64
 ; MIPS3-NEXT:    bnez $1, .LBB5_2
 ; MIPS3-NEXT:    dsrav $3, $4, $7
 ; MIPS3-NEXT:  # %bb.1: # %entry

 ; MIPS64R6-NEXT:    or $3, $3, $5
 ; MIPS64R6-NEXT:    seleqz $3, $3, $6
 ; MIPS64R6-NEXT:    selnez $1, $1, $6
 ; MIPS64R6-NEXT:    jr $ra
 ; MIPS64R6-NEXT:    or $3, $1, $3
 ;
 ; MMR3-LABEL: ashr_i128:
 ; MMR3:       # %bb.0: # %entry
-; MMR3-NEXT:    addiusp -48
-; MMR3-NEXT:    .cfi_def_cfa_offset 48
-; MMR3-NEXT:    swp $16, 40($sp)
+; MMR3-NEXT:    addiusp -40
+; MMR3-NEXT:    .cfi_def_cfa_offset 40
+; MMR3-NEXT:    swp $16, 32($sp)
 ; MMR3-NEXT:    .cfi_offset 17, -4
 ; MMR3-NEXT:    .cfi_offset 16, -8
-; MMR3-NEXT:    move $8, $7
-; MMR3-NEXT:    sw $6, 32($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sw $5, 36($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sw $4, 8($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    lw $16, 76($sp)
-; MMR3-NEXT:    srlv $4, $7, $16
-; MMR3-NEXT:    not16 $3, $16
-; MMR3-NEXT:    sw $3, 24($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sll16 $2, $6, 1
-; MMR3-NEXT:    sllv $3, $2, $3
-; MMR3-NEXT:    li16 $2, 64
-; MMR3-NEXT:    or16 $3, $4
-; MMR3-NEXT:    srlv $6, $6, $16
-; MMR3-NEXT:    sw $6, 12($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    subu16 $7, $2, $16
-; MMR3-NEXT:    sllv $9, $5, $7
-; MMR3-NEXT:    andi16 $2, $7, 32
-; MMR3-NEXT:    sw $2, 28($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    andi16 $5, $16, 32
-; MMR3-NEXT:    sw $5, 16($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    move $4, $9
-; MMR3-NEXT:    li16 $17, 0
-; MMR3-NEXT:    movn $4, $17, $2
-; MMR3-NEXT:    movn $3, $6, $5
-; MMR3-NEXT:    addiu $2, $16, -64
-; MMR3-NEXT:    lw $5, 36($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    srlv $5, $5, $2
-; MMR3-NEXT:    sw $5, 20($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    lw $17, 8($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    sll16 $6, $17, 1
-; MMR3-NEXT:    sw $6, 4($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    not16 $5, $2
-; MMR3-NEXT:    sllv $5, $6, $5
-; MMR3-NEXT:    or16 $3, $4
-; MMR3-NEXT:    lw $4, 20($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    or16 $5, $4
-; MMR3-NEXT:    srav $1, $17, $2
-; MMR3-NEXT:    andi16 $2, $2, 32
-; MMR3-NEXT:    sw $2, 20($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $5, $1, $2
-; MMR3-NEXT:    sllv $2, $17, $7
-; MMR3-NEXT:    not16 $4, $7
-; MMR3-NEXT:    lw $7, 36($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    srl16 $6, $7, 1
-; MMR3-NEXT:    srlv $6, $6, $4
-; MMR3-NEXT:    sltiu $10, $16, 64
-; MMR3-NEXT:    movn $5, $3, $10
-; MMR3-NEXT:    or16 $6, $2
-; MMR3-NEXT:    srlv $2, $7, $16
-; MMR3-NEXT:    lw $3, 24($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    lw $4, 4($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    sllv $3, $4, $3
-; MMR3-NEXT:    or16 $3, $2
-; MMR3-NEXT:    srav $11, $17, $16
-; MMR3-NEXT:    lw $4, 16($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $3, $11, $4
-; MMR3-NEXT:    sra $2, $17, 31
-; MMR3-NEXT:    movz $5, $8, $16
-; MMR3-NEXT:    move $8, $2
-; MMR3-NEXT:    movn $8, $3, $10
-; MMR3-NEXT:    lw $3, 28($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $6, $9, $3
-; MMR3-NEXT:    li16 $3, 0
-; MMR3-NEXT:    lw $7, 12($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $7, $3, $4
-; MMR3-NEXT:    or16 $7, $6
-; MMR3-NEXT:    lw $3, 20($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $1, $2, $3
-; MMR3-NEXT:    movn $1, $7, $10
-; MMR3-NEXT:    lw $3, 32($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movz $1, $3, $16
-; MMR3-NEXT:    movn $11, $2, $4
-; MMR3-NEXT:    movn $2, $11, $10
-; MMR3-NEXT:    move $3, $8
-; MMR3-NEXT:    move $4, $1
-; MMR3-NEXT:    lwp $16, 40($sp)
-; MMR3-NEXT:    addiusp 48
+; MMR3-NEXT:    swl $7, 28($sp)
+; MMR3-NEXT:    swl $6, 24($sp)
+; MMR3-NEXT:    swl $5, 20($sp)
+; MMR3-NEXT:    sra $1, $4, 31
+; MMR3-NEXT:    swl $4, 16($sp)
+; MMR3-NEXT:    swl $1, 12($sp)
+; MMR3-NEXT:    swl $1, 8($sp)
+; MMR3-NEXT:    swl $1, 4($sp)
+; MMR3-NEXT:    swl $1, 0($sp)
+; MMR3-NEXT:    swr $7, 31($sp)
+; MMR3-NEXT:    swr $6, 27($sp)
+; MMR3-NEXT:    swr $5, 23($sp)
+; MMR3-NEXT:    swr $4, 19($sp)
+; MMR3-NEXT:    swr $1, 15($sp)
+; MMR3-NEXT:    swr $1, 11($sp)
+; MMR3-NEXT:    swr $1, 7($sp)
+; MMR3-NEXT:    swr $1, 3($sp)
+; MMR3-NEXT:    addiur1sp $2, 0
+; MMR3-NEXT:    addiur2 $2, $2, 16
+; MMR3-NEXT:    lw $3, 68($sp)
+; MMR3-NEXT:    ext $4, $3, 3, 4
+; MMR3-NEXT:    subu16 $2, $2, $4
+; MMR3-NEXT:    lwl $7, 4($2)
+; MMR3-NEXT:    lwr $7, 7($2)
+; MMR3-NEXT:    sll16 $4, $7, 1
+; MMR3-NEXT:    lwl $5, 8($2)
+; MMR3-NEXT:    lwr $5, 11($2)
+; MMR3-NEXT:    andi16 $6, $3, 7
+; MMR3-NEXT:    not16 $3, $6
+; MMR3-NEXT:    andi16 $3, $3, 31
+; MMR3-NEXT:    srlv $16, $5, $6
+; MMR3-NEXT:    sllv $4, $4, $3
+; MMR3-NEXT:    srlv $17, $7, $6
+; MMR3-NEXT:    lwl $7, 0($2)
+; MMR3-NEXT:    lwr $7, 3($2)
+; MMR3-NEXT:    sll16 $3, $7, 1
+; MMR3-NEXT:    xori $1, $6, 31
+; MMR3-NEXT:    sllv $3, $3, $1
+; MMR3-NEXT:    or16 $3, $17
+; MMR3-NEXT:    or16 $4, $16
+; MMR3-NEXT:    lwl $8, 12($2)
+; MMR3-NEXT:    lwr $8, 15($2)
+; MMR3-NEXT:    srlv $2, $8, $6
+; MMR3-NEXT:    sll16 $5, $5, 1
+; MMR3-NEXT:    sllv $5, $5, $1
+; MMR3-NEXT:    or16 $5, $2
+; MMR3-NEXT:    srav $2, $7, $6
+; MMR3-NEXT:    lwp $16, 32($sp)
+; MMR3-NEXT:    addiusp 40
 ; MMR3-NEXT:    jrc $ra
 ;
 ; MMR6-LABEL: ashr_i128:
 ; MMR6:       # %bb.0: # %entry
-; MMR6-NEXT:    addiu $sp, $sp, -16
-; MMR6-NEXT:    .cfi_def_cfa_offset 16
-; MMR6-NEXT:    sw $17, 12($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    sw $16, 8($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    .cfi_offset 17, -4
-; MMR6-NEXT:    .cfi_offset 16, -8
-; MMR6-NEXT:    move $1, $7
-; MMR6-NEXT:    lw $3, 44($sp)
-; MMR6-NEXT:    li16 $2, 64
-; MMR6-NEXT:    subu16 $7, $2, $3
-; MMR6-NEXT:    sllv $8, $5, $7
-; MMR6-NEXT:    andi16 $2, $7, 32
-; MMR6-NEXT:    selnez $9, $8, $2
-; MMR6-NEXT:    sllv $10, $4, $7
-; MMR6-NEXT:    not16 $7, $7
-; MMR6-NEXT:    srl16 $16, $5, 1
-; MMR6-NEXT:    srlv $7, $16, $7
-; MMR6-NEXT:    or $7, $10, $7
-; MMR6-NEXT:    seleqz $7, $7, $2
-; MMR6-NEXT:    or $7, $9, $7
-; MMR6-NEXT:    srlv $9, $1, $3
-; MMR6-NEXT:    not16 $16, $3
-; MMR6-NEXT:    sw $16, 4($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    sll16 $17, $6, 1
-; MMR6-NEXT:    sllv $10, $17, $16
-; MMR6-NEXT:    or $9, $10, $9
-; MMR6-NEXT:    andi16 $17, $3, 32
-; MMR6-NEXT:    seleqz $9, $9, $17
-; MMR6-NEXT:    srlv $10, $6, $3
-; MMR6-NEXT:    selnez $11, $10, $17
-; MMR6-NEXT:    seleqz $10, $10, $17
-; MMR6-NEXT:    or $10, $10, $7
-; MMR6-NEXT:    seleqz $12, $8, $2
-; MMR6-NEXT:    or $8, $11, $9
-; MMR6-NEXT:    addiu $2, $3, -64
-; MMR6-NEXT:    srlv $9, $5, $2
-; MMR6-NEXT:    sll16 $7, $4, 1
-; MMR6-NEXT:    not16 $16, $2
-; MMR6-NEXT:    sllv $11, $7, $16
-; MMR6-NEXT:    sltiu $13, $3, 64
-; MMR6-NEXT:    or $8, $8, $12
-; MMR6-NEXT:    selnez $10, $10, $13
-; MMR6-NEXT:    or $9, $11, $9
-; MMR6-NEXT:    srav $11, $4, $2
-; MMR6-NEXT:    andi16 $2, $2, 32
-; MMR6-NEXT:    seleqz $12, $11, $2
-; MMR6-NEXT:    sra $14, $4, 31
-; MMR6-NEXT:    selnez $15, $14, $2
-; MMR6-NEXT:    seleqz $9, $9, $2
-; MMR6-NEXT:    or $12, $15, $12
-; MMR6-NEXT:    seleqz $12, $12, $13
-; MMR6-NEXT:    selnez $2, $11, $2
-; MMR6-NEXT:    seleqz $11, $14, $13
-; MMR6-NEXT:    or $10, $10, $12
-; MMR6-NEXT:    selnez $10, $10, $3
-; MMR6-NEXT:    selnez $8, $8, $13
-; MMR6-NEXT:    or $2, $2, $9
-; MMR6-NEXT:    srav $9, $4, $3
-; MMR6-NEXT:    seleqz $4, $9, $17
-; MMR6-NEXT:    selnez $12, $14, $17
-; MMR6-NEXT:    or $4, $12, $4
-; MMR6-NEXT:    selnez $12, $4, $13
-; MMR6-NEXT:    seleqz $2, $2, $13
-; MMR6-NEXT:    seleqz $4, $6, $3
-; MMR6-NEXT:    seleqz $1, $1, $3
-; MMR6-NEXT:    or $2, $8, $2
-; MMR6-NEXT:    selnez $2, $2, $3
-; MMR6-NEXT:    or $1, $1, $2
-; MMR6-NEXT:    or $4, $4, $10
-; MMR6-NEXT:    or $2, $12, $11
-; MMR6-NEXT:    srlv $3, $5, $3
-; MMR6-NEXT:    lw $5, 4($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    sllv $5, $7, $5
-; MMR6-NEXT:    or $3, $5, $3
-; MMR6-NEXT:    seleqz $3, $3, $17
-; MMR6-NEXT:    selnez $5, $9, $17
-; MMR6-NEXT:    or $3, $5, $3
-; MMR6-NEXT:    selnez $3, $3, $13
-; MMR6-NEXT:    or $3, $3, $11
-; MMR6-NEXT:    move $5, $1
-; MMR6-NEXT:    lw $16, 8($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    lw $17, 12($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    addiu $sp, $sp, 16
+; MMR6-NEXT:    addiu $sp, $sp, -40
+; MMR6-NEXT:    .cfi_def_cfa_offset 40
+; MMR6-NEXT:    sw $16, 36($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    .cfi_offset 16, -4
+; MMR6-NEXT:    sra $1, $4, 31
+; MMR6-NEXT:    sw $7, 32($sp)
+; MMR6-NEXT:    sw $6, 28($sp)
+; MMR6-NEXT:    sw $5, 24($sp)
+; MMR6-NEXT:    sw $4, 20($sp)
+; MMR6-NEXT:    sw $1, 16($sp)
+; MMR6-NEXT:    sw $1, 12($sp)
+; MMR6-NEXT:    sw $1, 8($sp)
+; MMR6-NEXT:    sw $1, 4($sp)
+; MMR6-NEXT:    addiu $2, $sp, 4
+; MMR6-NEXT:    addiur2 $2, $2, 16
+; MMR6-NEXT:    lw $3, 68($sp)
+; MMR6-NEXT:    ext $4, $3, 3, 4
+; MMR6-NEXT:    subu16 $5, $2, $4
+; MMR6-NEXT:    lw16 $4, 4($5)
+; MMR6-NEXT:    sll16 $6, $4, 1
+; MMR6-NEXT:    lw16 $7, 8($5)
+; MMR6-NEXT:    andi16 $2, $3, 7
+; MMR6-NEXT:    not16 $3, $2
+; MMR6-NEXT:    andi16 $3, $3, 31
+; MMR6-NEXT:    srlv $1, $7, $2
+; MMR6-NEXT:    sllv $6, $6, $3
+; MMR6-NEXT:    srlv $3, $4, $2
+; MMR6-NEXT:    lw16 $16, 0($5)
+; MMR6-NEXT:    sll16 $4, $16, 1
+; MMR6-NEXT:    xori $8, $2, 31
+; MMR6-NEXT:    sllv $4, $4, $8
+; MMR6-NEXT:    or $3, $3, $4
+; MMR6-NEXT:    or $4, $1, $6
+; MMR6-NEXT:    lw16 $5, 12($5)
+; MMR6-NEXT:    srlv $1, $5, $2
+; MMR6-NEXT:    sll16 $5, $7, 1
+; MMR6-NEXT:    sllv $5, $5, $8
+; MMR6-NEXT:    or $5, $1, $5
+; MMR6-NEXT:    srav $2, $16, $2
+; MMR6-NEXT:    lw $16, 36($sp) # 4-byte Folded Reload
+; MMR6-NEXT:    addiu $sp, $sp, 40
 ; MMR6-NEXT:    jrc $ra
 entry:
   %r = ashr i128 %a, %b
   ret i128 %r
 }

llvm/test/CodeGen/Mips/llvm-ir/lshr.ll

entry:
 
   %r = lshr i64 %a, %b
   ret i64 %r
 }
 
 define signext i128 @lshr_i128(i128 signext %a, i128 signext %b) {
 ; MIPS2-LABEL: lshr_i128:
 ; MIPS2:       # %bb.0: # %entry
-; MIPS2-NEXT:    lw $2, 28($sp)
-; MIPS2-NEXT:    addiu $1, $zero, 64
-; MIPS2-NEXT:    subu $12, $1, $2
-; MIPS2-NEXT:    sllv $10, $5, $12
-; MIPS2-NEXT:    andi $15, $12, 32
-; MIPS2-NEXT:    andi $8, $2, 32
-; MIPS2-NEXT:    addiu $3, $zero, 0
-; MIPS2-NEXT:    bnez $15, $BB5_2
-; MIPS2-NEXT:    addiu $13, $zero, 0
-; MIPS2-NEXT:  # %bb.1: # %entry
-; MIPS2-NEXT:    move $13, $10
-; MIPS2-NEXT:  $BB5_2: # %entry
-; MIPS2-NEXT:    not $9, $2
-; MIPS2-NEXT:    bnez $8, $BB5_5
-; MIPS2-NEXT:    srlv $24, $6, $2
-; MIPS2-NEXT:  # %bb.3: # %entry
-; MIPS2-NEXT:    sll $1, $6, 1
-; MIPS2-NEXT:    srlv $11, $7, $2
-; MIPS2-NEXT:    sllv $1, $1, $9
-; MIPS2-NEXT:    or $14, $1, $11
-; MIPS2-NEXT:    bnez $15, $BB5_7
-; MIPS2-NEXT:    move $11, $24
-; MIPS2-NEXT:  # %bb.4: # %entry
-; MIPS2-NEXT:    b $BB5_6
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  $BB5_5:
-; MIPS2-NEXT:    addiu $11, $zero, 0
-; MIPS2-NEXT:    bnez $15, $BB5_7
-; MIPS2-NEXT:    move $14, $24
-; MIPS2-NEXT:  $BB5_6: # %entry
-; MIPS2-NEXT:    sllv $1, $4, $12
-; MIPS2-NEXT:    not $10, $12
-; MIPS2-NEXT:    srl $12, $5, 1
-; MIPS2-NEXT:    srlv $10, $12, $10
-; MIPS2-NEXT:    or $10, $1, $10
-; MIPS2-NEXT:  $BB5_7: # %entry
-; MIPS2-NEXT:    addiu $15, $2, -64
-; MIPS2-NEXT:    sll $12, $4, 1
-; MIPS2-NEXT:    andi $1, $15, 32
-; MIPS2-NEXT:    bnez $1, $BB5_10
-; MIPS2-NEXT:    srlv $25, $4, $15
-; MIPS2-NEXT:  # %bb.8: # %entry
-; MIPS2-NEXT:    srlv $1, $5, $15
-; MIPS2-NEXT:    not $15, $15
-; MIPS2-NEXT:    sllv $15, $12, $15
-; MIPS2-NEXT:    or $24, $15, $1
-; MIPS2-NEXT:    move $15, $25
-; MIPS2-NEXT:    sltiu $25, $2, 64
-; MIPS2-NEXT:    beqz $25, $BB5_12
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  # %bb.9: # %entry
-; MIPS2-NEXT:    b $BB5_11
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  $BB5_10:
-; MIPS2-NEXT:    move $24, $25
-; MIPS2-NEXT:    sltiu $25, $2, 64
-; MIPS2-NEXT:    beqz $25, $BB5_12
-; MIPS2-NEXT:    addiu $15, $zero, 0
-; MIPS2-NEXT:  $BB5_11:
-; MIPS2-NEXT:    or $24, $14, $13
-; MIPS2-NEXT:  $BB5_12: # %entry
-; MIPS2-NEXT:    sltiu $13, $2, 1
-; MIPS2-NEXT:    beqz $13, $BB5_19
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  # %bb.13: # %entry
-; MIPS2-NEXT:    bnez $25, $BB5_20
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  $BB5_14: # %entry
-; MIPS2-NEXT:    bnez $13, $BB5_16
-; MIPS2-NEXT:    addiu $10, $zero, 63
-; MIPS2-NEXT:  $BB5_15: # %entry
-; MIPS2-NEXT:    move $6, $15
-; MIPS2-NEXT:  $BB5_16: # %entry
-; MIPS2-NEXT:    sltu $10, $10, $2
-; MIPS2-NEXT:    bnez $8, $BB5_22
-; MIPS2-NEXT:    srlv $11, $4, $2
-; MIPS2-NEXT:  # %bb.17: # %entry
-; MIPS2-NEXT:    srlv $1, $5, $2
-; MIPS2-NEXT:    sllv $2, $12, $9
-; MIPS2-NEXT:    or $4, $2, $1
-; MIPS2-NEXT:    move $5, $11
-; MIPS2-NEXT:    bnez $10, $BB5_24
-; MIPS2-NEXT:    addiu $2, $zero, 0
-; MIPS2-NEXT:  # %bb.18: # %entry
-; MIPS2-NEXT:    b $BB5_23
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  $BB5_19: # %entry
-; MIPS2-NEXT:    beqz $25, $BB5_14
-; MIPS2-NEXT:    move $7, $24
-; MIPS2-NEXT:  $BB5_20:
-; MIPS2-NEXT:    or $15, $11, $10
-; MIPS2-NEXT:    bnez $13, $BB5_16
-; MIPS2-NEXT:    addiu $10, $zero, 63
-; MIPS2-NEXT:  # %bb.21:
-; MIPS2-NEXT:    b $BB5_15
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  $BB5_22:
-; MIPS2-NEXT:    addiu $5, $zero, 0
-; MIPS2-NEXT:    move $4, $11
-; MIPS2-NEXT:    bnez $10, $BB5_24
-; MIPS2-NEXT:    addiu $2, $zero, 0
-; MIPS2-NEXT:  $BB5_23: # %entry
-; MIPS2-NEXT:    move $2, $5
-; MIPS2-NEXT:  $BB5_24: # %entry
-; MIPS2-NEXT:    bnez $10, $BB5_26
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  # %bb.25: # %entry
-; MIPS2-NEXT:    move $3, $4
-; MIPS2-NEXT:  $BB5_26: # %entry
-; MIPS2-NEXT:    move $4, $6
+; MIPS2-NEXT:    addiu $sp, $sp, -32
+; MIPS2-NEXT:    .cfi_def_cfa_offset 32
+; MIPS2-NEXT:    swl $7, 28($sp)
+; MIPS2-NEXT:    swl $6, 24($sp)
+; MIPS2-NEXT:    swl $5, 20($sp)
+; MIPS2-NEXT:    swl $4, 16($sp)
+; MIPS2-NEXT:    swl $zero, 12($sp)
+; MIPS2-NEXT:    swl $zero, 8($sp)
+; MIPS2-NEXT:    swl $zero, 4($sp)
+; MIPS2-NEXT:    swl $zero, 0($sp)
+; MIPS2-NEXT:    addiu $1, $sp, 0
+; MIPS2-NEXT:    swr $7, 31($sp)
+; MIPS2-NEXT:    swr $6, 27($sp)
+; MIPS2-NEXT:    swr $5, 23($sp)
+; MIPS2-NEXT:    swr $4, 19($sp)
+; MIPS2-NEXT:    swr $zero, 15($sp)
+; MIPS2-NEXT:    swr $zero, 11($sp)
+; MIPS2-NEXT:    swr $zero, 7($sp)
+; MIPS2-NEXT:    swr $zero, 3($sp)
+; MIPS2-NEXT:    addiu $1, $1, 16
+; MIPS2-NEXT:    lw $2, 60($sp)
+; MIPS2-NEXT:    srl $3, $2, 3
+; MIPS2-NEXT:    andi $3, $3, 15
+; MIPS2-NEXT:    subu $1, $1, $3
+; MIPS2-NEXT:    lwl $3, 4($1)
+; MIPS2-NEXT:    lwr $3, 7($1)
+; MIPS2-NEXT:    sll $4, $3, 1
+; MIPS2-NEXT:    lwl $5, 8($1)
+; MIPS2-NEXT:    lwr $5, 11($1)
+; MIPS2-NEXT:    andi $2, $2, 7
+; MIPS2-NEXT:    not $6, $2
+; MIPS2-NEXT:    andi $6, $6, 31
+; MIPS2-NEXT:    srlv $7, $5, $2
+; MIPS2-NEXT:    sllv $4, $4, $6
+; MIPS2-NEXT:    srlv $3, $3, $2
+; MIPS2-NEXT:    lwl $6, 0($1)
+; MIPS2-NEXT:    lwr $6, 3($1)
+; MIPS2-NEXT:    sll $8, $6, 1
+; MIPS2-NEXT:    xori $9, $2, 31
+; MIPS2-NEXT:    sllv $8, $8, $9
+; MIPS2-NEXT:    or $3, $3, $8
+; MIPS2-NEXT:    or $4, $7, $4
+; MIPS2-NEXT:    lwl $7, 12($1)
+; MIPS2-NEXT:    lwr $7, 15($1)
+; MIPS2-NEXT:    srlv $1, $7, $2
+; MIPS2-NEXT:    sll $5, $5, 1
+; MIPS2-NEXT:    sllv $5, $5, $9
+; MIPS2-NEXT:    or $5, $1, $5
+; MIPS2-NEXT:    srlv $2, $6, $2
 ; MIPS2-NEXT:    jr $ra
-; MIPS2-NEXT:    move $5, $7
+; MIPS2-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS32-LABEL: lshr_i128:
 ; MIPS32:       # %bb.0: # %entry
-; MIPS32-NEXT:    lw $9, 28($sp)
-; MIPS32-NEXT:    addiu $1, $zero, 64
-; MIPS32-NEXT:    subu $2, $1, $9
-; MIPS32-NEXT:    sllv $10, $5, $2
-; MIPS32-NEXT:    andi $11, $2, 32
-; MIPS32-NEXT:    move $1, $10
-; MIPS32-NEXT:    movn $1, $zero, $11
-; MIPS32-NEXT:    srlv $3, $7, $9
-; MIPS32-NEXT:    not $12, $9
+; MIPS32-NEXT:    addiu $sp, $sp, -32
+; MIPS32-NEXT:    .cfi_def_cfa_offset 32
+; MIPS32-NEXT:    swl $7, 28($sp)
+; MIPS32-NEXT:    swl $6, 24($sp)
+; MIPS32-NEXT:    swl $5, 20($sp)
+; MIPS32-NEXT:    swl $4, 16($sp)
+; MIPS32-NEXT:    swl $zero, 12($sp)
+; MIPS32-NEXT:    swl $zero, 8($sp)
+; MIPS32-NEXT:    swl $zero, 4($sp)
+; MIPS32-NEXT:    swl $zero, 0($sp)
+; MIPS32-NEXT:    addiu $1, $sp, 0
+; MIPS32-NEXT:    swr $7, 31($sp)
+; MIPS32-NEXT:    swr $6, 27($sp)
+; MIPS32-NEXT:    swr $5, 23($sp)
+; MIPS32-NEXT:    swr $4, 19($sp)
+; MIPS32-NEXT:    swr $zero, 15($sp)
+; MIPS32-NEXT:    swr $zero, 11($sp)
+; MIPS32-NEXT:    swr $zero, 7($sp)
+; MIPS32-NEXT:    swr $zero, 3($sp)
+; MIPS32-NEXT:    addiu $1, $1, 16
+; MIPS32-NEXT:    lw $2, 60($sp)
+; MIPS32-NEXT:    srl $3, $2, 3
+; MIPS32-NEXT:    andi $3, $3, 15
+; MIPS32-NEXT:    subu $1, $1, $3
+; MIPS32-NEXT:    lwl $3, 4($1)
+; MIPS32-NEXT:    lwr $3, 7($1)
+; MIPS32-NEXT:    sll $4, $3, 1
+; MIPS32-NEXT:    lwl $5, 8($1)
+; MIPS32-NEXT:    lwr $5, 11($1)
+; MIPS32-NEXT:    andi $2, $2, 7
+; MIPS32-NEXT:    not $6, $2
+; MIPS32-NEXT:    andi $6, $6, 31
+; MIPS32-NEXT:    srlv $7, $5, $2
+; MIPS32-NEXT:    sllv $4, $4, $6
+; MIPS32-NEXT:    srlv $3, $3, $2
+; MIPS32-NEXT:    lwl $6, 0($1)
+; MIPS32-NEXT:    lwr $6, 3($1)
 ; MIPS32-NEXT:    sll $8, $6, 1
-; MIPS32-NEXT:    sllv $8, $8, $12
-; MIPS32-NEXT:    or $3, $8, $3
-; MIPS32-NEXT:    srlv $13, $6, $9
-; MIPS32-NEXT:    andi $14, $9, 32
-; MIPS32-NEXT:    movn $3, $13, $14
-; MIPS32-NEXT:    addiu $15, $9, -64
-; MIPS32-NEXT:    or $3, $3, $1
-; MIPS32-NEXT:    srlv $1, $5, $15
-; MIPS32-NEXT:    sll $24, $4, 1
-; MIPS32-NEXT:    not $8, $15
-; MIPS32-NEXT:    sllv $8, $24, $8
-; MIPS32-NEXT:    or $1, $8, $1
-; MIPS32-NEXT:    srlv $8, $4, $15
-; MIPS32-NEXT:    andi $15, $15, 32
-; MIPS32-NEXT:    movn $1, $8, $15
-; MIPS32-NEXT:    sltiu $25, $9, 64
-; MIPS32-NEXT:    movn $1, $3, $25
-; MIPS32-NEXT:    sllv $3, $4, $2
-; MIPS32-NEXT:    not $2, $2
-; MIPS32-NEXT:    srl $gp, $5, 1
-; MIPS32-NEXT:    srlv $2, $gp, $2
-; MIPS32-NEXT:    or $gp, $3, $2
-; MIPS32-NEXT:    srlv $2, $5, $9
-; MIPS32-NEXT:    sllv $3, $24, $12
-; MIPS32-NEXT:    or $3, $3, $2
-; MIPS32-NEXT:    srlv $2, $4, $9
-; MIPS32-NEXT:    movn $3, $2, $14
-; MIPS32-NEXT:    movz $1, $7, $9
-; MIPS32-NEXT:    movz $3, $zero, $25
-; MIPS32-NEXT:    movn $gp, $10, $11
-; MIPS32-NEXT:    movn $13, $zero, $14
-; MIPS32-NEXT:    or $4, $13, $gp
-; MIPS32-NEXT:    movn $8, $zero, $15
-; MIPS32-NEXT:    movn $8, $4, $25
-; MIPS32-NEXT:    movz $8, $6, $9
-; MIPS32-NEXT:    movn $2, $zero, $14
-; MIPS32-NEXT:    movz $2, $zero, $25
-; MIPS32-NEXT:    move $4, $8
+; MIPS32-NEXT:    xori $9, $2, 31
+; MIPS32-NEXT:    sllv $8, $8, $9
+; MIPS32-NEXT:    or $3, $3, $8
+; MIPS32-NEXT:    or $4, $7, $4
+; MIPS32-NEXT:    lwl $7, 12($1)
+; MIPS32-NEXT:    lwr $7, 15($1)
+; MIPS32-NEXT:    srlv $1, $7, $2
+; MIPS32-NEXT:    sll $5, $5, 1
+; MIPS32-NEXT:    sllv $5, $5, $9
+; MIPS32-NEXT:    or $5, $1, $5
+; MIPS32-NEXT:    srlv $2, $6, $2
 ; MIPS32-NEXT:    jr $ra
-; MIPS32-NEXT:    move $5, $1
+; MIPS32-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS32R2-LABEL: lshr_i128:
 ; MIPS32R2:       # %bb.0: # %entry
-; MIPS32R2-NEXT:    lw $9, 28($sp)
-; MIPS32R2-NEXT:    addiu $1, $zero, 64
-; MIPS32R2-NEXT:    subu $2, $1, $9
-; MIPS32R2-NEXT:    sllv $10, $5, $2
-; MIPS32R2-NEXT:    andi $11, $2, 32
-; MIPS32R2-NEXT:    move $1, $10
-; MIPS32R2-NEXT:    movn $1, $zero, $11
-; MIPS32R2-NEXT:    srlv $3, $7, $9
-; MIPS32R2-NEXT:    not $12, $9
+; MIPS32R2-NEXT:    addiu $sp, $sp, -32
+; MIPS32R2-NEXT:    .cfi_def_cfa_offset 32
+; MIPS32R2-NEXT:    swl $7, 28($sp)
+; MIPS32R2-NEXT:    swl $6, 24($sp)
+; MIPS32R2-NEXT:    swl $5, 20($sp)
+; MIPS32R2-NEXT:    swl $4, 16($sp)
+; MIPS32R2-NEXT:    swl $zero, 12($sp)
+; MIPS32R2-NEXT:    swl $zero, 8($sp)
+; MIPS32R2-NEXT:    swl $zero, 4($sp)
+; MIPS32R2-NEXT:    swl $zero, 0($sp)
+; MIPS32R2-NEXT:    swr $7, 31($sp)
+; MIPS32R2-NEXT:    swr $6, 27($sp)
+; MIPS32R2-NEXT:    swr $5, 23($sp)
+; MIPS32R2-NEXT:    swr $4, 19($sp)
+; MIPS32R2-NEXT:    swr $zero, 15($sp)
+; MIPS32R2-NEXT:    swr $zero, 11($sp)
+; MIPS32R2-NEXT:    swr $zero, 7($sp)
+; MIPS32R2-NEXT:    swr $zero, 3($sp)
+; MIPS32R2-NEXT:    addiu $1, $sp, 0
+; MIPS32R2-NEXT:    addiu $1, $1, 16
+; MIPS32R2-NEXT:    lw $2, 60($sp)
+; MIPS32R2-NEXT:    ext $3, $2, 3, 4
+; MIPS32R2-NEXT:    subu $1, $1, $3
+; MIPS32R2-NEXT:    lwl $3, 4($1)
+; MIPS32R2-NEXT:    lwr $3, 7($1)
+; MIPS32R2-NEXT:    sll $4, $3, 1
+; MIPS32R2-NEXT:    lwl $5, 8($1)
+; MIPS32R2-NEXT:    lwr $5, 11($1)
+; MIPS32R2-NEXT:    andi $2, $2, 7
+; MIPS32R2-NEXT:    not $6, $2
+; MIPS32R2-NEXT:    andi $6, $6, 31
+; MIPS32R2-NEXT:    srlv $7, $5, $2
+; MIPS32R2-NEXT:    sllv $4, $4, $6
+; MIPS32R2-NEXT:    srlv $3, $3, $2
+; MIPS32R2-NEXT:    lwl $6, 0($1)
+; MIPS32R2-NEXT:    lwr $6, 3($1)
 ; MIPS32R2-NEXT:    sll $8, $6, 1
-; MIPS32R2-NEXT:    sllv $8, $8, $12
-; MIPS32R2-NEXT:    or $3, $8, $3
-; MIPS32R2-NEXT:    srlv $13, $6, $9
-; MIPS32R2-NEXT:    andi $14, $9, 32
-; MIPS32R2-NEXT:    movn $3, $13, $14
-; MIPS32R2-NEXT:    addiu $15, $9, -64
-; MIPS32R2-NEXT:    or $3, $3, $1
-; MIPS32R2-NEXT:    srlv $1, $5, $15
-; MIPS32R2-NEXT:    sll $24, $4, 1
-; MIPS32R2-NEXT:    not $8, $15
-; MIPS32R2-NEXT:    sllv $8, $24, $8
-; MIPS32R2-NEXT:    or $1, $8, $1
-; MIPS32R2-NEXT:    srlv $8, $4, $15
-; MIPS32R2-NEXT:    andi $15, $15, 32
-; MIPS32R2-NEXT:    movn $1, $8, $15
-; MIPS32R2-NEXT:    sltiu $25, $9, 64
-; MIPS32R2-NEXT:    movn $1, $3, $25
-; MIPS32R2-NEXT:    sllv $3, $4, $2
-; MIPS32R2-NEXT:    not $2, $2
-; MIPS32R2-NEXT:    srl $gp, $5, 1
-; MIPS32R2-NEXT:    srlv $2, $gp, $2
-; MIPS32R2-NEXT:    or $gp, $3, $2
-; MIPS32R2-NEXT:    srlv $2, $5, $9
-; MIPS32R2-NEXT:    sllv $3, $24, $12
-; MIPS32R2-NEXT:    or $3, $3, $2
-; MIPS32R2-NEXT:    srlv $2, $4, $9
-; MIPS32R2-NEXT:    movn $3, $2, $14
-; MIPS32R2-NEXT:    movz $1, $7, $9
-; MIPS32R2-NEXT:    movz $3, $zero, $25
-; MIPS32R2-NEXT:    movn $gp, $10, $11
-; MIPS32R2-NEXT:    movn $13, $zero, $14
-; MIPS32R2-NEXT:    or $4, $13, $gp
-; MIPS32R2-NEXT:    movn $8, $zero, $15
-; MIPS32R2-NEXT:    movn $8, $4, $25
-; MIPS32R2-NEXT:    movz $8, $6, $9
-; MIPS32R2-NEXT:    movn $2, $zero, $14
-; MIPS32R2-NEXT:    movz $2, $zero, $25
-; MIPS32R2-NEXT:    move $4, $8
+; MIPS32R2-NEXT:    xori $9, $2, 31
+; MIPS32R2-NEXT:    sllv $8, $8, $9
+; MIPS32R2-NEXT:    or $3, $3, $8
+; MIPS32R2-NEXT:    or $4, $7, $4
+; MIPS32R2-NEXT:    lwl $7, 12($1)
+; MIPS32R2-NEXT:    lwr $7, 15($1)
+; MIPS32R2-NEXT:    srlv $1, $7, $2
+; MIPS32R2-NEXT:    sll $5, $5, 1
+; MIPS32R2-NEXT:    sllv $5, $5, $9
+; MIPS32R2-NEXT:    or $5, $1, $5
+; MIPS32R2-NEXT:    srlv $2, $6, $2
 ; MIPS32R2-NEXT:    jr $ra
-; MIPS32R2-NEXT:    move $5, $1
+; MIPS32R2-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS32R6-LABEL: lshr_i128:
 ; MIPS32R6:       # %bb.0: # %entry
-; MIPS32R6-NEXT:    addiu $sp, $sp, -8
-; MIPS32R6-NEXT:    .cfi_def_cfa_offset 8
-; MIPS32R6-NEXT:    sw $16, 4($sp) # 4-byte Folded Spill
-; MIPS32R6-NEXT:    .cfi_offset 16, -4
-; MIPS32R6-NEXT:    lw $1, 36($sp)
-; MIPS32R6-NEXT:    srlv $2, $7, $1
-; MIPS32R6-NEXT:    not $3, $1
+; MIPS32R6-NEXT:    addiu $sp, $sp, -32
+; MIPS32R6-NEXT:    .cfi_def_cfa_offset 32
+; MIPS32R6-NEXT:    addiu $1, $sp, 0
+; MIPS32R6-NEXT:    sw $7, 28($sp)
+; MIPS32R6-NEXT:    sw $6, 24($sp)
+; MIPS32R6-NEXT:    sw $5, 20($sp)
+; MIPS32R6-NEXT:    sw $4, 16($sp)
+; MIPS32R6-NEXT:    addiu $1, $1, 16
+; MIPS32R6-NEXT:    lw $2, 60($sp)
+; MIPS32R6-NEXT:    ext $3, $2, 3, 4
+; MIPS32R6-NEXT:    subu $1, $1, $3
+; MIPS32R6-NEXT:    sw $zero, 12($sp)
+; MIPS32R6-NEXT:    sw $zero, 8($sp)
+; MIPS32R6-NEXT:    sw $zero, 4($sp)
+; MIPS32R6-NEXT:    sw $zero, 0($sp)
+; MIPS32R6-NEXT:    lw $3, 4($1)
+; MIPS32R6-NEXT:    sll $4, $3, 1
+; MIPS32R6-NEXT:    lw $5, 8($1)
+; MIPS32R6-NEXT:    andi $2, $2, 7
+; MIPS32R6-NEXT:    not $6, $2
+; MIPS32R6-NEXT:    andi $6, $6, 31
+; MIPS32R6-NEXT:    srlv $7, $5, $2
+; MIPS32R6-NEXT:    sllv $4, $4, $6
+; MIPS32R6-NEXT:    srlv $3, $3, $2
+; MIPS32R6-NEXT:    lw $6, 0($1)
 ; MIPS32R6-NEXT:    sll $8, $6, 1
-; MIPS32R6-NEXT:    sllv $8, $8, $3
-; MIPS32R6-NEXT:    or $2, $8, $2
-; MIPS32R6-NEXT:    addiu $8, $1, -64
-; MIPS32R6-NEXT:    srlv $9, $5, $8
-; MIPS32R6-NEXT:    sll $10, $4, 1
-; MIPS32R6-NEXT:    not $11, $8
-; MIPS32R6-NEXT:    sllv $11, $10, $11
-; MIPS32R6-NEXT:    andi $12, $1, 32
-; MIPS32R6-NEXT:    seleqz $2, $2, $12
-; MIPS32R6-NEXT:    or $9, $11, $9
-; MIPS32R6-NEXT:    srlv $11, $6, $1
-; MIPS32R6-NEXT:    selnez $13, $11, $12
-; MIPS32R6-NEXT:    addiu $14, $zero, 64
-; MIPS32R6-NEXT:    subu $14, $14, $1
-; MIPS32R6-NEXT:    sllv $15, $5, $14
-; MIPS32R6-NEXT:    andi $24, $14, 32
-; MIPS32R6-NEXT:    andi $25, $8, 32
-; MIPS32R6-NEXT:    seleqz $9, $9, $25
-; MIPS32R6-NEXT:    seleqz $gp, $15, $24
-; MIPS32R6-NEXT:    or $2, $13, $2
-; MIPS32R6-NEXT:    selnez $13, $15, $24
-; MIPS32R6-NEXT:    sllv $15, $4, $14
-; MIPS32R6-NEXT:    not $14, $14
-; MIPS32R6-NEXT:    srl $16, $5, 1
-; MIPS32R6-NEXT:    srlv $14, $16, $14
-; MIPS32R6-NEXT:    or $14, $15, $14
-; MIPS32R6-NEXT:    seleqz $14, $14, $24
-; MIPS32R6-NEXT:    srlv $8, $4, $8
-; MIPS32R6-NEXT:    or $13, $13, $14
-; MIPS32R6-NEXT:    or $2, $2, $gp
-; MIPS32R6-NEXT:    srlv $5, $5, $1
-; MIPS32R6-NEXT:    selnez $14, $8, $25
-; MIPS32R6-NEXT:    sltiu $15, $1, 64
-; MIPS32R6-NEXT:    selnez $2, $2, $15
-; MIPS32R6-NEXT:    or $9, $14, $9
-; MIPS32R6-NEXT:    sllv $3, $10, $3
-; MIPS32R6-NEXT:    seleqz $10, $11, $12
-; MIPS32R6-NEXT:    or $10, $10, $13
-; MIPS32R6-NEXT:    or $3, $3, $5
-; MIPS32R6-NEXT:    seleqz $5, $9, $15
-; MIPS32R6-NEXT:    seleqz $9, $zero, $15
-; MIPS32R6-NEXT:    srlv $4, $4, $1
-; MIPS32R6-NEXT:    seleqz $11, $4, $12
-; MIPS32R6-NEXT:    selnez $11, $11, $15
-; MIPS32R6-NEXT:    seleqz $7, $7, $1
-; MIPS32R6-NEXT:    or $2, $2, $5
-; MIPS32R6-NEXT:    selnez $2, $2, $1
-; MIPS32R6-NEXT:    or $5, $7, $2
-; MIPS32R6-NEXT:    or $2, $9, $11
-; MIPS32R6-NEXT:    seleqz $3, $3, $12
-; MIPS32R6-NEXT:    selnez $7, $4, $12
-; MIPS32R6-NEXT:    seleqz $4, $6, $1
-; MIPS32R6-NEXT:    selnez $6, $10, $15
-; MIPS32R6-NEXT:    seleqz $8, $8, $25
-; MIPS32R6-NEXT:    seleqz $8, $8, $15
-; MIPS32R6-NEXT:    or $6, $6, $8
-; MIPS32R6-NEXT:    selnez $1, $6, $1
-; MIPS32R6-NEXT:    or $4, $4, $1
-; MIPS32R6-NEXT:    or $1, $7, $3
-; MIPS32R6-NEXT:    selnez $1, $1, $15
-; MIPS32R6-NEXT:    or $3, $9, $1
-; MIPS32R6-NEXT:    lw $16, 4($sp) # 4-byte Folded Reload
+; MIPS32R6-NEXT:    xori $9, $2, 31
+; MIPS32R6-NEXT:    sllv $8, $8, $9
+; MIPS32R6-NEXT:    or $3, $3, $8
+; MIPS32R6-NEXT:    or $4, $7, $4
+; MIPS32R6-NEXT:    lw $1, 12($1)
+; MIPS32R6-NEXT:    srlv $1, $1, $2
+; MIPS32R6-NEXT:    sll $5, $5, 1
+; MIPS32R6-NEXT:    sllv $5, $5, $9
+; MIPS32R6-NEXT:    or $5, $1, $5
+; MIPS32R6-NEXT:    srlv $2, $6, $2
 ; MIPS32R6-NEXT:    jr $ra
-; MIPS32R6-NEXT:    addiu $sp, $sp, 8
+; MIPS32R6-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS3-LABEL: lshr_i128:
 ; MIPS3:       # %bb.0: # %entry
 ; MIPS3-NEXT:    sll $3, $7, 0
 ; MIPS3-NEXT:    dsrlv $6, $4, $7
 ; MIPS3-NEXT:    andi $1, $3, 64
 ; MIPS3-NEXT:    bnez $1, .LBB5_2
 ; MIPS3-NEXT:    daddiu $2, $zero, 0

 ;
 ; MMR3-LABEL: lshr_i128:
 ; MMR3:       # %bb.0: # %entry
 ; MMR3-NEXT:    addiusp -40
 ; MMR3-NEXT:    .cfi_def_cfa_offset 40
 ; MMR3-NEXT:    swp $16, 32($sp)
 ; MMR3-NEXT:    .cfi_offset 17, -4
 ; MMR3-NEXT:    .cfi_offset 16, -8
-; MMR3-NEXT:    move $8, $7
-; MMR3-NEXT:    sw $6, 24($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sw $4, 28($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    lw $16, 68($sp)
-; MMR3-NEXT:    li16 $2, 64
-; MMR3-NEXT:    subu16 $7, $2, $16
-; MMR3-NEXT:    sllv $9, $5, $7
-; MMR3-NEXT:    move $17, $5
-; MMR3-NEXT:    sw $5, 0($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    andi16 $3, $7, 32
-; MMR3-NEXT:    sw $3, 20($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    swl $7, 28($sp)
+; MMR3-NEXT:    swl $6, 24($sp)
+; MMR3-NEXT:    swl $5, 20($sp)
 ; MMR3-NEXT:    li16 $2, 0
-; MMR3-NEXT:    move $4, $9
-; MMR3-NEXT:    movn $4, $2, $3
-; MMR3-NEXT:    srlv $5, $8, $16
-; MMR3-NEXT:    not16 $3, $16
-; MMR3-NEXT:    sw $3, 16($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sll16 $2, $6, 1
-; MMR3-NEXT:    sllv $2, $2, $3
-; MMR3-NEXT:    or16 $2, $5
-; MMR3-NEXT:    srlv $5, $6, $16
-; MMR3-NEXT:    sw $5, 4($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    andi16 $3, $16, 32
-; MMR3-NEXT:    sw $3, 12($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $2, $5, $3
-; MMR3-NEXT:    addiu $3, $16, -64
-; MMR3-NEXT:    or16 $2, $4
-; MMR3-NEXT:    srlv $4, $17, $3
-; MMR3-NEXT:    sw $4, 8($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    lw $4, 28($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    sll16 $6, $4, 1
-; MMR3-NEXT:    not16 $5, $3
-; MMR3-NEXT:    sllv $5, $6, $5
-; MMR3-NEXT:    lw $17, 8($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    or16 $5, $17
-; MMR3-NEXT:    srlv $1, $4, $3
-; MMR3-NEXT:    andi16 $3, $3, 32
-; MMR3-NEXT:    sw $3, 8($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $5, $1, $3
-; MMR3-NEXT:    sltiu $10, $16, 64
-; MMR3-NEXT:    movn $5, $2, $10
-; MMR3-NEXT:    sllv $2, $4, $7
-; MMR3-NEXT:    not16 $3, $7
-; MMR3-NEXT:    lw $7, 0($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    srl16 $4, $7, 1
-; MMR3-NEXT:    srlv $4, $4, $3
-; MMR3-NEXT:    or16 $4, $2
-; MMR3-NEXT:    srlv $2, $7, $16
-; MMR3-NEXT:    lw $3, 16($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    sllv $3, $6, $3
-; MMR3-NEXT:    or16 $3, $2
-; MMR3-NEXT:    lw $2, 28($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    srlv $2, $2, $16
-; MMR3-NEXT:    lw $17, 12($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $3, $2, $17
-; MMR3-NEXT:    movz $5, $8, $16
-; MMR3-NEXT:    li16 $6, 0
-; MMR3-NEXT:    movz $3, $6, $10
-; MMR3-NEXT:    lw $7, 20($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $4, $9, $7
-; MMR3-NEXT:    lw $6, 4($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    li16 $7, 0
-; MMR3-NEXT:    movn $6, $7, $17
-; MMR3-NEXT:    or16 $6, $4
-; MMR3-NEXT:    lw $4, 8($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $1, $7, $4
-; MMR3-NEXT:    movn $1, $6, $10
-; MMR3-NEXT:    lw $4, 24($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movz $1, $4, $16
-; MMR3-NEXT:    movn $2, $7, $17
-; MMR3-NEXT:    li16 $4, 0
-; MMR3-NEXT:    movz $2, $4, $10
-; MMR3-NEXT:    move $4, $1
+; MMR3-NEXT:    swl $4, 16($sp)
+; MMR3-NEXT:    swl $2, 12($sp)
+; MMR3-NEXT:    swl $2, 8($sp)
+; MMR3-NEXT:    swl $2, 4($sp)
+; MMR3-NEXT:    swl $2, 0($sp)
+; MMR3-NEXT:    swr $7, 31($sp)
+; MMR3-NEXT:    swr $6, 27($sp)
+; MMR3-NEXT:    swr $5, 23($sp)
+; MMR3-NEXT:    swr $4, 19($sp)
+; MMR3-NEXT:    swr $2, 15($sp)
+; MMR3-NEXT:    swr $2, 11($sp)
+; MMR3-NEXT:    swr $2, 7($sp)
+; MMR3-NEXT:    swr $2, 3($sp)
+; MMR3-NEXT:    addiur1sp $2, 0
+; MMR3-NEXT:    addiur2 $2, $2, 16
+; MMR3-NEXT:    lw $3, 68($sp)
+; MMR3-NEXT:    ext $4, $3, 3, 4
+; MMR3-NEXT:    subu16 $2, $2, $4
+; MMR3-NEXT:    lwl $7, 4($2)
+; MMR3-NEXT:    lwr $7, 7($2)
+; MMR3-NEXT:    sll16 $4, $7, 1
+; MMR3-NEXT:    lwl $5, 8($2)
+; MMR3-NEXT:    lwr $5, 11($2)
+; MMR3-NEXT:    andi16 $6, $3, 7
+; MMR3-NEXT:    not16 $3, $6
+; MMR3-NEXT:    andi16 $3, $3, 31
+; MMR3-NEXT:    srlv $16, $5, $6
+; MMR3-NEXT:    sllv $4, $4, $3
+; MMR3-NEXT:    srlv $17, $7, $6
+; MMR3-NEXT:    lwl $7, 0($2)
+; MMR3-NEXT:    lwr $7, 3($2)
+; MMR3-NEXT:    sll16 $3, $7, 1
+; MMR3-NEXT:    xori $1, $6, 31
+; MMR3-NEXT:    sllv $3, $3, $1
+; MMR3-NEXT:    or16 $3, $17
+; MMR3-NEXT:    or16 $4, $16
+; MMR3-NEXT:    lwl $8, 12($2)
+; MMR3-NEXT:    lwr $8, 15($2)
+; MMR3-NEXT:    srlv $2, $8, $6
+; MMR3-NEXT:    sll16 $5, $5, 1
+; MMR3-NEXT:    sllv $5, $5, $1
+; MMR3-NEXT:    or16 $5, $2
+; MMR3-NEXT:    srlv $2, $7, $6
 ; MMR3-NEXT:    lwp $16, 32($sp)
 ; MMR3-NEXT:    addiusp 40
 ; MMR3-NEXT:    jrc $ra
 ;
 ; MMR6-LABEL: lshr_i128:
 ; MMR6:       # %bb.0: # %entry
-; MMR6-NEXT:    addiu $sp, $sp, -32
-; MMR6-NEXT:    .cfi_def_cfa_offset 32
-; MMR6-NEXT:    sw $17, 28($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    sw $16, 24($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    .cfi_offset 17, -4
-; MMR6-NEXT:    .cfi_offset 16, -8
-; MMR6-NEXT:    move $1, $7
-; MMR6-NEXT:    move $7, $5
-; MMR6-NEXT:    lw $3, 60($sp)
-; MMR6-NEXT:    srlv $2, $1, $3
-; MMR6-NEXT:    not16 $5, $3
-; MMR6-NEXT:    sw $5, 12($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    move $17, $6
-; MMR6-NEXT:    sw $6, 16($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    sll16 $6, $6, 1
-; MMR6-NEXT:    sllv $6, $6, $5
-; MMR6-NEXT:    or $8, $6, $2
-; MMR6-NEXT:    addiu $5, $3, -64
-; MMR6-NEXT:    srlv $9, $7, $5
-; MMR6-NEXT:    move $6, $4
-; MMR6-NEXT:    sll16 $2, $4, 1
-; MMR6-NEXT:    sw $2, 8($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    not16 $16, $5
-; MMR6-NEXT:    sllv $10, $2, $16
-; MMR6-NEXT:    andi16 $16, $3, 32
-; MMR6-NEXT:    seleqz $8, $8, $16
-; MMR6-NEXT:    or $9, $10, $9
-; MMR6-NEXT:    srlv $10, $17, $3
-; MMR6-NEXT:    selnez $11, $10, $16
-; MMR6-NEXT:    li16 $17, 64
-; MMR6-NEXT:    subu16 $2, $17, $3
-; MMR6-NEXT:    sllv $12, $7, $2
-; MMR6-NEXT:    move $17, $7
-; MMR6-NEXT:    andi16 $4, $2, 32
-; MMR6-NEXT:    andi16 $7, $5, 32
-; MMR6-NEXT:    sw $7, 20($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    seleqz $9, $9, $7
-; MMR6-NEXT:    seleqz $13, $12, $4
-; MMR6-NEXT:    or $8, $11, $8
-; MMR6-NEXT:    selnez $11, $12, $4
-; MMR6-NEXT:    sllv $12, $6, $2
-; MMR6-NEXT:    move $7, $6
-; MMR6-NEXT:    sw $6, 4($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    not16 $2, $2
-; MMR6-NEXT:    srl16 $6, $17, 1
-; MMR6-NEXT:    srlv $2, $6, $2
-; MMR6-NEXT:    or $2, $12, $2
-; MMR6-NEXT:    seleqz $2, $2, $4
-; MMR6-NEXT:    srlv $4, $7, $5
-; MMR6-NEXT:    or $11, $11, $2
-; MMR6-NEXT:    or $5, $8, $13
-; MMR6-NEXT:    srlv $6, $17, $3
-; MMR6-NEXT:    lw $2, 20($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    selnez $7, $4, $2
-; MMR6-NEXT:    sltiu $8, $3, 64
-; MMR6-NEXT:    selnez $12, $5, $8
-; MMR6-NEXT:    or $7, $7, $9
-; MMR6-NEXT:    lw $5, 12($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    lw $2, 8($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    sllv $9, $2, $5
-; MMR6-NEXT:    seleqz $10, $10, $16
-; MMR6-NEXT:    li16 $5, 0
-; MMR6-NEXT:    or $10, $10, $11
-; MMR6-NEXT:    or $6, $9, $6
-; MMR6-NEXT:    seleqz $2, $7, $8
-; MMR6-NEXT:    seleqz $7, $5, $8
-; MMR6-NEXT:    lw $5, 4($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    srlv $9, $5, $3
-; MMR6-NEXT:    seleqz $11, $9, $16
-; MMR6-NEXT:    selnez $11, $11, $8
-; MMR6-NEXT:    seleqz $1, $1, $3
-; MMR6-NEXT:    or $2, $12, $2
-; MMR6-NEXT:    selnez $2, $2, $3
-; MMR6-NEXT:    or $5, $1, $2
-; MMR6-NEXT:    or $2, $7, $11
-; MMR6-NEXT:    seleqz $1, $6, $16
-; MMR6-NEXT:    selnez $6, $9, $16
-; MMR6-NEXT:    lw $16, 16($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    seleqz $9, $16, $3
-; MMR6-NEXT:    selnez $10, $10, $8
-; MMR6-NEXT:    lw $16, 20($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    seleqz $4, $4, $16
-; MMR6-NEXT:    seleqz $4, $4, $8
-; MMR6-NEXT:    or $4, $10, $4
-; MMR6-NEXT:    selnez $3, $4, $3
-; MMR6-NEXT:    or $4, $9, $3
-; MMR6-NEXT:    or $1, $6, $1
-; MMR6-NEXT:    selnez $1, $1, $8
-; MMR6-NEXT:    or $3, $7, $1
-; MMR6-NEXT:    lw $16, 24($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    lw $17, 28($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    addiu $sp, $sp, 32
+; MMR6-NEXT:    addiu $sp, $sp, -40
+; MMR6-NEXT:    .cfi_def_cfa_offset 40
+; MMR6-NEXT:    sw $16, 36($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    .cfi_offset 16, -4
+; MMR6-NEXT:    li16 $2, 0
+; MMR6-NEXT:    sw $7, 32($sp)
+; MMR6-NEXT:    sw $6, 28($sp)
+; MMR6-NEXT:    sw $5, 24($sp)
+; MMR6-NEXT:    sw $4, 20($sp)
+; MMR6-NEXT:    sw $2, 16($sp)
+; MMR6-NEXT:    sw $2, 12($sp)
+; MMR6-NEXT:    sw $2, 8($sp)
+; MMR6-NEXT:    sw $2, 4($sp)
+; MMR6-NEXT:    addiu $2, $sp, 4
+; MMR6-NEXT:    addiur2 $2, $2, 16
+; MMR6-NEXT:    lw $3, 68($sp)
+; MMR6-NEXT:    ext $4, $3, 3, 4
+; MMR6-NEXT:    subu16 $5, $2, $4
+; MMR6-NEXT:    lw16 $4, 4($5)
+; MMR6-NEXT:    sll16 $6, $4, 1
+; MMR6-NEXT:    lw16 $7, 8($5)
+; MMR6-NEXT:    andi16 $2, $3, 7
+; MMR6-NEXT:    not16 $3, $2
+; MMR6-NEXT:    andi16 $3, $3, 31
+; MMR6-NEXT:    srlv $1, $7, $2
+; MMR6-NEXT:    sllv $6, $6, $3
+; MMR6-NEXT:    srlv $3, $4, $2
+; MMR6-NEXT:    lw16 $16, 0($5)
+; MMR6-NEXT:    sll16 $4, $16, 1
+; MMR6-NEXT:    xori $8, $2, 31
+; MMR6-NEXT:    sllv $4, $4, $8
+; MMR6-NEXT:    or $3, $3, $4
+; MMR6-NEXT:    or $4, $1, $6
+; MMR6-NEXT:    lw16 $5, 12($5)
+; MMR6-NEXT:    srlv $1, $5, $2
+; MMR6-NEXT:    sll16 $5, $7, 1
+; MMR6-NEXT:    sllv $5, $5, $8
+; MMR6-NEXT:    or $5, $1, $5
+; MMR6-NEXT:    srlv $2, $16, $2
+; MMR6-NEXT:    lw $16, 36($sp) # 4-byte Folded Reload
+; MMR6-NEXT:    addiu $sp, $sp, 40
 ; MMR6-NEXT:    jrc $ra
 entry:
 
 ; o32 shouldn't use TImode helpers.
 ; GP32-NOT:       lw        $25, %call16(__lshrti3)($gp)
 ; MM-NOT:         lw        $25, %call16(__lshrti3)($2)
 
   %r = lshr i128 %a, %b
   ret i128 %r
 }

llvm/test/CodeGen/Mips/llvm-ir/shl.ll

entry:
 
   %r = shl i64 %a, %b
   ret i64 %r
 }
 
 define signext i128 @shl_i128(i128 signext %a, i128 signext %b) {
 ; MIPS2-LABEL: shl_i128:
 ; MIPS2:       # %bb.0: # %entry
-; MIPS2-NEXT:    addiu $sp, $sp, -8
-; MIPS2-NEXT:    .cfi_def_cfa_offset 8
-; MIPS2-NEXT:    sw $17, 4($sp) # 4-byte Folded Spill
-; MIPS2-NEXT:    sw $16, 0($sp) # 4-byte Folded Spill
-; MIPS2-NEXT:    .cfi_offset 17, -4
-; MIPS2-NEXT:    .cfi_offset 16, -8
-; MIPS2-NEXT:    lw $8, 36($sp)
-; MIPS2-NEXT:    addiu $1, $zero, 64
-; MIPS2-NEXT:    subu $3, $1, $8
-; MIPS2-NEXT:    srlv $9, $6, $3
-; MIPS2-NEXT:    andi $1, $3, 32
-; MIPS2-NEXT:    bnez $1, $BB5_2
-; MIPS2-NEXT:    addiu $2, $zero, 0
-; MIPS2-NEXT:  # %bb.1: # %entry
-; MIPS2-NEXT:    srlv $1, $7, $3
-; MIPS2-NEXT:    not $3, $3
-; MIPS2-NEXT:    sll $10, $6, 1
-; MIPS2-NEXT:    sllv $3, $10, $3
-; MIPS2-NEXT:    or $3, $3, $1
-; MIPS2-NEXT:    b $BB5_3
-; MIPS2-NEXT:    move $15, $9
-; MIPS2-NEXT:  $BB5_2:
-; MIPS2-NEXT:    addiu $15, $zero, 0
-; MIPS2-NEXT:    move $3, $9
-; MIPS2-NEXT:  $BB5_3: # %entry
-; MIPS2-NEXT:    not $13, $8
-; MIPS2-NEXT:    sllv $9, $5, $8
-; MIPS2-NEXT:    andi $10, $8, 32
-; MIPS2-NEXT:    bnez $10, $BB5_5
-; MIPS2-NEXT:    move $25, $9
-; MIPS2-NEXT:  # %bb.4: # %entry
-; MIPS2-NEXT:    sllv $1, $4, $8
-; MIPS2-NEXT:    srl $11, $5, 1
-; MIPS2-NEXT:    srlv $11, $11, $13
-; MIPS2-NEXT:    or $25, $1, $11
-; MIPS2-NEXT:  $BB5_5: # %entry
-; MIPS2-NEXT:    addiu $14, $8, -64
-; MIPS2-NEXT:    srl $24, $7, 1
-; MIPS2-NEXT:    sllv $11, $7, $14
-; MIPS2-NEXT:    andi $12, $14, 32
-; MIPS2-NEXT:    bnez $12, $BB5_7
-; MIPS2-NEXT:    move $gp, $11
-; MIPS2-NEXT:  # %bb.6: # %entry
-; MIPS2-NEXT:    sllv $1, $6, $14
-; MIPS2-NEXT:    not $14, $14
-; MIPS2-NEXT:    srlv $14, $24, $14
-; MIPS2-NEXT:    or $gp, $1, $14
-; MIPS2-NEXT:  $BB5_7: # %entry
-; MIPS2-NEXT:    sltiu $14, $8, 64
-; MIPS2-NEXT:    beqz $14, $BB5_9
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  # %bb.8:
-; MIPS2-NEXT:    or $gp, $25, $15
-; MIPS2-NEXT:  $BB5_9: # %entry
-; MIPS2-NEXT:    sllv $25, $7, $8
-; MIPS2-NEXT:    bnez $10, $BB5_11
-; MIPS2-NEXT:    addiu $17, $zero, 0
-; MIPS2-NEXT:  # %bb.10: # %entry
-; MIPS2-NEXT:    move $17, $25
-; MIPS2-NEXT:  $BB5_11: # %entry
-; MIPS2-NEXT:    addiu $1, $zero, 63
-; MIPS2-NEXT:    sltiu $15, $8, 1
-; MIPS2-NEXT:    beqz $15, $BB5_21
-; MIPS2-NEXT:    sltu $16, $1, $8
-; MIPS2-NEXT:  # %bb.12: # %entry
-; MIPS2-NEXT:    beqz $16, $BB5_22
-; MIPS2-NEXT:    addiu $7, $zero, 0
-; MIPS2-NEXT:  $BB5_13: # %entry
-; MIPS2-NEXT:    beqz $10, $BB5_23
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  $BB5_14: # %entry
-; MIPS2-NEXT:    beqz $16, $BB5_24
-; MIPS2-NEXT:    addiu $6, $zero, 0
-; MIPS2-NEXT:  $BB5_15: # %entry
-; MIPS2-NEXT:    beqz $10, $BB5_25
-; MIPS2-NEXT:    addiu $8, $zero, 0
-; MIPS2-NEXT:  $BB5_16: # %entry
-; MIPS2-NEXT:    beqz $12, $BB5_26
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  $BB5_17: # %entry
-; MIPS2-NEXT:    bnez $14, $BB5_27
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  $BB5_18: # %entry
-; MIPS2-NEXT:    bnez $15, $BB5_20
-; MIPS2-NEXT:    nop
-; MIPS2-NEXT:  $BB5_19: # %entry
-; MIPS2-NEXT:    move $5, $2
-; MIPS2-NEXT:  $BB5_20: # %entry
-; MIPS2-NEXT:    move $2, $4
-; MIPS2-NEXT:    move $3, $5
-; MIPS2-NEXT:    move $4, $6
-; MIPS2-NEXT:    move $5, $7
-; MIPS2-NEXT:    lw $16, 0($sp) # 4-byte Folded Reload
-; MIPS2-NEXT:    lw $17, 4($sp) # 4-byte Folded Reload
+; MIPS2-NEXT:    addiu $sp, $sp, -32
+; MIPS2-NEXT:    .cfi_def_cfa_offset 32
+; MIPS2-NEXT:    swl $zero, 28($sp)
+; MIPS2-NEXT:    swl $zero, 24($sp)
+; MIPS2-NEXT:    swl $zero, 20($sp)
+; MIPS2-NEXT:    swl $zero, 16($sp)
+; MIPS2-NEXT:    swl $7, 12($sp)
+; MIPS2-NEXT:    swl $6, 8($sp)
+; MIPS2-NEXT:    swl $5, 4($sp)
+; MIPS2-NEXT:    swl $4, 0($sp)
+; MIPS2-NEXT:    swr $zero, 31($sp)
+; MIPS2-NEXT:    swr $zero, 27($sp)
+; MIPS2-NEXT:    swr $zero, 23($sp)
+; MIPS2-NEXT:    swr $zero, 19($sp)
+; MIPS2-NEXT:    swr $7, 15($sp)
+; MIPS2-NEXT:    swr $6, 11($sp)
+; MIPS2-NEXT:    swr $5, 7($sp)
+; MIPS2-NEXT:    swr $4, 3($sp)
+; MIPS2-NEXT:    lw $1, 60($sp)
+; MIPS2-NEXT:    srl $2, $1, 3
+; MIPS2-NEXT:    andi $2, $2, 15
+; MIPS2-NEXT:    addiu $3, $sp, 0
+; MIPS2-NEXT:    addu $4, $3, $2
+; MIPS2-NEXT:    lwl $5, 8($4)
+; MIPS2-NEXT:    lwr $5, 11($4)
+; MIPS2-NEXT:    srl $2, $5, 1
+; MIPS2-NEXT:    lwl $3, 4($4)
+; MIPS2-NEXT:    lwr $3, 7($4)
+; MIPS2-NEXT:    andi $1, $1, 7
+; MIPS2-NEXT:    not $6, $1
+; MIPS2-NEXT:    andi $6, $6, 31
+; MIPS2-NEXT:    sllv $7, $3, $1
+; MIPS2-NEXT:    srlv $6, $2, $6
+; MIPS2-NEXT:    lwl $2, 0($4)
+; MIPS2-NEXT:    lwr $2, 3($4)
+; MIPS2-NEXT:    sllv $2, $2, $1
+; MIPS2-NEXT:    srl $3, $3, 1
+; MIPS2-NEXT:    xori $8, $1, 31
+; MIPS2-NEXT:    srlv $3, $3, $8
+; MIPS2-NEXT:    or $2, $2, $3
+; MIPS2-NEXT:    or $3, $7, $6
+; MIPS2-NEXT:    sllv $5, $5, $1
+; MIPS2-NEXT:    lwl $6, 12($4)
+; MIPS2-NEXT:    lwr $6, 15($4)
+; MIPS2-NEXT:    srl $4, $6, 1
+; MIPS2-NEXT:    srlv $4, $4, $8
+; MIPS2-NEXT:    or $4, $5, $4
+; MIPS2-NEXT:    sllv $5, $6, $1
 ; MIPS2-NEXT:    jr $ra
-; MIPS2-NEXT:    addiu $sp, $sp, 8
-; MIPS2-NEXT:  $BB5_21: # %entry
-; MIPS2-NEXT:    move $4, $gp
-; MIPS2-NEXT:    bnez $16, $BB5_13
-; MIPS2-NEXT:    addiu $7, $zero, 0
-; MIPS2-NEXT:  $BB5_22: # %entry
-; MIPS2-NEXT:    bnez $10, $BB5_14
-; MIPS2-NEXT:    move $7, $17
-; MIPS2-NEXT:  $BB5_23: # %entry
-; MIPS2-NEXT:    sllv $1, $6, $8
-; MIPS2-NEXT:    srlv $6, $24, $13
-; MIPS2-NEXT:    or $25, $1, $6
-; MIPS2-NEXT:    bnez $16, $BB5_15
-; MIPS2-NEXT:    addiu $6, $zero, 0
-; MIPS2-NEXT:  $BB5_24: # %entry
-; MIPS2-NEXT:    move $6, $25
-; MIPS2-NEXT:    bnez $10, $BB5_16
-; MIPS2-NEXT:    addiu $8, $zero, 0
-; MIPS2-NEXT:  $BB5_25: # %entry
-; MIPS2-NEXT:    bnez $12, $BB5_17
-; MIPS2-NEXT:    move $8, $9
-; MIPS2-NEXT:  $BB5_26: # %entry
-; MIPS2-NEXT:    beqz $14, $BB5_18
-; MIPS2-NEXT:    move $2, $11
-; MIPS2-NEXT:  $BB5_27:
-; MIPS2-NEXT:    bnez $15, $BB5_20
-; MIPS2-NEXT:    or $2, $8, $3
-; MIPS2-NEXT:  # %bb.28:
-; MIPS2-NEXT:    b $BB5_19
-; MIPS2-NEXT:    nop
+; MIPS2-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS32-LABEL: shl_i128:
 ; MIPS32:       # %bb.0: # %entry
-; MIPS32-NEXT:    lw $8, 28($sp)
-; MIPS32-NEXT:    addiu $1, $zero, 64
-; MIPS32-NEXT:    subu $1, $1, $8
-; MIPS32-NEXT:    srlv $9, $6, $1
-; MIPS32-NEXT:    andi $10, $1, 32
-; MIPS32-NEXT:    move $2, $9
-; MIPS32-NEXT:    movn $2, $zero, $10
-; MIPS32-NEXT:    sllv $3, $4, $8
-; MIPS32-NEXT:    not $11, $8
-; MIPS32-NEXT:    srl $12, $5, 1
-; MIPS32-NEXT:    srlv $12, $12, $11
-; MIPS32-NEXT:    or $3, $3, $12
-; MIPS32-NEXT:    sllv $12, $5, $8
-; MIPS32-NEXT:    andi $13, $8, 32
-; MIPS32-NEXT:    movn $3, $12, $13
-; MIPS32-NEXT:    addiu $14, $8, -64
-; MIPS32-NEXT:    or $15, $3, $2
-; MIPS32-NEXT:    sllv $2, $6, $14
-; MIPS32-NEXT:    srl $24, $7, 1
-; MIPS32-NEXT:    not $3, $14
-; MIPS32-NEXT:    srlv $3, $24, $3
+; MIPS32-NEXT:    addiu $sp, $sp, -32
+; MIPS32-NEXT:    .cfi_def_cfa_offset 32
+; MIPS32-NEXT:    swl $zero, 28($sp)
+; MIPS32-NEXT:    swl $zero, 24($sp)
+; MIPS32-NEXT:    swl $zero, 20($sp)
+; MIPS32-NEXT:    swl $zero, 16($sp)
+; MIPS32-NEXT:    swl $7, 12($sp)
+; MIPS32-NEXT:    swl $6, 8($sp)
+; MIPS32-NEXT:    swl $5, 4($sp)
+; MIPS32-NEXT:    swl $4, 0($sp)
+; MIPS32-NEXT:    swr $zero, 31($sp)
+; MIPS32-NEXT:    swr $zero, 27($sp)
+; MIPS32-NEXT:    swr $zero, 23($sp)
+; MIPS32-NEXT:    swr $zero, 19($sp)
+; MIPS32-NEXT:    swr $7, 15($sp)
+; MIPS32-NEXT:    swr $6, 11($sp)
+; MIPS32-NEXT:    swr $5, 7($sp)
+; MIPS32-NEXT:    swr $4, 3($sp)
+; MIPS32-NEXT:    lw $1, 60($sp)
+; MIPS32-NEXT:    srl $2, $1, 3
+; MIPS32-NEXT:    andi $2, $2, 15
+; MIPS32-NEXT:    addiu $3, $sp, 0
+; MIPS32-NEXT:    addu $4, $3, $2
+; MIPS32-NEXT:    lwl $5, 8($4)
+; MIPS32-NEXT:    lwr $5, 11($4)
+; MIPS32-NEXT:    srl $2, $5, 1
+; MIPS32-NEXT:    lwl $3, 4($4)
+; MIPS32-NEXT:    lwr $3, 7($4)
+; MIPS32-NEXT:    andi $1, $1, 7
+; MIPS32-NEXT:    not $6, $1
+; MIPS32-NEXT:    andi $6, $6, 31
+; MIPS32-NEXT:    sllv $7, $3, $1
+; MIPS32-NEXT:    srlv $6, $2, $6
+; MIPS32-NEXT:    lwl $2, 0($4)
+; MIPS32-NEXT:    lwr $2, 3($4)
+; MIPS32-NEXT:    sllv $2, $2, $1
+; MIPS32-NEXT:    srl $3, $3, 1
+; MIPS32-NEXT:    xori $8, $1, 31
+; MIPS32-NEXT:    srlv $3, $3, $8
 ; MIPS32-NEXT:    or $2, $2, $3
-; MIPS32-NEXT:    sllv $3, $7, $14
-; MIPS32-NEXT:    andi $14, $14, 32
-; MIPS32-NEXT:    movn $2, $3, $14
-; MIPS32-NEXT:    sltiu $25, $8, 64
-; MIPS32-NEXT:    movn $2, $15, $25
-; MIPS32-NEXT:    srlv $15, $7, $1
-; MIPS32-NEXT:    not $1, $1
-; MIPS32-NEXT:    sll $gp, $6, 1
-; MIPS32-NEXT:    sllv $1, $gp, $1
-; MIPS32-NEXT:    or $15, $1, $15
-; MIPS32-NEXT:    sllv $1, $6, $8
-; MIPS32-NEXT:    srlv $6, $24, $11
-; MIPS32-NEXT:    or $1, $1, $6
-; MIPS32-NEXT:    sllv $6, $7, $8
-; MIPS32-NEXT:    movn $1, $6, $13
-; MIPS32-NEXT:    movz $2, $4, $8
-; MIPS32-NEXT:    movz $1, $zero, $25
-; MIPS32-NEXT:    movn $15, $9, $10
-; MIPS32-NEXT:    movn $12, $zero, $13
-; MIPS32-NEXT:    or $4, $12, $15
-; MIPS32-NEXT:    movn $3, $zero, $14
-; MIPS32-NEXT:    movn $3, $4, $25
-; MIPS32-NEXT:    movz $3, $5, $8
-; MIPS32-NEXT:    movn $6, $zero, $13
-; MIPS32-NEXT:    movz $6, $zero, $25
-; MIPS32-NEXT:    move $4, $1
+; MIPS32-NEXT:    or $3, $7, $6
+; MIPS32-NEXT:    sllv $5, $5, $1
+; MIPS32-NEXT:    lwl $6, 12($4)
+; MIPS32-NEXT:    lwr $6, 15($4)
+; MIPS32-NEXT:    srl $4, $6, 1
+; MIPS32-NEXT:    srlv $4, $4, $8
+; MIPS32-NEXT:    or $4, $5, $4
+; MIPS32-NEXT:    sllv $5, $6, $1
 ; MIPS32-NEXT:    jr $ra
-; MIPS32-NEXT:    move $5, $6
+; MIPS32-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS32R2-LABEL: shl_i128:
 ; MIPS32R2:       # %bb.0: # %entry
-; MIPS32R2-NEXT:    lw $8, 28($sp)
-; MIPS32R2-NEXT:    addiu $1, $zero, 64
-; MIPS32R2-NEXT:    subu $1, $1, $8
-; MIPS32R2-NEXT:    srlv $9, $6, $1
-; MIPS32R2-NEXT:    andi $10, $1, 32
-; MIPS32R2-NEXT:    move $2, $9
-; MIPS32R2-NEXT:    movn $2, $zero, $10
-; MIPS32R2-NEXT:    sllv $3, $4, $8
-; MIPS32R2-NEXT:    not $11, $8
-; MIPS32R2-NEXT:    srl $12, $5, 1
-; MIPS32R2-NEXT:    srlv $12, $12, $11
-; MIPS32R2-NEXT:    or $3, $3, $12
-; MIPS32R2-NEXT:    sllv $12, $5, $8
-; MIPS32R2-NEXT:    andi $13, $8, 32
-; MIPS32R2-NEXT:    movn $3, $12, $13
-; MIPS32R2-NEXT:    addiu $14, $8, -64
-; MIPS32R2-NEXT:    or $15, $3, $2
-; MIPS32R2-NEXT:    sllv $2, $6, $14
-; MIPS32R2-NEXT:    srl $24, $7, 1
-; MIPS32R2-NEXT:    not $3, $14
-; MIPS32R2-NEXT:    srlv $3, $24, $3
+; MIPS32R2-NEXT:    addiu $sp, $sp, -32
+; MIPS32R2-NEXT:    .cfi_def_cfa_offset 32
+; MIPS32R2-NEXT:    swl $zero, 28($sp)
+; MIPS32R2-NEXT:    swl $zero, 24($sp)
+; MIPS32R2-NEXT:    swl $zero, 20($sp)
+; MIPS32R2-NEXT:    swl $zero, 16($sp)
+; MIPS32R2-NEXT:    swl $7, 12($sp)
+; MIPS32R2-NEXT:    swl $6, 8($sp)
+; MIPS32R2-NEXT:    swl $5, 4($sp)
+; MIPS32R2-NEXT:    swl $4, 0($sp)
+; MIPS32R2-NEXT:    swr $zero, 31($sp)
+; MIPS32R2-NEXT:    swr $zero, 27($sp)
+; MIPS32R2-NEXT:    swr $zero, 23($sp)
+; MIPS32R2-NEXT:    swr $zero, 19($sp)
+; MIPS32R2-NEXT:    swr $7, 15($sp)
+; MIPS32R2-NEXT:    swr $6, 11($sp)
+; MIPS32R2-NEXT:    swr $5, 7($sp)
+; MIPS32R2-NEXT:    swr $4, 3($sp)
+; MIPS32R2-NEXT:    lw $1, 60($sp)
+; MIPS32R2-NEXT:    ext $2, $1, 3, 4
+; MIPS32R2-NEXT:    addiu $3, $sp, 0
+; MIPS32R2-NEXT:    addu $4, $3, $2
+; MIPS32R2-NEXT:    lwl $5, 8($4)
+; MIPS32R2-NEXT:    lwr $5, 11($4)
+; MIPS32R2-NEXT:    srl $2, $5, 1
+; MIPS32R2-NEXT:    lwl $3, 4($4)
+; MIPS32R2-NEXT:    lwr $3, 7($4)
+; MIPS32R2-NEXT:    andi $1, $1, 7
+; MIPS32R2-NEXT:    not $6, $1
+; MIPS32R2-NEXT:    andi $6, $6, 31
+; MIPS32R2-NEXT:    sllv $7, $3, $1
+; MIPS32R2-NEXT:    srlv $6, $2, $6
+; MIPS32R2-NEXT:    lwl $2, 0($4)
+; MIPS32R2-NEXT:    lwr $2, 3($4)
+; MIPS32R2-NEXT:    sllv $2, $2, $1
+; MIPS32R2-NEXT:    srl $3, $3, 1
+; MIPS32R2-NEXT:    xori $8, $1, 31
+; MIPS32R2-NEXT:    srlv $3, $3, $8
 ; MIPS32R2-NEXT:    or $2, $2, $3
-; MIPS32R2-NEXT:    sllv $3, $7, $14
-; MIPS32R2-NEXT:    andi $14, $14, 32
-; MIPS32R2-NEXT:    movn $2, $3, $14
-; MIPS32R2-NEXT:    sltiu $25, $8, 64
-; MIPS32R2-NEXT:    movn $2, $15, $25
-; MIPS32R2-NEXT:    srlv $15, $7, $1
-; MIPS32R2-NEXT:    not $1, $1
-; MIPS32R2-NEXT:    sll $gp, $6, 1
-; MIPS32R2-NEXT:    sllv $1, $gp, $1
-; MIPS32R2-NEXT:    or $15, $1, $15
-; MIPS32R2-NEXT:    sllv $1, $6, $8
-; MIPS32R2-NEXT:    srlv $6, $24, $11
-; MIPS32R2-NEXT:    or $1, $1, $6
-; MIPS32R2-NEXT:    sllv $6, $7, $8
-; MIPS32R2-NEXT:    movn $1, $6, $13
-; MIPS32R2-NEXT:    movz $2, $4, $8
-; MIPS32R2-NEXT:    movz $1, $zero, $25
-; MIPS32R2-NEXT:    movn $15, $9, $10
-; MIPS32R2-NEXT:    movn $12, $zero, $13
-; MIPS32R2-NEXT:    or $4, $12, $15
-; MIPS32R2-NEXT:    movn $3, $zero, $14
-; MIPS32R2-NEXT:    movn $3, $4, $25
-; MIPS32R2-NEXT:    movz $3, $5, $8
-; MIPS32R2-NEXT:    movn $6, $zero, $13
-; MIPS32R2-NEXT:    movz $6, $zero, $25
-; MIPS32R2-NEXT:    move $4, $1
+; MIPS32R2-NEXT:    or $3, $7, $6
+; MIPS32R2-NEXT:    sllv $5, $5, $1
+; MIPS32R2-NEXT:    lwl $6, 12($4)
+; MIPS32R2-NEXT:    lwr $6, 15($4)
+; MIPS32R2-NEXT:    srl $4, $6, 1
+; MIPS32R2-NEXT:    srlv $4, $4, $8
+; MIPS32R2-NEXT:    or $4, $5, $4
+; MIPS32R2-NEXT:    sllv $5, $6, $1
 ; MIPS32R2-NEXT:    jr $ra
-; MIPS32R2-NEXT:    move $5, $6
+; MIPS32R2-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS32R6-LABEL: shl_i128:
 ; MIPS32R6:       # %bb.0: # %entry
-; MIPS32R6-NEXT:    lw $3, 28($sp)
-; MIPS32R6-NEXT:    sllv $1, $4, $3
-; MIPS32R6-NEXT:    not $2, $3
-; MIPS32R6-NEXT:    srl $8, $5, 1
-; MIPS32R6-NEXT:    srlv $8, $8, $2
-; MIPS32R6-NEXT:    or $1, $1, $8
-; MIPS32R6-NEXT:    sllv $8, $5, $3
-; MIPS32R6-NEXT:    andi $9, $3, 32
-; MIPS32R6-NEXT:    seleqz $1, $1, $9
-; MIPS32R6-NEXT:    selnez $10, $8, $9
-; MIPS32R6-NEXT:    addiu $11, $zero, 64
-; MIPS32R6-NEXT:    subu $11, $11, $3
-; MIPS32R6-NEXT:    srlv $12, $6, $11
-; MIPS32R6-NEXT:    andi $13, $11, 32
-; MIPS32R6-NEXT:    seleqz $14, $12, $13
-; MIPS32R6-NEXT:    or $1, $10, $1
-; MIPS32R6-NEXT:    selnez $10, $12, $13
-; MIPS32R6-NEXT:    srlv $12, $7, $11
-; MIPS32R6-NEXT:    not $11, $11
-; MIPS32R6-NEXT:    sll $15, $6, 1
-; MIPS32R6-NEXT:    sllv $11, $15, $11
-; MIPS32R6-NEXT:    or $11, $11, $12
-; MIPS32R6-NEXT:    seleqz $11, $11, $13
-; MIPS32R6-NEXT:    addiu $12, $3, -64
-; MIPS32R6-NEXT:    or $10, $10, $11
-; MIPS32R6-NEXT:    or $1, $1, $14
-; MIPS32R6-NEXT:    sllv $11, $6, $12
-; MIPS32R6-NEXT:    srl $13, $7, 1
-; MIPS32R6-NEXT:    not $14, $12
-; MIPS32R6-NEXT:    srlv $14, $13, $14
-; MIPS32R6-NEXT:    or $11, $11, $14
-; MIPS32R6-NEXT:    andi $14, $12, 32
-; MIPS32R6-NEXT:    seleqz $11, $11, $14
-; MIPS32R6-NEXT:    sllv $12, $7, $12
-; MIPS32R6-NEXT:    selnez $15, $12, $14
-; MIPS32R6-NEXT:    sltiu $24, $3, 64
-; MIPS32R6-NEXT:    selnez $1, $1, $24
-; MIPS32R6-NEXT:    or $11, $15, $11
-; MIPS32R6-NEXT:    sllv $6, $6, $3
-; MIPS32R6-NEXT:    srlv $2, $13, $2
-; MIPS32R6-NEXT:    seleqz $8, $8, $9
-; MIPS32R6-NEXT:    or $8, $8, $10
-; MIPS32R6-NEXT:    or $6, $6, $2
-; MIPS32R6-NEXT:    seleqz $2, $11, $24
-; MIPS32R6-NEXT:    seleqz $10, $zero, $24
-; MIPS32R6-NEXT:    sllv $7, $7, $3
-; MIPS32R6-NEXT:    seleqz $11, $7, $9
-; MIPS32R6-NEXT:    selnez $11, $11, $24
-; MIPS32R6-NEXT:    seleqz $4, $4, $3
-; MIPS32R6-NEXT:    or $1, $1, $2
-; MIPS32R6-NEXT:    selnez $1, $1, $3
-; MIPS32R6-NEXT:    or $2, $4, $1
-; MIPS32R6-NEXT:    or $1, $10, $11
-; MIPS32R6-NEXT:    seleqz $4, $6, $9
-; MIPS32R6-NEXT:    selnez $6, $7, $9
-; MIPS32R6-NEXT:    seleqz $5, $5, $3
-; MIPS32R6-NEXT:    selnez $7, $8, $24
-; MIPS32R6-NEXT:    seleqz $8, $12, $14
-; MIPS32R6-NEXT:    seleqz $8, $8, $24
-; MIPS32R6-NEXT:    or $7, $7, $8
-; MIPS32R6-NEXT:    selnez $3, $7, $3
-; MIPS32R6-NEXT:    or $3, $5, $3
-; MIPS32R6-NEXT:    or $4, $6, $4
-; MIPS32R6-NEXT:    selnez $4, $4, $24
-; MIPS32R6-NEXT:    or $4, $10, $4
+; MIPS32R6-NEXT:    addiu $sp, $sp, -32
+; MIPS32R6-NEXT:    .cfi_def_cfa_offset 32
+; MIPS32R6-NEXT:    lw $1, 60($sp)
+; MIPS32R6-NEXT:    sw $7, 12($sp)
+; MIPS32R6-NEXT:    sw $6, 8($sp)
+; MIPS32R6-NEXT:    sw $5, 4($sp)
+; MIPS32R6-NEXT:    sw $4, 0($sp)
+; MIPS32R6-NEXT:    ext $2, $1, 3, 4
+; MIPS32R6-NEXT:    addiu $3, $sp, 0
+; MIPS32R6-NEXT:    addu $4, $3, $2
+; MIPS32R6-NEXT:    sw $zero, 28($sp)
+; MIPS32R6-NEXT:    sw $zero, 24($sp)
+; MIPS32R6-NEXT:    sw $zero, 20($sp)
+; MIPS32R6-NEXT:    sw $zero, 16($sp)
+; MIPS32R6-NEXT:    lw $5, 8($4)
+; MIPS32R6-NEXT:    srl $2, $5, 1
+; MIPS32R6-NEXT:    lw $3, 4($4)
+; MIPS32R6-NEXT:    andi $1, $1, 7
+; MIPS32R6-NEXT:    not $6, $1
+; MIPS32R6-NEXT:    andi $6, $6, 31
+; MIPS32R6-NEXT:    sllv $7, $3, $1
+; MIPS32R6-NEXT:    srlv $6, $2, $6
+; MIPS32R6-NEXT:    lw $2, 0($4)
+; MIPS32R6-NEXT:    sllv $2, $2, $1
+; MIPS32R6-NEXT:    srl $3, $3, 1
+; MIPS32R6-NEXT:    xori $8, $1, 31
+; MIPS32R6-NEXT:    srlv $3, $3, $8
+; MIPS32R6-NEXT:    or $2, $2, $3
+; MIPS32R6-NEXT:    or $3, $7, $6
+; MIPS32R6-NEXT:    sllv $5, $5, $1
+; MIPS32R6-NEXT:    lw $6, 12($4)
+; MIPS32R6-NEXT:    srl $4, $6, 1
+; MIPS32R6-NEXT:    srlv $4, $4, $8
+; MIPS32R6-NEXT:    or $4, $5, $4
+; MIPS32R6-NEXT:    sllv $5, $6, $1
 ; MIPS32R6-NEXT:    jr $ra
-; MIPS32R6-NEXT:    move $5, $1
+; MIPS32R6-NEXT:    addiu $sp, $sp, 32
 ;
 ; MIPS3-LABEL: shl_i128:
 ; MIPS3:       # %bb.0: # %entry
 ; MIPS3-NEXT:    sll $3, $7, 0
 ; MIPS3-NEXT:    dsllv $6, $5, $7
 ; MIPS3-NEXT:    andi $8, $3, 64
 ; MIPS3-NEXT:    beqz $8, .LBB5_3
 ; MIPS3-NEXT:    move $2, $6

 ;
 ; MMR3-LABEL: shl_i128:
 ; MMR3:       # %bb.0: # %entry
 ; MMR3-NEXT:    addiusp -40
 ; MMR3-NEXT:    .cfi_def_cfa_offset 40
 ; MMR3-NEXT:    swp $16, 32($sp)
 ; MMR3-NEXT:    .cfi_offset 17, -4
 ; MMR3-NEXT:    .cfi_offset 16, -8
-; MMR3-NEXT:    move $17, $7
-; MMR3-NEXT:    sw $7, 4($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    move $7, $6
-; MMR3-NEXT:    move $1, $4
-; MMR3-NEXT:    lw $16, 68($sp)
-; MMR3-NEXT:    li16 $2, 64
-; MMR3-NEXT:    subu16 $6, $2, $16
-; MMR3-NEXT:    srlv $9, $7, $6
-; MMR3-NEXT:    andi16 $4, $6, 32
-; MMR3-NEXT:    sw $4, 24($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    li16 $3, 0
-; MMR3-NEXT:    move $2, $9
-; MMR3-NEXT:    movn $2, $3, $4
-; MMR3-NEXT:    sllv $3, $1, $16
-; MMR3-NEXT:    sw $3, 16($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    not16 $4, $16
-; MMR3-NEXT:    sw $4, 20($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sw $5, 28($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    srl16 $3, $5, 1
-; MMR3-NEXT:    srlv $3, $3, $4
-; MMR3-NEXT:    lw $4, 16($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    or16 $3, $4
-; MMR3-NEXT:    sllv $5, $5, $16
-; MMR3-NEXT:    sw $5, 8($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    andi16 $4, $16, 32
-; MMR3-NEXT:    sw $4, 16($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $3, $5, $4
-; MMR3-NEXT:    addiu $4, $16, -64
-; MMR3-NEXT:    or16 $3, $2
-; MMR3-NEXT:    sllv $2, $7, $4
-; MMR3-NEXT:    sw $2, 12($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    srl16 $5, $17, 1
-; MMR3-NEXT:    not16 $2, $4
-; MMR3-NEXT:    srlv $2, $5, $2
-; MMR3-NEXT:    lw $17, 12($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    li16 $2, 0
+; MMR3-NEXT:    swl $2, 28($sp)
+; MMR3-NEXT:    swl $2, 24($sp)
+; MMR3-NEXT:    swl $2, 20($sp)
+; MMR3-NEXT:    swl $2, 16($sp)
+; MMR3-NEXT:    swl $7, 12($sp)
+; MMR3-NEXT:    swl $6, 8($sp)
+; MMR3-NEXT:    swl $5, 4($sp)
+; MMR3-NEXT:    swl $4, 0($sp)
+; MMR3-NEXT:    swr $2, 31($sp)
+; MMR3-NEXT:    swr $2, 27($sp)
+; MMR3-NEXT:    swr $2, 23($sp)
+; MMR3-NEXT:    swr $2, 19($sp)
+; MMR3-NEXT:    swr $7, 15($sp)
+; MMR3-NEXT:    swr $6, 11($sp)
+; MMR3-NEXT:    swr $5, 7($sp)
+; MMR3-NEXT:    swr $4, 3($sp)
+; MMR3-NEXT:    lw $2, 68($sp)
+; MMR3-NEXT:    ext $3, $2, 3, 4
+; MMR3-NEXT:    addiur1sp $4, 0
+; MMR3-NEXT:    addu16 $4, $4, $3
+; MMR3-NEXT:    lwl $6, 8($4)
+; MMR3-NEXT:    lwr $6, 11($4)
+; MMR3-NEXT:    srl16 $3, $6, 1
+; MMR3-NEXT:    lwl $7, 4($4)
+; MMR3-NEXT:    lwr $7, 7($4)
+; MMR3-NEXT:    andi16 $5, $2, 7
+; MMR3-NEXT:    not16 $2, $5
+; MMR3-NEXT:    andi16 $2, $2, 31
+; MMR3-NEXT:    sllv $16, $7, $5
+; MMR3-NEXT:    srlv $3, $3, $2
+; MMR3-NEXT:    lwl $1, 0($4)
+; MMR3-NEXT:    lwr $1, 3($4)
+; MMR3-NEXT:    sllv $17, $1, $5
+; MMR3-NEXT:    srl16 $2, $7, 1
+; MMR3-NEXT:    xori $1, $5, 31
+; MMR3-NEXT:    srlv $2, $2, $1
 ; MMR3-NEXT:    or16 $2, $17
-; MMR3-NEXT:    lw $17, 4($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    sllv $8, $17, $4
-; MMR3-NEXT:    andi16 $4, $4, 32
-; MMR3-NEXT:    sw $4, 12($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $2, $8, $4
-; MMR3-NEXT:    sltiu $10, $16, 64
-; MMR3-NEXT:    movn $2, $3, $10
-; MMR3-NEXT:    srlv $4, $17, $6
-; MMR3-NEXT:    not16 $3, $6
-; MMR3-NEXT:    sll16 $6, $7, 1
-; MMR3-NEXT:    sllv $3, $6, $3
-; MMR3-NEXT:    or16 $3, $4
-; MMR3-NEXT:    sllv $6, $7, $16
-; MMR3-NEXT:    lw $4, 20($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    srlv $4, $5, $4
+; MMR3-NEXT:    or16 $3, $16
+; MMR3-NEXT:    sllv $6, $6, $5
+; MMR3-NEXT:    lwl $7, 12($4)
+; MMR3-NEXT:    lwr $7, 15($4)
+; MMR3-NEXT:    srl16 $4, $7, 1
+; MMR3-NEXT:    srlv $4, $4, $1
 ; MMR3-NEXT:    or16 $4, $6
-; MMR3-NEXT:    sllv $6, $17, $16
-; MMR3-NEXT:    lw $17, 16($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $4, $6, $17
-; MMR3-NEXT:    movz $2, $1, $16
-; MMR3-NEXT:    li16 $5, 0
-; MMR3-NEXT:    movz $4, $5, $10
-; MMR3-NEXT:    lw $7, 24($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $3, $9, $7
-; MMR3-NEXT:    lw $5, 8($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    li16 $7, 0
-; MMR3-NEXT:    movn $5, $7, $17
-; MMR3-NEXT:    or16 $5, $3
-; MMR3-NEXT:    lw $3, 12($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $8, $7, $3
-; MMR3-NEXT:    movn $8, $5, $10
-; MMR3-NEXT:    lw $3, 28($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movz $8, $3, $16
-; MMR3-NEXT:    movn $6, $7, $17
-; MMR3-NEXT:    li16 $3, 0
-; MMR3-NEXT:    movz $6, $3, $10
-; MMR3-NEXT:    move $3, $8
-; MMR3-NEXT:    move $5, $6
+; MMR3-NEXT:    sllv $5, $7, $5
 ; MMR3-NEXT:    lwp $16, 32($sp)
 ; MMR3-NEXT:    addiusp 40
 ; MMR3-NEXT:    jrc $ra
 ;
 ; MMR6-LABEL: shl_i128:
 ; MMR6:       # %bb.0: # %entry
-; MMR6-NEXT:    addiu $sp, $sp, -16
-; MMR6-NEXT:    .cfi_def_cfa_offset 16
-; MMR6-NEXT:    sw $17, 12($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    sw $16, 8($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    .cfi_offset 17, -4
-; MMR6-NEXT:    .cfi_offset 16, -8
-; MMR6-NEXT:    move $11, $4
-; MMR6-NEXT:    lw $3, 44($sp)
-; MMR6-NEXT:    sllv $1, $4, $3
-; MMR6-NEXT:    not16 $2, $3
-; MMR6-NEXT:    sw $2, 4($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    srl16 $16, $5, 1
-; MMR6-NEXT:    srlv $8, $16, $2
-; MMR6-NEXT:    or $1, $1, $8
-; MMR6-NEXT:    sllv $8, $5, $3
-; MMR6-NEXT:    andi16 $16, $3, 32
-; MMR6-NEXT:    seleqz $1, $1, $16
-; MMR6-NEXT:    selnez $9, $8, $16
-; MMR6-NEXT:    li16 $17, 64
-; MMR6-NEXT:    subu16 $17, $17, $3
-; MMR6-NEXT:    srlv $10, $6, $17
-; MMR6-NEXT:    andi16 $2, $17, 32
-; MMR6-NEXT:    seleqz $12, $10, $2
-; MMR6-NEXT:    or $1, $9, $1
-; MMR6-NEXT:    selnez $9, $10, $2
-; MMR6-NEXT:    srlv $10, $7, $17
-; MMR6-NEXT:    not16 $17, $17
-; MMR6-NEXT:    sll16 $4, $6, 1
-; MMR6-NEXT:    sllv $4, $4, $17
-; MMR6-NEXT:    or $4, $4, $10
-; MMR6-NEXT:    seleqz $2, $4, $2
-; MMR6-NEXT:    addiu $4, $3, -64
-; MMR6-NEXT:    or $10, $9, $2
-; MMR6-NEXT:    or $1, $1, $12
-; MMR6-NEXT:    sllv $9, $6, $4
-; MMR6-NEXT:    srl16 $2, $7, 1
-; MMR6-NEXT:    not16 $17, $4
-; MMR6-NEXT:    srlv $12, $2, $17
-; MMR6-NEXT:    or $9, $9, $12
-; MMR6-NEXT:    andi16 $17, $4, 32
-; MMR6-NEXT:    seleqz $9, $9, $17
-; MMR6-NEXT:    sllv $14, $7, $4
-; MMR6-NEXT:    selnez $12, $14, $17
-; MMR6-NEXT:    sltiu $13, $3, 64
-; MMR6-NEXT:    selnez $1, $1, $13
-; MMR6-NEXT:    or $9, $12, $9
-; MMR6-NEXT:    sllv $6, $6, $3
-; MMR6-NEXT:    lw $4, 4($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    srlv $2, $2, $4
-; MMR6-NEXT:    seleqz $8, $8, $16
-; MMR6-NEXT:    li16 $4, 0
-; MMR6-NEXT:    or $8, $8, $10
-; MMR6-NEXT:    or $6, $6, $2
-; MMR6-NEXT:    seleqz $2, $9, $13
-; MMR6-NEXT:    seleqz $9, $4, $13
-; MMR6-NEXT:    sllv $7, $7, $3
-; MMR6-NEXT:    seleqz $10, $7, $16
-; MMR6-NEXT:    selnez $10, $10, $13
-; MMR6-NEXT:    seleqz $11, $11, $3
-; MMR6-NEXT:    or $1, $1, $2
-; MMR6-NEXT:    selnez $1, $1, $3
-; MMR6-NEXT:    or $2, $11, $1
-; MMR6-NEXT:    or $1, $9, $10
-; MMR6-NEXT:    seleqz $6, $6, $16
-; MMR6-NEXT:    selnez $7, $7, $16
-; MMR6-NEXT:    seleqz $5, $5, $3
-; MMR6-NEXT:    selnez $8, $8, $13
-; MMR6-NEXT:    seleqz $4, $14, $17
-; MMR6-NEXT:    seleqz $4, $4, $13
-; MMR6-NEXT:    or $4, $8, $4
-; MMR6-NEXT:    selnez $3, $4, $3
-; MMR6-NEXT:    or $3, $5, $3
-; MMR6-NEXT:    or $4, $7, $6
-; MMR6-NEXT:    selnez $4, $4, $13
-; MMR6-NEXT:    or $4, $9, $4
-; MMR6-NEXT:    move $5, $1
-; MMR6-NEXT:    lw $16, 8($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    lw $17, 12($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    addiu $sp, $sp, 16
+; MMR6-NEXT:    addiu $sp, $sp, -32
+; MMR6-NEXT:    .cfi_def_cfa_offset 32
+; MMR6-NEXT:    li16 $2, 0
+; MMR6-NEXT:    sw $2, 28($sp)
+; MMR6-NEXT:    sw $2, 24($sp)
+; MMR6-NEXT:    sw $2, 20($sp)
+; MMR6-NEXT:    sw $2, 16($sp)
+; MMR6-NEXT:    sw $7, 12($sp)
+; MMR6-NEXT:    sw $6, 8($sp)
+; MMR6-NEXT:    sw $5, 4($sp)
+; MMR6-NEXT:    sw $4, 0($sp)
+; MMR6-NEXT:    lw $2, 60($sp)
+; MMR6-NEXT:    ext $3, $2, 3, 4
+; MMR6-NEXT:    addiu $4, $sp, 0
+; MMR6-NEXT:    addu16 $4, $4, $3
+; MMR6-NEXT:    lw16 $6, 8($4)
+; MMR6-NEXT:    srl16 $3, $6, 1
+; MMR6-NEXT:    lw16 $7, 4($4)
+; MMR6-NEXT:    andi16 $5, $2, 7
+; MMR6-NEXT:    not16 $2, $5
+; MMR6-NEXT:    andi16 $2, $2, 31
+; MMR6-NEXT:    sllv $1, $7, $5
+; MMR6-NEXT:    srlv $3, $3, $2
+; MMR6-NEXT:    lw16 $2, 0($4)
+; MMR6-NEXT:    sllv $2, $2, $5
+; MMR6-NEXT:    srl16 $7, $7, 1
+; MMR6-NEXT:    xori $8, $5, 31
+; MMR6-NEXT:    srlv $7, $7, $8
+; MMR6-NEXT:    or $2, $2, $7
+; MMR6-NEXT:    or $3, $1, $3
+; MMR6-NEXT:    sllv $1, $6, $5
+; MMR6-NEXT:    lw16 $6, 12($4)
+; MMR6-NEXT:    srl16 $4, $6, 1
+; MMR6-NEXT:    srlv $4, $4, $8
+; MMR6-NEXT:    or $4, $1, $4
+; MMR6-NEXT:    sllv $5, $6, $5
+; MMR6-NEXT:    addiu $sp, $sp, 32
 ; MMR6-NEXT:    jrc $ra
 entry:
 
 ; o32 shouldn't use TImode helpers.
 ; GP32-NOT:       lw        $25, %call16(__ashlti3)($gp)
 ; MM-NOT:         lw        $25, %call16(__ashlti3)($2)
 
   %r = shl i128 %a, %b
   ret i128 %r
 }

llvm/test/CodeGen/PowerPC/ctrloop-sh.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs < %s | FileCheck %s
 target datalayout = "E-m:e-p:32:32-i128:64-n32"
 target triple = "powerpc-ellcc-linux"
 
 ; Function Attrs: nounwind
 define void @foo1(ptr %a, ptr readonly %b, ptr readonly %c) #0 {
 ; CHECK-LABEL: foo1:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    stwu 1, -48(1)
-; CHECK-NEXT:    stw 24, 16(1) # 4-byte Folded Spill
-; CHECK-NEXT:    li 6, 2048
-; CHECK-NEXT:    stw 25, 20(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 26, 24(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 27, 28(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 28, 32(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 29, 36(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 30, 40(1) # 4-byte Folded Spill
-; CHECK-NEXT:    mtctr 6
+; CHECK-NEXT:    stwu 1, -64(1)
+; CHECK-NEXT:    stw 28, 48(1) # 4-byte Folded Spill
+; CHECK-NEXT:    li 8, 2048
+; CHECK-NEXT:    stw 29, 52(1) # 4-byte Folded Spill
 ; CHECK-NEXT:    li 6, 0
+; CHECK-NEXT:    stw 30, 56(1) # 4-byte Folded Spill
+; CHECK-NEXT:    li 7, 7
+; CHECK-NEXT:    mtctr 8
+; CHECK-NEXT:    addi 8, 1, 16
 ; CHECK-NEXT:  .LBB0_1: # %for.body
 ; CHECK-NEXT:    #
-; CHECK-NEXT:    lwz 9, 12(5)
-; CHECK-NEXT:    lwz 10, 8(4)
-; CHECK-NEXT:    lwz 11, 12(4)
-; CHECK-NEXT:    subfic 12, 9, 96
-; CHECK-NEXT:    lwz 7, 4(4)
-; CHECK-NEXT:    addi 0, 9, -64
-; CHECK-NEXT:    lwz 8, 0(4)
-; CHECK-NEXT:    subfic 28, 9, 32
-; CHECK-NEXT:    cmplwi 9, 64
-; CHECK-NEXT:    slw 26, 11, 9
-; CHECK-NEXT:    srw 12, 11, 12
-; CHECK-NEXT:    slw 25, 10, 0
-; CHECK-NEXT:    addi 30, 9, -96
-; CHECK-NEXT:    slw 29, 8, 9
-; CHECK-NEXT:    or 12, 25, 12
-; CHECK-NEXT:    srw 25, 7, 28
-; CHECK-NEXT:    bc 12, 0, .LBB0_3
-; CHECK-NEXT:  # %bb.2: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 26, 6, 0
-; CHECK-NEXT:    b .LBB0_3
-; CHECK-NEXT:  .LBB0_3: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    slw 27, 10, 9
-; CHECK-NEXT:    or 29, 29, 25
-; CHECK-NEXT:    srw 25, 11, 28
-; CHECK-NEXT:    stw 26, 12(3)
-; CHECK-NEXT:    subfic 26, 9, 64
-; CHECK-NEXT:    slw 30, 11, 30
-; CHECK-NEXT:    or 27, 27, 25
-; CHECK-NEXT:    addi 25, 9, -32
-; CHECK-NEXT:    or 12, 12, 30
-; CHECK-NEXT:    subfic 30, 26, 32
-; CHECK-NEXT:    srw 28, 10, 28
-; CHECK-NEXT:    slw 30, 10, 30
-; CHECK-NEXT:    srw 10, 10, 26
-; CHECK-NEXT:    srw 26, 11, 26
-; CHECK-NEXT:    slw 24, 11, 0
-; CHECK-NEXT:    slw 0, 7, 25
-; CHECK-NEXT:    or 0, 29, 0
-; CHECK-NEXT:    or 30, 26, 30
-; CHECK-NEXT:    cmplwi 1, 9, 0
-; CHECK-NEXT:    slw 9, 7, 9
-; CHECK-NEXT:    or 10, 0, 10
-; CHECK-NEXT:    or 0, 30, 28
-; CHECK-NEXT:    slw 11, 11, 25
-; CHECK-NEXT:    or 9, 9, 0
-; CHECK-NEXT:    or 11, 27, 11
-; CHECK-NEXT:    bc 12, 0, .LBB0_5
-; CHECK-NEXT:  # %bb.4: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 10, 12, 0
-; CHECK-NEXT:    ori 9, 24, 0
-; CHECK-NEXT:    ori 11, 6, 0
-; CHECK-NEXT:    b .LBB0_5
-; CHECK-NEXT:  .LBB0_5: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    bc 12, 6, .LBB0_7
-; CHECK-NEXT:  # %bb.6: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 8, 10, 0
-; CHECK-NEXT:    ori 7, 9, 0
-; CHECK-NEXT:    b .LBB0_7
-; CHECK-NEXT:  .LBB0_7: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    stw 11, 8(3)
-; CHECK-NEXT:    stw 8, 0(3)
-; CHECK-NEXT:    stw 7, 4(3)
+; CHECK-NEXT:    lwz 9, 0(4)
+; CHECK-NEXT:    lwz 10, 4(4)
+; CHECK-NEXT:    lwz 11, 8(4)
+; CHECK-NEXT:    lwz 12, 12(4)
+; CHECK-NEXT:    lwz 0, 12(5)
+; CHECK-NEXT:    stw 6, 44(1)
+; CHECK-NEXT:    stw 6, 40(1)
+; CHECK-NEXT:    stw 6, 36(1)
+; CHECK-NEXT:    stw 6, 32(1)
+; CHECK-NEXT:    stw 12, 28(1)
+; CHECK-NEXT:    clrlwi 12, 0, 29
+; CHECK-NEXT:    stw 11, 24(1)
+; CHECK-NEXT:    nand 11, 0, 7
+; CHECK-NEXT:    stw 10, 20(1)
+; CHECK-NEXT:    subfic 29, 12, 32
+; CHECK-NEXT:    stw 9, 16(1)
+; CHECK-NEXT:    rlwinm 9, 0, 29, 28, 31
+; CHECK-NEXT:    lwzux 10, 9, 8
+; CHECK-NEXT:    clrlwi 11, 11, 27
+; CHECK-NEXT:    lwz 0, 8(9)
+; CHECK-NEXT:    slw 10, 10, 12
+; CHECK-NEXT:    lwz 30, 4(9)
+; CHECK-NEXT:    lwz 9, 12(9)
+; CHECK-NEXT:    slw 28, 30, 12
+; CHECK-NEXT:    srw 30, 30, 29
+; CHECK-NEXT:    srw 29, 9, 29
+; CHECK-NEXT:    slw 9, 9, 12
+; CHECK-NEXT:    slw 12, 0, 12
+; CHECK-NEXT:    srwi 0, 0, 1
+; CHECK-NEXT:    stw 9, 12(3)
+; CHECK-NEXT:    or 9, 12, 29
+; CHECK-NEXT:    srw 11, 0, 11
+; CHECK-NEXT:    stw 9, 8(3)
+; CHECK-NEXT:    or 9, 10, 30
+; CHECK-NEXT:    stw 9, 0(3)
+; CHECK-NEXT:    or 9, 28, 11
+; CHECK-NEXT:    stw 9, 4(3)
 ; CHECK-NEXT:    bdnz .LBB0_1
-; CHECK-NEXT:  # %bb.8: # %for.end
-; CHECK-NEXT:    lwz 30, 40(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 29, 36(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 28, 32(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 27, 28(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 26, 24(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 25, 20(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 24, 16(1) # 4-byte Folded Reload
-; CHECK-NEXT:    addi 1, 1, 48
+; CHECK-NEXT:  # %bb.2: # %for.end
+; CHECK-NEXT:    lwz 30, 56(1) # 4-byte Folded Reload
+; CHECK-NEXT:    lwz 29, 52(1) # 4-byte Folded Reload
+; CHECK-NEXT:    lwz 28, 48(1) # 4-byte Folded Reload
+; CHECK-NEXT:    addi 1, 1, 64
 ; CHECK-NEXT:    blr
 entry:
   br label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %i.02 = phi i32 [ 0, %entry ], [ %inc, %for.body ]
   %0 = load i128, ptr %b, align 16
   %1 = load i128, ptr %c, align 16
   %shl = shl i128 %0, %1
   store i128 %shl, ptr %a, align 16
   %inc = add nsw i32 %i.02, 1
   %exitcond = icmp eq i32 %inc, 2048
   br i1 %exitcond, label %for.end, label %for.body
 
 for.end:                                          ; preds = %for.body
   ret void
 }
 
 ; Function Attrs: nounwind
 define void @foo2(ptr %a, ptr readonly %b, ptr readonly %c) #0 {
 ; CHECK-LABEL: foo2:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    stwu 1, -48(1)
-; CHECK-NEXT:    stw 24, 16(1) # 4-byte Folded Spill
-; CHECK-NEXT:    li 6, 2048
-; CHECK-NEXT:    stw 25, 20(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 26, 24(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 27, 28(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 28, 32(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 29, 36(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 30, 40(1) # 4-byte Folded Spill
-; CHECK-NEXT:    mtctr 6
+; CHECK-NEXT:    stwu 1, -64(1)
+; CHECK-NEXT:    stw 29, 52(1) # 4-byte Folded Spill
+; CHECK-NEXT:    li 7, 2048
+; CHECK-NEXT:    stw 30, 56(1) # 4-byte Folded Spill
+; CHECK-NEXT:    li 6, 7
+; CHECK-NEXT:    mtctr 7
+; CHECK-NEXT:    addi 7, 1, 36
 ; CHECK-NEXT:  .LBB1_1: # %for.body
 ; CHECK-NEXT:    #
-; CHECK-NEXT:    lwz 8, 12(5)
+; CHECK-NEXT:    lwz 8, 0(4)
+; CHECK-NEXT:    lwz 10, 8(4)
+; CHECK-NEXT:    lwz 12, 12(5)
 ; CHECK-NEXT:    lwz 9, 4(4)
-; CHECK-NEXT:    lwz 10, 0(4)
-; CHECK-NEXT:    subfic 11, 8, 96
-; CHECK-NEXT:    lwz 6, 8(4)
-; CHECK-NEXT:    addi 12, 8, -64
-; CHECK-NEXT:    lwz 7, 12(4)
-; CHECK-NEXT:    subfic 29, 8, 32
-; CHECK-NEXT:    slw 11, 10, 11
-; CHECK-NEXT:    srw 25, 9, 12
-; CHECK-NEXT:    srw 30, 7, 8
-; CHECK-NEXT:    or 11, 25, 11
-; CHECK-NEXT:    slw 25, 6, 29
-; CHECK-NEXT:    srw 27, 9, 8
-; CHECK-NEXT:    or 30, 30, 25
-; CHECK-NEXT:    slw 25, 10, 29
-; CHECK-NEXT:    addi 0, 8, -96
-; CHECK-NEXT:    cmplwi 8, 64
-; CHECK-NEXT:    srawi 26, 10, 31
-; CHECK-NEXT:    or 27, 27, 25
-; CHECK-NEXT:    sraw 25, 10, 8
-; CHECK-NEXT:    cmpwi 1, 0, 1
-; CHECK-NEXT:    sraw 24, 10, 0
-; CHECK-NEXT:    bc 12, 0, .LBB1_3
-; CHECK-NEXT:  # %bb.2: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 0, 26, 0
-; CHECK-NEXT:    b .LBB1_4
-; CHECK-NEXT:  .LBB1_3: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    addi 0, 25, 0
-; CHECK-NEXT:  .LBB1_4: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    addi 28, 8, -32
+; CHECK-NEXT:    lwz 11, 12(4)
+; CHECK-NEXT:    stw 10, 44(1)
+; CHECK-NEXT:    rlwinm 10, 12, 29, 28, 31
+; CHECK-NEXT:    stw 8, 36(1)
+; CHECK-NEXT:    srawi 8, 8, 31
+; CHECK-NEXT:    stw 11, 48(1)
+; CHECK-NEXT:    clrlwi 11, 12, 29
+; CHECK-NEXT:    stw 9, 40(1)
+; CHECK-NEXT:    nand 9, 12, 6
+; CHECK-NEXT:    stw 8, 32(1)
+; CHECK-NEXT:    subfic 30, 11, 32
+; CHECK-NEXT:    stw 8, 28(1)
+; CHECK-NEXT:    clrlwi 9, 9, 27
+; CHECK-NEXT:    stw 8, 24(1)
+; CHECK-NEXT:    stw 8, 20(1)
+; CHECK-NEXT:    sub 8, 7, 10
+; CHECK-NEXT:    lwz 10, 4(8)
+; CHECK-NEXT:    lwz 12, 8(8)
+; CHECK-NEXT:    lwz 0, 0(8)
+; CHECK-NEXT:    lwz 8, 12(8)
+; CHECK-NEXT:    srw 29, 12, 11
+; CHECK-NEXT:    slw 12, 12, 30
+; CHECK-NEXT:    slw 30, 0, 30
+; CHECK-NEXT:    srw 8, 8, 11
+; CHECK-NEXT:    sraw 0, 0, 11
+; CHECK-NEXT:    srw 11, 10, 11
+; CHECK-NEXT:    slwi 10, 10, 1
+; CHECK-NEXT:    or 8, 12, 8
+; CHECK-NEXT:    slw 9, 10, 9
+; CHECK-NEXT:    stw 8, 12(3)
+; CHECK-NEXT:    or 8, 30, 11
+; CHECK-NEXT:    stw 8, 4(3)
+; CHECK-NEXT:    or 8, 29, 9
 ; CHECK-NEXT:    stw 0, 0(3)
-; CHECK-NEXT:    subfic 0, 8, 64
-; CHECK-NEXT:    subfic 25, 0, 32
-; CHECK-NEXT:    slw 29, 9, 29
-; CHECK-NEXT:    srw 25, 9, 25
-; CHECK-NEXT:    slw 9, 9, 0
-; CHECK-NEXT:    slw 0, 10, 0
-; CHECK-NEXT:    bc 12, 4, .LBB1_6
-; CHECK-NEXT:  # %bb.5: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 11, 24, 0
-; CHECK-NEXT:    b .LBB1_6
-; CHECK-NEXT:  .LBB1_6: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    sraw 12, 10, 12
-; CHECK-NEXT:    sraw 10, 10, 28
-; CHECK-NEXT:    cmpwi 1, 28, 1
-; CHECK-NEXT:    srw 28, 6, 28
-; CHECK-NEXT:    or 0, 0, 25
-; CHECK-NEXT:    or 30, 30, 28
-; CHECK-NEXT:    bc 12, 4, .LBB1_7
-; CHECK-NEXT:    b .LBB1_8
-; CHECK-NEXT:  .LBB1_7: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    addi 10, 27, 0
-; CHECK-NEXT:  .LBB1_8: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    cmplwi 1, 8, 0
-; CHECK-NEXT:    srw 8, 6, 8
-; CHECK-NEXT:    or 0, 0, 29
-; CHECK-NEXT:    or 9, 30, 9
-; CHECK-NEXT:    or 8, 8, 0
-; CHECK-NEXT:    bc 12, 0, .LBB1_10
-; CHECK-NEXT:  # %bb.9: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 9, 11, 0
-; CHECK-NEXT:    ori 8, 12, 0
-; CHECK-NEXT:    ori 10, 26, 0
-; CHECK-NEXT:    b .LBB1_10
-; CHECK-NEXT:  .LBB1_10: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    bc 12, 6, .LBB1_12
-; CHECK-NEXT:  # %bb.11: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 7, 9, 0
-; CHECK-NEXT:    ori 6, 8, 0
-; CHECK-NEXT:    b .LBB1_12
-; CHECK-NEXT:  .LBB1_12: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    stw 10, 4(3)
-; CHECK-NEXT:    stw 7, 12(3)
-; CHECK-NEXT:    stw 6, 8(3)
+; CHECK-NEXT:    stw 8, 8(3)
 ; CHECK-NEXT:    bdnz .LBB1_1
-; CHECK-NEXT:  # %bb.13: # %for.end
-; CHECK-NEXT:    lwz 30, 40(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 29, 36(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 28, 32(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 27, 28(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 26, 24(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 25, 20(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 24, 16(1) # 4-byte Folded Reload
-; CHECK-NEXT:    addi 1, 1, 48
+; CHECK-NEXT:  # %bb.2: # %for.end
+; CHECK-NEXT:    lwz 30, 56(1) # 4-byte Folded Reload
+; CHECK-NEXT:    lwz 29, 52(1) # 4-byte Folded Reload
+; CHECK-NEXT:    addi 1, 1, 64
 ; CHECK-NEXT:    blr
 entry:
   br label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %i.02 = phi i32 [ 0, %entry ], [ %inc, %for.body ]
   %0 = load i128, ptr %b, align 16
   %1 = load i128, ptr %c, align 16
   %shl = ashr i128 %0, %1
   store i128 %shl, ptr %a, align 16
   %inc = add nsw i32 %i.02, 1
   %exitcond = icmp eq i32 %inc, 2048
   br i1 %exitcond, label %for.end, label %for.body
 
 for.end:                                          ; preds = %for.body
   ret void
 }
 
 ; Function Attrs: nounwind
 define void @foo3(ptr %a, ptr readonly %b, ptr readonly %c) #0 {
 ; CHECK-LABEL: foo3:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    stwu 1, -48(1)
-; CHECK-NEXT:    stw 24, 16(1) # 4-byte Folded Spill
-; CHECK-NEXT:    li 6, 2048
-; CHECK-NEXT:    stw 25, 20(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 26, 24(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 27, 28(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 28, 32(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 29, 36(1) # 4-byte Folded Spill
-; CHECK-NEXT:    stw 30, 40(1) # 4-byte Folded Spill
-; CHECK-NEXT:    mtctr 6
+; CHECK-NEXT:    stwu 1, -64(1)
+; CHECK-NEXT:    stw 28, 48(1) # 4-byte Folded Spill
+; CHECK-NEXT:    li 8, 2048
+; CHECK-NEXT:    stw 29, 52(1) # 4-byte Folded Spill
 ; CHECK-NEXT:    li 6, 0
+; CHECK-NEXT:    stw 30, 56(1) # 4-byte Folded Spill
+; CHECK-NEXT:    li 7, 7
+; CHECK-NEXT:    mtctr 8
+; CHECK-NEXT:    addi 8, 1, 32
 ; CHECK-NEXT:  .LBB2_1: # %for.body
 ; CHECK-NEXT:    #
-; CHECK-NEXT:    lwz 9, 12(5)
 ; CHECK-NEXT:    lwz 10, 4(4)
-; CHECK-NEXT:    lwz 11, 0(4)
-; CHECK-NEXT:    subfic 12, 9, 96
-; CHECK-NEXT:    lwz 7, 8(4)
-; CHECK-NEXT:    addi 0, 9, -64
-; CHECK-NEXT:    lwz 8, 12(4)
-; CHECK-NEXT:    subfic 28, 9, 32
-; CHECK-NEXT:    cmplwi 9, 64
-; CHECK-NEXT:    srw 26, 11, 9
-; CHECK-NEXT:    slw 12, 11, 12
-; CHECK-NEXT:    srw 25, 10, 0
-; CHECK-NEXT:    addi 30, 9, -96
-; CHECK-NEXT:    srw 29, 8, 9
-; CHECK-NEXT:    or 12, 25, 12
-; CHECK-NEXT:    slw 25, 7, 28
-; CHECK-NEXT:    bc 12, 0, .LBB2_3
-; CHECK-NEXT:  # %bb.2: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 26, 6, 0
-; CHECK-NEXT:    b .LBB2_3
-; CHECK-NEXT:  .LBB2_3: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    srw 27, 10, 9
-; CHECK-NEXT:    or 29, 29, 25
-; CHECK-NEXT:    slw 25, 11, 28
-; CHECK-NEXT:    stw 26, 0(3)
-; CHECK-NEXT:    subfic 26, 9, 64
-; CHECK-NEXT:    srw 30, 11, 30
-; CHECK-NEXT:    or 27, 27, 25
-; CHECK-NEXT:    addi 25, 9, -32
-; CHECK-NEXT:    or 12, 12, 30
-; CHECK-NEXT:    subfic 30, 26, 32
-; CHECK-NEXT:    slw 28, 10, 28
-; CHECK-NEXT:    srw 30, 10, 30
-; CHECK-NEXT:    slw 10, 10, 26
-; CHECK-NEXT:    slw 26, 11, 26
-; CHECK-NEXT:    srw 24, 11, 0
-; CHECK-NEXT:    srw 0, 7, 25
-; CHECK-NEXT:    or 0, 29, 0
-; CHECK-NEXT:    or 30, 26, 30
-; CHECK-NEXT:    cmplwi 1, 9, 0
-; CHECK-NEXT:    srw 9, 7, 9
+; CHECK-NEXT:    lwz 0, 12(5)
+; CHECK-NEXT:    lwz 9, 0(4)
+; CHECK-NEXT:    lwz 11, 8(4)
+; CHECK-NEXT:    lwz 12, 12(4)
+; CHECK-NEXT:    stw 10, 36(1)
+; CHECK-NEXT:    rlwinm 10, 0, 29, 28, 31
+; CHECK-NEXT:    stw 6, 28(1)
+; CHECK-NEXT:    sub 10, 8, 10
+; CHECK-NEXT:    stw 6, 24(1)
+; CHECK-NEXT:    stw 6, 20(1)
+; CHECK-NEXT:    stw 6, 16(1)
+; CHECK-NEXT:    stw 12, 44(1)
+; CHECK-NEXT:    clrlwi 12, 0, 29
+; CHECK-NEXT:    stw 11, 40(1)
+; CHECK-NEXT:    subfic 29, 12, 32
+; CHECK-NEXT:    stw 9, 32(1)
+; CHECK-NEXT:    nand 9, 0, 7
+; CHECK-NEXT:    lwz 11, 4(10)
+; CHECK-NEXT:    clrlwi 9, 9, 27
+; CHECK-NEXT:    lwz 0, 8(10)
+; CHECK-NEXT:    lwz 30, 0(10)
+; CHECK-NEXT:    lwz 10, 12(10)
+; CHECK-NEXT:    srw 28, 0, 12
+; CHECK-NEXT:    slw 0, 0, 29
+; CHECK-NEXT:    slw 29, 30, 29
+; CHECK-NEXT:    srw 10, 10, 12
+; CHECK-NEXT:    srw 30, 30, 12
+; CHECK-NEXT:    srw 12, 11, 12
+; CHECK-NEXT:    slwi 11, 11, 1
+; CHECK-NEXT:    slw 9, 11, 9
 ; CHECK-NEXT:    or 10, 0, 10
-; CHECK-NEXT:    or 0, 30, 28
-; CHECK-NEXT:    srw 11, 11, 25
-; CHECK-NEXT:    or 9, 9, 0
-; CHECK-NEXT:    or 11, 27, 11
-; CHECK-NEXT:    bc 12, 0, .LBB2_5
-; CHECK-NEXT:  # %bb.4: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 10, 12, 0
-; CHECK-NEXT:    ori 9, 24, 0
-; CHECK-NEXT:    ori 11, 6, 0
-; CHECK-NEXT:    b .LBB2_5
-; CHECK-NEXT:  .LBB2_5: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    bc 12, 6, .LBB2_7
-; CHECK-NEXT:  # %bb.6: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    ori 8, 10, 0
-; CHECK-NEXT:    ori 7, 9, 0
-; CHECK-NEXT:    b .LBB2_7
-; CHECK-NEXT:  .LBB2_7: # %for.body
-; CHECK-NEXT:    #
-; CHECK-NEXT:    stw 11, 4(3)
-; CHECK-NEXT:    stw 8, 12(3)
-; CHECK-NEXT:    stw 7, 8(3)
+; CHECK-NEXT:    stw 10, 12(3)
+; CHECK-NEXT:    or 10, 29, 12
+; CHECK-NEXT:    or 9, 28, 9
+; CHECK-NEXT:    stw 30, 0(3)
+; CHECK-NEXT:    stw 10, 4(3)
+; CHECK-NEXT:    stw 9, 8(3)
 ; CHECK-NEXT:    bdnz .LBB2_1
-; CHECK-NEXT:  # %bb.8: # %for.end
-; CHECK-NEXT:    lwz 30, 40(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 29, 36(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 28, 32(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 27, 28(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 26, 24(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 25, 20(1) # 4-byte Folded Reload
-; CHECK-NEXT:    lwz 24, 16(1) # 4-byte Folded Reload
-; CHECK-NEXT:    addi 1, 1, 48
+; CHECK-NEXT:  # %bb.2: # %for.end
+; CHECK-NEXT:    lwz 30, 56(1) # 4-byte Folded Reload
+; CHECK-NEXT:    lwz 29, 52(1) # 4-byte Folded Reload
+; CHECK-NEXT:    lwz 28, 48(1) # 4-byte Folded Reload
+; CHECK-NEXT:    addi 1, 1, 64
 ; CHECK-NEXT:    blr
 entry:
   br label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %i.02 = phi i32 [ 0, %entry ], [ %inc, %for.body ]
   %0 = load i128, ptr %b, align 16
   %1 = load i128, ptr %c, align 16

llvm/test/CodeGen/PowerPC/wide-scalar-shift-by-byte-multiple-legalization.ll

 ; BE-NEXT:    srd 4, 6, 4
 ; BE-NEXT:    or 3, 3, 7
 ; BE-NEXT:    std 4, 0(5)
 ; BE-NEXT:    std 3, 8(5)
 ; BE-NEXT:    blr
 ;
 ; LE-32BIT-LABEL: lshr_16bytes:
 ; LE-32BIT:       # %bb.0:
-; LE-32BIT-NEXT:    stwu 1, -32(1)
+; LE-32BIT-NEXT:    stwu 1, -48(1)
+; LE-32BIT-NEXT:    lwz 7, 0(3)
+; LE-32BIT-NEXT:    li 6, 0
+; LE-32BIT-NEXT:    lwz 8, 4(3)
+; LE-32BIT-NEXT:    lwz 9, 8(3)
+; LE-32BIT-NEXT:    lwz 3, 12(3)
 ; LE-32BIT-NEXT:    lwz 4, 12(4)
-; LE-32BIT-NEXT:    li 8, 0
-; LE-32BIT-NEXT:    lwz 6, 8(3)
-; LE-32BIT-NEXT:    lwz 7, 12(3)
-; LE-32BIT-NEXT:    rlwinm. 4, 4, 3, 0, 28
-; LE-32BIT-NEXT:    lwz 9, 4(3)
-; LE-32BIT-NEXT:    subfic 10, 4, 96
-; LE-32BIT-NEXT:    lwz 3, 0(3)
-; LE-32BIT-NEXT:    addi 11, 4, -64
-; LE-32BIT-NEXT:    stw 27, 12(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    cmplwi 1, 4, 64
-; LE-32BIT-NEXT:    stw 28, 16(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 28, 3, 4
-; LE-32BIT-NEXT:    stw 30, 24(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    subfic 30, 4, 32
-; LE-32BIT-NEXT:    slw 10, 3, 10
-; LE-32BIT-NEXT:    srw 27, 9, 11
-; LE-32BIT-NEXT:    addi 12, 4, -96
-; LE-32BIT-NEXT:    srw 0, 7, 4
-; LE-32BIT-NEXT:    or 10, 27, 10
-; LE-32BIT-NEXT:    slw 27, 6, 30
-; LE-32BIT-NEXT:    bc 12, 4, .LBB6_2
-; LE-32BIT-NEXT:  # %bb.1:
-; LE-32BIT-NEXT:    ori 28, 8, 0
-; LE-32BIT-NEXT:    b .LBB6_2
-; LE-32BIT-NEXT:  .LBB6_2:
-; LE-32BIT-NEXT:    stw 29, 20(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 29, 9, 4
-; LE-32BIT-NEXT:    or 0, 0, 27
-; LE-32BIT-NEXT:    slw 27, 3, 30
-; LE-32BIT-NEXT:    stw 28, 0(5)
-; LE-32BIT-NEXT:    subfic 28, 4, 64
-; LE-32BIT-NEXT:    srw 12, 3, 12
-; LE-32BIT-NEXT:    or 29, 29, 27
-; LE-32BIT-NEXT:    addi 27, 4, -32
-; LE-32BIT-NEXT:    or 10, 10, 12
-; LE-32BIT-NEXT:    subfic 12, 28, 32
-; LE-32BIT-NEXT:    slw 30, 9, 30
-; LE-32BIT-NEXT:    srw 12, 9, 12
-; LE-32BIT-NEXT:    slw 9, 9, 28
-; LE-32BIT-NEXT:    slw 28, 3, 28
-; LE-32BIT-NEXT:    srw 11, 3, 11
-; LE-32BIT-NEXT:    srw 3, 3, 27
-; LE-32BIT-NEXT:    srw 27, 6, 27
-; LE-32BIT-NEXT:    or 0, 0, 27
-; LE-32BIT-NEXT:    or 12, 28, 12
-; LE-32BIT-NEXT:    srw 4, 6, 4
-; LE-32BIT-NEXT:    or 3, 29, 3
-; LE-32BIT-NEXT:    or 9, 0, 9
-; LE-32BIT-NEXT:    or 12, 12, 30
-; LE-32BIT-NEXT:    bc 12, 4, .LBB6_4
-; LE-32BIT-NEXT:  # %bb.3:
-; LE-32BIT-NEXT:    ori 3, 8, 0
-; LE-32BIT-NEXT:    ori 8, 10, 0
-; LE-32BIT-NEXT:    b .LBB6_5
-; LE-32BIT-NEXT:  .LBB6_4:
-; LE-32BIT-NEXT:    addi 8, 9, 0
-; LE-32BIT-NEXT:  .LBB6_5:
-; LE-32BIT-NEXT:    or 4, 4, 12
-; LE-32BIT-NEXT:    stw 3, 4(5)
-; LE-32BIT-NEXT:    bc 12, 2, .LBB6_7
-; LE-32BIT-NEXT:  # %bb.6:
-; LE-32BIT-NEXT:    ori 3, 8, 0
-; LE-32BIT-NEXT:    b .LBB6_8
-; LE-32BIT-NEXT:  .LBB6_7:
-; LE-32BIT-NEXT:    addi 3, 7, 0
-; LE-32BIT-NEXT:  .LBB6_8:
-; LE-32BIT-NEXT:    bc 12, 4, .LBB6_10
-; LE-32BIT-NEXT:  # %bb.9:
-; LE-32BIT-NEXT:    ori 4, 11, 0
-; LE-32BIT-NEXT:    b .LBB6_10
-; LE-32BIT-NEXT:  .LBB6_10:
+; LE-32BIT-NEXT:    stw 3, 44(1)
+; LE-32BIT-NEXT:    addi 3, 1, 32
+; LE-32BIT-NEXT:    clrlwi 4, 4, 28
+; LE-32BIT-NEXT:    stw 6, 28(1)
+; LE-32BIT-NEXT:    sub 3, 3, 4
+; LE-32BIT-NEXT:    stw 6, 24(1)
+; LE-32BIT-NEXT:    stw 6, 20(1)
+; LE-32BIT-NEXT:    stw 6, 16(1)
+; LE-32BIT-NEXT:    stw 9, 40(1)
+; LE-32BIT-NEXT:    stw 8, 36(1)
+; LE-32BIT-NEXT:    stw 7, 32(1)
+; LE-32BIT-NEXT:    lwz 4, 4(3)
+; LE-32BIT-NEXT:    lwz 6, 0(3)
+; LE-32BIT-NEXT:    lwz 7, 8(3)
+; LE-32BIT-NEXT:    lwz 3, 12(3)
+; LE-32BIT-NEXT:    stw 7, 8(5)
 ; LE-32BIT-NEXT:    stw 3, 12(5)
-; LE-32BIT-NEXT:    bc 12, 2, .LBB6_12
-; LE-32BIT-NEXT:  # %bb.11:
-; LE-32BIT-NEXT:    ori 3, 4, 0
-; LE-32BIT-NEXT:    b .LBB6_13
-; LE-32BIT-NEXT:  .LBB6_12:
-; LE-32BIT-NEXT:    addi 3, 6, 0
-; LE-32BIT-NEXT:  .LBB6_13:
-; LE-32BIT-NEXT:    stw 3, 8(5)
-; LE-32BIT-NEXT:    lwz 30, 24(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 29, 20(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 28, 16(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 27, 12(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    addi 1, 1, 32
+; LE-32BIT-NEXT:    stw 6, 0(5)
+; LE-32BIT-NEXT:    stw 4, 4(5)
+; LE-32BIT-NEXT:    addi 1, 1, 48
 ; LE-32BIT-NEXT:    blr
   %src = load i128, ptr %src.ptr, align 1
   %byteOff = load i128, ptr %byteOff.ptr, align 1
   %bitOff = shl i128 %byteOff, 3
   %res = lshr i128 %src, %bitOff
   store i128 %res, ptr %dst, align 1
   ret void
 }

 ; BE-NEXT:    sld 4, 6, 4
 ; BE-NEXT:    or 3, 3, 7
 ; BE-NEXT:    std 4, 8(5)
 ; BE-NEXT:    std 3, 0(5)
 ; BE-NEXT:    blr
 ;
 ; LE-32BIT-LABEL: shl_16bytes:
 ; LE-32BIT:       # %bb.0:
-; LE-32BIT-NEXT:    stwu 1, -32(1)
-; LE-32BIT-NEXT:    lwz 4, 12(4)
-; LE-32BIT-NEXT:    li 8, 0
-; LE-32BIT-NEXT:    lwz 6, 4(3)
+; LE-32BIT-NEXT:    stwu 1, -48(1)
 ; LE-32BIT-NEXT:    lwz 7, 0(3)
-; LE-32BIT-NEXT:    rlwinm. 4, 4, 3, 0, 28
+; LE-32BIT-NEXT:    li 6, 0
+; LE-32BIT-NEXT:    lwz 8, 4(3)
 ; LE-32BIT-NEXT:    lwz 9, 8(3)
-; LE-32BIT-NEXT:    subfic 10, 4, 96
 ; LE-32BIT-NEXT:    lwz 3, 12(3)
-; LE-32BIT-NEXT:    addi 11, 4, -64
-; LE-32BIT-NEXT:    stw 27, 12(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    cmplwi 1, 4, 64
-; LE-32BIT-NEXT:    stw 28, 16(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    slw 28, 3, 4
-; LE-32BIT-NEXT:    stw 30, 24(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    subfic 30, 4, 32
-; LE-32BIT-NEXT:    srw 10, 3, 10
-; LE-32BIT-NEXT:    slw 27, 9, 11
-; LE-32BIT-NEXT:    addi 12, 4, -96
-; LE-32BIT-NEXT:    slw 0, 7, 4
-; LE-32BIT-NEXT:    or 10, 27, 10
-; LE-32BIT-NEXT:    srw 27, 6, 30
-; LE-32BIT-NEXT:    bc 12, 4, .LBB7_2
-; LE-32BIT-NEXT:  # %bb.1:
-; LE-32BIT-NEXT:    ori 28, 8, 0
-; LE-32BIT-NEXT:    b .LBB7_2
-; LE-32BIT-NEXT:  .LBB7_2:
-; LE-32BIT-NEXT:    stw 29, 20(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    slw 29, 9, 4
-; LE-32BIT-NEXT:    or 0, 0, 27
-; LE-32BIT-NEXT:    srw 27, 3, 30
-; LE-32BIT-NEXT:    stw 28, 12(5)
-; LE-32BIT-NEXT:    subfic 28, 4, 64
-; LE-32BIT-NEXT:    slw 12, 3, 12
-; LE-32BIT-NEXT:    or 29, 29, 27
-; LE-32BIT-NEXT:    addi 27, 4, -32
-; LE-32BIT-NEXT:    or 10, 10, 12
-; LE-32BIT-NEXT:    subfic 12, 28, 32
-; LE-32BIT-NEXT:    srw 30, 9, 30
-; LE-32BIT-NEXT:    slw 12, 9, 12
-; LE-32BIT-NEXT:    srw 9, 9, 28
-; LE-32BIT-NEXT:    srw 28, 3, 28
-; LE-32BIT-NEXT:    slw 11, 3, 11
-; LE-32BIT-NEXT:    slw 3, 3, 27
-; LE-32BIT-NEXT:    slw 27, 6, 27
-; LE-32BIT-NEXT:    or 0, 0, 27
-; LE-32BIT-NEXT:    or 12, 28, 12
-; LE-32BIT-NEXT:    slw 4, 6, 4
-; LE-32BIT-NEXT:    or 3, 29, 3
-; LE-32BIT-NEXT:    or 9, 0, 9
-; LE-32BIT-NEXT:    or 12, 12, 30
-; LE-32BIT-NEXT:    bc 12, 4, .LBB7_4
-; LE-32BIT-NEXT:  # %bb.3:
-; LE-32BIT-NEXT:    ori 3, 8, 0
-; LE-32BIT-NEXT:    ori 8, 10, 0
-; LE-32BIT-NEXT:    b .LBB7_5
-; LE-32BIT-NEXT:  .LBB7_4:
-; LE-32BIT-NEXT:    addi 8, 9, 0
-; LE-32BIT-NEXT:  .LBB7_5:
-; LE-32BIT-NEXT:    or 4, 4, 12
-; LE-32BIT-NEXT:    stw 3, 8(5)
-; LE-32BIT-NEXT:    bc 12, 2, .LBB7_7
-; LE-32BIT-NEXT:  # %bb.6:
-; LE-32BIT-NEXT:    ori 3, 8, 0
-; LE-32BIT-NEXT:    b .LBB7_8
-; LE-32BIT-NEXT:  .LBB7_7:
-; LE-32BIT-NEXT:    addi 3, 7, 0
-; LE-32BIT-NEXT:  .LBB7_8:
-; LE-32BIT-NEXT:    bc 12, 4, .LBB7_10
-; LE-32BIT-NEXT:  # %bb.9:
-; LE-32BIT-NEXT:    ori 4, 11, 0
-; LE-32BIT-NEXT:    b .LBB7_10
-; LE-32BIT-NEXT:  .LBB7_10:
+; LE-32BIT-NEXT:    lwz 4, 12(4)
+; LE-32BIT-NEXT:    stw 6, 44(1)
+; LE-32BIT-NEXT:    stw 6, 40(1)
+; LE-32BIT-NEXT:    clrlwi 4, 4, 28
+; LE-32BIT-NEXT:    stw 6, 36(1)
+; LE-32BIT-NEXT:    stw 6, 32(1)
+; LE-32BIT-NEXT:    stw 3, 28(1)
+; LE-32BIT-NEXT:    addi 3, 1, 16
+; LE-32BIT-NEXT:    stw 9, 24(1)
+; LE-32BIT-NEXT:    stw 8, 20(1)
+; LE-32BIT-NEXT:    stw 7, 16(1)
+; LE-32BIT-NEXT:    lwzux 3, 4, 3
+; LE-32BIT-NEXT:    lwz 6, 4(4)
+; LE-32BIT-NEXT:    lwz 7, 12(4)
+; LE-32BIT-NEXT:    lwz 4, 8(4)
 ; LE-32BIT-NEXT:    stw 3, 0(5)
-; LE-32BIT-NEXT:    bc 12, 2, .LBB7_12
-; LE-32BIT-NEXT:  # %bb.11:
-; LE-32BIT-NEXT:    ori 3, 4, 0
-; LE-32BIT-NEXT:    b .LBB7_13
-; LE-32BIT-NEXT:  .LBB7_12:
-; LE-32BIT-NEXT:    addi 3, 6, 0
-; LE-32BIT-NEXT:  .LBB7_13:
-; LE-32BIT-NEXT:    stw 3, 4(5)
-; LE-32BIT-NEXT:    lwz 30, 24(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 29, 20(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 28, 16(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 27, 12(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    addi 1, 1, 32
+; LE-32BIT-NEXT:    stw 4, 8(5)
+; LE-32BIT-NEXT:    stw 7, 12(5)
+; LE-32BIT-NEXT:    stw 6, 4(5)
+; LE-32BIT-NEXT:    addi 1, 1, 48
 ; LE-32BIT-NEXT:    blr
   %src = load i128, ptr %src.ptr, align 1
   %byteOff = load i128, ptr %byteOff.ptr, align 1
   %bitOff = shl i128 %byteOff, 3
   %res = shl i128 %src, %bitOff
   store i128 %res, ptr %dst, align 1
   ret void
 }

 ; BE-NEXT:    b .LBB8_2
 ; BE-NEXT:  .LBB8_2:
 ; BE-NEXT:    std 3, 0(5)
 ; BE-NEXT:    std 6, 8(5)
 ; BE-NEXT:    blr
 ;
 ; LE-32BIT-LABEL: ashr_16bytes:
 ; LE-32BIT:       # %bb.0:
-; LE-32BIT-NEXT:    stwu 1, -32(1)
+; LE-32BIT-NEXT:    stwu 1, -48(1)
+; LE-32BIT-NEXT:    lwz 7, 0(3)
+; LE-32BIT-NEXT:    addi 6, 1, 32
+; LE-32BIT-NEXT:    lwz 8, 4(3)
+; LE-32BIT-NEXT:    lwz 9, 8(3)
+; LE-32BIT-NEXT:    lwz 3, 12(3)
 ; LE-32BIT-NEXT:    lwz 4, 12(4)
-; LE-32BIT-NEXT:    lwz 8, 0(3)
-; LE-32BIT-NEXT:    lwz 9, 4(3)
-; LE-32BIT-NEXT:    lwz 6, 8(3)
-; LE-32BIT-NEXT:    lwz 7, 12(3)
-; LE-32BIT-NEXT:    rlwinm. 3, 4, 3, 0, 28
-; LE-32BIT-NEXT:    subfic 10, 3, 96
-; LE-32BIT-NEXT:    addi 11, 3, -64
-; LE-32BIT-NEXT:    stw 27, 12(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    addi 12, 3, -96
-; LE-32BIT-NEXT:    stw 30, 24(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    subfic 30, 3, 32
-; LE-32BIT-NEXT:    slw 10, 8, 10
-; LE-32BIT-NEXT:    srw 27, 9, 11
-; LE-32BIT-NEXT:    stw 26, 8(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    sraw 26, 8, 12
-; LE-32BIT-NEXT:    stw 28, 16(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 28, 9, 3
-; LE-32BIT-NEXT:    stw 29, 20(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    addi 29, 3, -32
-; LE-32BIT-NEXT:    cmpwi 1, 12, 1
-; LE-32BIT-NEXT:    slw 12, 8, 30
-; LE-32BIT-NEXT:    or 10, 27, 10
-; LE-32BIT-NEXT:    srw 0, 7, 3
-; LE-32BIT-NEXT:    sraw 27, 8, 29
-; LE-32BIT-NEXT:    bc 12, 4, .LBB8_2
-; LE-32BIT-NEXT:  # %bb.1:
-; LE-32BIT-NEXT:    ori 10, 26, 0
-; LE-32BIT-NEXT:    b .LBB8_2
-; LE-32BIT-NEXT:  .LBB8_2:
-; LE-32BIT-NEXT:    cmpwi 1, 29, 1
-; LE-32BIT-NEXT:    or 12, 28, 12
-; LE-32BIT-NEXT:    subfic 28, 3, 64
-; LE-32BIT-NEXT:    slw 26, 6, 30
-; LE-32BIT-NEXT:    srawi 4, 8, 31
-; LE-32BIT-NEXT:    bc 12, 4, .LBB8_4
-; LE-32BIT-NEXT:  # %bb.3:
-; LE-32BIT-NEXT:    ori 12, 27, 0
-; LE-32BIT-NEXT:    b .LBB8_4
-; LE-32BIT-NEXT:  .LBB8_4:
-; LE-32BIT-NEXT:    sraw 27, 8, 3
-; LE-32BIT-NEXT:    or 0, 0, 26
-; LE-32BIT-NEXT:    slw 26, 9, 28
-; LE-32BIT-NEXT:    sraw 11, 8, 11
-; LE-32BIT-NEXT:    slw 8, 8, 28
-; LE-32BIT-NEXT:    subfic 28, 28, 32
-; LE-32BIT-NEXT:    slw 30, 9, 30
-; LE-32BIT-NEXT:    srw 9, 9, 28
-; LE-32BIT-NEXT:    srw 29, 6, 29
-; LE-32BIT-NEXT:    or 8, 8, 9
-; LE-32BIT-NEXT:    cmplwi 1, 3, 64
-; LE-32BIT-NEXT:    or 0, 0, 29
-; LE-32BIT-NEXT:    srw 3, 6, 3
-; LE-32BIT-NEXT:    or 8, 8, 30
-; LE-32BIT-NEXT:    or 9, 0, 26
-; LE-32BIT-NEXT:    or 3, 3, 8
-; LE-32BIT-NEXT:    bc 12, 4, .LBB8_6
-; LE-32BIT-NEXT:  # %bb.5:
-; LE-32BIT-NEXT:    ori 28, 4, 0
-; LE-32BIT-NEXT:    ori 9, 10, 0
-; LE-32BIT-NEXT:    ori 3, 11, 0
-; LE-32BIT-NEXT:    b .LBB8_7
-; LE-32BIT-NEXT:  .LBB8_6:
-; LE-32BIT-NEXT:    addi 28, 27, 0
-; LE-32BIT-NEXT:    addi 4, 12, 0
-; LE-32BIT-NEXT:  .LBB8_7:
-; LE-32BIT-NEXT:    lwz 30, 24(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    bc 12, 2, .LBB8_8
-; LE-32BIT-NEXT:    b .LBB8_9
-; LE-32BIT-NEXT:  .LBB8_8:
-; LE-32BIT-NEXT:    addi 3, 6, 0
-; LE-32BIT-NEXT:  .LBB8_9:
+; LE-32BIT-NEXT:    stw 3, 44(1)
+; LE-32BIT-NEXT:    srawi 3, 7, 31
+; LE-32BIT-NEXT:    clrlwi 4, 4, 28
+; LE-32BIT-NEXT:    stw 9, 40(1)
+; LE-32BIT-NEXT:    stw 8, 36(1)
+; LE-32BIT-NEXT:    stw 7, 32(1)
+; LE-32BIT-NEXT:    stw 3, 28(1)
+; LE-32BIT-NEXT:    stw 3, 24(1)
+; LE-32BIT-NEXT:    stw 3, 20(1)
+; LE-32BIT-NEXT:    stw 3, 16(1)
+; LE-32BIT-NEXT:    sub 3, 6, 4
+; LE-32BIT-NEXT:    lwz 4, 4(3)
+; LE-32BIT-NEXT:    lwz 6, 0(3)
+; LE-32BIT-NEXT:    lwz 7, 8(3)
+; LE-32BIT-NEXT:    lwz 3, 12(3)
+; LE-32BIT-NEXT:    stw 7, 8(5)
+; LE-32BIT-NEXT:    stw 3, 12(5)
+; LE-32BIT-NEXT:    stw 6, 0(5)
 ; LE-32BIT-NEXT:    stw 4, 4(5)
-; LE-32BIT-NEXT:    bc 12, 2, .LBB8_11
-; LE-32BIT-NEXT:  # %bb.10:
-; LE-32BIT-NEXT:    ori 4, 9, 0
-; LE-32BIT-NEXT:    b .LBB8_12
-; LE-32BIT-NEXT:  .LBB8_11:
-; LE-32BIT-NEXT:    addi 4, 7, 0
-; LE-32BIT-NEXT:  .LBB8_12:
-; LE-32BIT-NEXT:    stw 28, 0(5)
-; LE-32BIT-NEXT:    stw 4, 12(5)
-; LE-32BIT-NEXT:    stw 3, 8(5)
-; LE-32BIT-NEXT:    lwz 29, 20(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 28, 16(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 27, 12(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 26, 8(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    addi 1, 1, 32
+; LE-32BIT-NEXT:    addi 1, 1, 48
 ; LE-32BIT-NEXT:    blr
   %src = load i128, ptr %src.ptr, align 1
   %byteOff = load i128, ptr %byteOff.ptr, align 1
   %bitOff = shl i128 %byteOff, 3
   %res = ashr i128 %src, %bitOff
   store i128 %res, ptr %dst, align 1
   ret void
 }
 
 define void @lshr_32bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
 ; LE-64BIT-LABEL: lshr_32bytes:
 ; LE-64BIT:       # %bb.0:
-; LE-64BIT-NEXT:    lwz 4, 0(4)
-; LE-64BIT-NEXT:    ld 7, 0(3)
-; LE-64BIT-NEXT:    ld 8, 8(3)
-; LE-64BIT-NEXT:    ld 9, 16(3)
-; LE-64BIT-NEXT:    li 6, 0
-; LE-64BIT-NEXT:    ld 3, 24(3)
-; LE-64BIT-NEXT:    std 28, -32(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 21, -88(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 24, -64(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    rlwinm. 4, 4, 3, 0, 28
-; LE-64BIT-NEXT:    std 25, -56(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    subfic 28, 4, 64
-; LE-64BIT-NEXT:    subfic 11, 4, 192
-; LE-64BIT-NEXT:    std 27, -40(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 29, -24(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    addi 0, 4, -128
-; LE-64BIT-NEXT:    srd 29, 9, 4
-; LE-64BIT-NEXT:    addi 27, 4, -64
-; LE-64BIT-NEXT:    subfic 25, 4, 128
-; LE-64BIT-NEXT:    sld 24, 8, 28
-; LE-64BIT-NEXT:    std 22, -80(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 26, -48(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    sld 21, 9, 28
-; LE-64BIT-NEXT:    sld 28, 3, 28
-; LE-64BIT-NEXT:    std 30, -16(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    srd 10, 7, 4
-; LE-64BIT-NEXT:    addi 30, 4, -192
-; LE-64BIT-NEXT:    subfic 22, 25, 64
-; LE-64BIT-NEXT:    sld 11, 3, 11
-; LE-64BIT-NEXT:    srd 26, 9, 0
-; LE-64BIT-NEXT:    or 29, 29, 28
-; LE-64BIT-NEXT:    std 23, -72(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    or 10, 10, 24
-; LE-64BIT-NEXT:    srd 28, 3, 27
-; LE-64BIT-NEXT:    srd 30, 3, 30
-; LE-64BIT-NEXT:    or 11, 26, 11
-; LE-64BIT-NEXT:    ld 26, -48(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    srd 23, 8, 27
-; LE-64BIT-NEXT:    srd 27, 9, 22
-; LE-64BIT-NEXT:    or 29, 29, 28
-; LE-64BIT-NEXT:    or 11, 11, 30
-; LE-64BIT-NEXT:    ld 24, -64(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    sld 28, 3, 25
-; LE-64BIT-NEXT:    or 10, 10, 23
-; LE-64BIT-NEXT:    ld 23, -72(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    ld 22, -80(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    sld 9, 9, 25
-; LE-64BIT-NEXT:    or 30, 28, 27
-; LE-64BIT-NEXT:    ld 28, -32(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    ld 27, -40(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    ld 25, -56(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    cmplwi 1, 4, 128
-; LE-64BIT-NEXT:    srd 12, 8, 4
-; LE-64BIT-NEXT:    or 9, 10, 9
-; LE-64BIT-NEXT:    or 30, 30, 21
-; LE-64BIT-NEXT:    ld 21, -88(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    srd 10, 3, 0
-; LE-64BIT-NEXT:    isel 9, 9, 11, 4
-; LE-64BIT-NEXT:    or 11, 12, 30
-; LE-64BIT-NEXT:    ld 30, -16(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    iseleq 7, 7, 9
-; LE-64BIT-NEXT:    srd 3, 3, 4
-; LE-64BIT-NEXT:    isel 9, 11, 10, 4
-; LE-64BIT-NEXT:    std 7, 0(5)
-; LE-64BIT-NEXT:    isel 0, 29, 6, 4
-; LE-64BIT-NEXT:    ld 29, -24(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    iseleq 4, 8, 9
-; LE-64BIT-NEXT:    std 0, 16(5)
-; LE-64BIT-NEXT:    isel 3, 3, 6, 4
-; LE-64BIT-NEXT:    std 4, 8(5)
-; LE-64BIT-NEXT:    std 3, 24(5)
+; LE-64BIT-NEXT:    li 6, 16
+; LE-64BIT-NEXT:    lxvd2x 1, 0, 3
+; LE-64BIT-NEXT:    xxlxor 2, 2, 2
+; LE-64BIT-NEXT:    addi 7, 1, -64
+; LE-64BIT-NEXT:    li 8, 32
+; LE-64BIT-NEXT:    lxvd2x 0, 3, 6
+; LE-64BIT-NEXT:    lwz 3, 0(4)
+; LE-64BIT-NEXT:    li 4, 48
+; LE-64BIT-NEXT:    stxvd2x 2, 7, 4
+; LE-64BIT-NEXT:    stxvd2x 2, 7, 8
+; LE-64BIT-NEXT:    clrldi 3, 3, 59
+; LE-64BIT-NEXT:    stxvd2x 0, 7, 6
+; LE-64BIT-NEXT:    stxvd2x 1, 0, 7
+; LE-64BIT-NEXT:    add 4, 7, 3
+; LE-64BIT-NEXT:    lxvd2x 0, 7, 3
+; LE-64BIT-NEXT:    lxvd2x 1, 4, 6
+; LE-64BIT-NEXT:    stxvd2x 1, 5, 6
+; LE-64BIT-NEXT:    stxvd2x 0, 0, 5
 ; LE-64BIT-NEXT:    blr
 ;
 ; BE-LABEL: lshr_32bytes:
 ; BE:       # %bb.0:
+; BE-NEXT:    ld 6, 0(3)
+; BE-NEXT:    ld 7, 8(3)
+; BE-NEXT:    ld 8, 16(3)
+; BE-NEXT:    ld 3, 24(3)
 ; BE-NEXT:    lwz 4, 28(4)
-; BE-NEXT:    ld 7, 16(3)
-; BE-NEXT:    ld 8, 24(3)
-; BE-NEXT:    ld 9, 8(3)
-; BE-NEXT:    ld 3, 0(3)
-; BE-NEXT:    std 27, -40(1) # 8-byte Folded Spill
-; BE-NEXT:    std 30, -16(1) # 8-byte Folded Spill
-; BE-NEXT:    std 28, -32(1) # 8-byte Folded Spill
-; BE-NEXT:    std 29, -24(1) # 8-byte Folded Spill
-; BE-NEXT:    li 6, 0
-; BE-NEXT:    rlwinm. 4, 4, 3, 0, 28
-; BE-NEXT:    subfic 10, 4, 192
-; BE-NEXT:    addi 11, 4, -128
-; BE-NEXT:    addi 12, 4, -192
-; BE-NEXT:    subfic 30, 4, 64
-; BE-NEXT:    sld 10, 3, 10
-; BE-NEXT:    srd 27, 9, 11
-; BE-NEXT:    srd 0, 8, 4
-; BE-NEXT:    addi 29, 4, -64
-; BE-NEXT:    subfic 28, 4, 128
-; BE-NEXT:    srd 12, 3, 12
-; BE-NEXT:    or 10, 27, 10
-; BE-NEXT:    sld 27, 7, 30
-; BE-NEXT:    or 10, 10, 12
-; BE-NEXT:    or 0, 0, 27
-; BE-NEXT:    srd 27, 7, 29
-; BE-NEXT:    subfic 12, 28, 64
-; BE-NEXT:    or 0, 0, 27
-; BE-NEXT:    sld 27, 3, 28
-; BE-NEXT:    srd 12, 9, 12
-; BE-NEXT:    sld 28, 9, 28
-; BE-NEXT:    cmplwi 1, 4, 128
-; BE-NEXT:    or 12, 27, 12
-; BE-NEXT:    or 28, 0, 28
-; BE-NEXT:    sld 0, 9, 30
-; BE-NEXT:    srd 9, 9, 4
-; BE-NEXT:    srd 11, 3, 11
-; BE-NEXT:    bc 12, 4, .LBB9_1
-; BE-NEXT:    b .LBB9_2
-; BE-NEXT:  .LBB9_1:
-; BE-NEXT:    addi 10, 28, 0
-; BE-NEXT:  .LBB9_2:
-; BE-NEXT:    ld 28, -32(1) # 8-byte Folded Reload
-; BE-NEXT:    ld 27, -40(1) # 8-byte Folded Reload
-; BE-NEXT:    or 12, 12, 0
-; BE-NEXT:    srd 0, 7, 4
-; BE-NEXT:    or 12, 0, 12
-; BE-NEXT:    sld 0, 3, 30
-; BE-NEXT:    srd 30, 3, 29
-; BE-NEXT:    bc 12, 4, .LBB9_3
-; BE-NEXT:    b .LBB9_4
-; BE-NEXT:  .LBB9_3:
-; BE-NEXT:    addi 11, 12, 0
-; BE-NEXT:  .LBB9_4:
-; BE-NEXT:    srd 3, 3, 4
-; BE-NEXT:    bc 12, 2, .LBB9_6
-; BE-NEXT:  # %bb.5:
-; BE-NEXT:    ori 4, 10, 0
-; BE-NEXT:    b .LBB9_7
-; BE-NEXT:  .LBB9_6:
-; BE-NEXT:    addi 4, 8, 0
-; BE-NEXT:  .LBB9_7:
-; BE-NEXT:    ld 29, -24(1) # 8-byte Folded Reload
-; BE-NEXT:    or 9, 9, 0
-; BE-NEXT:    or 9, 9, 30
-; BE-NEXT:    bc 12, 2, .LBB9_9
-; BE-NEXT:  # %bb.8:
-; BE-NEXT:    ori 7, 11, 0
-; BE-NEXT:    b .LBB9_9
-; BE-NEXT:  .LBB9_9:
-; BE-NEXT:    bc 12, 4, .LBB9_11
-; BE-NEXT:  # %bb.10:
-; BE-NEXT:    ori 8, 6, 0
-; BE-NEXT:    ori 3, 6, 0
-; BE-NEXT:    b .LBB9_12
-; BE-NEXT:  .LBB9_11:
-; BE-NEXT:    addi 8, 9, 0
-; BE-NEXT:  .LBB9_12:
-; BE-NEXT:    std 4, 24(5)
-; BE-NEXT:    ld 30, -16(1) # 8-byte Folded Reload
+; BE-NEXT:    addi 9, 1, -64
+; BE-NEXT:    li 10, 0
+; BE-NEXT:    std 10, 24(9)
+; BE-NEXT:    std 10, 16(9)
+; BE-NEXT:    std 10, 8(9)
+; BE-NEXT:    std 10, -64(1)
+; BE-NEXT:    std 3, 56(9)
+; BE-NEXT:    clrlwi 3, 4, 27
+; BE-NEXT:    neg 3, 3
+; BE-NEXT:    std 8, 48(9)
+; BE-NEXT:    std 7, 40(9)
+; BE-NEXT:    std 6, 32(9)
+; BE-NEXT:    extsw 3, 3
+; BE-NEXT:    addi 4, 1, -32
+; BE-NEXT:    ldux 3, 4, 3
+; BE-NEXT:    ld 6, 8(4)
+; BE-NEXT:    ld 7, 24(4)
+; BE-NEXT:    ld 4, 16(4)
 ; BE-NEXT:    std 3, 0(5)
-; BE-NEXT:    std 8, 8(5)
-; BE-NEXT:    std 7, 16(5)
+; BE-NEXT:    std 4, 16(5)
+; BE-NEXT:    std 7, 24(5)
+; BE-NEXT:    std 6, 8(5)
 ; BE-NEXT:    blr
 ;
 ; LE-32BIT-LABEL: lshr_32bytes:
 ; LE-32BIT:       # %bb.0:
-; LE-32BIT-NEXT:    stwu 1, -144(1)
-; LE-32BIT-NEXT:    mfcr 12
-; LE-32BIT-NEXT:    stw 14, 72(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 15, 76(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 16, 80(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 17, 84(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 18, 88(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 19, 92(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 20, 96(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 21, 100(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 22, 104(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 23, 108(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 24, 112(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 25, 116(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 26, 120(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 27, 124(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 28, 128(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 29, 132(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 30, 136(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 31, 140(1) # 4-byte Folded Spill
+; LE-32BIT-NEXT:    stwu 1, -80(1)
+; LE-32BIT-NEXT:    lwz 7, 0(3)
+; LE-32BIT-NEXT:    li 6, 0
+; LE-32BIT-NEXT:    lwz 8, 4(3)
+; LE-32BIT-NEXT:    lwz 9, 8(3)
+; LE-32BIT-NEXT:    lwz 10, 12(3)
+; LE-32BIT-NEXT:    lwz 11, 16(3)
+; LE-32BIT-NEXT:    lwz 12, 20(3)
+; LE-32BIT-NEXT:    lwz 0, 24(3)
+; LE-32BIT-NEXT:    lwz 3, 28(3)
+; LE-32BIT-NEXT:    lwz 4, 28(4)
+; LE-32BIT-NEXT:    stw 3, 76(1)
+; LE-32BIT-NEXT:    addi 3, 1, 48
+; LE-32BIT-NEXT:    clrlwi 4, 4, 27
+; LE-32BIT-NEXT:    stw 6, 44(1)
+; LE-32BIT-NEXT:    sub 3, 3, 4
+; LE-32BIT-NEXT:    stw 6, 40(1)
+; LE-32BIT-NEXT:    stw 6, 36(1)
+; LE-32BIT-NEXT:    stw 6, 32(1)
+; LE-32BIT-NEXT:    stw 6, 28(1)
+; LE-32BIT-NEXT:    stw 6, 24(1)
+; LE-32BIT-NEXT:    stw 6, 20(1)
+; LE-32BIT-NEXT:    stw 6, 16(1)
+; LE-32BIT-NEXT:    stw 0, 72(1)
 ; LE-32BIT-NEXT:    stw 12, 68(1)
-; LE-32BIT-NEXT:    lwz 0, 28(4)
-; LE-32BIT-NEXT:    lwz 11, 4(3)
+; LE-32BIT-NEXT:    stw 11, 64(1)
+; LE-32BIT-NEXT:    stw 10, 60(1)
+; LE-32BIT-NEXT:    stw 9, 56(1)
+; LE-32BIT-NEXT:    stw 8, 52(1)
+; LE-32BIT-NEXT:    stw 7, 48(1)
+; LE-32BIT-NEXT:    lwz 4, 4(3)
 ; LE-32BIT-NEXT:    lwz 6, 0(3)
-; LE-32BIT-NEXT:    rlwinm. 30, 0, 3, 0, 28
-; LE-32BIT-NEXT:    stw 5, 64(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    subfic 21, 30, 224
-; LE-32BIT-NEXT:    lwz 5, 24(3)
-; LE-32BIT-NEXT:    subfic 4, 30, 160
-; LE-32BIT-NEXT:    lwz 7, 28(3)
-; LE-32BIT-NEXT:    addi 0, 30, -128
-; LE-32BIT-NEXT:    lwz 10, 20(3)
-; LE-32BIT-NEXT:    subfic 28, 30, 96
-; LE-32BIT-NEXT:    lwz 8, 16(3)
-; LE-32BIT-NEXT:    addi 29, 30, -64
-; LE-32BIT-NEXT:    lwz 27, 12(3)
-; LE-32BIT-NEXT:    subfic 12, 30, 32
-; LE-32BIT-NEXT:    lwz 9, 8(3)
-; LE-32BIT-NEXT:    addi 3, 30, -192
-; LE-32BIT-NEXT:    slw 21, 6, 21
-; LE-32BIT-NEXT:    srw 16, 11, 3
-; LE-32BIT-NEXT:    stw 3, 56(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 20, 7, 30
-; LE-32BIT-NEXT:    slw 15, 9, 4
-; LE-32BIT-NEXT:    srw 14, 27, 0
-; LE-32BIT-NEXT:    slw 31, 8, 28
-; LE-32BIT-NEXT:    srw 3, 10, 29
-; LE-32BIT-NEXT:    or 21, 16, 21
-; LE-32BIT-NEXT:    slw 16, 5, 12
-; LE-32BIT-NEXT:    srw 19, 10, 30
-; LE-32BIT-NEXT:    or 15, 14, 15
-; LE-32BIT-NEXT:    slw 14, 8, 12
-; LE-32BIT-NEXT:    or 3, 3, 31
-; LE-32BIT-NEXT:    slw 31, 6, 4
-; LE-32BIT-NEXT:    or 20, 20, 16
-; LE-32BIT-NEXT:    srw 16, 11, 0
-; LE-32BIT-NEXT:    stw 7, 60(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    addi 26, 30, -224
-; LE-32BIT-NEXT:    mr 7, 10
-; LE-32BIT-NEXT:    mr 10, 12
-; LE-32BIT-NEXT:    or 19, 19, 14
-; LE-32BIT-NEXT:    slw 14, 6, 28
-; LE-32BIT-NEXT:    or 16, 16, 31
-; LE-32BIT-NEXT:    srw 31, 11, 29
-; LE-32BIT-NEXT:    addi 23, 30, -160
-; LE-32BIT-NEXT:    srw 18, 27, 30
-; LE-32BIT-NEXT:    stw 0, 40(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    mr 12, 28
-; LE-32BIT-NEXT:    or 14, 31, 14
-; LE-32BIT-NEXT:    mr 28, 9
-; LE-32BIT-NEXT:    slw 31, 9, 10
-; LE-32BIT-NEXT:    srw 0, 6, 26
-; LE-32BIT-NEXT:    addi 25, 30, -96
-; LE-32BIT-NEXT:    srw 17, 11, 30
-; LE-32BIT-NEXT:    stw 4, 36(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 18, 18, 31
-; LE-32BIT-NEXT:    slw 31, 6, 10
-; LE-32BIT-NEXT:    or 4, 21, 0
-; LE-32BIT-NEXT:    srw 0, 28, 23
-; LE-32BIT-NEXT:    or 17, 17, 31
-; LE-32BIT-NEXT:    addi 31, 30, -32
-; LE-32BIT-NEXT:    or 0, 15, 0
-; LE-32BIT-NEXT:    srw 15, 8, 25
-; LE-32BIT-NEXT:    or 3, 3, 15
-; LE-32BIT-NEXT:    srw 15, 5, 31
-; LE-32BIT-NEXT:    or 20, 20, 15
-; LE-32BIT-NEXT:    srw 15, 8, 31
-; LE-32BIT-NEXT:    stw 3, 24(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 3, 19, 15
-; LE-32BIT-NEXT:    srw 23, 6, 23
-; LE-32BIT-NEXT:    stw 3, 48(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    subfic 15, 30, 64
-; LE-32BIT-NEXT:    or 3, 16, 23
-; LE-32BIT-NEXT:    stw 3, 44(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    subfic 3, 15, 32
-; LE-32BIT-NEXT:    slw 16, 28, 15
-; LE-32BIT-NEXT:    srw 22, 27, 3
-; LE-32BIT-NEXT:    stw 4, 32(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 4, 16, 22
-; LE-32BIT-NEXT:    subfic 16, 30, 128
-; LE-32BIT-NEXT:    stw 5, 28(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    subfic 5, 16, 32
-; LE-32BIT-NEXT:    stw 4, 20(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    slw 4, 6, 16
-; LE-32BIT-NEXT:    srw 24, 11, 5
-; LE-32BIT-NEXT:    stw 29, 52(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    mr 29, 27
-; LE-32BIT-NEXT:    or 22, 4, 24
-; LE-32BIT-NEXT:    slw 24, 28, 16
-; LE-32BIT-NEXT:    srw 27, 27, 5
-; LE-32BIT-NEXT:    or 27, 24, 27
-; LE-32BIT-NEXT:    slw 24, 8, 15
-; LE-32BIT-NEXT:    srw 26, 7, 3
-; LE-32BIT-NEXT:    or 26, 24, 26
-; LE-32BIT-NEXT:    subfic 24, 30, 192
-; LE-32BIT-NEXT:    mr 9, 10
-; LE-32BIT-NEXT:    mr 10, 28
-; LE-32BIT-NEXT:    subfic 28, 24, 32
-; LE-32BIT-NEXT:    srw 28, 11, 28
-; LE-32BIT-NEXT:    slw 19, 6, 24
-; LE-32BIT-NEXT:    or 28, 19, 28
-; LE-32BIT-NEXT:    srw 19, 6, 25
-; LE-32BIT-NEXT:    or 19, 14, 19
-; LE-32BIT-NEXT:    srw 14, 10, 31
-; LE-32BIT-NEXT:    lwz 4, 64(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 18, 18, 14
-; LE-32BIT-NEXT:    srw 3, 11, 3
-; LE-32BIT-NEXT:    slw 14, 6, 15
-; LE-32BIT-NEXT:    cmplwi 5, 30, 64
-; LE-32BIT-NEXT:    cmplwi 1, 30, 128
-; LE-32BIT-NEXT:    slw 24, 11, 24
-; LE-32BIT-NEXT:    mr 21, 8
-; LE-32BIT-NEXT:    or 8, 14, 3
-; LE-32BIT-NEXT:    srw 14, 6, 31
-; LE-32BIT-NEXT:    crnand 21, 4, 20
-; LE-32BIT-NEXT:    srw 31, 6, 30
-; LE-32BIT-NEXT:    or 24, 0, 24
-; LE-32BIT-NEXT:    slw 0, 7, 15
-; LE-32BIT-NEXT:    mr 23, 7
-; LE-32BIT-NEXT:    or 17, 17, 14
-; LE-32BIT-NEXT:    bc 12, 21, .LBB9_2
-; LE-32BIT-NEXT:  # %bb.1:
-; LE-32BIT-NEXT:    ori 14, 31, 0
-; LE-32BIT-NEXT:    b .LBB9_3
-; LE-32BIT-NEXT:  .LBB9_2:
-; LE-32BIT-NEXT:    li 14, 0
-; LE-32BIT-NEXT:  .LBB9_3:
-; LE-32BIT-NEXT:    or 20, 20, 0
-; LE-32BIT-NEXT:    subfic 0, 16, 64
-; LE-32BIT-NEXT:    lwz 7, 20(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    srw 31, 29, 0
-; LE-32BIT-NEXT:    stw 14, 0(4)
-; LE-32BIT-NEXT:    subfic 14, 0, 32
-; LE-32BIT-NEXT:    slw 14, 10, 14
-; LE-32BIT-NEXT:    or 14, 31, 14
-; LE-32BIT-NEXT:    slw 31, 29, 9
-; LE-32BIT-NEXT:    lwz 3, 36(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 7, 7, 31
-; LE-32BIT-NEXT:    slw 31, 11, 12
-; LE-32BIT-NEXT:    stw 7, 20(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 7, 22, 31
-; LE-32BIT-NEXT:    slw 31, 29, 12
-; LE-32BIT-NEXT:    or 27, 27, 31
-; LE-32BIT-NEXT:    slw 31, 23, 9
-; LE-32BIT-NEXT:    or 26, 26, 31
-; LE-32BIT-NEXT:    slw 31, 11, 3
-; LE-32BIT-NEXT:    or 28, 28, 31
-; LE-32BIT-NEXT:    slw 31, 11, 15
-; LE-32BIT-NEXT:    lwz 22, 28(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 18, 18, 31
-; LE-32BIT-NEXT:    lwz 31, 40(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    srw 0, 10, 0
-; LE-32BIT-NEXT:    or 7, 7, 0
-; LE-32BIT-NEXT:    srw 0, 22, 30
-; LE-32BIT-NEXT:    slw 25, 11, 9
-; LE-32BIT-NEXT:    or 26, 0, 26
-; LE-32BIT-NEXT:    srw 0, 10, 31
-; LE-32BIT-NEXT:    or 3, 8, 25
-; LE-32BIT-NEXT:    or 28, 0, 28
-; LE-32BIT-NEXT:    srw 0, 10, 30
-; LE-32BIT-NEXT:    srw 5, 10, 5
-; LE-32BIT-NEXT:    or 3, 0, 3
-; LE-32BIT-NEXT:    bc 12, 21, .LBB9_5
-; LE-32BIT-NEXT:  # %bb.4:
-; LE-32BIT-NEXT:    ori 0, 17, 0
-; LE-32BIT-NEXT:    b .LBB9_6
-; LE-32BIT-NEXT:  .LBB9_5:
-; LE-32BIT-NEXT:    li 0, 0
-; LE-32BIT-NEXT:  .LBB9_6:
-; LE-32BIT-NEXT:    lwz 8, 32(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 5, 14, 5
-; LE-32BIT-NEXT:    mr 14, 4
-; LE-32BIT-NEXT:    stw 0, 4(4)
-; LE-32BIT-NEXT:    slw 0, 11, 16
-; LE-32BIT-NEXT:    lwz 4, 52(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    cmplwi 6, 31, 64
-; LE-32BIT-NEXT:    mr 9, 21
-; LE-32BIT-NEXT:    or 5, 0, 5
-; LE-32BIT-NEXT:    lwz 0, 56(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    bc 12, 24, .LBB9_8
-; LE-32BIT-NEXT:  # %bb.7:
-; LE-32BIT-NEXT:    ori 25, 8, 0
-; LE-32BIT-NEXT:    b .LBB9_9
-; LE-32BIT-NEXT:  .LBB9_8:
-; LE-32BIT-NEXT:    addi 25, 24, 0
-; LE-32BIT-NEXT:  .LBB9_9:
-; LE-32BIT-NEXT:    bc 12, 20, .LBB9_11
-; LE-32BIT-NEXT:  # %bb.10:
-; LE-32BIT-NEXT:    ori 24, 19, 0
-; LE-32BIT-NEXT:    b .LBB9_12
-; LE-32BIT-NEXT:  .LBB9_11:
-; LE-32BIT-NEXT:    addi 24, 18, 0
-; LE-32BIT-NEXT:  .LBB9_12:
-; LE-32BIT-NEXT:    srw 19, 9, 4
-; LE-32BIT-NEXT:    srw 17, 6, 4
-; LE-32BIT-NEXT:    lwz 4, 20(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    srw 30, 21, 30
-; LE-32BIT-NEXT:    lwz 8, 24(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    slw 21, 29, 16
-; LE-32BIT-NEXT:    cmplwi 2, 16, 64
-; LE-32BIT-NEXT:    cmplwi 3, 16, 0
-; LE-32BIT-NEXT:    li 16, 0
-; LE-32BIT-NEXT:    srw 18, 6, 0
-; LE-32BIT-NEXT:    bc 12, 8, .LBB9_14
-; LE-32BIT-NEXT:  # %bb.13:
-; LE-32BIT-NEXT:    ori 0, 16, 0
-; LE-32BIT-NEXT:    b .LBB9_15
-; LE-32BIT-NEXT:  .LBB9_14:
-; LE-32BIT-NEXT:    addi 0, 21, 0
-; LE-32BIT-NEXT:  .LBB9_15:
-; LE-32BIT-NEXT:    lwz 21, 60(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    bc 12, 8, .LBB9_16
-; LE-32BIT-NEXT:    b .LBB9_17
-; LE-32BIT-NEXT:  .LBB9_16:
-; LE-32BIT-NEXT:    addi 4, 7, 0
-; LE-32BIT-NEXT:  .LBB9_17:
-; LE-32BIT-NEXT:    bc 12, 20, .LBB9_18
-; LE-32BIT-NEXT:    b .LBB9_19
-; LE-32BIT-NEXT:  .LBB9_18:
-; LE-32BIT-NEXT:    addi 8, 20, 0
-; LE-32BIT-NEXT:  .LBB9_19:
-; LE-32BIT-NEXT:    mr 12, 29
-; LE-32BIT-NEXT:    lwz 7, 48(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    slw 20, 29, 15
-; LE-32BIT-NEXT:    srw 29, 6, 31
-; LE-32BIT-NEXT:    bc 12, 2, .LBB9_20
-; LE-32BIT-NEXT:    b .LBB9_21
-; LE-32BIT-NEXT:  .LBB9_20:
-; LE-32BIT-NEXT:    addi 8, 21, 0
-; LE-32BIT-NEXT:  .LBB9_21:
-; LE-32BIT-NEXT:    cmplwi 7, 31, 0
-; LE-32BIT-NEXT:    bc 12, 20, .LBB9_23
-; LE-32BIT-NEXT:  # %bb.22:
-; LE-32BIT-NEXT:    ori 26, 19, 0
-; LE-32BIT-NEXT:    ori 3, 17, 0
-; LE-32BIT-NEXT:    b .LBB9_23
-; LE-32BIT-NEXT:  .LBB9_23:
-; LE-32BIT-NEXT:    or 8, 8, 0
-; LE-32BIT-NEXT:    bc 12, 20, .LBB9_25
-; LE-32BIT-NEXT:  # %bb.24:
-; LE-32BIT-NEXT:    ori 0, 16, 0
-; LE-32BIT-NEXT:    b .LBB9_26
-; LE-32BIT-NEXT:  .LBB9_25:
-; LE-32BIT-NEXT:    addi 0, 30, 0
-; LE-32BIT-NEXT:  .LBB9_26:
-; LE-32BIT-NEXT:    bc 12, 24, .LBB9_28
-; LE-32BIT-NEXT:  # %bb.27:
-; LE-32BIT-NEXT:    ori 30, 16, 0
-; LE-32BIT-NEXT:    b .LBB9_29
-; LE-32BIT-NEXT:  .LBB9_28:
-; LE-32BIT-NEXT:    addi 30, 29, 0
-; LE-32BIT-NEXT:  .LBB9_29:
-; LE-32BIT-NEXT:    bc 12, 20, .LBB9_31
-; LE-32BIT-NEXT:  # %bb.30:
-; LE-32BIT-NEXT:    ori 29, 16, 0
-; LE-32BIT-NEXT:    b .LBB9_32
-; LE-32BIT-NEXT:  .LBB9_31:
-; LE-32BIT-NEXT:    addi 29, 7, 0
-; LE-32BIT-NEXT:  .LBB9_32:
-; LE-32BIT-NEXT:    lwz 7, 44(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    bc 12, 30, .LBB9_33
-; LE-32BIT-NEXT:    b .LBB9_34
-; LE-32BIT-NEXT:  .LBB9_33:
-; LE-32BIT-NEXT:    addi 25, 12, 0
-; LE-32BIT-NEXT:  .LBB9_34:
-; LE-32BIT-NEXT:    bc 12, 14, .LBB9_35
-; LE-32BIT-NEXT:    b .LBB9_36
-; LE-32BIT-NEXT:  .LBB9_35:
-; LE-32BIT-NEXT:    addi 4, 6, 0
-; LE-32BIT-NEXT:  .LBB9_36:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB9_38
-; LE-32BIT-NEXT:  # %bb.37:
-; LE-32BIT-NEXT:    ori 6, 26, 0
-; LE-32BIT-NEXT:    b .LBB9_39
-; LE-32BIT-NEXT:  .LBB9_38:
-; LE-32BIT-NEXT:    addi 6, 22, 0
-; LE-32BIT-NEXT:  .LBB9_39:
-; LE-32BIT-NEXT:    li 26, 0
-; LE-32BIT-NEXT:    bc 12, 2, .LBB9_40
-; LE-32BIT-NEXT:    b .LBB9_41
-; LE-32BIT-NEXT:  .LBB9_40:
-; LE-32BIT-NEXT:    addi 3, 10, 0
-; LE-32BIT-NEXT:  .LBB9_41:
-; LE-32BIT-NEXT:    bc 12, 8, .LBB9_43
-; LE-32BIT-NEXT:  # %bb.42:
-; LE-32BIT-NEXT:    ori 5, 20, 0
-; LE-32BIT-NEXT:    b .LBB9_43
-; LE-32BIT-NEXT:  .LBB9_43:
-; LE-32BIT-NEXT:    bc 12, 4, .LBB9_45
-; LE-32BIT-NEXT:  # %bb.44:
-; LE-32BIT-NEXT:    ori 8, 25, 0
-; LE-32BIT-NEXT:    b .LBB9_45
-; LE-32BIT-NEXT:  .LBB9_45:
-; LE-32BIT-NEXT:    bc 12, 24, .LBB9_47
-; LE-32BIT-NEXT:  # %bb.46:
-; LE-32BIT-NEXT:    ori 28, 18, 0
-; LE-32BIT-NEXT:    b .LBB9_47
-; LE-32BIT-NEXT:  .LBB9_47:
-; LE-32BIT-NEXT:    bc 12, 8, .LBB9_49
-; LE-32BIT-NEXT:  # %bb.48:
-; LE-32BIT-NEXT:    ori 27, 16, 0
-; LE-32BIT-NEXT:    b .LBB9_49
-; LE-32BIT-NEXT:  .LBB9_49:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB9_51
-; LE-32BIT-NEXT:  # %bb.50:
-; LE-32BIT-NEXT:    ori 12, 24, 0
-; LE-32BIT-NEXT:    b .LBB9_51
-; LE-32BIT-NEXT:  .LBB9_51:
-; LE-32BIT-NEXT:    bc 12, 4, .LBB9_53
-; LE-32BIT-NEXT:  # %bb.52:
-; LE-32BIT-NEXT:    ori 3, 26, 0
-; LE-32BIT-NEXT:    b .LBB9_53
-; LE-32BIT-NEXT:  .LBB9_53:
-; LE-32BIT-NEXT:    bc 12, 14, .LBB9_54
-; LE-32BIT-NEXT:    b .LBB9_55
-; LE-32BIT-NEXT:  .LBB9_54:
-; LE-32BIT-NEXT:    addi 5, 11, 0
-; LE-32BIT-NEXT:  .LBB9_55:
-; LE-32BIT-NEXT:    bc 12, 30, .LBB9_56
-; LE-32BIT-NEXT:    b .LBB9_57
-; LE-32BIT-NEXT:  .LBB9_56:
-; LE-32BIT-NEXT:    addi 28, 10, 0
-; LE-32BIT-NEXT:  .LBB9_57:
-; LE-32BIT-NEXT:    or 6, 6, 27
-; LE-32BIT-NEXT:    stw 3, 8(14)
-; LE-32BIT-NEXT:    or 3, 0, 4
-; LE-32BIT-NEXT:    bc 12, 2, .LBB9_59
-; LE-32BIT-NEXT:  # %bb.58:
-; LE-32BIT-NEXT:    ori 4, 8, 0
-; LE-32BIT-NEXT:    b .LBB9_60
-; LE-32BIT-NEXT:  .LBB9_59:
-; LE-32BIT-NEXT:    addi 4, 21, 0
-; LE-32BIT-NEXT:  .LBB9_60:
-; LE-32BIT-NEXT:    bc 12, 24, .LBB9_62
-; LE-32BIT-NEXT:  # %bb.61:
-; LE-32BIT-NEXT:    ori 24, 16, 0
-; LE-32BIT-NEXT:    b .LBB9_63
-; LE-32BIT-NEXT:  .LBB9_62:
-; LE-32BIT-NEXT:    addi 24, 7, 0
-; LE-32BIT-NEXT:  .LBB9_63:
-; LE-32BIT-NEXT:    bc 12, 4, .LBB9_65
-; LE-32BIT-NEXT:  # %bb.64:
-; LE-32BIT-NEXT:    ori 3, 30, 0
-; LE-32BIT-NEXT:    ori 6, 28, 0
-; LE-32BIT-NEXT:    ori 12, 16, 0
-; LE-32BIT-NEXT:    b .LBB9_65
-; LE-32BIT-NEXT:  .LBB9_65:
-; LE-32BIT-NEXT:    stw 4, 28(14)
-; LE-32BIT-NEXT:    or 4, 29, 5
-; LE-32BIT-NEXT:    bc 12, 4, .LBB9_67
-; LE-32BIT-NEXT:  # %bb.66:
-; LE-32BIT-NEXT:    ori 4, 24, 0
-; LE-32BIT-NEXT:    b .LBB9_67
-; LE-32BIT-NEXT:  .LBB9_67:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB9_69
-; LE-32BIT-NEXT:  # %bb.68:
-; LE-32BIT-NEXT:    ori 5, 6, 0
-; LE-32BIT-NEXT:    b .LBB9_70
-; LE-32BIT-NEXT:  .LBB9_69:
-; LE-32BIT-NEXT:    addi 3, 9, 0
-; LE-32BIT-NEXT:    addi 5, 22, 0
-; LE-32BIT-NEXT:  .LBB9_70:
-; LE-32BIT-NEXT:    stw 12, 12(14)
-; LE-32BIT-NEXT:    stw 3, 16(14)
-; LE-32BIT-NEXT:    bc 12, 2, .LBB9_72
-; LE-32BIT-NEXT:  # %bb.71:
-; LE-32BIT-NEXT:    ori 3, 4, 0
-; LE-32BIT-NEXT:    b .LBB9_73
-; LE-32BIT-NEXT:  .LBB9_72:
-; LE-32BIT-NEXT:    addi 3, 23, 0
-; LE-32BIT-NEXT:  .LBB9_73:
-; LE-32BIT-NEXT:    stw 5, 24(14)
-; LE-32BIT-NEXT:    stw 3, 20(14)
-; LE-32BIT-NEXT:    lwz 12, 68(1)
-; LE-32BIT-NEXT:    lwz 31, 140(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    mtcrf 32, 12 # cr2
-; LE-32BIT-NEXT:    mtcrf 16, 12 # cr3
-; LE-32BIT-NEXT:    lwz 30, 136(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 29, 132(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 28, 128(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 27, 124(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 26, 120(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 25, 116(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 24, 112(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 23, 108(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 22, 104(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 21, 100(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 20, 96(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 19, 92(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 18, 88(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 17, 84(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 16, 80(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 15, 76(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 14, 72(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    addi 1, 1, 144
+; LE-32BIT-NEXT:    lwz 7, 12(3)
+; LE-32BIT-NEXT:    lwz 8, 8(3)
+; LE-32BIT-NEXT:    lwz 9, 20(3)
+; LE-32BIT-NEXT:    lwz 10, 16(3)
+; LE-32BIT-NEXT:    lwz 11, 24(3)
+; LE-32BIT-NEXT:    lwz 3, 28(3)
+; LE-32BIT-NEXT:    stw 11, 24(5)
+; LE-32BIT-NEXT:    stw 3, 28(5)
+; LE-32BIT-NEXT:    stw 10, 16(5)
+; LE-32BIT-NEXT:    stw 9, 20(5)
+; LE-32BIT-NEXT:    stw 8, 8(5)
+; LE-32BIT-NEXT:    stw 7, 12(5)
+; LE-32BIT-NEXT:    stw 6, 0(5)
+; LE-32BIT-NEXT:    stw 4, 4(5)
+; LE-32BIT-NEXT:    addi 1, 1, 80
 ; LE-32BIT-NEXT:    blr
   %src = load i256, ptr %src.ptr, align 1
   %byteOff = load i256, ptr %byteOff.ptr, align 1
   %bitOff = shl i256 %byteOff, 3
   %res = lshr i256 %src, %bitOff
   store i256 %res, ptr %dst, align 1
   ret void
 }
 define void @shl_32bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
 ; LE-64BIT-LABEL: shl_32bytes:
 ; LE-64BIT:       # %bb.0:
+; LE-64BIT-NEXT:    li 6, 16
 ; LE-64BIT-NEXT:    lwz 4, 0(4)
-; LE-64BIT-NEXT:    ld 7, 24(3)
-; LE-64BIT-NEXT:    ld 8, 16(3)
-; LE-64BIT-NEXT:    ld 9, 8(3)
-; LE-64BIT-NEXT:    li 6, 0
-; LE-64BIT-NEXT:    ld 3, 0(3)
-; LE-64BIT-NEXT:    std 28, -32(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 21, -88(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 24, -64(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    rlwinm. 4, 4, 3, 0, 28
-; LE-64BIT-NEXT:    std 25, -56(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    subfic 28, 4, 64
-; LE-64BIT-NEXT:    subfic 11, 4, 192
-; LE-64BIT-NEXT:    std 27, -40(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 29, -24(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    addi 0, 4, -128
-; LE-64BIT-NEXT:    sld 29, 9, 4
-; LE-64BIT-NEXT:    addi 27, 4, -64
-; LE-64BIT-NEXT:    subfic 25, 4, 128
-; LE-64BIT-NEXT:    srd 24, 8, 28
-; LE-64BIT-NEXT:    std 22, -80(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 26, -48(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    srd 21, 9, 28
-; LE-64BIT-NEXT:    srd 28, 3, 28
-; LE-64BIT-NEXT:    std 30, -16(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    sld 10, 7, 4
-; LE-64BIT-NEXT:    addi 30, 4, -192
-; LE-64BIT-NEXT:    subfic 22, 25, 64
-; LE-64BIT-NEXT:    srd 11, 3, 11
-; LE-64BIT-NEXT:    sld 26, 9, 0
-; LE-64BIT-NEXT:    or 29, 29, 28
-; LE-64BIT-NEXT:    std 23, -72(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    or 10, 10, 24
-; LE-64BIT-NEXT:    sld 28, 3, 27
-; LE-64BIT-NEXT:    sld 30, 3, 30
-; LE-64BIT-NEXT:    or 11, 26, 11
-; LE-64BIT-NEXT:    ld 26, -48(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    sld 23, 8, 27
-; LE-64BIT-NEXT:    sld 27, 9, 22
-; LE-64BIT-NEXT:    or 29, 29, 28
-; LE-64BIT-NEXT:    or 11, 11, 30
-; LE-64BIT-NEXT:    ld 24, -64(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    srd 28, 3, 25
-; LE-64BIT-NEXT:    or 10, 10, 23
-; LE-64BIT-NEXT:    ld 23, -72(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    ld 22, -80(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    srd 9, 9, 25
-; LE-64BIT-NEXT:    or 30, 28, 27
-; LE-64BIT-NEXT:    ld 28, -32(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    ld 27, -40(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    ld 25, -56(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    cmplwi 1, 4, 128
-; LE-64BIT-NEXT:    sld 12, 8, 4
-; LE-64BIT-NEXT:    or 9, 10, 9
-; LE-64BIT-NEXT:    or 30, 30, 21
-; LE-64BIT-NEXT:    ld 21, -88(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    sld 10, 3, 0
-; LE-64BIT-NEXT:    isel 9, 9, 11, 4
-; LE-64BIT-NEXT:    or 11, 12, 30
-; LE-64BIT-NEXT:    ld 30, -16(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    iseleq 7, 7, 9
-; LE-64BIT-NEXT:    sld 3, 3, 4
-; LE-64BIT-NEXT:    isel 9, 11, 10, 4
-; LE-64BIT-NEXT:    std 7, 24(5)
-; LE-64BIT-NEXT:    isel 0, 29, 6, 4
-; LE-64BIT-NEXT:    ld 29, -24(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    iseleq 4, 8, 9
-; LE-64BIT-NEXT:    std 0, 8(5)
-; LE-64BIT-NEXT:    isel 3, 3, 6, 4
-; LE-64BIT-NEXT:    std 4, 16(5)
-; LE-64BIT-NEXT:    std 3, 0(5)
+; LE-64BIT-NEXT:    xxlxor 1, 1, 1
+; LE-64BIT-NEXT:    lxvd2x 2, 0, 3
+; LE-64BIT-NEXT:    li 7, 48
+; LE-64BIT-NEXT:    addi 8, 1, -32
+; LE-64BIT-NEXT:    lxvd2x 0, 3, 6
+; LE-64BIT-NEXT:    addi 3, 1, -64
+; LE-64BIT-NEXT:    clrlwi 4, 4, 27
+; LE-64BIT-NEXT:    stxvd2x 1, 3, 6
+; LE-64BIT-NEXT:    neg 4, 4
+; LE-64BIT-NEXT:    stxvd2x 0, 3, 7
+; LE-64BIT-NEXT:    li 7, 32
+; LE-64BIT-NEXT:    extsw 4, 4
+; LE-64BIT-NEXT:    stxvd2x 2, 3, 7
+; LE-64BIT-NEXT:    stxvd2x 1, 0, 3
+; LE-64BIT-NEXT:    add 3, 8, 4
+; LE-64BIT-NEXT:    lxvd2x 0, 8, 4
+; LE-64BIT-NEXT:    lxvd2x 1, 3, 6
+; LE-64BIT-NEXT:    stxvd2x 1, 5, 6
+; LE-64BIT-NEXT:    stxvd2x 0, 0, 5
 ; LE-64BIT-NEXT:    blr
 ;
 ; BE-LABEL: shl_32bytes:
 ; BE:       # %bb.0:
-; BE-NEXT:    lwz 4, 28(4)
+; BE-NEXT:    ld 6, 0(3)
 ; BE-NEXT:    ld 7, 8(3)
-; BE-NEXT:    ld 8, 0(3)
-; BE-NEXT:    ld 9, 16(3)
+; BE-NEXT:    ld 8, 16(3)
 ; BE-NEXT:    ld 3, 24(3)
-; BE-NEXT:    std 27, -40(1) # 8-byte Folded Spill
-; BE-NEXT:    std 30, -16(1) # 8-byte Folded Spill
-; BE-NEXT:    std 28, -32(1) # 8-byte Folded Spill
-; BE-NEXT:    std 29, -24(1) # 8-byte Folded Spill
-; BE-NEXT:    li 6, 0
-; BE-NEXT:    rlwinm. 4, 4, 3, 0, 28
-; BE-NEXT:    subfic 10, 4, 192
-; BE-NEXT:    addi 11, 4, -128
-; BE-NEXT:    addi 12, 4, -192
-; BE-NEXT:    subfic 30, 4, 64
-; BE-NEXT:    srd 10, 3, 10
-; BE-NEXT:    sld 27, 9, 11
-; BE-NEXT:    sld 0, 8, 4
-; BE-NEXT:    addi 29, 4, -64
-; BE-NEXT:    subfic 28, 4, 128
-; BE-NEXT:    sld 12, 3, 12
-; BE-NEXT:    or 10, 27, 10
-; BE-NEXT:    srd 27, 7, 30
-; BE-NEXT:    or 10, 10, 12
-; BE-NEXT:    or 0, 0, 27
-; BE-NEXT:    sld 27, 7, 29
-; BE-NEXT:    subfic 12, 28, 64
-; BE-NEXT:    or 0, 0, 27
-; BE-NEXT:    srd 27, 3, 28
-; BE-NEXT:    sld 12, 9, 12
-; BE-NEXT:    srd 28, 9, 28
-; BE-NEXT:    cmplwi 1, 4, 128
-; BE-NEXT:    or 12, 27, 12
-; BE-NEXT:    or 28, 0, 28
-; BE-NEXT:    srd 0, 9, 30
-; BE-NEXT:    sld 9, 9, 4
-; BE-NEXT:    sld 11, 3, 11
-; BE-NEXT:    bc 12, 4, .LBB10_1
-; BE-NEXT:    b .LBB10_2
-; BE-NEXT:  .LBB10_1:
-; BE-NEXT:    addi 10, 28, 0
-; BE-NEXT:  .LBB10_2:
-; BE-NEXT:    ld 28, -32(1) # 8-byte Folded Reload
-; BE-NEXT:    ld 27, -40(1) # 8-byte Folded Reload
-; BE-NEXT:    or 12, 12, 0
-; BE-NEXT:    sld 0, 7, 4
-; BE-NEXT:    or 12, 0, 12
-; BE-NEXT:    srd 0, 3, 30
-; BE-NEXT:    sld 30, 3, 29
-; BE-NEXT:    bc 12, 4, .LBB10_3
-; BE-NEXT:    b .LBB10_4
-; BE-NEXT:  .LBB10_3:
-; BE-NEXT:    addi 11, 12, 0
-; BE-NEXT:  .LBB10_4:
-; BE-NEXT:    sld 3, 3, 4
-; BE-NEXT:    bc 12, 2, .LBB10_6
-; BE-NEXT:  # %bb.5:
-; BE-NEXT:    ori 4, 10, 0
-; BE-NEXT:    b .LBB10_7
-; BE-NEXT:  .LBB10_6:
-; BE-NEXT:    addi 4, 8, 0
-; BE-NEXT:  .LBB10_7:
-; BE-NEXT:    ld 29, -24(1) # 8-byte Folded Reload
-; BE-NEXT:    or 9, 9, 0
-; BE-NEXT:    or 9, 9, 30
-; BE-NEXT:    bc 12, 2, .LBB10_9
-; BE-NEXT:  # %bb.8:
-; BE-NEXT:    ori 7, 11, 0
-; BE-NEXT:    b .LBB10_9
-; BE-NEXT:  .LBB10_9:
-; BE-NEXT:    bc 12, 4, .LBB10_11
-; BE-NEXT:  # %bb.10:
-; BE-NEXT:    ori 8, 6, 0
-; BE-NEXT:    ori 3, 6, 0
-; BE-NEXT:    b .LBB10_12
-; BE-NEXT:  .LBB10_11:
-; BE-NEXT:    addi 8, 9, 0
-; BE-NEXT:  .LBB10_12:
+; BE-NEXT:    lwz 4, 28(4)
+; BE-NEXT:    addi 9, 1, -64
+; BE-NEXT:    li 10, 0
+; BE-NEXT:    std 10, 56(9)
+; BE-NEXT:    std 10, 48(9)
+; BE-NEXT:    std 10, 40(9)
+; BE-NEXT:    std 10, 32(9)
+; BE-NEXT:    std 3, 24(9)
+; BE-NEXT:    std 8, 16(9)
+; BE-NEXT:    std 7, 8(9)
+; BE-NEXT:    std 6, -64(1)
+; BE-NEXT:    clrldi 3, 4, 59
+; BE-NEXT:    ldux 4, 3, 9
+; BE-NEXT:    ld 6, 8(3)
+; BE-NEXT:    ld 7, 24(3)
+; BE-NEXT:    ld 3, 16(3)
 ; BE-NEXT:    std 4, 0(5)
-; BE-NEXT:    ld 30, -16(1) # 8-byte Folded Reload
-; BE-NEXT:    std 3, 24(5)
-; BE-NEXT:    std 8, 16(5)
-; BE-NEXT:    std 7, 8(5)
+; BE-NEXT:    std 3, 16(5)
+; BE-NEXT:    std 7, 24(5)
+; BE-NEXT:    std 6, 8(5)
 ; BE-NEXT:    blr
 ;
 ; LE-32BIT-LABEL: shl_32bytes:
 ; LE-32BIT:       # %bb.0:
-; LE-32BIT-NEXT:    stwu 1, -144(1)
-; LE-32BIT-NEXT:    mfcr 12
-; LE-32BIT-NEXT:    stw 14, 72(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 15, 76(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 16, 80(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 17, 84(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 18, 88(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 19, 92(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 20, 96(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 21, 100(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 22, 104(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 23, 108(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 24, 112(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 25, 116(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 26, 120(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 27, 124(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 28, 128(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 29, 132(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 30, 136(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 31, 140(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 12, 68(1)
-; LE-32BIT-NEXT:    lwz 0, 28(4)
-; LE-32BIT-NEXT:    stw 5, 64(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    lwz 6, 24(3)
-; LE-32BIT-NEXT:    rlwinm. 30, 0, 3, 0, 28
-; LE-32BIT-NEXT:    lwz 5, 28(3)
-; LE-32BIT-NEXT:    subfic 21, 30, 224
-; LE-32BIT-NEXT:    lwz 7, 4(3)
-; LE-32BIT-NEXT:    subfic 0, 30, 160
-; LE-32BIT-NEXT:    lwz 9, 0(3)
-; LE-32BIT-NEXT:    addi 4, 30, -128
-; LE-32BIT-NEXT:    lwz 10, 8(3)
-; LE-32BIT-NEXT:    subfic 28, 30, 96
-; LE-32BIT-NEXT:    lwz 8, 12(3)
-; LE-32BIT-NEXT:    addi 29, 30, -64
-; LE-32BIT-NEXT:    lwz 12, 16(3)
-; LE-32BIT-NEXT:    subfic 25, 30, 32
-; LE-32BIT-NEXT:    lwz 11, 20(3)
-; LE-32BIT-NEXT:    addi 3, 30, -192
-; LE-32BIT-NEXT:    srw 21, 5, 21
-; LE-32BIT-NEXT:    slw 16, 6, 3
-; LE-32BIT-NEXT:    stw 3, 56(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    slw 20, 9, 30
-; LE-32BIT-NEXT:    srw 15, 11, 0
-; LE-32BIT-NEXT:    slw 14, 12, 4
-; LE-32BIT-NEXT:    srw 31, 8, 28
-; LE-32BIT-NEXT:    slw 3, 10, 29
-; LE-32BIT-NEXT:    or 21, 16, 21
-; LE-32BIT-NEXT:    srw 16, 7, 25
-; LE-32BIT-NEXT:    slw 19, 10, 30
-; LE-32BIT-NEXT:    or 15, 14, 15
-; LE-32BIT-NEXT:    srw 14, 8, 25
-; LE-32BIT-NEXT:    or 3, 3, 31
-; LE-32BIT-NEXT:    srw 31, 5, 0
-; LE-32BIT-NEXT:    or 20, 20, 16
-; LE-32BIT-NEXT:    slw 16, 6, 4
-; LE-32BIT-NEXT:    addi 27, 30, -224
-; LE-32BIT-NEXT:    or 19, 19, 14
-; LE-32BIT-NEXT:    srw 14, 5, 28
-; LE-32BIT-NEXT:    or 16, 16, 31
-; LE-32BIT-NEXT:    slw 31, 6, 29
-; LE-32BIT-NEXT:    addi 23, 30, -160
-; LE-32BIT-NEXT:    slw 18, 12, 30
-; LE-32BIT-NEXT:    stw 0, 40(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 14, 31, 14
-; LE-32BIT-NEXT:    srw 31, 11, 25
-; LE-32BIT-NEXT:    slw 0, 5, 27
-; LE-32BIT-NEXT:    addi 26, 30, -96
-; LE-32BIT-NEXT:    slw 17, 6, 30
-; LE-32BIT-NEXT:    or 18, 18, 31
-; LE-32BIT-NEXT:    srw 31, 5, 25
-; LE-32BIT-NEXT:    or 21, 21, 0
-; LE-32BIT-NEXT:    slw 0, 11, 23
-; LE-32BIT-NEXT:    or 17, 17, 31
-; LE-32BIT-NEXT:    addi 31, 30, -32
-; LE-32BIT-NEXT:    or 0, 15, 0
-; LE-32BIT-NEXT:    slw 15, 8, 26
-; LE-32BIT-NEXT:    stw 29, 52(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 29, 3, 15
-; LE-32BIT-NEXT:    slw 15, 7, 31
-; LE-32BIT-NEXT:    or 20, 20, 15
-; LE-32BIT-NEXT:    slw 15, 8, 31
-; LE-32BIT-NEXT:    or 3, 19, 15
-; LE-32BIT-NEXT:    subfic 15, 30, 128
-; LE-32BIT-NEXT:    slw 23, 5, 23
-; LE-32BIT-NEXT:    stw 3, 48(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 3, 16, 23
-; LE-32BIT-NEXT:    subfic 16, 15, 32
-; LE-32BIT-NEXT:    stw 3, 44(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 3, 11, 15
-; LE-32BIT-NEXT:    slw 22, 12, 16
-; LE-32BIT-NEXT:    or 23, 3, 22
-; LE-32BIT-NEXT:    subfic 22, 30, 64
-; LE-32BIT-NEXT:    stw 9, 60(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    mr 9, 10
-; LE-32BIT-NEXT:    subfic 3, 22, 32
-; LE-32BIT-NEXT:    stw 4, 36(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 4, 8, 22
-; LE-32BIT-NEXT:    slw 24, 9, 3
-; LE-32BIT-NEXT:    or 4, 4, 24
-; LE-32BIT-NEXT:    subfic 24, 30, 192
-; LE-32BIT-NEXT:    subfic 27, 24, 32
-; LE-32BIT-NEXT:    mr 10, 26
-; LE-32BIT-NEXT:    slw 27, 6, 27
-; LE-32BIT-NEXT:    srw 26, 5, 24
-; LE-32BIT-NEXT:    stw 28, 24(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 27, 26, 27
-; LE-32BIT-NEXT:    srw 26, 11, 22
-; LE-32BIT-NEXT:    slw 28, 12, 3
-; LE-32BIT-NEXT:    or 28, 26, 28
-; LE-32BIT-NEXT:    srw 26, 5, 15
-; LE-32BIT-NEXT:    slw 19, 6, 16
-; LE-32BIT-NEXT:    or 26, 26, 19
-; LE-32BIT-NEXT:    slw 19, 5, 10
-; LE-32BIT-NEXT:    stw 7, 32(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    mr 7, 9
-; LE-32BIT-NEXT:    or 19, 14, 19
-; LE-32BIT-NEXT:    slw 14, 11, 31
-; LE-32BIT-NEXT:    lwz 9, 64(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 18, 18, 14
-; LE-32BIT-NEXT:    slw 3, 6, 3
-; LE-32BIT-NEXT:    srw 14, 5, 22
-; LE-32BIT-NEXT:    cmplwi 5, 30, 64
-; LE-32BIT-NEXT:    cmplwi 1, 30, 128
-; LE-32BIT-NEXT:    srw 24, 6, 24
-; LE-32BIT-NEXT:    or 10, 14, 3
-; LE-32BIT-NEXT:    slw 14, 5, 31
-; LE-32BIT-NEXT:    crnand 21, 4, 20
-; LE-32BIT-NEXT:    slw 31, 5, 30
-; LE-32BIT-NEXT:    or 24, 0, 24
-; LE-32BIT-NEXT:    mr 3, 7
-; LE-32BIT-NEXT:    stw 7, 28(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 0, 7, 22
-; LE-32BIT-NEXT:    lwz 7, 24(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 17, 17, 14
-; LE-32BIT-NEXT:    bc 12, 21, .LBB10_2
-; LE-32BIT-NEXT:  # %bb.1:
-; LE-32BIT-NEXT:    ori 14, 31, 0
-; LE-32BIT-NEXT:    b .LBB10_3
-; LE-32BIT-NEXT:  .LBB10_2:
-; LE-32BIT-NEXT:    li 14, 0
-; LE-32BIT-NEXT:  .LBB10_3:
-; LE-32BIT-NEXT:    or 20, 20, 0
-; LE-32BIT-NEXT:    subfic 0, 15, 64
-; LE-32BIT-NEXT:    stw 14, 28(9)
-; LE-32BIT-NEXT:    subfic 14, 0, 32
-; LE-32BIT-NEXT:    srw 14, 11, 14
-; LE-32BIT-NEXT:    slw 31, 12, 0
-; LE-32BIT-NEXT:    or 14, 31, 14
-; LE-32BIT-NEXT:    srw 31, 12, 7
-; LE-32BIT-NEXT:    or 23, 23, 31
-; LE-32BIT-NEXT:    srw 31, 3, 25
-; LE-32BIT-NEXT:    lwz 3, 40(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 4, 4, 31
-; LE-32BIT-NEXT:    slw 0, 11, 0
-; LE-32BIT-NEXT:    cmplwi 2, 15, 64
-; LE-32BIT-NEXT:    srw 31, 6, 3
-; LE-32BIT-NEXT:    or 27, 27, 31
-; LE-32BIT-NEXT:    srw 31, 12, 25
-; LE-32BIT-NEXT:    or 28, 28, 31
-; LE-32BIT-NEXT:    srw 31, 6, 7
-; LE-32BIT-NEXT:    or 26, 26, 31
-; LE-32BIT-NEXT:    srw 31, 6, 22
-; LE-32BIT-NEXT:    or 18, 18, 31
-; LE-32BIT-NEXT:    lwz 31, 36(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    srw 25, 6, 25
-; LE-32BIT-NEXT:    or 3, 10, 25
-; LE-32BIT-NEXT:    or 26, 26, 0
-; LE-32BIT-NEXT:    cmplwi 6, 31, 64
-; LE-32BIT-NEXT:    slw 0, 11, 30
-; LE-32BIT-NEXT:    bc 12, 24, .LBB10_5
-; LE-32BIT-NEXT:  # %bb.4:
-; LE-32BIT-NEXT:    ori 25, 21, 0
-; LE-32BIT-NEXT:    b .LBB10_6
-; LE-32BIT-NEXT:  .LBB10_5:
-; LE-32BIT-NEXT:    addi 25, 24, 0
-; LE-32BIT-NEXT:  .LBB10_6:
-; LE-32BIT-NEXT:    slw 24, 11, 16
-; LE-32BIT-NEXT:    lwz 10, 32(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 3, 0, 3
-; LE-32BIT-NEXT:    bc 12, 21, .LBB10_8
-; LE-32BIT-NEXT:  # %bb.7:
-; LE-32BIT-NEXT:    ori 0, 17, 0
-; LE-32BIT-NEXT:    b .LBB10_9
-; LE-32BIT-NEXT:  .LBB10_8:
-; LE-32BIT-NEXT:    li 0, 0
-; LE-32BIT-NEXT:  .LBB10_9:
-; LE-32BIT-NEXT:    or 24, 14, 24
-; LE-32BIT-NEXT:    stw 0, 24(9)
-; LE-32BIT-NEXT:    srw 0, 6, 15
-; LE-32BIT-NEXT:    or 24, 0, 24
-; LE-32BIT-NEXT:    lwz 0, 56(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    slw 21, 10, 30
-; LE-32BIT-NEXT:    bc 12, 20, .LBB10_11
-; LE-32BIT-NEXT:  # %bb.10:
-; LE-32BIT-NEXT:    ori 7, 29, 0
-; LE-32BIT-NEXT:    b .LBB10_12
-; LE-32BIT-NEXT:  .LBB10_11:
-; LE-32BIT-NEXT:    addi 7, 20, 0
-; LE-32BIT-NEXT:  .LBB10_12:
-; LE-32BIT-NEXT:    or 4, 21, 4
-; LE-32BIT-NEXT:    slw 21, 11, 31
-; LE-32BIT-NEXT:    srw 20, 12, 15
-; LE-32BIT-NEXT:    cmplwi 3, 15, 0
-; LE-32BIT-NEXT:    li 15, 0
-; LE-32BIT-NEXT:    or 27, 21, 27
-; LE-32BIT-NEXT:    bc 12, 20, .LBB10_14
-; LE-32BIT-NEXT:  # %bb.13:
-; LE-32BIT-NEXT:    ori 21, 19, 0
-; LE-32BIT-NEXT:    b .LBB10_15
-; LE-32BIT-NEXT:  .LBB10_14:
-; LE-32BIT-NEXT:    addi 21, 18, 0
-; LE-32BIT-NEXT:  .LBB10_15:
-; LE-32BIT-NEXT:    mr 16, 9
-; LE-32BIT-NEXT:    lwz 9, 52(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    slw 18, 5, 0
-; LE-32BIT-NEXT:    bc 12, 8, .LBB10_17
-; LE-32BIT-NEXT:  # %bb.16:
-; LE-32BIT-NEXT:    ori 0, 15, 0
-; LE-32BIT-NEXT:    b .LBB10_18
-; LE-32BIT-NEXT:  .LBB10_17:
-; LE-32BIT-NEXT:    addi 0, 20, 0
-; LE-32BIT-NEXT:  .LBB10_18:
-; LE-32BIT-NEXT:    lwz 20, 60(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    slw 30, 8, 30
-; LE-32BIT-NEXT:    slw 19, 8, 9
-; LE-32BIT-NEXT:    slw 17, 5, 9
-; LE-32BIT-NEXT:    bc 12, 2, .LBB10_20
-; LE-32BIT-NEXT:  # %bb.19:
-; LE-32BIT-NEXT:    ori 9, 7, 0
-; LE-32BIT-NEXT:    b .LBB10_21
-; LE-32BIT-NEXT:  .LBB10_20:
-; LE-32BIT-NEXT:    addi 9, 20, 0
-; LE-32BIT-NEXT:  .LBB10_21:
-; LE-32BIT-NEXT:    lwz 7, 48(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    slw 29, 5, 31
-; LE-32BIT-NEXT:    or 9, 9, 0
-; LE-32BIT-NEXT:    bc 12, 20, .LBB10_23
-; LE-32BIT-NEXT:  # %bb.22:
-; LE-32BIT-NEXT:    ori 0, 15, 0
-; LE-32BIT-NEXT:    b .LBB10_24
-; LE-32BIT-NEXT:  .LBB10_23:
-; LE-32BIT-NEXT:    addi 0, 30, 0
-; LE-32BIT-NEXT:  .LBB10_24:
-; LE-32BIT-NEXT:    bc 12, 24, .LBB10_26
-; LE-32BIT-NEXT:  # %bb.25:
-; LE-32BIT-NEXT:    ori 30, 15, 0
-; LE-32BIT-NEXT:    b .LBB10_27
-; LE-32BIT-NEXT:  .LBB10_26:
-; LE-32BIT-NEXT:    addi 30, 29, 0
-; LE-32BIT-NEXT:  .LBB10_27:
-; LE-32BIT-NEXT:    bc 12, 8, .LBB10_28
-; LE-32BIT-NEXT:    b .LBB10_29
-; LE-32BIT-NEXT:  .LBB10_28:
-; LE-32BIT-NEXT:    addi 28, 26, 0
-; LE-32BIT-NEXT:  .LBB10_29:
-; LE-32BIT-NEXT:    bc 12, 20, .LBB10_31
-; LE-32BIT-NEXT:  # %bb.30:
-; LE-32BIT-NEXT:    ori 3, 17, 0
-; LE-32BIT-NEXT:    b .LBB10_31
-; LE-32BIT-NEXT:  .LBB10_31:
-; LE-32BIT-NEXT:    srw 22, 12, 22
-; LE-32BIT-NEXT:    bc 12, 20, .LBB10_33
-; LE-32BIT-NEXT:  # %bb.32:
-; LE-32BIT-NEXT:    ori 29, 15, 0
-; LE-32BIT-NEXT:    b .LBB10_34
-; LE-32BIT-NEXT:  .LBB10_33:
-; LE-32BIT-NEXT:    addi 29, 7, 0
-; LE-32BIT-NEXT:  .LBB10_34:
-; LE-32BIT-NEXT:    lwz 7, 44(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    bc 12, 20, .LBB10_36
-; LE-32BIT-NEXT:  # %bb.35:
-; LE-32BIT-NEXT:    ori 4, 19, 0
-; LE-32BIT-NEXT:    b .LBB10_36
-; LE-32BIT-NEXT:  .LBB10_36:
-; LE-32BIT-NEXT:    bc 12, 14, .LBB10_38
-; LE-32BIT-NEXT:  # %bb.37:
-; LE-32BIT-NEXT:    ori 5, 28, 0
-; LE-32BIT-NEXT:    b .LBB10_38
-; LE-32BIT-NEXT:  .LBB10_38:
-; LE-32BIT-NEXT:    li 28, 0
-; LE-32BIT-NEXT:    bc 12, 2, .LBB10_39
-; LE-32BIT-NEXT:    b .LBB10_40
-; LE-32BIT-NEXT:  .LBB10_39:
-; LE-32BIT-NEXT:    addi 3, 11, 0
-; LE-32BIT-NEXT:  .LBB10_40:
-; LE-32BIT-NEXT:    cmplwi 7, 31, 0
-; LE-32BIT-NEXT:    bc 12, 24, .LBB10_42
-; LE-32BIT-NEXT:  # %bb.41:
-; LE-32BIT-NEXT:    ori 27, 18, 0
-; LE-32BIT-NEXT:    b .LBB10_42
-; LE-32BIT-NEXT:  .LBB10_42:
-; LE-32BIT-NEXT:    bc 12, 8, .LBB10_44
-; LE-32BIT-NEXT:  # %bb.43:
-; LE-32BIT-NEXT:    ori 26, 22, 0
-; LE-32BIT-NEXT:    b .LBB10_45
-; LE-32BIT-NEXT:  .LBB10_44:
-; LE-32BIT-NEXT:    addi 26, 24, 0
-; LE-32BIT-NEXT:  .LBB10_45:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB10_46
-; LE-32BIT-NEXT:    b .LBB10_47
-; LE-32BIT-NEXT:  .LBB10_46:
-; LE-32BIT-NEXT:    addi 4, 10, 0
-; LE-32BIT-NEXT:  .LBB10_47:
-; LE-32BIT-NEXT:    bc 12, 4, .LBB10_49
-; LE-32BIT-NEXT:  # %bb.48:
-; LE-32BIT-NEXT:    ori 3, 28, 0
-; LE-32BIT-NEXT:    b .LBB10_49
-; LE-32BIT-NEXT:  .LBB10_49:
-; LE-32BIT-NEXT:    bc 12, 30, .LBB10_50
-; LE-32BIT-NEXT:    b .LBB10_51
-; LE-32BIT-NEXT:  .LBB10_50:
-; LE-32BIT-NEXT:    addi 25, 12, 0
-; LE-32BIT-NEXT:  .LBB10_51:
-; LE-32BIT-NEXT:    or 5, 0, 5
-; LE-32BIT-NEXT:    bc 12, 24, .LBB10_53
-; LE-32BIT-NEXT:  # %bb.52:
-; LE-32BIT-NEXT:    ori 24, 15, 0
-; LE-32BIT-NEXT:    b .LBB10_54
-; LE-32BIT-NEXT:  .LBB10_53:
-; LE-32BIT-NEXT:    addi 24, 7, 0
-; LE-32BIT-NEXT:  .LBB10_54:
-; LE-32BIT-NEXT:    bc 12, 8, .LBB10_56
-; LE-32BIT-NEXT:  # %bb.55:
-; LE-32BIT-NEXT:    ori 7, 15, 0
-; LE-32BIT-NEXT:    b .LBB10_57
-; LE-32BIT-NEXT:  .LBB10_56:
-; LE-32BIT-NEXT:    addi 7, 23, 0
-; LE-32BIT-NEXT:  .LBB10_57:
-; LE-32BIT-NEXT:    bc 12, 30, .LBB10_58
-; LE-32BIT-NEXT:    b .LBB10_59
-; LE-32BIT-NEXT:  .LBB10_58:
-; LE-32BIT-NEXT:    addi 27, 11, 0
-; LE-32BIT-NEXT:  .LBB10_59:
-; LE-32BIT-NEXT:    stw 3, 20(16)
-; LE-32BIT-NEXT:    or 3, 4, 7
-; LE-32BIT-NEXT:    bc 12, 4, .LBB10_61
-; LE-32BIT-NEXT:  # %bb.60:
-; LE-32BIT-NEXT:    ori 3, 27, 0
-; LE-32BIT-NEXT:    ori 9, 25, 0
-; LE-32BIT-NEXT:    b .LBB10_61
-; LE-32BIT-NEXT:  .LBB10_61:
-; LE-32BIT-NEXT:    bc 12, 14, .LBB10_63
-; LE-32BIT-NEXT:  # %bb.62:
-; LE-32BIT-NEXT:    ori 6, 26, 0
-; LE-32BIT-NEXT:    b .LBB10_63
-; LE-32BIT-NEXT:  .LBB10_63:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB10_65
-; LE-32BIT-NEXT:  # %bb.64:
-; LE-32BIT-NEXT:    ori 12, 21, 0
-; LE-32BIT-NEXT:    b .LBB10_65
-; LE-32BIT-NEXT:  .LBB10_65:
-; LE-32BIT-NEXT:    bc 12, 4, .LBB10_67
-; LE-32BIT-NEXT:  # %bb.66:
-; LE-32BIT-NEXT:    ori 5, 30, 0
-; LE-32BIT-NEXT:    b .LBB10_67
-; LE-32BIT-NEXT:  .LBB10_67:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB10_69
-; LE-32BIT-NEXT:  # %bb.68:
-; LE-32BIT-NEXT:    ori 4, 9, 0
-; LE-32BIT-NEXT:    b .LBB10_70
-; LE-32BIT-NEXT:  .LBB10_69:
-; LE-32BIT-NEXT:    addi 3, 10, 0
-; LE-32BIT-NEXT:    addi 4, 20, 0
-; LE-32BIT-NEXT:  .LBB10_70:
-; LE-32BIT-NEXT:    bc 12, 4, .LBB10_72
-; LE-32BIT-NEXT:  # %bb.71:
-; LE-32BIT-NEXT:    ori 12, 15, 0
-; LE-32BIT-NEXT:    b .LBB10_72
-; LE-32BIT-NEXT:  .LBB10_72:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB10_73
-; LE-32BIT-NEXT:    b .LBB10_74
-; LE-32BIT-NEXT:  .LBB10_73:
-; LE-32BIT-NEXT:    addi 5, 8, 0
-; LE-32BIT-NEXT:  .LBB10_74:
-; LE-32BIT-NEXT:    stw 3, 4(16)
-; LE-32BIT-NEXT:    lwz 3, 28(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    stw 4, 0(16)
-; LE-32BIT-NEXT:    or 4, 29, 6
-; LE-32BIT-NEXT:    bc 12, 4, .LBB10_76
-; LE-32BIT-NEXT:  # %bb.75:
-; LE-32BIT-NEXT:    ori 4, 24, 0
-; LE-32BIT-NEXT:    b .LBB10_76
-; LE-32BIT-NEXT:  .LBB10_76:
-; LE-32BIT-NEXT:    stw 12, 16(16)
-; LE-32BIT-NEXT:    bc 12, 2, .LBB10_78
-; LE-32BIT-NEXT:  # %bb.77:
-; LE-32BIT-NEXT:    ori 3, 4, 0
-; LE-32BIT-NEXT:    b .LBB10_78
-; LE-32BIT-NEXT:  .LBB10_78:
-; LE-32BIT-NEXT:    stw 5, 12(16)
-; LE-32BIT-NEXT:    stw 3, 8(16)
-; LE-32BIT-NEXT:    lwz 12, 68(1)
-; LE-32BIT-NEXT:    lwz 31, 140(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    mtcrf 32, 12 # cr2
-; LE-32BIT-NEXT:    mtcrf 16, 12 # cr3
-; LE-32BIT-NEXT:    lwz 30, 136(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 29, 132(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 28, 128(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 27, 124(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 26, 120(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 25, 116(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 24, 112(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 23, 108(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 22, 104(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 21, 100(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 20, 96(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 19, 92(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 18, 88(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 17, 84(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 16, 80(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 15, 76(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 14, 72(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    addi 1, 1, 144
+; LE-32BIT-NEXT:    stwu 1, -80(1)
+; LE-32BIT-NEXT:    lwz 7, 0(3)
+; LE-32BIT-NEXT:    li 6, 0
+; LE-32BIT-NEXT:    lwz 8, 4(3)
+; LE-32BIT-NEXT:    lwz 9, 8(3)
+; LE-32BIT-NEXT:    lwz 10, 12(3)
+; LE-32BIT-NEXT:    lwz 11, 16(3)
+; LE-32BIT-NEXT:    lwz 12, 20(3)
+; LE-32BIT-NEXT:    lwz 0, 24(3)
+; LE-32BIT-NEXT:    lwz 3, 28(3)
+; LE-32BIT-NEXT:    lwz 4, 28(4)
+; LE-32BIT-NEXT:    stw 6, 76(1)
+; LE-32BIT-NEXT:    stw 6, 72(1)
+; LE-32BIT-NEXT:    clrlwi 4, 4, 27
+; LE-32BIT-NEXT:    stw 6, 68(1)
+; LE-32BIT-NEXT:    stw 6, 64(1)
+; LE-32BIT-NEXT:    stw 6, 60(1)
+; LE-32BIT-NEXT:    stw 6, 56(1)
+; LE-32BIT-NEXT:    stw 6, 52(1)
+; LE-32BIT-NEXT:    stw 6, 48(1)
+; LE-32BIT-NEXT:    stw 3, 44(1)
+; LE-32BIT-NEXT:    addi 3, 1, 16
+; LE-32BIT-NEXT:    stw 0, 40(1)
+; LE-32BIT-NEXT:    stw 12, 36(1)
+; LE-32BIT-NEXT:    stw 11, 32(1)
+; LE-32BIT-NEXT:    stw 10, 28(1)
+; LE-32BIT-NEXT:    stw 9, 24(1)
+; LE-32BIT-NEXT:    stw 8, 20(1)
+; LE-32BIT-NEXT:    stw 7, 16(1)
+; LE-32BIT-NEXT:    lwzux 3, 4, 3
+; LE-32BIT-NEXT:    lwz 6, 4(4)
+; LE-32BIT-NEXT:    lwz 7, 12(4)
+; LE-32BIT-NEXT:    lwz 8, 8(4)
+; LE-32BIT-NEXT:    lwz 9, 20(4)
+; LE-32BIT-NEXT:    lwz 10, 16(4)
+; LE-32BIT-NEXT:    lwz 11, 28(4)
+; LE-32BIT-NEXT:    lwz 4, 24(4)
+; LE-32BIT-NEXT:    stw 3, 0(5)
+; LE-32BIT-NEXT:    stw 4, 24(5)
+; LE-32BIT-NEXT:    stw 11, 28(5)
+; LE-32BIT-NEXT:    stw 10, 16(5)
+; LE-32BIT-NEXT:    stw 9, 20(5)
+; LE-32BIT-NEXT:    stw 8, 8(5)
+; LE-32BIT-NEXT:    stw 7, 12(5)
+; LE-32BIT-NEXT:    stw 6, 4(5)
+; LE-32BIT-NEXT:    addi 1, 1, 80
 ; LE-32BIT-NEXT:    blr
   %src = load i256, ptr %src.ptr, align 1
   %byteOff = load i256, ptr %byteOff.ptr, align 1
   %bitOff = shl i256 %byteOff, 3
   %res = shl i256 %src, %bitOff
   store i256 %res, ptr %dst, align 1
   ret void
 }
 define void @ashr_32bytes(ptr %src.ptr, ptr %byteOff.ptr, ptr %dst) nounwind {
 ; LE-64BIT-LABEL: ashr_32bytes:
 ; LE-64BIT:       # %bb.0:
+; LE-64BIT-NEXT:    ld 7, 16(3)
+; LE-64BIT-NEXT:    ld 8, 24(3)
+; LE-64BIT-NEXT:    lxvd2x 0, 0, 3
 ; LE-64BIT-NEXT:    lwz 4, 0(4)
-; LE-64BIT-NEXT:    ld 6, 24(3)
-; LE-64BIT-NEXT:    ld 8, 16(3)
-; LE-64BIT-NEXT:    std 28, -32(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 29, -24(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 30, -16(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    std 26, -48(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    rlwinm. 4, 4, 3, 0, 28
-; LE-64BIT-NEXT:    sradi 9, 6, 63
-; LE-64BIT-NEXT:    subfic 10, 4, 192
-; LE-64BIT-NEXT:    addi 11, 4, -128
-; LE-64BIT-NEXT:    addi 30, 4, -192
-; LE-64BIT-NEXT:    sld 10, 6, 10
-; LE-64BIT-NEXT:    srd 29, 8, 11
-; LE-64BIT-NEXT:    subfic 28, 4, 64
-; LE-64BIT-NEXT:    std 27, -40(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    ld 7, 0(3)
-; LE-64BIT-NEXT:    ld 3, 8(3)
-; LE-64BIT-NEXT:    srd 0, 8, 4
-; LE-64BIT-NEXT:    srad 27, 6, 30
-; LE-64BIT-NEXT:    or 10, 29, 10
-; LE-64BIT-NEXT:    std 25, -56(1) # 8-byte Folded Spill
-; LE-64BIT-NEXT:    cmpwi 1, 30, 1
-; LE-64BIT-NEXT:    sld 26, 6, 28
-; LE-64BIT-NEXT:    addi 30, 4, -64
-; LE-64BIT-NEXT:    isel 10, 10, 27, 4
-; LE-64BIT-NEXT:    or 27, 0, 26
-; LE-64BIT-NEXT:    subfic 0, 4, 128
-; LE-64BIT-NEXT:    srd 12, 7, 4
-; LE-64BIT-NEXT:    sld 26, 3, 28
-; LE-64BIT-NEXT:    subfic 25, 0, 64
-; LE-64BIT-NEXT:    srad 29, 6, 30
-; LE-64BIT-NEXT:    cmpwi 1, 30, 1
-; LE-64BIT-NEXT:    or 12, 12, 26
-; LE-64BIT-NEXT:    srd 30, 3, 30
-; LE-64BIT-NEXT:    sld 28, 8, 28
-; LE-64BIT-NEXT:    srd 26, 8, 25
-; LE-64BIT-NEXT:    sld 8, 8, 0
-; LE-64BIT-NEXT:    or 12, 12, 30
-; LE-64BIT-NEXT:    ld 30, -16(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    ld 25, -56(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    sld 0, 6, 0
-; LE-64BIT-NEXT:    isel 29, 27, 29, 4
-; LE-64BIT-NEXT:    or 8, 12, 8
-; LE-64BIT-NEXT:    or 0, 0, 26
-; LE-64BIT-NEXT:    cmplwi 1, 4, 128
-; LE-64BIT-NEXT:    ld 26, -48(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    srd 27, 3, 4
-; LE-64BIT-NEXT:    or 0, 0, 28
-; LE-64BIT-NEXT:    ld 28, -32(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    srad 11, 6, 11
-; LE-64BIT-NEXT:    isel 8, 8, 10, 4
-; LE-64BIT-NEXT:    or 10, 27, 0
-; LE-64BIT-NEXT:    ld 27, -40(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    iseleq 7, 7, 8
-; LE-64BIT-NEXT:    srad 4, 6, 4
-; LE-64BIT-NEXT:    isel 8, 10, 11, 4
-; LE-64BIT-NEXT:    std 7, 0(5)
-; LE-64BIT-NEXT:    isel 12, 29, 9, 4
-; LE-64BIT-NEXT:    ld 29, -24(1) # 8-byte Folded Reload
-; LE-64BIT-NEXT:    iseleq 3, 3, 8
-; LE-64BIT-NEXT:    std 12, 16(5)
-; LE-64BIT-NEXT:    isel 4, 4, 9, 4
-; LE-64BIT-NEXT:    std 3, 8(5)
-; LE-64BIT-NEXT:    std 4, 24(5)
+; LE-64BIT-NEXT:    addi 6, 1, -64
+; LE-64BIT-NEXT:    sradi 3, 8, 63
+; LE-64BIT-NEXT:    clrldi 4, 4, 59
+; LE-64BIT-NEXT:    std 8, 24(6)
+; LE-64BIT-NEXT:    std 7, 16(6)
+; LE-64BIT-NEXT:    std 3, 56(6)
+; LE-64BIT-NEXT:    std 3, 48(6)
+; LE-64BIT-NEXT:    li 7, 16
+; LE-64BIT-NEXT:    std 3, 40(6)
+; LE-64BIT-NEXT:    std 3, 32(6)
+; LE-64BIT-NEXT:    add 3, 6, 4
+; LE-64BIT-NEXT:    stxvd2x 0, 0, 6
+; LE-64BIT-NEXT:    lxvd2x 0, 6, 4
+; LE-64BIT-NEXT:    lxvd2x 1, 3, 7
+; LE-64BIT-NEXT:    stxvd2x 1, 5, 7
+; LE-64BIT-NEXT:    stxvd2x 0, 0, 5
 ; LE-64BIT-NEXT:    blr
 ;
 ; BE-LABEL: ashr_32bytes:
 ; BE:       # %bb.0:
-; BE-NEXT:    lwz 4, 28(4)
-; BE-NEXT:    ld 6, 16(3)
-; BE-NEXT:    ld 7, 24(3)
+; BE-NEXT:    ld 7, 0(3)
 ; BE-NEXT:    ld 8, 8(3)
-; BE-NEXT:    ld 3, 0(3)
-; BE-NEXT:    std 27, -40(1) # 8-byte Folded Spill
-; BE-NEXT:    std 29, -24(1) # 8-byte Folded Spill
-; BE-NEXT:    std 28, -32(1) # 8-byte Folded Spill
-; BE-NEXT:    std 30, -16(1) # 8-byte Folded Spill
-; BE-NEXT:    rlwinm. 4, 4, 3, 0, 28
-; BE-NEXT:    subfic 9, 4, 192
-; BE-NEXT:    addi 10, 4, -128
-; BE-NEXT:    addi 11, 4, -192
-; BE-NEXT:    subfic 0, 4, 64
-; BE-NEXT:    sld 9, 3, 9
-; BE-NEXT:    srd 27, 8, 10
-; BE-NEXT:    srd 12, 7, 4
-; BE-NEXT:    subfic 29, 4, 128
-; BE-NEXT:    cmpwi 1, 11, 1
-; BE-NEXT:    srad 11, 3, 11
-; BE-NEXT:    or 9, 27, 9
-; BE-NEXT:    sld 27, 6, 0
-; BE-NEXT:    addi 30, 4, -64
-; BE-NEXT:    srd 28, 8, 4
-; BE-NEXT:    or 12, 12, 27
-; BE-NEXT:    sld 27, 3, 0
-; BE-NEXT:    bc 12, 4, .LBB11_2
-; BE-NEXT:  # %bb.1:
-; BE-NEXT:    ori 9, 11, 0
-; BE-NEXT:    b .LBB11_2
-; BE-NEXT:  .LBB11_2:
-; BE-NEXT:    subfic 11, 29, 64
-; BE-NEXT:    or 28, 28, 27
-; BE-NEXT:    srd 27, 6, 30
-; BE-NEXT:    sld 0, 8, 0
-; BE-NEXT:    srd 11, 8, 11
-; BE-NEXT:    sld 8, 8, 29
-; BE-NEXT:    sld 29, 3, 29
-; BE-NEXT:    cmplwi 1, 4, 128
-; BE-NEXT:    or 12, 12, 27
-; BE-NEXT:    or 11, 29, 11
-; BE-NEXT:    or 8, 12, 8
-; BE-NEXT:    srd 12, 6, 4
-; BE-NEXT:    or 11, 11, 0
-; BE-NEXT:    srad 10, 3, 10
-; BE-NEXT:    srad 29, 3, 30
-; BE-NEXT:    or 11, 12, 11
-; BE-NEXT:    cmpwi 5, 30, 1
-; BE-NEXT:    bc 12, 20, .LBB11_4
-; BE-NEXT:  # %bb.3:
-; BE-NEXT:    ori 12, 29, 0
-; BE-NEXT:    b .LBB11_5
-; BE-NEXT:  .LBB11_4:
-; BE-NEXT:    addi 12, 28, 0
-; BE-NEXT:  .LBB11_5:
-; BE-NEXT:    bc 12, 4, .LBB11_7
-; BE-NEXT:  # %bb.6:
-; BE-NEXT:    ori 8, 9, 0
-; BE-NEXT:    ori 9, 10, 0
-; BE-NEXT:    b .LBB11_8
-; BE-NEXT:  .LBB11_7:
-; BE-NEXT:    addi 9, 11, 0
-; BE-NEXT:  .LBB11_8:
-; BE-NEXT:    sradi 10, 3, 63
-; BE-NEXT:    srad 3, 3, 4
-; BE-NEXT:    ld 30, -16(1) # 8-byte Folded Reload
-; BE-NEXT:    ld 29, -24(1) # 8-byte Folded Reload
-; BE-NEXT:    ld 28, -32(1) # 8-byte Folded Reload
-; BE-NEXT:    ld 27, -40(1) # 8-byte Folded Reload
-; BE-NEXT:    bc 12, 2, .LBB11_10
-; BE-NEXT:  # %bb.9:
-; BE-NEXT:    ori 4, 8, 0
-; BE-NEXT:    ori 6, 9, 0
-; BE-NEXT:    b .LBB11_11
-; BE-NEXT:  .LBB11_10:
-; BE-NEXT:    addi 4, 7, 0
-; BE-NEXT:  .LBB11_11:
-; BE-NEXT:    bc 12, 4, .LBB11_13
-; BE-NEXT:  # %bb.12:
-; BE-NEXT:    ori 7, 10, 0
-; BE-NEXT:    ori 3, 10, 0
-; BE-NEXT:    b .LBB11_14
-; BE-NEXT:  .LBB11_13:
-; BE-NEXT:    addi 7, 12, 0
-; BE-NEXT:  .LBB11_14:
+; BE-NEXT:    ld 9, 16(3)
+; BE-NEXT:    ld 3, 24(3)
+; BE-NEXT:    lwz 4, 28(4)
+; BE-NEXT:    addi 6, 1, -64
+; BE-NEXT:    std 3, 56(6)
+; BE-NEXT:    sradi 3, 7, 63
+; BE-NEXT:    clrlwi 4, 4, 27
+; BE-NEXT:    std 3, 24(6)
+; BE-NEXT:    std 3, 16(6)
+; BE-NEXT:    std 3, 8(6)
+; BE-NEXT:    std 3, -64(1)
+; BE-NEXT:    neg 3, 4
+; BE-NEXT:    std 9, 48(6)
+; BE-NEXT:    std 8, 40(6)
+; BE-NEXT:    std 7, 32(6)
+; BE-NEXT:    extsw 3, 3
+; BE-NEXT:    addi 4, 1, -32
+; BE-NEXT:    ldux 3, 4, 3
+; BE-NEXT:    ld 6, 8(4)
+; BE-NEXT:    ld 7, 24(4)
+; BE-NEXT:    ld 4, 16(4)
 ; BE-NEXT:    std 3, 0(5)
-; BE-NEXT:    std 7, 8(5)
-; BE-NEXT:    std 4, 24(5)
-; BE-NEXT:    std 6, 16(5)
+; BE-NEXT:    std 4, 16(5)
+; BE-NEXT:    std 7, 24(5)
+; BE-NEXT:    std 6, 8(5)
 ; BE-NEXT:    blr
 ;
 ; LE-32BIT-LABEL: ashr_32bytes:
 ; LE-32BIT:       # %bb.0:
-; LE-32BIT-NEXT:    stwu 1, -144(1)
-; LE-32BIT-NEXT:    mfcr 12
-; LE-32BIT-NEXT:    stw 14, 72(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 15, 76(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 16, 80(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 17, 84(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 18, 88(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 19, 92(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 20, 96(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 21, 100(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 22, 104(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 23, 108(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 24, 112(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 25, 116(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 26, 120(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 27, 124(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 28, 128(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 29, 132(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 30, 136(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    stw 31, 140(1) # 4-byte Folded Spill
+; LE-32BIT-NEXT:    stwu 1, -80(1)
+; LE-32BIT-NEXT:    lwz 7, 0(3)
+; LE-32BIT-NEXT:    addi 6, 1, 48
+; LE-32BIT-NEXT:    lwz 8, 4(3)
+; LE-32BIT-NEXT:    lwz 9, 8(3)
+; LE-32BIT-NEXT:    lwz 10, 12(3)
+; LE-32BIT-NEXT:    lwz 11, 16(3)
+; LE-32BIT-NEXT:    lwz 12, 20(3)
+; LE-32BIT-NEXT:    lwz 0, 24(3)
+; LE-32BIT-NEXT:    lwz 3, 28(3)
+; LE-32BIT-NEXT:    lwz 4, 28(4)
+; LE-32BIT-NEXT:    stw 3, 76(1)
+; LE-32BIT-NEXT:    srawi 3, 7, 31
+; LE-32BIT-NEXT:    clrlwi 4, 4, 27
+; LE-32BIT-NEXT:    stw 0, 72(1)
 ; LE-32BIT-NEXT:    stw 12, 68(1)
-; LE-32BIT-NEXT:    lwz 0, 28(4)
-; LE-32BIT-NEXT:    lwz 29, 4(3)
-; LE-32BIT-NEXT:    lwz 12, 0(3)
-; LE-32BIT-NEXT:    rlwinm. 30, 0, 3, 0, 28
-; LE-32BIT-NEXT:    stw 5, 64(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    subfic 23, 30, 224
-; LE-32BIT-NEXT:    lwz 5, 24(3)
-; LE-32BIT-NEXT:    addi 21, 30, -224
-; LE-32BIT-NEXT:    lwz 8, 28(3)
-; LE-32BIT-NEXT:    subfic 4, 30, 160
-; LE-32BIT-NEXT:    lwz 10, 20(3)
-; LE-32BIT-NEXT:    addi 11, 30, -128
-; LE-32BIT-NEXT:    lwz 9, 16(3)
-; LE-32BIT-NEXT:    subfic 25, 30, 96
-; LE-32BIT-NEXT:    lwz 26, 12(3)
-; LE-32BIT-NEXT:    addi 0, 30, -64
-; LE-32BIT-NEXT:    lwz 7, 8(3)
-; LE-32BIT-NEXT:    addi 3, 30, -192
-; LE-32BIT-NEXT:    subfic 27, 30, 32
-; LE-32BIT-NEXT:    slw 23, 12, 23
-; LE-32BIT-NEXT:    srw 16, 29, 3
-; LE-32BIT-NEXT:    stw 3, 56(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 20, 8, 30
-; LE-32BIT-NEXT:    sraw 15, 12, 21
-; LE-32BIT-NEXT:    cmpwi 1, 21, 1
-; LE-32BIT-NEXT:    slw 21, 7, 4
-; LE-32BIT-NEXT:    srw 14, 26, 11
-; LE-32BIT-NEXT:    slw 31, 9, 25
-; LE-32BIT-NEXT:    srw 3, 10, 0
-; LE-32BIT-NEXT:    or 23, 16, 23
-; LE-32BIT-NEXT:    slw 16, 5, 27
-; LE-32BIT-NEXT:    srw 19, 10, 30
-; LE-32BIT-NEXT:    or 21, 14, 21
-; LE-32BIT-NEXT:    slw 14, 9, 27
-; LE-32BIT-NEXT:    or 3, 3, 31
-; LE-32BIT-NEXT:    slw 31, 12, 4
-; LE-32BIT-NEXT:    or 20, 20, 16
-; LE-32BIT-NEXT:    srw 16, 29, 11
-; LE-32BIT-NEXT:    or 19, 19, 14
-; LE-32BIT-NEXT:    slw 14, 12, 25
-; LE-32BIT-NEXT:    or 16, 16, 31
-; LE-32BIT-NEXT:    srw 31, 29, 0
-; LE-32BIT-NEXT:    addi 24, 30, -160
-; LE-32BIT-NEXT:    srw 18, 26, 30
-; LE-32BIT-NEXT:    or 14, 31, 14
-; LE-32BIT-NEXT:    slw 31, 7, 27
-; LE-32BIT-NEXT:    addi 28, 30, -96
-; LE-32BIT-NEXT:    srw 17, 29, 30
-; LE-32BIT-NEXT:    stw 4, 32(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 18, 18, 31
-; LE-32BIT-NEXT:    slw 31, 12, 27
-; LE-32BIT-NEXT:    bc 12, 4, .LBB11_2
-; LE-32BIT-NEXT:  # %bb.1:
-; LE-32BIT-NEXT:    ori 4, 15, 0
-; LE-32BIT-NEXT:    b .LBB11_3
-; LE-32BIT-NEXT:  .LBB11_2:
-; LE-32BIT-NEXT:    addi 4, 23, 0
-; LE-32BIT-NEXT:  .LBB11_3:
-; LE-32BIT-NEXT:    srw 15, 7, 24
-; LE-32BIT-NEXT:    or 17, 17, 31
-; LE-32BIT-NEXT:    addi 31, 30, -32
-; LE-32BIT-NEXT:    or 21, 21, 15
-; LE-32BIT-NEXT:    srw 15, 9, 28
-; LE-32BIT-NEXT:    or 3, 3, 15
-; LE-32BIT-NEXT:    srw 15, 5, 31
-; LE-32BIT-NEXT:    or 20, 20, 15
-; LE-32BIT-NEXT:    srw 15, 9, 31
-; LE-32BIT-NEXT:    stw 3, 28(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    or 3, 19, 15
-; LE-32BIT-NEXT:    subfic 15, 30, 64
-; LE-32BIT-NEXT:    stw 4, 24(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    cmpwi 1, 24, 1
-; LE-32BIT-NEXT:    sraw 24, 12, 24
-; LE-32BIT-NEXT:    subfic 4, 15, 32
-; LE-32BIT-NEXT:    stw 0, 52(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 0, 26, 4
-; LE-32BIT-NEXT:    stw 3, 48(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    bc 12, 4, .LBB11_5
-; LE-32BIT-NEXT:  # %bb.4:
-; LE-32BIT-NEXT:    ori 3, 24, 0
-; LE-32BIT-NEXT:    b .LBB11_6
-; LE-32BIT-NEXT:  .LBB11_5:
-; LE-32BIT-NEXT:    addi 3, 16, 0
-; LE-32BIT-NEXT:  .LBB11_6:
-; LE-32BIT-NEXT:    slw 16, 7, 15
-; LE-32BIT-NEXT:    or 0, 16, 0
-; LE-32BIT-NEXT:    subfic 16, 30, 128
-; LE-32BIT-NEXT:    stw 5, 36(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    subfic 5, 16, 32
-; LE-32BIT-NEXT:    stw 3, 44(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    slw 3, 12, 16
-; LE-32BIT-NEXT:    srw 22, 29, 5
-; LE-32BIT-NEXT:    stw 8, 60(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    mr 8, 10
-; LE-32BIT-NEXT:    mr 10, 27
-; LE-32BIT-NEXT:    or 23, 3, 22
-; LE-32BIT-NEXT:    slw 22, 7, 16
-; LE-32BIT-NEXT:    srw 27, 26, 5
-; LE-32BIT-NEXT:    stw 11, 40(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    mr 6, 26
-; LE-32BIT-NEXT:    or 11, 22, 27
-; LE-32BIT-NEXT:    slw 22, 9, 15
-; LE-32BIT-NEXT:    srw 26, 8, 4
-; LE-32BIT-NEXT:    subfic 3, 30, 192
-; LE-32BIT-NEXT:    or 26, 22, 26
-; LE-32BIT-NEXT:    cmpwi 1, 28, 1
-; LE-32BIT-NEXT:    sraw 22, 12, 28
-; LE-32BIT-NEXT:    subfic 19, 3, 32
-; LE-32BIT-NEXT:    srw 4, 29, 4
-; LE-32BIT-NEXT:    slw 28, 12, 15
-; LE-32BIT-NEXT:    stw 9, 20(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    srw 19, 29, 19
-; LE-32BIT-NEXT:    slw 24, 12, 3
-; LE-32BIT-NEXT:    or 9, 28, 4
-; LE-32BIT-NEXT:    lwz 4, 64(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 24, 24, 19
-; LE-32BIT-NEXT:    bc 12, 4, .LBB11_7
-; LE-32BIT-NEXT:    b .LBB11_8
-; LE-32BIT-NEXT:  .LBB11_7:
-; LE-32BIT-NEXT:    addi 22, 14, 0
-; LE-32BIT-NEXT:  .LBB11_8:
-; LE-32BIT-NEXT:    srw 19, 7, 31
-; LE-32BIT-NEXT:    cmplwi 5, 30, 64
-; LE-32BIT-NEXT:    cmplwi 1, 30, 128
-; LE-32BIT-NEXT:    slw 3, 29, 3
-; LE-32BIT-NEXT:    or 19, 18, 19
-; LE-32BIT-NEXT:    cmpwi 6, 31, 1
-; LE-32BIT-NEXT:    sraw 18, 12, 31
-; LE-32BIT-NEXT:    crand 21, 4, 20
-; LE-32BIT-NEXT:    srawi 14, 12, 31
-; LE-32BIT-NEXT:    sraw 31, 12, 30
-; LE-32BIT-NEXT:    or 3, 21, 3
-; LE-32BIT-NEXT:    slw 21, 8, 15
-; LE-32BIT-NEXT:    bc 12, 24, .LBB11_10
-; LE-32BIT-NEXT:  # %bb.9:
-; LE-32BIT-NEXT:    ori 28, 18, 0
-; LE-32BIT-NEXT:    b .LBB11_11
-; LE-32BIT-NEXT:  .LBB11_10:
-; LE-32BIT-NEXT:    addi 28, 17, 0
-; LE-32BIT-NEXT:  .LBB11_11:
-; LE-32BIT-NEXT:    bc 12, 21, .LBB11_13
-; LE-32BIT-NEXT:  # %bb.12:
-; LE-32BIT-NEXT:    ori 18, 14, 0
-; LE-32BIT-NEXT:    b .LBB11_14
-; LE-32BIT-NEXT:  .LBB11_13:
-; LE-32BIT-NEXT:    addi 18, 31, 0
-; LE-32BIT-NEXT:  .LBB11_14:
-; LE-32BIT-NEXT:    or 21, 20, 21
-; LE-32BIT-NEXT:    subfic 20, 16, 64
-; LE-32BIT-NEXT:    stw 18, 0(4)
-; LE-32BIT-NEXT:    subfic 18, 20, 32
-; LE-32BIT-NEXT:    slw 18, 7, 18
-; LE-32BIT-NEXT:    srw 17, 6, 20
-; LE-32BIT-NEXT:    or 18, 17, 18
-; LE-32BIT-NEXT:    slw 17, 6, 10
-; LE-32BIT-NEXT:    or 27, 0, 17
-; LE-32BIT-NEXT:    slw 0, 29, 25
-; LE-32BIT-NEXT:    mr 31, 8
-; LE-32BIT-NEXT:    or 8, 23, 0
-; LE-32BIT-NEXT:    slw 0, 6, 25
-; LE-32BIT-NEXT:    or 11, 11, 0
-; LE-32BIT-NEXT:    stw 11, 16(1) # 4-byte Folded Spill
-; LE-32BIT-NEXT:    slw 0, 31, 10
-; LE-32BIT-NEXT:    lwz 11, 32(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 0, 26, 0
-; LE-32BIT-NEXT:    slw 25, 29, 10
-; LE-32BIT-NEXT:    or 23, 9, 25
-; LE-32BIT-NEXT:    slw 26, 29, 11
-; LE-32BIT-NEXT:    or 26, 24, 26
-; LE-32BIT-NEXT:    slw 24, 29, 15
-; LE-32BIT-NEXT:    or 24, 19, 24
-; LE-32BIT-NEXT:    lwz 19, 40(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    srw 25, 7, 20
-; LE-32BIT-NEXT:    lwz 9, 24(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 8, 8, 25
-; LE-32BIT-NEXT:    cmplwi 6, 19, 64
-; LE-32BIT-NEXT:    srw 5, 7, 5
-; LE-32BIT-NEXT:    bc 12, 24, .LBB11_16
-; LE-32BIT-NEXT:  # %bb.15:
-; LE-32BIT-NEXT:    ori 3, 9, 0
-; LE-32BIT-NEXT:    b .LBB11_16
-; LE-32BIT-NEXT:  .LBB11_16:
-; LE-32BIT-NEXT:    lwz 9, 28(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 5, 18, 5
-; LE-32BIT-NEXT:    lwz 17, 20(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    mr 18, 4
-; LE-32BIT-NEXT:    bc 12, 20, .LBB11_18
-; LE-32BIT-NEXT:  # %bb.17:
-; LE-32BIT-NEXT:    ori 10, 9, 0
-; LE-32BIT-NEXT:    b .LBB11_19
-; LE-32BIT-NEXT:  .LBB11_18:
-; LE-32BIT-NEXT:    addi 10, 21, 0
-; LE-32BIT-NEXT:  .LBB11_19:
-; LE-32BIT-NEXT:    lwz 9, 36(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    bc 12, 20, .LBB11_21
-; LE-32BIT-NEXT:  # %bb.20:
-; LE-32BIT-NEXT:    ori 24, 22, 0
-; LE-32BIT-NEXT:    b .LBB11_21
-; LE-32BIT-NEXT:  .LBB11_21:
-; LE-32BIT-NEXT:    cmplwi 7, 19, 0
-; LE-32BIT-NEXT:    cmplwi 2, 16, 64
-; LE-32BIT-NEXT:    bc 12, 30, .LBB11_22
-; LE-32BIT-NEXT:    b .LBB11_23
-; LE-32BIT-NEXT:  .LBB11_22:
-; LE-32BIT-NEXT:    addi 3, 6, 0
-; LE-32BIT-NEXT:  .LBB11_23:
-; LE-32BIT-NEXT:    cmplwi 3, 16, 0
-; LE-32BIT-NEXT:    srw 25, 9, 30
-; LE-32BIT-NEXT:    or 25, 25, 0
-; LE-32BIT-NEXT:    srw 0, 7, 19
-; LE-32BIT-NEXT:    or 26, 0, 26
-; LE-32BIT-NEXT:    srw 0, 7, 30
-; LE-32BIT-NEXT:    or 11, 0, 23
-; LE-32BIT-NEXT:    bc 12, 21, .LBB11_25
-; LE-32BIT-NEXT:  # %bb.24:
-; LE-32BIT-NEXT:    ori 0, 14, 0
-; LE-32BIT-NEXT:    b .LBB11_26
-; LE-32BIT-NEXT:  .LBB11_25:
-; LE-32BIT-NEXT:    addi 0, 28, 0
-; LE-32BIT-NEXT:  .LBB11_26:
-; LE-32BIT-NEXT:    slw 28, 6, 16
-; LE-32BIT-NEXT:    stw 0, 4(4)
-; LE-32BIT-NEXT:    slw 0, 29, 16
-; LE-32BIT-NEXT:    lwz 4, 52(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    or 5, 0, 5
-; LE-32BIT-NEXT:    lwz 0, 56(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    slw 23, 6, 15
-; LE-32BIT-NEXT:    srw 22, 17, 4
-; LE-32BIT-NEXT:    li 15, 0
-; LE-32BIT-NEXT:    sraw 21, 12, 0
-; LE-32BIT-NEXT:    bc 12, 8, .LBB11_28
-; LE-32BIT-NEXT:  # %bb.27:
-; LE-32BIT-NEXT:    ori 0, 15, 0
-; LE-32BIT-NEXT:    b .LBB11_29
-; LE-32BIT-NEXT:  .LBB11_28:
-; LE-32BIT-NEXT:    addi 0, 28, 0
-; LE-32BIT-NEXT:  .LBB11_29:
-; LE-32BIT-NEXT:    bc 12, 20, .LBB11_31
-; LE-32BIT-NEXT:  # %bb.30:
-; LE-32BIT-NEXT:    ori 28, 22, 0
-; LE-32BIT-NEXT:    b .LBB11_32
-; LE-32BIT-NEXT:  .LBB11_31:
-; LE-32BIT-NEXT:    addi 28, 25, 0
-; LE-32BIT-NEXT:  .LBB11_32:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB11_34
-; LE-32BIT-NEXT:  # %bb.33:
-; LE-32BIT-NEXT:    ori 22, 24, 0
-; LE-32BIT-NEXT:    b .LBB11_35
-; LE-32BIT-NEXT:  .LBB11_34:
-; LE-32BIT-NEXT:    addi 22, 6, 0
-; LE-32BIT-NEXT:  .LBB11_35:
-; LE-32BIT-NEXT:    lwz 6, 48(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    sraw 20, 12, 4
-; LE-32BIT-NEXT:    lwz 16, 60(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    bc 12, 20, .LBB11_37
-; LE-32BIT-NEXT:  # %bb.36:
-; LE-32BIT-NEXT:    ori 4, 20, 0
-; LE-32BIT-NEXT:    b .LBB11_38
-; LE-32BIT-NEXT:  .LBB11_37:
-; LE-32BIT-NEXT:    addi 4, 11, 0
-; LE-32BIT-NEXT:  .LBB11_38:
-; LE-32BIT-NEXT:    srw 30, 17, 30
-; LE-32BIT-NEXT:    bc 12, 20, .LBB11_40
-; LE-32BIT-NEXT:  # %bb.39:
-; LE-32BIT-NEXT:    ori 25, 15, 0
-; LE-32BIT-NEXT:    b .LBB11_41
-; LE-32BIT-NEXT:  .LBB11_40:
-; LE-32BIT-NEXT:    addi 25, 6, 0
-; LE-32BIT-NEXT:  .LBB11_41:
-; LE-32BIT-NEXT:    lwz 6, 44(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    bc 12, 8, .LBB11_43
-; LE-32BIT-NEXT:  # %bb.42:
-; LE-32BIT-NEXT:    ori 8, 27, 0
-; LE-32BIT-NEXT:    ori 5, 23, 0
-; LE-32BIT-NEXT:    b .LBB11_43
-; LE-32BIT-NEXT:  .LBB11_43:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB11_44
-; LE-32BIT-NEXT:    b .LBB11_45
-; LE-32BIT-NEXT:  .LBB11_44:
-; LE-32BIT-NEXT:    addi 4, 7, 0
-; LE-32BIT-NEXT:  .LBB11_45:
-; LE-32BIT-NEXT:    sraw 19, 12, 19
-; LE-32BIT-NEXT:    bc 12, 2, .LBB11_46
-; LE-32BIT-NEXT:    b .LBB11_47
-; LE-32BIT-NEXT:  .LBB11_46:
-; LE-32BIT-NEXT:    addi 10, 16, 0
-; LE-32BIT-NEXT:  .LBB11_47:
-; LE-32BIT-NEXT:    bc 12, 24, .LBB11_49
-; LE-32BIT-NEXT:  # %bb.48:
-; LE-32BIT-NEXT:    ori 26, 21, 0
-; LE-32BIT-NEXT:    b .LBB11_49
-; LE-32BIT-NEXT:  .LBB11_49:
-; LE-32BIT-NEXT:    bc 12, 14, .LBB11_50
-; LE-32BIT-NEXT:    b .LBB11_51
-; LE-32BIT-NEXT:  .LBB11_50:
-; LE-32BIT-NEXT:    addi 5, 29, 0
-; LE-32BIT-NEXT:  .LBB11_51:
-; LE-32BIT-NEXT:    bc 12, 4, .LBB11_53
-; LE-32BIT-NEXT:  # %bb.52:
-; LE-32BIT-NEXT:    ori 4, 14, 0
-; LE-32BIT-NEXT:    b .LBB11_53
-; LE-32BIT-NEXT:  .LBB11_53:
-; LE-32BIT-NEXT:    or 10, 10, 0
-; LE-32BIT-NEXT:    bc 12, 24, .LBB11_55
-; LE-32BIT-NEXT:  # %bb.54:
-; LE-32BIT-NEXT:    ori 24, 14, 0
-; LE-32BIT-NEXT:    b .LBB11_56
-; LE-32BIT-NEXT:  .LBB11_55:
-; LE-32BIT-NEXT:    addi 24, 6, 0
-; LE-32BIT-NEXT:  .LBB11_56:
-; LE-32BIT-NEXT:    lwz 6, 16(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    bc 12, 4, .LBB11_57
-; LE-32BIT-NEXT:    b .LBB11_58
-; LE-32BIT-NEXT:  .LBB11_57:
-; LE-32BIT-NEXT:    addi 3, 10, 0
-; LE-32BIT-NEXT:  .LBB11_58:
-; LE-32BIT-NEXT:    bc 12, 20, .LBB11_60
-; LE-32BIT-NEXT:  # %bb.59:
-; LE-32BIT-NEXT:    ori 0, 15, 0
-; LE-32BIT-NEXT:    b .LBB11_61
-; LE-32BIT-NEXT:  .LBB11_60:
-; LE-32BIT-NEXT:    addi 0, 30, 0
-; LE-32BIT-NEXT:  .LBB11_61:
-; LE-32BIT-NEXT:    bc 12, 24, .LBB11_63
-; LE-32BIT-NEXT:  # %bb.62:
-; LE-32BIT-NEXT:    ori 30, 14, 0
-; LE-32BIT-NEXT:    b .LBB11_64
-; LE-32BIT-NEXT:  .LBB11_63:
-; LE-32BIT-NEXT:    addi 30, 19, 0
-; LE-32BIT-NEXT:  .LBB11_64:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB11_65
-; LE-32BIT-NEXT:    b .LBB11_66
-; LE-32BIT-NEXT:  .LBB11_65:
-; LE-32BIT-NEXT:    addi 3, 16, 0
-; LE-32BIT-NEXT:  .LBB11_66:
-; LE-32BIT-NEXT:    stw 4, 8(18)
-; LE-32BIT-NEXT:    bc 12, 8, .LBB11_68
-; LE-32BIT-NEXT:  # %bb.67:
-; LE-32BIT-NEXT:    ori 27, 15, 0
-; LE-32BIT-NEXT:    b .LBB11_69
-; LE-32BIT-NEXT:  .LBB11_68:
-; LE-32BIT-NEXT:    addi 27, 6, 0
-; LE-32BIT-NEXT:  .LBB11_69:
-; LE-32BIT-NEXT:    bc 12, 14, .LBB11_71
-; LE-32BIT-NEXT:  # %bb.70:
-; LE-32BIT-NEXT:    ori 6, 8, 0
-; LE-32BIT-NEXT:    b .LBB11_72
-; LE-32BIT-NEXT:  .LBB11_71:
-; LE-32BIT-NEXT:    addi 6, 12, 0
-; LE-32BIT-NEXT:  .LBB11_72:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB11_74
-; LE-32BIT-NEXT:  # %bb.73:
-; LE-32BIT-NEXT:    ori 8, 28, 0
-; LE-32BIT-NEXT:    b .LBB11_75
-; LE-32BIT-NEXT:  .LBB11_74:
-; LE-32BIT-NEXT:    addi 8, 9, 0
-; LE-32BIT-NEXT:  .LBB11_75:
-; LE-32BIT-NEXT:    bc 12, 30, .LBB11_77
-; LE-32BIT-NEXT:  # %bb.76:
-; LE-32BIT-NEXT:    ori 28, 26, 0
-; LE-32BIT-NEXT:    b .LBB11_78
-; LE-32BIT-NEXT:  .LBB11_77:
-; LE-32BIT-NEXT:    addi 28, 7, 0
-; LE-32BIT-NEXT:  .LBB11_78:
-; LE-32BIT-NEXT:    stw 3, 28(18)
-; LE-32BIT-NEXT:    or 7, 8, 27
-; LE-32BIT-NEXT:    or 4, 0, 6
-; LE-32BIT-NEXT:    or 3, 25, 5
-; LE-32BIT-NEXT:    bc 12, 4, .LBB11_80
-; LE-32BIT-NEXT:  # %bb.79:
-; LE-32BIT-NEXT:    ori 6, 28, 0
-; LE-32BIT-NEXT:    ori 4, 30, 0
-; LE-32BIT-NEXT:    ori 3, 24, 0
-; LE-32BIT-NEXT:    ori 12, 14, 0
-; LE-32BIT-NEXT:    b .LBB11_81
-; LE-32BIT-NEXT:  .LBB11_80:
-; LE-32BIT-NEXT:    addi 6, 7, 0
-; LE-32BIT-NEXT:    addi 12, 22, 0
-; LE-32BIT-NEXT:  .LBB11_81:
-; LE-32BIT-NEXT:    bc 12, 2, .LBB11_83
-; LE-32BIT-NEXT:  # %bb.82:
-; LE-32BIT-NEXT:    ori 5, 6, 0
-; LE-32BIT-NEXT:    b .LBB11_84
-; LE-32BIT-NEXT:  .LBB11_83:
-; LE-32BIT-NEXT:    addi 5, 9, 0
-; LE-32BIT-NEXT:    addi 4, 17, 0
-; LE-32BIT-NEXT:    addi 3, 31, 0
-; LE-32BIT-NEXT:  .LBB11_84:
-; LE-32BIT-NEXT:    stw 12, 12(18)
-; LE-32BIT-NEXT:    stw 5, 24(18)
-; LE-32BIT-NEXT:    stw 4, 16(18)
-; LE-32BIT-NEXT:    stw 3, 20(18)
-; LE-32BIT-NEXT:    lwz 12, 68(1)
-; LE-32BIT-NEXT:    lwz 31, 140(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    mtcrf 32, 12 # cr2
-; LE-32BIT-NEXT:    mtcrf 16, 12 # cr3
-; LE-32BIT-NEXT:    lwz 30, 136(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 29, 132(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 28, 128(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 27, 124(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 26, 120(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 25, 116(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 24, 112(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 23, 108(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 22, 104(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 21, 100(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 20, 96(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 19, 92(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 18, 88(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 17, 84(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 16, 80(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 15, 76(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    lwz 14, 72(1) # 4-byte Folded Reload
-; LE-32BIT-NEXT:    addi 1, 1, 144
+; LE-32BIT-NEXT:    stw 11, 64(1)
+; LE-32BIT-NEXT:    stw 10, 60(1)
+; LE-32BIT-NEXT:    stw 9, 56(1)
+; LE-32BIT-NEXT:    stw 8, 52(1)
+; LE-32BIT-NEXT:    stw 7, 48(1)
+; LE-32BIT-NEXT:    stw 3, 44(1)
+; LE-32BIT-NEXT:    stw 3, 40(1)
+; LE-32BIT-NEXT:    stw 3, 36(1)
+; LE-32BIT-NEXT:    stw 3, 32(1)
+; LE-32BIT-NEXT:    stw 3, 28(1)
+; LE-32BIT-NEXT:    stw 3, 24(1)
+; LE-32BIT-NEXT:    stw 3, 20(1)
+; LE-32BIT-NEXT:    stw 3, 16(1)
+; LE-32BIT-NEXT:    sub 3, 6, 4
+; LE-32BIT-NEXT:    lwz 4, 4(3)
+; LE-32BIT-NEXT:    lwz 6, 0(3)
+; LE-32BIT-NEXT:    lwz 7, 12(3)
+; LE-32BIT-NEXT:    lwz 8, 8(3)
+; LE-32BIT-NEXT:    lwz 9, 20(3)
+; LE-32BIT-NEXT:    lwz 10, 16(3)
+; LE-32BIT-NEXT:    lwz 11, 24(3)
+; LE-32BIT-NEXT:    lwz 3, 28(3)
+; LE-32BIT-NEXT:    stw 11, 24(5)
+; LE-32BIT-NEXT:    stw 3, 28(5)
+; LE-32BIT-NEXT:    stw 10, 16(5)
+; LE-32BIT-NEXT:    stw 9, 20(5)
+; LE-32BIT-NEXT:    stw 8, 8(5)
+; LE-32BIT-NEXT:    stw 7, 12(5)
+; LE-32BIT-NEXT:    stw 6, 0(5)
+; LE-32BIT-NEXT:    stw 4, 4(5)
+; LE-32BIT-NEXT:    addi 1, 1, 80
 ; LE-32BIT-NEXT:    blr
   %src = load i256, ptr %src.ptr, align 1
   %byteOff = load i256, ptr %byteOff.ptr, align 1
   %bitOff = shl i256 %byteOff, 3
   %res = ashr i256 %src, %bitOff
   store i256 %res, ptr %dst, align 1
   ret void
 }
 ;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
 ; LE: {{.*}}

llvm/test/CodeGen/RISCV/shifts.ll

 ; RV64I-NEXT:    ret
   %1 = shl i64 %a, %b
   ret i64 %1
 }
 
 define i128 @lshr128(i128 %a, i128 %b) nounwind {
 ; RV32I-LABEL: lshr128:
 ; RV32I:       # %bb.0:
-; RV32I-NEXT:    addi sp, sp, -16
-; RV32I-NEXT:    sw s0, 12(sp) # 4-byte Folded Spill
+; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    lw a2, 0(a2)
-; RV32I-NEXT:    lw t5, 4(a1)
-; RV32I-NEXT:    lw t3, 8(a1)
-; RV32I-NEXT:    lw t2, 12(a1)
-; RV32I-NEXT:    neg a3, a2
-; RV32I-NEXT:    li t0, 64
-; RV32I-NEXT:    li a4, 32
-; RV32I-NEXT:    sub a7, a4, a2
-; RV32I-NEXT:    sll a5, t3, a3
-; RV32I-NEXT:    bltz a7, .LBB6_2
-; RV32I-NEXT:  # %bb.1:
-; RV32I-NEXT:    mv t1, a5
-; RV32I-NEXT:    j .LBB6_3
-; RV32I-NEXT:  .LBB6_2:
-; RV32I-NEXT:    sll a3, t2, a3
-; RV32I-NEXT:    sub a4, t0, a2
-; RV32I-NEXT:    not a4, a4
-; RV32I-NEXT:    srli a6, t3, 1
-; RV32I-NEXT:    srl a4, a6, a4
-; RV32I-NEXT:    or t1, a3, a4
-; RV32I-NEXT:  .LBB6_3:
-; RV32I-NEXT:    srl t4, t5, a2
-; RV32I-NEXT:    addi a6, a2, -32
-; RV32I-NEXT:    slti a3, a6, 0
-; RV32I-NEXT:    neg a3, a3
-; RV32I-NEXT:    srl a4, t2, a2
-; RV32I-NEXT:    addi t6, a2, -96
-; RV32I-NEXT:    bltu a2, t0, .LBB6_5
-; RV32I-NEXT:  # %bb.4:
-; RV32I-NEXT:    slti t1, t6, 0
-; RV32I-NEXT:    neg s0, t1
-; RV32I-NEXT:    and s0, s0, a4
-; RV32I-NEXT:    mv t1, t5
-; RV32I-NEXT:    bnez a2, .LBB6_6
-; RV32I-NEXT:    j .LBB6_7
-; RV32I-NEXT:  .LBB6_5:
-; RV32I-NEXT:    and s0, a3, t4
-; RV32I-NEXT:    or s0, s0, t1
-; RV32I-NEXT:    mv t1, t5
-; RV32I-NEXT:    beqz a2, .LBB6_7
-; RV32I-NEXT:  .LBB6_6:
-; RV32I-NEXT:    mv t1, s0
-; RV32I-NEXT:  .LBB6_7:
-; RV32I-NEXT:    lw a1, 0(a1)
-; RV32I-NEXT:    not s0, a2
-; RV32I-NEXT:    bgez a6, .LBB6_9
-; RV32I-NEXT:  # %bb.8:
-; RV32I-NEXT:    srl t4, a1, a2
-; RV32I-NEXT:    slli t5, t5, 1
-; RV32I-NEXT:    sll t5, t5, s0
-; RV32I-NEXT:    or t4, t4, t5
-; RV32I-NEXT:  .LBB6_9:
-; RV32I-NEXT:    srl t3, t3, a2
-; RV32I-NEXT:    slli t2, t2, 1
-; RV32I-NEXT:    sll t2, t2, s0
-; RV32I-NEXT:    or t2, t3, t2
-; RV32I-NEXT:    mv t3, t2
-; RV32I-NEXT:    bgez t6, .LBB6_15
-; RV32I-NEXT:  # %bb.10:
-; RV32I-NEXT:    bltu a2, t0, .LBB6_16
-; RV32I-NEXT:  .LBB6_11:
-; RV32I-NEXT:    bnez a2, .LBB6_17
-; RV32I-NEXT:  .LBB6_12:
-; RV32I-NEXT:    bltz a6, .LBB6_14
-; RV32I-NEXT:  .LBB6_13:
-; RV32I-NEXT:    mv t2, a4
-; RV32I-NEXT:  .LBB6_14:
-; RV32I-NEXT:    sltiu a2, a2, 64
-; RV32I-NEXT:    neg a2, a2
-; RV32I-NEXT:    and a5, a2, t2
-; RV32I-NEXT:    and a3, a3, a4
-; RV32I-NEXT:    and a2, a2, a3
-; RV32I-NEXT:    sw a2, 12(a0)
+; RV32I-NEXT:    lw a3, 0(a1)
+; RV32I-NEXT:    lw a4, 4(a1)
+; RV32I-NEXT:    lw a5, 8(a1)
+; RV32I-NEXT:    lw a1, 12(a1)
+; RV32I-NEXT:    sb zero, 31(sp)
+; RV32I-NEXT:    sb zero, 30(sp)
+; RV32I-NEXT:    sb zero, 29(sp)
+; RV32I-NEXT:    sb zero, 28(sp)
+; RV32I-NEXT:    sb zero, 27(sp)
+; RV32I-NEXT:    sb zero, 26(sp)
+; RV32I-NEXT:    sb zero, 25(sp)
+; RV32I-NEXT:    sb zero, 24(sp)
+; RV32I-NEXT:    sb zero, 23(sp)
+; RV32I-NEXT:    sb zero, 22(sp)
+; RV32I-NEXT:    sb zero, 21(sp)
+; RV32I-NEXT:    sb zero, 20(sp)
+; RV32I-NEXT:    sb zero, 19(sp)
+; RV32I-NEXT:    sb zero, 18(sp)
+; RV32I-NEXT:    sb zero, 17(sp)
+; RV32I-NEXT:    sb zero, 16(sp)
+; RV32I-NEXT:    sb a1, 12(sp)
+; RV32I-NEXT:    sb a5, 8(sp)
+; RV32I-NEXT:    sb a4, 4(sp)
+; RV32I-NEXT:    sb a3, 0(sp)
+; RV32I-NEXT:    srli a6, a1, 24
+; RV32I-NEXT:    sb a6, 15(sp)
+; RV32I-NEXT:    srli a6, a1, 16
+; RV32I-NEXT:    sb a6, 14(sp)
+; RV32I-NEXT:    srli a1, a1, 8
+; RV32I-NEXT:    sb a1, 13(sp)
+; RV32I-NEXT:    srli a1, a5, 24
+; RV32I-NEXT:    sb a1, 11(sp)
+; RV32I-NEXT:    srli a1, a5, 16
+; RV32I-NEXT:    sb a1, 10(sp)
+; RV32I-NEXT:    srli a5, a5, 8
+; RV32I-NEXT:    sb a5, 9(sp)
+; RV32I-NEXT:    srli a1, a4, 24
+; RV32I-NEXT:    sb a1, 7(sp)
+; RV32I-NEXT:    srli a1, a4, 16
+; RV32I-NEXT:    sb a1, 6(sp)
+; RV32I-NEXT:    srli a4, a4, 8
+; RV32I-NEXT:    sb a4, 5(sp)
+; RV32I-NEXT:    srli a1, a3, 24
+; RV32I-NEXT:    sb a1, 3(sp)
+; RV32I-NEXT:    srli a1, a3, 16
+; RV32I-NEXT:    sb a1, 2(sp)
+; RV32I-NEXT:    srli a3, a3, 8
+; RV32I-NEXT:    sb a3, 1(sp)
+; RV32I-NEXT:    slli a1, a2, 25
+; RV32I-NEXT:    srli a1, a1, 28
+; RV32I-NEXT:    mv a3, sp
+; RV32I-NEXT:    add a1, a3, a1
+; RV32I-NEXT:    lbu a3, 1(a1)
+; RV32I-NEXT:    lbu a4, 0(a1)
+; RV32I-NEXT:    lbu a5, 2(a1)
+; RV32I-NEXT:    lbu a6, 3(a1)
+; RV32I-NEXT:    slli a3, a3, 8
+; RV32I-NEXT:    or a3, a3, a4
+; RV32I-NEXT:    slli a5, a5, 16
+; RV32I-NEXT:    slli a6, a6, 24
+; RV32I-NEXT:    or a3, a5, a3
+; RV32I-NEXT:    or a3, a6, a3
+; RV32I-NEXT:    andi a2, a2, 7
+; RV32I-NEXT:    srl a3, a3, a2
+; RV32I-NEXT:    lbu a4, 5(a1)
+; RV32I-NEXT:    lbu a5, 4(a1)
+; RV32I-NEXT:    lbu a6, 6(a1)
+; RV32I-NEXT:    lbu a7, 7(a1)
+; RV32I-NEXT:    slli a4, a4, 8
+; RV32I-NEXT:    or a4, a4, a5
+; RV32I-NEXT:    slli a6, a6, 16
+; RV32I-NEXT:    slli a7, a7, 24
+; RV32I-NEXT:    or a4, a6, a4
+; RV32I-NEXT:    or a4, a7, a4
+; RV32I-NEXT:    slli a5, a4, 1
+; RV32I-NEXT:    xori a6, a2, 31
+; RV32I-NEXT:    sll a5, a5, a6
+; RV32I-NEXT:    or a3, a3, a5
+; RV32I-NEXT:    srl a4, a4, a2
+; RV32I-NEXT:    lbu a5, 9(a1)
+; RV32I-NEXT:    lbu a7, 8(a1)
+; RV32I-NEXT:    lbu t0, 10(a1)
+; RV32I-NEXT:    lbu t1, 11(a1)
+; RV32I-NEXT:    slli a5, a5, 8
+; RV32I-NEXT:    or a5, a5, a7
+; RV32I-NEXT:    slli t0, t0, 16
+; RV32I-NEXT:    slli t1, t1, 24
+; RV32I-NEXT:    or a5, t0, a5
+; RV32I-NEXT:    or a5, t1, a5
+; RV32I-NEXT:    slli a7, a5, 1
+; RV32I-NEXT:    not t0, a2
+; RV32I-NEXT:    lbu t1, 13(a1)
+; RV32I-NEXT:    sll a7, a7, t0
+; RV32I-NEXT:    or a4, a4, a7
+; RV32I-NEXT:    lbu a7, 12(a1)
+; RV32I-NEXT:    slli t1, t1, 8
+; RV32I-NEXT:    lbu t0, 14(a1)
+; RV32I-NEXT:    lbu a1, 15(a1)
+; RV32I-NEXT:    or a7, t1, a7
+; RV32I-NEXT:    srl a5, a5, a2
+; RV32I-NEXT:    slli t0, t0, 16
+; RV32I-NEXT:    slli a1, a1, 24
+; RV32I-NEXT:    or a7, t0, a7
+; RV32I-NEXT:    or a1, a1, a7
+; RV32I-NEXT:    slli a7, a1, 1
+; RV32I-NEXT:    sll a6, a7, a6
+; RV32I-NEXT:    or a5, a5, a6
+; RV32I-NEXT:    srl a1, a1, a2
+; RV32I-NEXT:    sw a1, 12(a0)
 ; RV32I-NEXT:    sw a5, 8(a0)
-; RV32I-NEXT:    sw a1, 0(a0)
-; RV32I-NEXT:    sw t1, 4(a0)
-; RV32I-NEXT:    lw s0, 12(sp) # 4-byte Folded Reload
-; RV32I-NEXT:    addi sp, sp, 16
+; RV32I-NEXT:    sw a4, 4(a0)
+; RV32I-NEXT:    sw a3, 0(a0)
+; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
-; RV32I-NEXT:  .LBB6_15:
-; RV32I-NEXT:    mv t3, a4
-; RV32I-NEXT:    bgeu a2, t0, .LBB6_11
-; RV32I-NEXT:  .LBB6_16:
-; RV32I-NEXT:    slti a7, a7, 0
-; RV32I-NEXT:    neg a7, a7
-; RV32I-NEXT:    and a5, a7, a5
-; RV32I-NEXT:    or t3, t4, a5
-; RV32I-NEXT:    beqz a2, .LBB6_12
-; RV32I-NEXT:  .LBB6_17:
-; RV32I-NEXT:    mv a1, t3
-; RV32I-NEXT:    bgez a6, .LBB6_13
-; RV32I-NEXT:    j .LBB6_14
 ;
 ; RV64I-LABEL: lshr128:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    addi a4, a2, -64
 ; RV64I-NEXT:    srl a3, a1, a2
 ; RV64I-NEXT:    bltz a4, .LBB6_2
 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a0, a3

 ; RV64I-NEXT:    ret
   %1 = lshr i128 %a, %b
   ret i128 %1
 }
 
 define i128 @ashr128(i128 %a, i128 %b) nounwind {
 ; RV32I-LABEL: ashr128:
 ; RV32I:       # %bb.0:
-; RV32I-NEXT:    addi sp, sp, -16
-; RV32I-NEXT:    sw s0, 12(sp) # 4-byte Folded Spill
+; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    lw a2, 0(a2)
-; RV32I-NEXT:    lw t3, 8(a1)
-; RV32I-NEXT:    lw t2, 12(a1)
-; RV32I-NEXT:    neg a4, a2
-; RV32I-NEXT:    li a3, 64
-; RV32I-NEXT:    li a5, 32
-; RV32I-NEXT:    sub t1, a5, a2
-; RV32I-NEXT:    sll t0, t3, a4
-; RV32I-NEXT:    bltz t1, .LBB7_2
-; RV32I-NEXT:  # %bb.1:
-; RV32I-NEXT:    mv a7, t0
-; RV32I-NEXT:    j .LBB7_3
-; RV32I-NEXT:  .LBB7_2:
-; RV32I-NEXT:    sll a4, t2, a4
-; RV32I-NEXT:    sub a5, a3, a2
-; RV32I-NEXT:    not a5, a5
-; RV32I-NEXT:    srli a6, t3, 1
-; RV32I-NEXT:    srl a5, a6, a5
-; RV32I-NEXT:    or a7, a4, a5
-; RV32I-NEXT:  .LBB7_3:
-; RV32I-NEXT:    lw t6, 4(a1)
-; RV32I-NEXT:    sra a4, t2, a2
-; RV32I-NEXT:    addi t4, a2, -96
-; RV32I-NEXT:    srai a5, t2, 31
-; RV32I-NEXT:    mv s0, a4
-; RV32I-NEXT:    bltz t4, .LBB7_5
-; RV32I-NEXT:  # %bb.4:
-; RV32I-NEXT:    mv s0, a5
-; RV32I-NEXT:  .LBB7_5:
-; RV32I-NEXT:    addi a6, a2, -32
-; RV32I-NEXT:    srl t5, t6, a2
-; RV32I-NEXT:    bgeu a2, a3, .LBB7_7
-; RV32I-NEXT:  # %bb.6:
-; RV32I-NEXT:    slti s0, a6, 0
-; RV32I-NEXT:    neg s0, s0
-; RV32I-NEXT:    and s0, s0, t5
-; RV32I-NEXT:    or s0, s0, a7
-; RV32I-NEXT:  .LBB7_7:
-; RV32I-NEXT:    mv a7, t6
-; RV32I-NEXT:    beqz a2, .LBB7_9
-; RV32I-NEXT:  # %bb.8:
-; RV32I-NEXT:    mv a7, s0
-; RV32I-NEXT:  .LBB7_9:
+; RV32I-NEXT:    lw a3, 12(a1)
+; RV32I-NEXT:    lw a4, 8(a1)
+; RV32I-NEXT:    lw a5, 4(a1)
 ; RV32I-NEXT:    lw a1, 0(a1)
-; RV32I-NEXT:    not s0, a2
-; RV32I-NEXT:    bgez a6, .LBB7_11
-; RV32I-NEXT:  # %bb.10:
-; RV32I-NEXT:    srl t5, a1, a2
-; RV32I-NEXT:    slli t6, t6, 1
-; RV32I-NEXT:    sll t6, t6, s0
-; RV32I-NEXT:    or t5, t5, t6
-; RV32I-NEXT:  .LBB7_11:
-; RV32I-NEXT:    srl t3, t3, a2
-; RV32I-NEXT:    slli t2, t2, 1
-; RV32I-NEXT:    sll t2, t2, s0
-; RV32I-NEXT:    or t2, t3, t2
-; RV32I-NEXT:    mv t3, t2
-; RV32I-NEXT:    bgez t4, .LBB7_20
-; RV32I-NEXT:  # %bb.12:
-; RV32I-NEXT:    bltu a2, a3, .LBB7_21
-; RV32I-NEXT:  .LBB7_13:
-; RV32I-NEXT:    bnez a2, .LBB7_22
-; RV32I-NEXT:  .LBB7_14:
-; RV32I-NEXT:    bgez a6, .LBB7_23
-; RV32I-NEXT:  .LBB7_15:
-; RV32I-NEXT:    bgeu a2, a3, .LBB7_24
-; RV32I-NEXT:  .LBB7_16:
-; RV32I-NEXT:    bgez a6, .LBB7_25
-; RV32I-NEXT:  .LBB7_17:
-; RV32I-NEXT:    bltu a2, a3, .LBB7_19
-; RV32I-NEXT:  .LBB7_18:
-; RV32I-NEXT:    mv a4, a5
-; RV32I-NEXT:  .LBB7_19:
-; RV32I-NEXT:    sw a4, 12(a0)
-; RV32I-NEXT:    sw t2, 8(a0)
-; RV32I-NEXT:    sw a1, 0(a0)
-; RV32I-NEXT:    sw a7, 4(a0)
-; RV32I-NEXT:    lw s0, 12(sp) # 4-byte Folded Reload
-; RV32I-NEXT:    addi sp, sp, 16
+; RV32I-NEXT:    sb a3, 12(sp)
+; RV32I-NEXT:    sb a4, 8(sp)
+; RV32I-NEXT:    sb a5, 4(sp)
+; RV32I-NEXT:    sb a1, 0(sp)
+; RV32I-NEXT:    srai a6, a3, 31
+; RV32I-NEXT:    sb a6, 28(sp)
+; RV32I-NEXT:    sb a6, 24(sp)
+; RV32I-NEXT:    sb a6, 20(sp)
+; RV32I-NEXT:    sb a6, 16(sp)
+; RV32I-NEXT:    srli a7, a3, 24
+; RV32I-NEXT:    sb a7, 15(sp)
+; RV32I-NEXT:    srli a7, a3, 16
+; RV32I-NEXT:    sb a7, 14(sp)
+; RV32I-NEXT:    srli a3, a3, 8
+; RV32I-NEXT:    sb a3, 13(sp)
+; RV32I-NEXT:    srli a3, a4, 24
+; RV32I-NEXT:    sb a3, 11(sp)
+; RV32I-NEXT:    srli a3, a4, 16
+; RV32I-NEXT:    sb a3, 10(sp)
+; RV32I-NEXT:    srli a4, a4, 8
+; RV32I-NEXT:    sb a4, 9(sp)
+; RV32I-NEXT:    srli a3, a5, 24
+; RV32I-NEXT:    sb a3, 7(sp)
+; RV32I-NEXT:    srli a3, a5, 16
+; RV32I-NEXT:    sb a3, 6(sp)
+; RV32I-NEXT:    srli a5, a5, 8
+; RV32I-NEXT:    sb a5, 5(sp)
+; RV32I-NEXT:    srli a3, a1, 24
+; RV32I-NEXT:    sb a3, 3(sp)
+; RV32I-NEXT:    srli a3, a1, 16
+; RV32I-NEXT:    sb a3, 2(sp)
+; RV32I-NEXT:    srli a1, a1, 8
+; RV32I-NEXT:    sb a1, 1(sp)
+; RV32I-NEXT:    srli a1, a6, 24
+; RV32I-NEXT:    sb a1, 31(sp)
+; RV32I-NEXT:    srli a3, a6, 16
+; RV32I-NEXT:    sb a3, 30(sp)
+; RV32I-NEXT:    srli a4, a6, 8
+; RV32I-NEXT:    sb a4, 29(sp)
+; RV32I-NEXT:    sb a1, 27(sp)
+; RV32I-NEXT:    sb a3, 26(sp)
+; RV32I-NEXT:    sb a4, 25(sp)
+; RV32I-NEXT:    sb a1, 23(sp)
+; RV32I-NEXT:    sb a3, 22(sp)
+; RV32I-NEXT:    sb a4, 21(sp)
+; RV32I-NEXT:    sb a1, 19(sp)
+; RV32I-NEXT:    sb a3, 18(sp)
+; RV32I-NEXT:    sb a4, 17(sp)
+; RV32I-NEXT:    slli a1, a2, 25
+; RV32I-NEXT:    srli a1, a1, 28
+; RV32I-NEXT:    mv a3, sp
+; RV32I-NEXT:    add a1, a3, a1
+; RV32I-NEXT:    lbu a3, 1(a1)
+; RV32I-NEXT:    lbu a4, 0(a1)
+; RV32I-NEXT:    lbu a5, 2(a1)
+; RV32I-NEXT:    lbu a6, 3(a1)
+; RV32I-NEXT:    slli a3, a3, 8
+; RV32I-NEXT:    or a3, a3, a4
+; RV32I-NEXT:    slli a5, a5, 16
+; RV32I-NEXT:    slli a6, a6, 24
+; RV32I-NEXT:    or a3, a5, a3
+; RV32I-NEXT:    or a3, a6, a3
+; RV32I-NEXT:    andi a2, a2, 7
+; RV32I-NEXT:    srl a3, a3, a2
+; RV32I-NEXT:    lbu a4, 5(a1)
+; RV32I-NEXT:    lbu a5, 4(a1)
+; RV32I-NEXT:    lbu a6, 6(a1)
+; RV32I-NEXT:    lbu a7, 7(a1)
+; RV32I-NEXT:    slli a4, a4, 8
+; RV32I-NEXT:    or a4, a4, a5
+; RV32I-NEXT:    slli a6, a6, 16
+; RV32I-NEXT:    slli a7, a7, 24
+; RV32I-NEXT:    or a4, a6, a4
+; RV32I-NEXT:    or a4, a7, a4
+; RV32I-NEXT:    slli a5, a4, 1
+; RV32I-NEXT:    xori a6, a2, 31
+; RV32I-NEXT:    sll a5, a5, a6
+; RV32I-NEXT:    or a3, a3, a5
+; RV32I-NEXT:    srl a4, a4, a2
+; RV32I-NEXT:    lbu a5, 9(a1)
+; RV32I-NEXT:    lbu a7, 8(a1)
+; RV32I-NEXT:    lbu t0, 10(a1)
+; RV32I-NEXT:    lbu t1, 11(a1)
+; RV32I-NEXT:    slli a5, a5, 8
+; RV32I-NEXT:    or a5, a5, a7
+; RV32I-NEXT:    slli t0, t0, 16
+; RV32I-NEXT:    slli t1, t1, 24
+; RV32I-NEXT:    or a5, t0, a5
+; RV32I-NEXT:    or a5, t1, a5
+; RV32I-NEXT:    slli a7, a5, 1
+; RV32I-NEXT:    not t0, a2
+; RV32I-NEXT:    lbu t1, 13(a1)
+; RV32I-NEXT:    sll a7, a7, t0
+; RV32I-NEXT:    or a4, a4, a7
+; RV32I-NEXT:    lbu a7, 12(a1)
+; RV32I-NEXT:    slli t1, t1, 8
+; RV32I-NEXT:    lbu t0, 14(a1)
+; RV32I-NEXT:    lbu a1, 15(a1)
+; RV32I-NEXT:    or a7, t1, a7
+; RV32I-NEXT:    srl a5, a5, a2
+; RV32I-NEXT:    slli t0, t0, 16
+; RV32I-NEXT:    slli a1, a1, 24
+; RV32I-NEXT:    or a7, t0, a7
+; RV32I-NEXT:    or a1, a1, a7
+; RV32I-NEXT:    slli a7, a1, 1
+; RV32I-NEXT:    sll a6, a7, a6
+; RV32I-NEXT:    or a5, a5, a6
+; RV32I-NEXT:    sra a1, a1, a2
+; RV32I-NEXT:    sw a1, 12(a0)
+; RV32I-NEXT:    sw a5, 8(a0)
+; RV32I-NEXT:    sw a4, 4(a0)
+; RV32I-NEXT:    sw a3, 0(a0)
+; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
-; RV32I-NEXT:  .LBB7_20:
-; RV32I-NEXT:    mv t3, a4
-; RV32I-NEXT:    bgeu a2, a3, .LBB7_13
-; RV32I-NEXT:  .LBB7_21:
-; RV32I-NEXT:    slti t1, t1, 0
-; RV32I-NEXT:    neg t1, t1
-; RV32I-NEXT:    and t0, t1, t0
-; RV32I-NEXT:    or t3, t5, t0
-; RV32I-NEXT:    beqz a2, .LBB7_14
-; RV32I-NEXT:  .LBB7_22:
-; RV32I-NEXT:    mv a1, t3
-; RV32I-NEXT:    bltz a6, .LBB7_15
-; RV32I-NEXT:  .LBB7_23:
-; RV32I-NEXT:    mv t2, a4
-; RV32I-NEXT:    bltu a2, a3, .LBB7_16
-; RV32I-NEXT:  .LBB7_24:
-; RV32I-NEXT:    mv t2, a5
-; RV32I-NEXT:    bltz a6, .LBB7_17
-; RV32I-NEXT:  .LBB7_25:
-; RV32I-NEXT:    mv a4, a5
-; RV32I-NEXT:    bgeu a2, a3, .LBB7_18
-; RV32I-NEXT:    j .LBB7_19
 ;
 ; RV64I-LABEL: ashr128:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    mv a3, a1
 ; RV64I-NEXT:    addi a4, a2, -64
 ; RV64I-NEXT:    sra a1, a1, a2
 ; RV64I-NEXT:    bltz a4, .LBB7_2
 ; RV64I-NEXT:  # %bb.1:

 ; RV64I-NEXT:    ret
   %1 = ashr i128 %a, %b
   ret i128 %1
 }
 
 define i128 @shl128(i128 %a, i128 %b) nounwind {
 ; RV32I-LABEL: shl128:
 ; RV32I:       # %bb.0:
-; RV32I-NEXT:    addi sp, sp, -16
-; RV32I-NEXT:    sw s0, 12(sp) # 4-byte Folded Spill
+; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    lw a2, 0(a2)
-; RV32I-NEXT:    lw t5, 8(a1)
-; RV32I-NEXT:    lw t3, 4(a1)
-; RV32I-NEXT:    lw t2, 0(a1)
-; RV32I-NEXT:    neg a3, a2
-; RV32I-NEXT:    li a7, 64
-; RV32I-NEXT:    li a4, 32
-; RV32I-NEXT:    sub a6, a4, a2
-; RV32I-NEXT:    srl a5, t3, a3
-; RV32I-NEXT:    bltz a6, .LBB8_2
-; RV32I-NEXT:  # %bb.1:
-; RV32I-NEXT:    mv t1, a5
-; RV32I-NEXT:    j .LBB8_3
-; RV32I-NEXT:  .LBB8_2:
-; RV32I-NEXT:    srl a3, t2, a3
-; RV32I-NEXT:    sub a4, a7, a2
-; RV32I-NEXT:    not a4, a4
-; RV32I-NEXT:    slli t0, t3, 1
-; RV32I-NEXT:    sll a4, t0, a4
-; RV32I-NEXT:    or t1, a3, a4
-; RV32I-NEXT:  .LBB8_3:
-; RV32I-NEXT:    sll t4, t5, a2
-; RV32I-NEXT:    addi a3, a2, -32
-; RV32I-NEXT:    slti a4, a3, 0
-; RV32I-NEXT:    neg t0, a4
-; RV32I-NEXT:    sll a4, t2, a2
-; RV32I-NEXT:    addi t6, a2, -96
-; RV32I-NEXT:    bltu a2, a7, .LBB8_5
-; RV32I-NEXT:  # %bb.4:
-; RV32I-NEXT:    slti t1, t6, 0
-; RV32I-NEXT:    neg s0, t1
-; RV32I-NEXT:    and s0, s0, a4
-; RV32I-NEXT:    mv t1, t5
-; RV32I-NEXT:    bnez a2, .LBB8_6
-; RV32I-NEXT:    j .LBB8_7
-; RV32I-NEXT:  .LBB8_5:
-; RV32I-NEXT:    and s0, t0, t4
-; RV32I-NEXT:    or s0, s0, t1
-; RV32I-NEXT:    mv t1, t5
-; RV32I-NEXT:    beqz a2, .LBB8_7
-; RV32I-NEXT:  .LBB8_6:
-; RV32I-NEXT:    mv t1, s0
-; RV32I-NEXT:  .LBB8_7:
+; RV32I-NEXT:    lw a3, 0(a1)
+; RV32I-NEXT:    lw a4, 4(a1)
+; RV32I-NEXT:    lw a5, 8(a1)
 ; RV32I-NEXT:    lw a1, 12(a1)
-; RV32I-NEXT:    not s0, a2
-; RV32I-NEXT:    bgez a3, .LBB8_9
-; RV32I-NEXT:  # %bb.8:
-; RV32I-NEXT:    sll t4, a1, a2
-; RV32I-NEXT:    srli t5, t5, 1
-; RV32I-NEXT:    srl t5, t5, s0
-; RV32I-NEXT:    or t4, t4, t5
-; RV32I-NEXT:  .LBB8_9:
-; RV32I-NEXT:    sll t3, t3, a2
-; RV32I-NEXT:    srli t2, t2, 1
-; RV32I-NEXT:    srl t2, t2, s0
-; RV32I-NEXT:    or t2, t3, t2
-; RV32I-NEXT:    mv t3, t2
-; RV32I-NEXT:    bgez t6, .LBB8_16
-; RV32I-NEXT:  # %bb.10:
-; RV32I-NEXT:    bltu a2, a7, .LBB8_17
-; RV32I-NEXT:  .LBB8_11:
-; RV32I-NEXT:    beqz a2, .LBB8_13
-; RV32I-NEXT:  .LBB8_12:
-; RV32I-NEXT:    mv a1, t3
-; RV32I-NEXT:  .LBB8_13:
-; RV32I-NEXT:    and a6, t0, a4
-; RV32I-NEXT:    sltiu a2, a2, 64
-; RV32I-NEXT:    neg a5, a2
-; RV32I-NEXT:    and a2, a5, a6
-; RV32I-NEXT:    bltz a3, .LBB8_15
-; RV32I-NEXT:  # %bb.14:
-; RV32I-NEXT:    mv t2, a4
-; RV32I-NEXT:  .LBB8_15:
-; RV32I-NEXT:    and a3, a5, t2
-; RV32I-NEXT:    sw a3, 4(a0)
+; RV32I-NEXT:    sb zero, 15(sp)
+; RV32I-NEXT:    sb zero, 14(sp)
+; RV32I-NEXT:    sb zero, 13(sp)
+; RV32I-NEXT:    sb zero, 12(sp)
+; RV32I-NEXT:    sb zero, 11(sp)
+; RV32I-NEXT:    sb zero, 10(sp)
+; RV32I-NEXT:    sb zero, 9(sp)
+; RV32I-NEXT:    sb zero, 8(sp)
+; RV32I-NEXT:    sb zero, 7(sp)
+; RV32I-NEXT:    sb zero, 6(sp)
+; RV32I-NEXT:    sb zero, 5(sp)
+; RV32I-NEXT:    sb zero, 4(sp)
+; RV32I-NEXT:    sb zero, 3(sp)
+; RV32I-NEXT:    sb zero, 2(sp)
+; RV32I-NEXT:    sb zero, 1(sp)
+; RV32I-NEXT:    sb zero, 0(sp)
+; RV32I-NEXT:    sb a1, 28(sp)
+; RV32I-NEXT:    sb a5, 24(sp)
+; RV32I-NEXT:    sb a4, 20(sp)
+; RV32I-NEXT:    sb a3, 16(sp)
+; RV32I-NEXT:    srli a6, a1, 24
+; RV32I-NEXT:    sb a6, 31(sp)
+; RV32I-NEXT:    srli a6, a1, 16
+; RV32I-NEXT:    sb a6, 30(sp)
+; RV32I-NEXT:    srli a1, a1, 8
+; RV32I-NEXT:    sb a1, 29(sp)
+; RV32I-NEXT:    srli a1, a5, 24
+; RV32I-NEXT:    sb a1, 27(sp)
+; RV32I-NEXT:    srli a1, a5, 16
+; RV32I-NEXT:    sb a1, 26(sp)
+; RV32I-NEXT:    srli a5, a5, 8
+; RV32I-NEXT:    sb a5, 25(sp)
+; RV32I-NEXT:    srli a1, a4, 24
+; RV32I-NEXT:    sb a1, 23(sp)
+; RV32I-NEXT:    srli a1, a4, 16
+; RV32I-NEXT:    sb a1, 22(sp)
+; RV32I-NEXT:    srli a4, a4, 8
+; RV32I-NEXT:    sb a4, 21(sp)
+; RV32I-NEXT:    srli a1, a3, 24
+; RV32I-NEXT:    sb a1, 19(sp)
+; RV32I-NEXT:    srli a1, a3, 16
+; RV32I-NEXT:    sb a1, 18(sp)
+; RV32I-NEXT:    srli a3, a3, 8
+; RV32I-NEXT:    sb a3, 17(sp)
+; RV32I-NEXT:    slli a1, a2, 25
+; RV32I-NEXT:    srli a1, a1, 28
+; RV32I-NEXT:    addi a3, sp, 16
+; RV32I-NEXT:    sub a3, a3, a1
+; RV32I-NEXT:    lbu a1, 5(a3)
+; RV32I-NEXT:    lbu a4, 4(a3)
+; RV32I-NEXT:    lbu a5, 6(a3)
+; RV32I-NEXT:    lbu a6, 7(a3)
+; RV32I-NEXT:    slli a1, a1, 8
+; RV32I-NEXT:    or a1, a1, a4
+; RV32I-NEXT:    slli a5, a5, 16
+; RV32I-NEXT:    slli a6, a6, 24
+; RV32I-NEXT:    or a1, a5, a1
+; RV32I-NEXT:    or a1, a6, a1
+; RV32I-NEXT:    andi a2, a2, 7
+; RV32I-NEXT:    sll a4, a1, a2
+; RV32I-NEXT:    lbu a5, 1(a3)
+; RV32I-NEXT:    lbu a6, 0(a3)
+; RV32I-NEXT:    lbu a7, 2(a3)
+; RV32I-NEXT:    lbu t0, 3(a3)
+; RV32I-NEXT:    slli a5, a5, 8
+; RV32I-NEXT:    or a5, a5, a6
+; RV32I-NEXT:    slli a7, a7, 16
+; RV32I-NEXT:    slli t0, t0, 24
+; RV32I-NEXT:    or a5, a7, a5
+; RV32I-NEXT:    or a5, t0, a5
+; RV32I-NEXT:    srli a6, a5, 1
+; RV32I-NEXT:    xori a7, a2, 31
+; RV32I-NEXT:    srl a6, a6, a7
+; RV32I-NEXT:    or a4, a4, a6
+; RV32I-NEXT:    lbu a6, 9(a3)
+; RV32I-NEXT:    lbu t0, 8(a3)
+; RV32I-NEXT:    lbu t1, 10(a3)
+; RV32I-NEXT:    lbu t2, 11(a3)
+; RV32I-NEXT:    slli a6, a6, 8
+; RV32I-NEXT:    or a6, a6, t0
+; RV32I-NEXT:    slli t1, t1, 16
+; RV32I-NEXT:    slli t2, t2, 24
+; RV32I-NEXT:    or a6, t1, a6
+; RV32I-NEXT:    or a6, t2, a6
+; RV32I-NEXT:    sll t0, a6, a2
+; RV32I-NEXT:    srli a1, a1, 1
+; RV32I-NEXT:    not t1, a2
+; RV32I-NEXT:    srl a1, a1, t1
+; RV32I-NEXT:    or a1, t0, a1
+; RV32I-NEXT:    lbu t0, 13(a3)
+; RV32I-NEXT:    lbu t1, 12(a3)
+; RV32I-NEXT:    lbu t2, 14(a3)
+; RV32I-NEXT:    lbu a3, 15(a3)
+; RV32I-NEXT:    slli t0, t0, 8
+; RV32I-NEXT:    or t0, t0, t1
+; RV32I-NEXT:    slli t2, t2, 16
+; RV32I-NEXT:    slli a3, a3, 24
+; RV32I-NEXT:    or t0, t2, t0
+; RV32I-NEXT:    or a3, a3, t0
+; RV32I-NEXT:    sll a3, a3, a2
+; RV32I-NEXT:    srli a6, a6, 1
+; RV32I-NEXT:    srl a6, a6, a7
+; RV32I-NEXT:    or a3, a3, a6
+; RV32I-NEXT:    sll a2, a5, a2
 ; RV32I-NEXT:    sw a2, 0(a0)
-; RV32I-NEXT:    sw a1, 12(a0)
-; RV32I-NEXT:    sw t1, 8(a0)
-; RV32I-NEXT:    lw s0, 12(sp) # 4-byte Folded Reload
-; RV32I-NEXT:    addi sp, sp, 16
+; RV32I-NEXT:    sw a3, 12(a0)
+; RV32I-NEXT:    sw a1, 8(a0)
+; RV32I-NEXT:    sw a4, 4(a0)
+; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
-; RV32I-NEXT:  .LBB8_16:
-; RV32I-NEXT:    mv t3, a4
-; RV32I-NEXT:    bgeu a2, a7, .LBB8_11
-; RV32I-NEXT:  .LBB8_17:
-; RV32I-NEXT:    slti a6, a6, 0
-; RV32I-NEXT:    neg a6, a6
-; RV32I-NEXT:    and a5, a6, a5
-; RV32I-NEXT:    or t3, t4, a5
-; RV32I-NEXT:    bnez a2, .LBB8_12
-; RV32I-NEXT:    j .LBB8_13
 ;
 ; RV64I-LABEL: shl128:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    addi a4, a2, -64
 ; RV64I-NEXT:    sll a3, a0, a2
 ; RV64I-NEXT:    bltz a4, .LBB8_2
 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a1, a3

llvm/test/CodeGen/X86/div-rem-pair-recomposition-signed.ll

 ; X86 doesn't have __divti3, so the urem is expanded into a loop.
 define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
 ; X86-LABEL: scalar_i128:
 ; X86:       # %bb.0: # %_udiv-special-cases
 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
-; X86-NEXT:    subl $92, %esp
+; X86-NEXT:    subl $152, %esp
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    movl %edx, %eax
+; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    sarl $31, %eax
-; X86-NEXT:    movl %esi, %ecx
+; X86-NEXT:    movl %ebp, %ecx
 ; X86-NEXT:    sarl $31, %ecx
-; X86-NEXT:    movl %eax, %edi
-; X86-NEXT:    xorl %edx, %edi
-; X86-NEXT:    movl %edi, %ebp
 ; X86-NEXT:    movl %eax, %edx
-; X86-NEXT:    xorl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    xorl %esi, %edx
+; X86-NEXT:    movl %eax, %esi
+; X86-NEXT:    xorl {{[0-9]+}}(%esp), %esi
+; X86-NEXT:    movl %esi, %ebx
 ; X86-NEXT:    movl %eax, %edi
 ; X86-NEXT:    xorl {{[0-9]+}}(%esp), %edi
-; X86-NEXT:    movl %eax, %ebx
-; X86-NEXT:    xorl {{[0-9]+}}(%esp), %ebx
-; X86-NEXT:    subl %eax, %ebx
-; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %eax, %esi
+; X86-NEXT:    xorl {{[0-9]+}}(%esp), %esi
+; X86-NEXT:    subl %eax, %esi
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    sbbl %eax, %edi
 ; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    sbbl %eax, %ebx
+; X86-NEXT:    movl %ebx, (%esp) # 4-byte Spill
 ; X86-NEXT:    sbbl %eax, %edx
 ; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %eax, %ebp
-; X86-NEXT:    movl %ebp, %edx
-; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %ecx, %ebx
-; X86-NEXT:    xorl %esi, %ebx
-; X86-NEXT:    movl %ecx, %esi
-; X86-NEXT:    xorl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    movl %ecx, %edi
-; X86-NEXT:    xorl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    xorl %ebp, %edi
 ; X86-NEXT:    movl %ecx, %ebp
 ; X86-NEXT:    xorl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT:    subl %ecx, %ebp
-; X86-NEXT:    sbbl %ecx, %edi
-; X86-NEXT:    sbbl %ecx, %esi
+; X86-NEXT:    movl %ecx, %ebx
+; X86-NEXT:    xorl {{[0-9]+}}(%esp), %ebx
+; X86-NEXT:    movl %ecx, %esi
+; X86-NEXT:    xorl {{[0-9]+}}(%esp), %esi
+; X86-NEXT:    subl %ecx, %esi
 ; X86-NEXT:    sbbl %ecx, %ebx
+; X86-NEXT:    sbbl %ecx, %ebp
+; X86-NEXT:    sbbl %ecx, %edi
 ; X86-NEXT:    xorl %eax, %ecx
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %edi, %eax
-; X86-NEXT:    orl %ebx, %eax
-; X86-NEXT:    movl %ebp, %ecx
-; X86-NEXT:    orl %esi, %ecx
+; X86-NEXT:    movl %ebx, %eax
+; X86-NEXT:    orl %edi, %eax
+; X86-NEXT:    movl %esi, %ecx
+; X86-NEXT:    orl %ebp, %ecx
 ; X86-NEXT:    orl %eax, %ecx
 ; X86-NEXT:    sete {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    orl %edx, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
+; X86-NEXT:    orl (%esp), %edx # 4-byte Folded Reload
 ; X86-NEXT:    orl %ecx, %edx
 ; X86-NEXT:    sete {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Spill
-; X86-NEXT:    bsrl %ebx, %eax
+; X86-NEXT:    bsrl %edi, %eax
 ; X86-NEXT:    xorl $31, %eax
-; X86-NEXT:    bsrl %esi, %edx
+; X86-NEXT:    bsrl %ebp, %edx
 ; X86-NEXT:    xorl $31, %edx
 ; X86-NEXT:    addl $32, %edx
-; X86-NEXT:    testl %ebx, %ebx
+; X86-NEXT:    testl %edi, %edi
 ; X86-NEXT:    cmovnel %eax, %edx
-; X86-NEXT:    bsrl %edi, %eax
+; X86-NEXT:    bsrl %ebx, %eax
 ; X86-NEXT:    xorl $31, %eax
-; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    bsrl %ebp, %ebp
-; X86-NEXT:    xorl $31, %ebp
-; X86-NEXT:    addl $32, %ebp
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testl %edi, %edi
-; X86-NEXT:    cmovnel %eax, %ebp
-; X86-NEXT:    addl $64, %ebp
 ; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    bsrl %esi, %ecx
+; X86-NEXT:    xorl $31, %ecx
+; X86-NEXT:    addl $32, %ecx
 ; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    orl %ebx, %esi
-; X86-NEXT:    cmovnel %edx, %ebp
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    bsrl %eax, %esi
-; X86-NEXT:    xorl $31, %esi
+; X86-NEXT:    testl %ebx, %ebx
+; X86-NEXT:    cmovnel %eax, %ecx
+; X86-NEXT:    addl $64, %ecx
+; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    orl %edi, %ebp
+; X86-NEXT:    cmovnel %edx, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    bsrl %ebx, %edx
+; X86-NEXT:    bsrl %ebx, %esi
+; X86-NEXT:    xorl $31, %esi
+; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
+; X86-NEXT:    bsrl %eax, %edx
 ; X86-NEXT:    xorl $31, %edx
 ; X86-NEXT:    addl $32, %edx
-; X86-NEXT:    testl %eax, %eax
+; X86-NEXT:    testl %ebx, %ebx
 ; X86-NEXT:    cmovnel %esi, %edx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    bsrl %eax, %edi
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
+; X86-NEXT:    bsrl %ebp, %edi
 ; X86-NEXT:    xorl $31, %edi
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    bsrl %ecx, %esi
+; X86-NEXT:    bsrl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
 ; X86-NEXT:    xorl $31, %esi
 ; X86-NEXT:    addl $32, %esi
-; X86-NEXT:    testl %eax, %eax
+; X86-NEXT:    testl %ebp, %ebp
 ; X86-NEXT:    cmovnel %edi, %esi
 ; X86-NEXT:    addl $64, %esi
-; X86-NEXT:    movl %ebx, %edi
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
+; X86-NEXT:    movl %eax, %edi
+; X86-NEXT:    orl %ebx, %edi
 ; X86-NEXT:    cmovnel %edx, %esi
 ; X86-NEXT:    xorl %edi, %edi
-; X86-NEXT:    subl %esi, %ebp
-; X86-NEXT:    movl $0, %ebx
-; X86-NEXT:    sbbl %ebx, %ebx
+; X86-NEXT:    subl %esi, %ecx
+; X86-NEXT:    movl $0, %ebp
+; X86-NEXT:    sbbl %ebp, %ebp
 ; X86-NEXT:    movl $0, %eax
 ; X86-NEXT:    sbbl %eax, %eax
 ; X86-NEXT:    movl $0, %esi
 ; X86-NEXT:    sbbl %esi, %esi
 ; X86-NEXT:    movl $127, %edx
-; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    cmpl %ebp, %edx
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    cmpl %ecx, %edx
+; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    movl $0, %edx
-; X86-NEXT:    sbbl %ebx, %edx
+; X86-NEXT:    sbbl %ebp, %edx
 ; X86-NEXT:    movl $0, %edx
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    sbbl %eax, %edx
 ; X86-NEXT:    movl $0, %edx
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    sbbl %esi, %edx
 ; X86-NEXT:    setb %dl
 ; X86-NEXT:    orb {{[-0-9]+}}(%e{{[sb]}}p), %dl # 1-byte Folded Reload
 ; X86-NEXT:    orb {{[-0-9]+}}(%e{{[sb]}}p), %dl # 1-byte Folded Reload
+; X86-NEXT:    cmovnel %edi, %ebx
+; X86-NEXT:    movl (%esp), %esi # 4-byte Reload
+; X86-NEXT:    cmovnel %edi, %esi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    cmovnel %edi, %eax
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    cmovnel %edi, %eax
-; X86-NEXT:    movl %eax, (%esp) # 4-byte Spill
-; X86-NEXT:    movl %edx, %eax
-; X86-NEXT:    cmovnel %edi, %eax
-; X86-NEXT:    cmovel %ecx, %edi
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    cmovel {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
 ; X86-NEXT:    jne .LBB4_8
 ; X86-NEXT:  # %bb.1: # %_udiv-special-cases
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NEXT:    xorl $127, %edx
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
-; X86-NEXT:    movl %ebx, %ecx
-; X86-NEXT:    orl %esi, %ecx
+; X86-NEXT:    orl %ecx, %edx
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ebp, %ecx
+; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
 ; X86-NEXT:    orl %edx, %ecx
 ; X86-NEXT:    je .LBB4_8
 ; X86-NEXT:  # %bb.2: # %udiv-bb1
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    movl %eax, %ecx
-; X86-NEXT:    xorb $127, %cl
+; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    movl %esi, %edx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    shldl %cl, %esi, %eax
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; X86-NEXT:    shll %cl, %ebp
+; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
+; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %esi, %edi
-; X86-NEXT:    shll %cl, %edi
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    movb %cl, %ch
-; X86-NEXT:    movb %cl, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
-; X86-NEXT:    cmovnel %edi, %eax
-; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl $0, %edx
-; X86-NEXT:    cmovnel %edx, %ebp
-; X86-NEXT:    cmovnel %edx, %edi
-; X86-NEXT:    subb $64, %ch
-; X86-NEXT:    cmovael %edx, %edi
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    negb %ch
-; X86-NEXT:    movl %esi, %ebx
+; X86-NEXT:    movl %ecx, %ebp
+; X86-NEXT:    xorb $127, %al
+; X86-NEXT:    movb %al, %ch
+; X86-NEXT:    andb $7, %ch
+; X86-NEXT:    shrb $3, %al
+; X86-NEXT:    andb $15, %al
+; X86-NEXT:    negb %al
+; X86-NEXT:    movsbl %al, %edi
+; X86-NEXT:    movl 144(%esp,%edi), %edx
+; X86-NEXT:    movl 148(%esp,%edi), %ebx
 ; X86-NEXT:    movb %ch, %cl
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    shrdl %cl, %edx, %ebx
-; X86-NEXT:    shrl %cl, %edx
-; X86-NEXT:    testb $32, %ch
-; X86-NEXT:    cmovnel %edx, %ebx
-; X86-NEXT:    orl %ebp, %ebx
-; X86-NEXT:    movb {{[-0-9]+}}(%e{{[sb]}}p), %cl # 1-byte Reload
-; X86-NEXT:    addb $-64, %cl
-; X86-NEXT:    movl %esi, %ebp
-; X86-NEXT:    movb %cl, (%esp) # 1-byte Spill
-; X86-NEXT:    shll %cl, %ebp
+; X86-NEXT:    shldl %cl, %edx, %ebx
+; X86-NEXT:    shll %cl, %edx
+; X86-NEXT:    notb %cl
+; X86-NEXT:    movl 140(%esp,%edi), %eax
+; X86-NEXT:    movl %eax, %esi
+; X86-NEXT:    shrl %esi
+; X86-NEXT:    shrl %cl, %esi
+; X86-NEXT:    orl %edx, %esi
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl 136(%esp,%edi), %edx
+; X86-NEXT:    movb %ch, %cl
+; X86-NEXT:    shldl %cl, %edx, %eax
+; X86-NEXT:    shll %cl, %edx
+; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    addl $1, %ebp
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    movl $0, %edi
-; X86-NEXT:    cmovnel %edi, %ebp
-; X86-NEXT:    movb {{[-0-9]+}}(%e{{[sb]}}p), %cl # 1-byte Reload
-; X86-NEXT:    cmpb $64, %cl
-; X86-NEXT:    cmovbl %ebx, %ebp
-; X86-NEXT:    movl $0, %ebx
-; X86-NEXT:    cmovael %ebx, %eax
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testb $32, %ch
-; X86-NEXT:    cmovnel %ebx, %edx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT:    movl %edi, %eax
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    shldl %cl, %ebx, %eax
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    cmovnel {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    movzbl (%esp), %ecx # 1-byte Folded Reload
-; X86-NEXT:    shldl %cl, %esi, %ebx
-; X86-NEXT:    movl %eax, %ecx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    testb $32, (%esp) # 1-byte Folded Reload
-; X86-NEXT:    cmovnel {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
-; X86-NEXT:    orl %edx, %ecx
-; X86-NEXT:    movzbl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 1-byte Folded Reload
-; X86-NEXT:    cmpb $64, %dl
-; X86-NEXT:    cmovael %ebx, %ecx
-; X86-NEXT:    testb %dl, %dl
-; X86-NEXT:    cmovel %edi, %ecx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    cmovel %esi, %ebp
-; X86-NEXT:    movl %ebp, (%esp) # 4-byte Spill
-; X86-NEXT:    addl $1, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT:    adcl $0, %eax
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    adcl $0, %ebx
-; X86-NEXT:    adcl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
+; X86-NEXT:    adcl $0, %ebp
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    adcl $0, %ecx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT:    adcl $0, %edx
 ; X86-NEXT:    jae .LBB4_3
 ; X86-NEXT:  # %bb.6:
-; X86-NEXT:    xorl %ebx, %ebx
 ; X86-NEXT:    xorl %edx, %edx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    movl (%esp), %esi # 4-byte Reload
+; X86-NEXT:    xorl %ecx, %ecx
 ; X86-NEXT:    jmp .LBB4_7
 ; X86-NEXT:  .LBB4_3: # %udiv-preheader
-; X86-NEXT:    movl %esi, %edx
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    shrdl %cl, %edi, %edx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    shrl %cl, %esi
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %edi, %esi
-; X86-NEXT:    shrl %cl, %esi
-; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    testb $32, %cl
+; X86-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl (%esp), %esi # 4-byte Reload
+; X86-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    cmovnel %esi, %edx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; X86-NEXT:    cmovnel %eax, %ebp
-; X86-NEXT:    cmovnel %eax, %esi
-; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    movb %cl, %ch
-; X86-NEXT:    subb $64, %ch
-; X86-NEXT:    cmovael %eax, %esi
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    negb %ch
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    movl %edi, %esi
-; X86-NEXT:    movb %ch, %cl
-; X86-NEXT:    shldl %cl, %eax, %esi
-; X86-NEXT:    shll %cl, %eax
+; X86-NEXT:    movb %dl, %ch
+; X86-NEXT:    andb $7, %ch
+; X86-NEXT:    movb %dl, %cl
+; X86-NEXT:    shrb $3, %cl
+; X86-NEXT:    andb $15, %cl
+; X86-NEXT:    movzbl %cl, %edx
+; X86-NEXT:    movl 100(%esp,%edx), %esi
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testb $32, %ch
-; X86-NEXT:    cmovnel %eax, %esi
-; X86-NEXT:    orl %ebp, %esi
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    movb %al, %cl
-; X86-NEXT:    addb $-64, %cl
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; X86-NEXT:    movb %cl, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
-; X86-NEXT:    shrl %cl, %ebp
+; X86-NEXT:    movl 96(%esp,%edx), %ebp
+; X86-NEXT:    movl %edx, %eax
+; X86-NEXT:    movl %ebp, %edx
+; X86-NEXT:    movb %ch, %cl
+; X86-NEXT:    shrdl %cl, %esi, %edx
+; X86-NEXT:    movl %ebx, %edi
+; X86-NEXT:    movl 88(%esp,%eax), %ebx
+; X86-NEXT:    movl 92(%esp,%eax), %eax
+; X86-NEXT:    movl %eax, (%esp) # 4-byte Spill
+; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
+; X86-NEXT:    shrl %cl, %eax
+; X86-NEXT:    notb %cl
+; X86-NEXT:    addl %ebp, %ebp
+; X86-NEXT:    shll %cl, %ebp
+; X86-NEXT:    orl %eax, %ebp
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    movl $0, %edi
-; X86-NEXT:    cmovnel %edi, %ebp
-; X86-NEXT:    cmpb $64, %al
-; X86-NEXT:    cmovbl %esi, %ebp
-; X86-NEXT:    cmovael %edi, %edx
-; X86-NEXT:    xorl %esi, %esi
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testb $32, %ch
-; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    cmovnel %esi, %ecx
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    movl %eax, %ecx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    shrdl %cl, %edx, %esi
-; X86-NEXT:    testb $32, %al
-; X86-NEXT:    cmovnel {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
-; X86-NEXT:    movzbl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 1-byte Folded Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    shrdl %cl, %edx, %edi
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    cmovnel {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
-; X86-NEXT:    cmpb $64, %al
-; X86-NEXT:    cmovael %edi, %esi
-; X86-NEXT:    testb %al, %al
-; X86-NEXT:    cmovel {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
+; X86-NEXT:    movb %ch, %cl
+; X86-NEXT:    shrl %cl, %esi
 ; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    cmovel {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Folded Reload
-; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
+; X86-NEXT:    shrdl %cl, %eax, %ebx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    addl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
 ; X86-NEXT:    .p2align 4, 0x90
 ; X86-NEXT:  .LBB4_4: # %udiv-do-while
 ; X86-NEXT:    # =>This Inner Loop Header: Depth=1
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
+; X86-NEXT:    movl %edx, (%esp) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; X86-NEXT:    shldl $1, %ebp, %edi
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    shldl $1, %esi, %ebp
+; X86-NEXT:    shldl $1, %edx, %ebp
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    shldl $1, %edx, %esi
+; X86-NEXT:    shldl $1, %edx, (%esp) # 4-byte Folded Spill
+; X86-NEXT:    shldl $1, %ebx, %edx
+; X86-NEXT:    shldl $1, %edi, %ebx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    shldl $1, %esi, %edi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    shldl $1, %eax, %edx
-; X86-NEXT:    movl (%esp), %ebx # 4-byte Reload
-; X86-NEXT:    shldl $1, %ebx, %eax
+; X86-NEXT:    orl %eax, %edi
+; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ecx, %edi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    orl %ecx, %eax
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    shldl $1, %eax, %ebx
-; X86-NEXT:    orl %ecx, %ebx
-; X86-NEXT:    movl %ebx, (%esp) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    shldl $1, %ebx, %eax
-; X86-NEXT:    orl %ecx, %eax
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    addl %ebx, %ebx
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
-; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    cmpl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
+; X86-NEXT:    shldl $1, %ecx, %esi
+; X86-NEXT:    orl %eax, %esi
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    shldl $1, %esi, %ecx
+; X86-NEXT:    orl %eax, %ecx
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    addl %esi, %esi
+; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    cmpl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    sbbl %esi, %ecx
+; X86-NEXT:    sbbl %edx, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    sbbl %ebp, %ecx
+; X86-NEXT:    sbbl (%esp), %ecx # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    sbbl %edi, %ecx
+; X86-NEXT:    sbbl %ebp, %ecx
 ; X86-NEXT:    sarl $31, %ecx
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    andl $1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %ecx, %ebx
-; X86-NEXT:    andl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
+; X86-NEXT:    movl %ecx, %esi
+; X86-NEXT:    andl %edi, %esi
 ; X86-NEXT:    movl %ecx, %edi
 ; X86-NEXT:    andl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    andl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
 ; X86-NEXT:    andl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; X86-NEXT:    subl %ecx, %edx
+; X86-NEXT:    subl %ecx, %ebx
+; X86-NEXT:    sbbl %eax, %edx
 ; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    sbbl %eax, %esi
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %edi, %ebp
+; X86-NEXT:    movl (%esp), %edx # 4-byte Reload
+; X86-NEXT:    sbbl %edi, %edx
+; X86-NEXT:    sbbl %esi, %ebp
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    addl $-1, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %eax
-; X86-NEXT:    adcl $-1, %ebx
-; X86-NEXT:    adcl $-1, %edx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    adcl $-1, %esi
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
+; X86-NEXT:    adcl $-1, %edi
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    orl %edx, %eax
+; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    orl %edi, %eax
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    orl %ebx, %ecx
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    orl %esi, %ecx
 ; X86-NEXT:    orl %eax, %ecx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    jne .LBB4_4
 ; X86-NEXT:  # %bb.5:
+; X86-NEXT:    movl %edi, %ebx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    movl (%esp), %esi # 4-byte Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; X86-NEXT:  .LBB4_7: # %udiv-loop-exit
-; X86-NEXT:    shldl $1, %esi, %ecx
-; X86-NEXT:    orl %edx, %ecx
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    shldl $1, %esi, %ebx
+; X86-NEXT:    orl %ecx, %ebx
 ; X86-NEXT:    shldl $1, %eax, %esi
-; X86-NEXT:    orl %edx, %esi
-; X86-NEXT:    movl %esi, (%esp) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    shldl $1, %ecx, %eax
-; X86-NEXT:    orl %edx, %eax
-; X86-NEXT:    addl %ecx, %ecx
-; X86-NEXT:    orl %ebx, %ecx
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    orl %ecx, %esi
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
+; X86-NEXT:    shldl $1, %edi, %eax
+; X86-NEXT:    orl %ecx, %eax
+; X86-NEXT:    addl %edi, %edi
+; X86-NEXT:    orl %edx, %edi
 ; X86-NEXT:  .LBB4_8: # %udiv-end
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; X86-NEXT:    xorl %ecx, %ebp
-; X86-NEXT:    movl (%esp), %esi # 4-byte Reload
+; X86-NEXT:    xorl %ecx, %ebx
 ; X86-NEXT:    xorl %ecx, %esi
 ; X86-NEXT:    xorl %ecx, %eax
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT:    movl %edi, %edx
 ; X86-NEXT:    xorl %ecx, %edx
 ; X86-NEXT:    subl %ecx, %edx
 ; X86-NEXT:    sbbl %ecx, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    sbbl %ecx, %esi
-; X86-NEXT:    sbbl %ecx, %ebp
-; X86-NEXT:    movl %edx, (%edi)
-; X86-NEXT:    movl %eax, 4(%edi)
-; X86-NEXT:    movl %esi, 8(%edi)
-; X86-NEXT:    movl %ebp, 12(%edi)
+; X86-NEXT:    sbbl %ecx, %ebx
+; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl %edx, (%ecx)
+; X86-NEXT:    movl %eax, 4(%ecx)
+; X86-NEXT:    movl %esi, 8(%ecx)
+; X86-NEXT:    movl %ebx, 12(%ecx)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl %edx, %ebx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    mull %edi
 ; X86-NEXT:    movl %edx, %ecx
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ebx, %eax
 ; X86-NEXT:    movl %ebx, %edi
 ; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %ebx, %eax
 ; X86-NEXT:    mull {{[0-9]+}}(%esp)
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %eax, (%esp) # 4-byte Spill
 ; X86-NEXT:    movl %edx, %ebx
 ; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
 ; X86-NEXT:    adcl $0, %ecx
 ; X86-NEXT:    movl %edi, %eax
 ; X86-NEXT:    movl %esi, %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    mull %esi
 ; X86-NEXT:    addl %ebx, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    adcl %ecx, %edx
 ; X86-NEXT:    movl %edx, %ecx
-; X86-NEXT:    setb (%esp) # 1-byte Folded Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    movl %ebx, %eax
-; X86-NEXT:    mull %esi
+; X86-NEXT:    setb %bl
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    movl %esi, %eax
+; X86-NEXT:    mull {{[0-9]+}}(%esp)
 ; X86-NEXT:    addl %ecx, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movzbl (%esp), %eax # 1-byte Folded Reload
+; X86-NEXT:    movzbl %bl, %eax
 ; X86-NEXT:    adcl %eax, %edx
-; X86-NEXT:    movl %edx, (%esp) # 4-byte Spill
+; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    imull %eax, %ebp
+; X86-NEXT:    imull %eax, %ecx
 ; X86-NEXT:    mull %edi
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    imull %esi, %edi
-; X86-NEXT:    addl %ebp, %edi
+; X86-NEXT:    imull {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    addl %ecx, %edi
 ; X86-NEXT:    addl %edx, %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl %ebx, %edx
-; X86-NEXT:    imull %eax, %edx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    imull %ecx, %ebx
-; X86-NEXT:    addl %edx, %ebx
-; X86-NEXT:    mull %ecx
-; X86-NEXT:    addl %edx, %ebx
+; X86-NEXT:    imull %eax, %esi
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT:    imull %edx, %ebp
+; X86-NEXT:    addl %esi, %ebp
+; X86-NEXT:    mull %edx
+; X86-NEXT:    addl %edx, %ebp
 ; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; X86-NEXT:    adcl %edi, %ebx
+; X86-NEXT:    adcl %edi, %ebp
 ; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; X86-NEXT:    adcl (%esp), %ebx # 4-byte Folded Reload
+; X86-NEXT:    adcl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    subl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
+; X86-NEXT:    subl (%esp), %esi # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    sbbl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    sbbl %eax, %ecx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; X86-NEXT:    sbbl %ebx, %edi
+; X86-NEXT:    sbbl %ebp, %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %esi, (%eax)
 ; X86-NEXT:    movl %edx, 4(%eax)
 ; X86-NEXT:    movl %ecx, 8(%eax)
 ; X86-NEXT:    movl %edi, 12(%eax)
-; X86-NEXT:    addl $92, %esp
+; X86-NEXT:    addl $152, %esp
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx
 ; X86-NEXT:    popl %ebp
 ; X86-NEXT:    retl $4
 ;
 ; X64-LABEL: scalar_i128:
 ; X64:       # %bb.0:

llvm/test/CodeGen/X86/div-rem-pair-recomposition-unsigned.ll

 ; X86 doesn't have __divti3, so the urem is expanded into a loop.
 define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
 ; X86-LABEL: scalar_i128:
 ; X86:       # %bb.0: # %_udiv-special-cases
 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
-; X86-NEXT:    subl $72, %esp
+; X86-NEXT:    subl $132, %esp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    movl %ebx, %eax
+; X86-NEXT:    movl %edi, %eax
 ; X86-NEXT:    orl %ebp, %eax
-; X86-NEXT:    orl %edi, %ecx
+; X86-NEXT:    orl %ebx, %ecx
 ; X86-NEXT:    orl %eax, %ecx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    sete {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Spill
+; X86-NEXT:    sete (%esp) # 1-byte Folded Spill
 ; X86-NEXT:    orl %edx, %ecx
 ; X86-NEXT:    movl %edx, %eax
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    orl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    orl %ecx, %edx
 ; X86-NEXT:    sete {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Spill
 ; X86-NEXT:    bsrl %ebp, %esi
 ; X86-NEXT:    xorl $31, %esi
-; X86-NEXT:    bsrl %edi, %edx
+; X86-NEXT:    bsrl %ebx, %edx
 ; X86-NEXT:    xorl $31, %edx
 ; X86-NEXT:    addl $32, %edx
 ; X86-NEXT:    testl %ebp, %ebp
 ; X86-NEXT:    cmovnel %esi, %edx
-; X86-NEXT:    bsrl %ebx, %esi
+; X86-NEXT:    bsrl %edi, %esi
 ; X86-NEXT:    xorl $31, %esi
 ; X86-NEXT:    bsrl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    xorl $31, %ecx
 ; X86-NEXT:    addl $32, %ecx
-; X86-NEXT:    testl %ebx, %ebx
+; X86-NEXT:    testl %edi, %edi
 ; X86-NEXT:    cmovnel %esi, %ecx
 ; X86-NEXT:    addl $64, %ecx
-; X86-NEXT:    orl %ebp, %edi
+; X86-NEXT:    orl %ebp, %ebx
 ; X86-NEXT:    cmovnel %edx, %ecx
 ; X86-NEXT:    movl %eax, %ebx
 ; X86-NEXT:    bsrl %eax, %esi
 ; X86-NEXT:    xorl $31, %esi
-; X86-NEXT:    bsrl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    bsrl %edx, %edx
 ; X86-NEXT:    xorl $31, %edx
 ; X86-NEXT:    addl $32, %edx
 ; X86-NEXT:    testl %eax, %eax
 ; X86-NEXT:    cmovnel %esi, %edx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    bsrl %ebp, %edi
 ; X86-NEXT:    xorl $31, %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    bsrl %eax, %esi
 ; X86-NEXT:    xorl $31, %esi
 ; X86-NEXT:    addl $32, %esi
 ; X86-NEXT:    testl %ebp, %ebp
 ; X86-NEXT:    cmovnel %edi, %esi
 ; X86-NEXT:    addl $64, %esi
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl %eax, %edi
 ; X86-NEXT:    orl %ebx, %edi
 ; X86-NEXT:    cmovnel %edx, %esi
-; X86-NEXT:    xorl %ebp, %ebp
+; X86-NEXT:    xorl %edi, %edi
 ; X86-NEXT:    subl %esi, %ecx
-; X86-NEXT:    movl $0, %eax
-; X86-NEXT:    sbbl %eax, %eax
+; X86-NEXT:    movl %eax, %ebx
+; X86-NEXT:    movl $0, %ebp
+; X86-NEXT:    sbbl %ebp, %ebp
 ; X86-NEXT:    movl $0, %esi
 ; X86-NEXT:    sbbl %esi, %esi
-; X86-NEXT:    movl $0, %edi
-; X86-NEXT:    sbbl %edi, %edi
+; X86-NEXT:    movl $0, %eax
+; X86-NEXT:    sbbl %eax, %eax
 ; X86-NEXT:    movl $127, %edx
-; X86-NEXT:    movl %ecx, (%esp) # 4-byte Spill
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    cmpl %ecx, %edx
-; X86-NEXT:    movl %eax, %ecx
+; X86-NEXT:    movl %esi, %ecx
 ; X86-NEXT:    movl $0, %edx
-; X86-NEXT:    sbbl %eax, %edx
+; X86-NEXT:    sbbl %ebp, %edx
 ; X86-NEXT:    movl $0, %edx
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    sbbl %esi, %edx
 ; X86-NEXT:    movl $0, %edx
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %edi, %edx
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    sbbl %eax, %edx
 ; X86-NEXT:    setb %dl
 ; X86-NEXT:    orb {{[-0-9]+}}(%e{{[sb]}}p), %dl # 1-byte Folded Reload
-; X86-NEXT:    orb {{[-0-9]+}}(%e{{[sb]}}p), %dl # 1-byte Folded Reload
+; X86-NEXT:    orb (%esp), %dl # 1-byte Folded Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    cmovnel %ebp, %ebx
-; X86-NEXT:    movl %ebx, %edx
-; X86-NEXT:    movl %eax, %ebx
-; X86-NEXT:    cmovnel %ebp, %ebx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    cmovnel %ebp, %esi
-; X86-NEXT:    cmovel {{[0-9]+}}(%esp), %ebp
-; X86-NEXT:    jne .LBB4_8
-; X86-NEXT:  # %bb.1: # %_udiv-special-cases
-; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
+; X86-NEXT:    cmovnel %edi, %eax
+; X86-NEXT:    movl %ebx, %esi
+; X86-NEXT:    cmovnel %edi, %esi
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; X86-NEXT:    cmovnel %edi, %ebx
+; X86-NEXT:    cmovel {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    jne .LBB4_1
+; X86-NEXT:  # %bb.8: # %_udiv-special-cases
+; X86-NEXT:    movl %eax, %edx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    xorl $127, %eax
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; X86-NEXT:    movl %ecx, %edi
+; X86-NEXT:    orl %ecx, %eax
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ebp, %ecx
 ; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
 ; X86-NEXT:    orl %eax, %ecx
-; X86-NEXT:    je .LBB4_8
-; X86-NEXT:  # %bb.2: # %udiv-bb1
-; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
-; X86-NEXT:    movl %eax, %ecx
-; X86-NEXT:    xorb $127, %cl
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    shldl %cl, %edx, %ebp
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    shll %cl, %esi
+; X86-NEXT:    movl %edx, %eax
+; X86-NEXT:    movl %ebp, %ecx
+; X86-NEXT:    movl %ebx, %ebp
+; X86-NEXT:    je .LBB4_9
+; X86-NEXT:  # %bb.5: # %udiv-bb1
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT:    movl %ebx, %eax
+; X86-NEXT:    xorb $127, %al
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movb %al, %ch
+; X86-NEXT:    andb $7, %ch
+; X86-NEXT:    shrb $3, %al
+; X86-NEXT:    andb $15, %al
+; X86-NEXT:    negb %al
+; X86-NEXT:    movsbl %al, %eax
+; X86-NEXT:    movl 124(%esp,%eax), %edx
+; X86-NEXT:    movl 128(%esp,%eax), %esi
+; X86-NEXT:    movb %ch, %cl
+; X86-NEXT:    shldl %cl, %edx, %esi
 ; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    shll %cl, %edx
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    cmovnel %edx, %ebp
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %esi, %edi
-; X86-NEXT:    movl $0, %esi
-; X86-NEXT:    cmovnel %esi, %edi
-; X86-NEXT:    cmovnel %esi, %edx
-; X86-NEXT:    movb %cl, %ch
-; X86-NEXT:    movl %ecx, %eax
-; X86-NEXT:    movb %cl, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
-; X86-NEXT:    subb $64, %ch
-; X86-NEXT:    cmovael %esi, %edx
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    negb %ch
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; X86-NEXT:    notb %cl
+; X86-NEXT:    movl 120(%esp,%eax), %ebp
+; X86-NEXT:    movl %ebp, %esi
+; X86-NEXT:    shrl %esi
+; X86-NEXT:    shrl %cl, %esi
+; X86-NEXT:    orl %edx, %esi
+; X86-NEXT:    movl 116(%esp,%eax), %edi
 ; X86-NEXT:    movb %ch, %cl
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    shrdl %cl, %edx, %esi
-; X86-NEXT:    shrl %cl, %edx
-; X86-NEXT:    testb $32, %ch
-; X86-NEXT:    cmovnel %edx, %esi
-; X86-NEXT:    orl %edi, %esi
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; X86-NEXT:    movb %al, %cl
-; X86-NEXT:    addb $-64, %cl
-; X86-NEXT:    movb %cl, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
+; X86-NEXT:    shldl %cl, %edi, %ebp
 ; X86-NEXT:    shll %cl, %edi
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    movl %edi, %ebx
-; X86-NEXT:    movl $0, %eax
-; X86-NEXT:    cmovnel %eax, %ebx
-; X86-NEXT:    movb {{[-0-9]+}}(%e{{[sb]}}p), %cl # 1-byte Reload
-; X86-NEXT:    cmpb $64, %cl
-; X86-NEXT:    cmovbl %esi, %ebx
-; X86-NEXT:    movl $0, %esi
-; X86-NEXT:    cmovael %esi, %ebp
-; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testb $32, %ch
-; X86-NEXT:    cmovnel %esi, %edx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    shldl %cl, %esi, %ebp
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    movb %cl, %ch
-; X86-NEXT:    cmovnel {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Folded Reload
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movb {{[-0-9]+}}(%e{{[sb]}}p), %cl # 1-byte Reload
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    shldl %cl, %esi, %eax
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    cmovnel %edi, %eax
-; X86-NEXT:    orl %edx, %ebp
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    cmpb $64, %ch
-; X86-NEXT:    cmovael %eax, %ebp
-; X86-NEXT:    movl %ebp, %edx
-; X86-NEXT:    testb %ch, %ch
-; X86-NEXT:    cmovel {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    movl %esi, %ebp
-; X86-NEXT:    cmovel %esi, %ebx
-; X86-NEXT:    addl $1, (%esp) # 4-byte Folded Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    adcl $0, %esi
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT:    adcl $0, %edi
+; X86-NEXT:    addl $1, %ebx
+; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    adcl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    adcl $0, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    adcl $0, %eax
-; X86-NEXT:    jae .LBB4_3
+; X86-NEXT:    jae .LBB4_2
 ; X86-NEXT:  # %bb.6:
+; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    xorl %ecx, %ecx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    xorl %eax, %eax
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    jmp .LBB4_7
-; X86-NEXT:  .LBB4_3: # %udiv-preheader
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl (%esp), %ecx # 4-byte Reload
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    shrdl %cl, %esi, %ebp
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:  .LBB4_1:
+; X86-NEXT:    movl %ebx, %ebp
+; X86-NEXT:    jmp .LBB4_9
+; X86-NEXT:  .LBB4_2: # %udiv-preheader
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    shrl %cl, %edx
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    shrl %cl, %esi
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    movl %ecx, (%esp) # 4-byte Spill
-; X86-NEXT:    cmovnel %esi, %ebp
-; X86-NEXT:    movl $0, %eax
-; X86-NEXT:    cmovnel %eax, %edx
-; X86-NEXT:    cmovnel %eax, %esi
-; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    movb %cl, %ch
-; X86-NEXT:    subb $64, %ch
-; X86-NEXT:    cmovael %eax, %esi
-; X86-NEXT:    negb %ch
-; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; X86-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movb %al, %ch
+; X86-NEXT:    andb $7, %ch
+; X86-NEXT:    # kill: def $al killed $al killed $eax
+; X86-NEXT:    shrb $3, %al
+; X86-NEXT:    andb $15, %al
+; X86-NEXT:    movzbl %al, %eax
+; X86-NEXT:    movl 80(%esp,%eax), %edx
+; X86-NEXT:    movl %edx, (%esp) # 4-byte Spill
+; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl 76(%esp,%eax), %edi
+; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %edi, %ebx
 ; X86-NEXT:    movb %ch, %cl
+; X86-NEXT:    shrdl %cl, %edx, %ebx
+; X86-NEXT:    movl 68(%esp,%eax), %ebp
+; X86-NEXT:    movl 72(%esp,%eax), %edx
+; X86-NEXT:    movl %edx, %eax
+; X86-NEXT:    shrl %cl, %eax
+; X86-NEXT:    notb %cl
+; X86-NEXT:    addl %edi, %edi
+; X86-NEXT:    shll %cl, %edi
+; X86-NEXT:    orl %eax, %edi
+; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movb %ch, %cl
+; X86-NEXT:    shrl %cl, (%esp) # 4-byte Folded Spill
+; X86-NEXT:    shrdl %cl, %edx, %ebp
+; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    shldl %cl, %eax, %ebx
-; X86-NEXT:    shll %cl, %eax
+; X86-NEXT:    addl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testb $32, %ch
-; X86-NEXT:    cmovnel %eax, %ebx
-; X86-NEXT:    orl %edx, %ebx
-; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
-; X86-NEXT:    # kill: def $al killed $al killed $eax
-; X86-NEXT:    addb $-64, %al
-; X86-NEXT:    movb %al, %cl
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movb %cl, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
-; X86-NEXT:    shrl %cl, %eax
+; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    movl $0, %edx
-; X86-NEXT:    cmovnel %edx, %eax
-; X86-NEXT:    movl (%esp), %edx # 4-byte Reload
-; X86-NEXT:    cmpb $64, %dl
-; X86-NEXT:    cmovbl %ebx, %eax
-; X86-NEXT:    movl $0, %ebx
-; X86-NEXT:    cmovael %ebx, %ebp
-; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testb $32, %ch
-; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    cmovnel %ebx, %ecx
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; X86-NEXT:    movl %edx, %ecx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    shrdl %cl, %edx, %ebx
-; X86-NEXT:    testb $32, (%esp) # 1-byte Folded Reload
-; X86-NEXT:    cmovnel {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    movzbl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 1-byte Folded Reload
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT:    shrdl %cl, %ebp, %edx
-; X86-NEXT:    testb $32, %cl
-; X86-NEXT:    cmovnel {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
-; X86-NEXT:    movl (%esp), %ecx # 4-byte Reload
-; X86-NEXT:    cmpb $64, %cl
-; X86-NEXT:    cmovael %edx, %ebx
-; X86-NEXT:    testb %cl, %cl
-; X86-NEXT:    cmovel {{[0-9]+}}(%esp), %ebx
-; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    cmovel {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    addl $-1, %ecx
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    adcl $-1, %ecx
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    adcl $-1, %ecx
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NEXT:    .p2align 4, 0x90
-; X86-NEXT:  .LBB4_4: # %udiv-do-while
+; X86-NEXT:  .LBB4_3: # %udiv-do-while
 ; X86-NEXT:    # =>This Inner Loop Header: Depth=1
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; X86-NEXT:    shldl $1, %ebp, %esi
-; X86-NEXT:    movl %esi, %edi
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ebx, %edi
+; X86-NEXT:    shldl $1, %ebx, (%esp) # 4-byte Folded Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    shldl $1, %ebx, %ebp
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    shldl $1, %eax, %ebx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    shldl $1, %ecx, %eax
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    shldl $1, %esi, %ecx
+; X86-NEXT:    shldl $1, %ebx, %edi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    orl %edx, %ecx
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    shldl $1, %edx, %ebx
+; X86-NEXT:    shldl $1, %eax, %edx
+; X86-NEXT:    shldl $1, %esi, %eax
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
+; X86-NEXT:    orl %ebp, %eax
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    shldl $1, %ecx, %esi
-; X86-NEXT:    orl %edx, %esi
+; X86-NEXT:    orl %ebp, %esi
 ; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    shldl $1, %esi, %ecx
-; X86-NEXT:    orl %edx, %ecx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    shldl $1, %eax, %ecx
+; X86-NEXT:    orl %ebp, %ecx
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    addl %esi, %esi
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    cmpl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
+; X86-NEXT:    addl %eax, %eax
+; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    cmpl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    sbbl %ebx, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    sbbl %ebp, %ecx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    sbbl %edi, %ecx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    sbbl (%esp), %ecx # 4-byte Folded Reload
 ; X86-NEXT:    sarl $31, %ecx
-; X86-NEXT:    movl %ecx, %edx
-; X86-NEXT:    andl $1, %edx
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ecx, %eax
+; X86-NEXT:    andl $1, %eax
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ecx, %ebp
+; X86-NEXT:    andl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl %ecx, %esi
 ; X86-NEXT:    andl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    movl %ecx, %edi
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %edi
-; X86-NEXT:    movl %ecx, %edx
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl %ecx, %eax
+; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    andl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    subl %ecx, %eax
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %edx, %ebx
+; X86-NEXT:    subl %ecx, %edx
+; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT:    sbbl %eax, %ebx
 ; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %edi, %ebp
-; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    sbbl %esi, %eax
-; X86-NEXT:    movl %eax, %esi
-; X86-NEXT:    movl (%esp), %edx # 4-byte Reload
-; X86-NEXT:    addl $-1, %edx
+; X86-NEXT:    sbbl %esi, %edi
+; X86-NEXT:    movl %edi, %ebx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    sbbl %ebp, (%esp) # 4-byte Folded Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    adcl $-1, %ecx
+; X86-NEXT:    addl $-1, %ecx
+; X86-NEXT:    adcl $-1, %edx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %edi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %ebp
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %edx, %eax
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    orl %ebp, %ecx
-; X86-NEXT:    movl %edx, (%esp) # 4-byte Spill
-; X86-NEXT:    orl %edi, %edx
-; X86-NEXT:    orl %ecx, %edx
-; X86-NEXT:    jne .LBB4_4
-; X86-NEXT:  # %bb.5:
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT:    orl %ebp, %eax
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    orl %edi, %ecx
+; X86-NEXT:    orl %eax, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    jne .LBB4_3
+; X86-NEXT:  # %bb.4:
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NEXT:  .LBB4_7: # %udiv-loop-exit
-; X86-NEXT:    shldl $1, %ebx, %edx
-; X86-NEXT:    orl %eax, %edx
-; X86-NEXT:    shldl $1, %esi, %ebx
-; X86-NEXT:    orl %eax, %ebx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
+; X86-NEXT:    shldl $1, %esi, %eax
+; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    shldl $1, %ebp, %esi
-; X86-NEXT:    orl %eax, %esi
-; X86-NEXT:    addl %ebp, %ebp
+; X86-NEXT:    orl %ecx, %esi
+; X86-NEXT:    shldl $1, %edi, %ebp
 ; X86-NEXT:    orl %ecx, %ebp
-; X86-NEXT:  .LBB4_8: # %udiv-end
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl %ebp, (%eax)
-; X86-NEXT:    movl %esi, 4(%eax)
-; X86-NEXT:    movl %ebx, 8(%eax)
-; X86-NEXT:    movl %edx, 12(%eax)
-; X86-NEXT:    movl %edx, %edi
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl %eax, %ecx
-; X86-NEXT:    imull %esi, %ecx
+; X86-NEXT:    addl %edi, %edi
+; X86-NEXT:    orl %edx, %edi
+; X86-NEXT:  .LBB4_9: # %udiv-end
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl %edi, (%ecx)
+; X86-NEXT:    movl %ebp, 4(%ecx)
+; X86-NEXT:    movl %esi, 8(%ecx)
+; X86-NEXT:    movl %eax, 12(%ecx)
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    imull %ebp, %edx
-; X86-NEXT:    addl %ecx, %edx
 ; X86-NEXT:    movl %edx, %ecx
-; X86-NEXT:    mull %ebp
+; X86-NEXT:    imull %ebp, %ecx
+; X86-NEXT:    movl %edx, %eax
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; X86-NEXT:    imull %edi, %ebp
+; X86-NEXT:    addl %ecx, %ebp
+; X86-NEXT:    mull %edi
 ; X86-NEXT:    movl %eax, (%esp) # 4-byte Spill
-; X86-NEXT:    addl %edx, %ecx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    movl %esi, %eax
-; X86-NEXT:    mull %ebx
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    imull %esi, %edi
-; X86-NEXT:    movl %edi, %edx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; X86-NEXT:    imull %edi, %ebx
-; X86-NEXT:    addl %edx, %ebx
-; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
-; X86-NEXT:    addl (%esp), %eax # 4-byte Folded Reload
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    adcl %ecx, %ebx
-; X86-NEXT:    movl %ebp, %eax
+; X86-NEXT:    addl %edx, %ebp
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; X86-NEXT:    movl %ebx, %eax
 ; X86-NEXT:    mull %esi
+; X86-NEXT:    movl %eax, %ecx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    imull %ebx, %eax
+; X86-NEXT:    movl %edi, %ebx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    imull %edi, %esi
+; X86-NEXT:    addl %eax, %esi
+; X86-NEXT:    addl %edx, %esi
+; X86-NEXT:    addl (%esp), %ecx # 4-byte Folded Reload
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    adcl %ebp, %esi
+; X86-NEXT:    movl %ebx, %eax
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    mull %ecx
 ; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %eax, (%esp) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    mull %esi
+; X86-NEXT:    mull %ecx
 ; X86-NEXT:    movl %edx, %ecx
-; X86-NEXT:    movl %eax, %esi
-; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
+; X86-NEXT:    movl %eax, %ebp
+; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Folded Reload
 ; X86-NEXT:    adcl $0, %ecx
-; X86-NEXT:    movl %ebp, %eax
+; X86-NEXT:    movl %ebx, %eax
 ; X86-NEXT:    mull %edi
-; X86-NEXT:    movl %edx, %ebp
-; X86-NEXT:    addl %esi, %eax
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    adcl %ecx, %ebp
+; X86-NEXT:    movl %edx, %ebx
+; X86-NEXT:    addl %ebp, %eax
+; X86-NEXT:    movl %eax, %ebp
+; X86-NEXT:    adcl %ecx, %ebx
 ; X86-NEXT:    setb %cl
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    mull %edi
-; X86-NEXT:    addl %ebp, %eax
+; X86-NEXT:    addl %ebx, %eax
 ; X86-NEXT:    movzbl %cl, %ecx
 ; X86-NEXT:    adcl %ecx, %edx
 ; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; X86-NEXT:    adcl %ebx, %edx
+; X86-NEXT:    adcl %esi, %edx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X86-NEXT:    subl (%esp), %ebx # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    sbbl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
+; X86-NEXT:    sbbl %ebp, %esi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    sbbl %eax, %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    sbbl %edx, %ecx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %ebx, (%eax)
 ; X86-NEXT:    movl %esi, 4(%eax)
 ; X86-NEXT:    movl %edi, 8(%eax)
 ; X86-NEXT:    movl %ecx, 12(%eax)
-; X86-NEXT:    addl $72, %esp
+; X86-NEXT:    addl $132, %esp
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx
 ; X86-NEXT:    popl %ebp
 ; X86-NEXT:    retl $4
 ;
 ; X64-LABEL: scalar_i128:
 ; X64:       # %bb.0:

llvm/test/CodeGen/X86/scheduler-backtracking.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64-- < %s -pre-RA-sched=list-ilp    | FileCheck %s --check-prefix=ILP
 ; RUN: llc -mtriple=x86_64-- < %s -pre-RA-sched=list-hybrid | FileCheck %s --check-prefix=HYBRID
 ; RUN: llc -mtriple=x86_64-- < %s -pre-RA-sched=list-burr   | FileCheck %s --check-prefix=BURR
 ; RUN: llc -mtriple=x86_64-- < %s -pre-RA-sched=source      | FileCheck %s --check-prefix=SRC
 ; FIXME: Fix machine verifier issues and remove -verify-machineinstrs=0. PR39452.
 ; RUN: llc -mtriple=x86_64-- < %s -pre-RA-sched=linearize -verify-machineinstrs=0 | FileCheck %s --check-prefix=LIN
 
 ; PR22304 https://llvm.org/bugs/show_bug.cgi?id=22304
 ; Tests checking backtracking in source scheduler. llc used to crash on them.
 
 define i256 @test1(i256 %a) nounwind {
 ; ILP-LABEL: test1:
 ; ILP:       # %bb.0:
-; ILP-NEXT:    pushq %r14
-; ILP-NEXT:    pushq %rbx
 ; ILP-NEXT:    movq %rdi, %rax
-; ILP-NEXT:    xorl %r8d, %r8d
-; ILP-NEXT:    addl %esi, %esi
-; ILP-NEXT:    leal 3(%rsi), %edx
-; ILP-NEXT:    movl $1, %r9d
-; ILP-NEXT:    xorl %r10d, %r10d
-; ILP-NEXT:    movl %edx, %ecx
-; ILP-NEXT:    shldq %cl, %r9, %r10
-; ILP-NEXT:    movl $1, %r11d
-; ILP-NEXT:    shlq %cl, %r11
-; ILP-NEXT:    leal -125(%rsi), %edi
-; ILP-NEXT:    xorl %ebx, %ebx
-; ILP-NEXT:    movl %edi, %ecx
-; ILP-NEXT:    shldq %cl, %r9, %rbx
-; ILP-NEXT:    testb $64, %dl
-; ILP-NEXT:    cmovneq %r11, %r10
-; ILP-NEXT:    cmovneq %r8, %r11
-; ILP-NEXT:    movl $1, %r14d
-; ILP-NEXT:    shlq %cl, %r14
-; ILP-NEXT:    movb $125, %cl
-; ILP-NEXT:    subb %sil, %cl
-; ILP-NEXT:    shrdq %cl, %r8, %r9
-; ILP-NEXT:    testb $64, %cl
-; ILP-NEXT:    cmovneq %r8, %r9
-; ILP-NEXT:    testb $64, %dil
-; ILP-NEXT:    cmovneq %r14, %rbx
-; ILP-NEXT:    cmovneq %r8, %r14
-; ILP-NEXT:    testb %dl, %dl
-; ILP-NEXT:    cmovsq %r8, %r10
-; ILP-NEXT:    cmovsq %r8, %r11
+; ILP-NEXT:    leal (%rsi,%rsi), %ecx
+; ILP-NEXT:    addb $3, %cl
+; ILP-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; ILP-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; ILP-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; ILP-NEXT:    movq $1, -{{[0-9]+}}(%rsp)
+; ILP-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; ILP-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; ILP-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; ILP-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; ILP-NEXT:    movl %ecx, %edx
+; ILP-NEXT:    shrb $3, %dl
+; ILP-NEXT:    andb $7, %cl
+; ILP-NEXT:    negb %dl
+; ILP-NEXT:    movsbq %dl, %rdx
+; ILP-NEXT:    movq -16(%rsp,%rdx), %rsi
+; ILP-NEXT:    movq -8(%rsp,%rdx), %rdi
+; ILP-NEXT:    shldq %cl, %rsi, %rdi
+; ILP-NEXT:    movq -32(%rsp,%rdx), %r8
+; ILP-NEXT:    movq -24(%rsp,%rdx), %rdx
+; ILP-NEXT:    movq %r8, %r9
+; ILP-NEXT:    shlq %cl, %r9
+; ILP-NEXT:    movq %rdx, %r10
+; ILP-NEXT:    shldq %cl, %r8, %r10
+; ILP-NEXT:    movq %rdi, 24(%rax)
 ; ILP-NEXT:    movq %r10, 8(%rax)
-; ILP-NEXT:    movq %r11, (%rax)
-; ILP-NEXT:    cmovnsq %r8, %rbx
-; ILP-NEXT:    cmoveq %r8, %rbx
-; ILP-NEXT:    movq %rbx, 24(%rax)
-; ILP-NEXT:    cmovnsq %r9, %r14
-; ILP-NEXT:    cmoveq %r8, %r14
-; ILP-NEXT:    movq %r14, 16(%rax)
-; ILP-NEXT:    popq %rbx
-; ILP-NEXT:    popq %r14
+; ILP-NEXT:    movq %r9, (%rax)
+; ILP-NEXT:    shlq %cl, %rsi
+; ILP-NEXT:    notb %cl
+; ILP-NEXT:    shrq %rdx
+; ILP-NEXT:    # kill: def $cl killed $cl killed $ecx
+; ILP-NEXT:    shrq %cl, %rdx
+; ILP-NEXT:    orq %rsi, %rdx
+; ILP-NEXT:    movq %rdx, 16(%rax)
 ; ILP-NEXT:    retq
 ;
 ; HYBRID-LABEL: test1:
 ; HYBRID:       # %bb.0:
-; HYBRID-NEXT:    pushq %rbx
 ; HYBRID-NEXT:    movq %rdi, %rax
+; HYBRID-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; HYBRID-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; HYBRID-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; HYBRID-NEXT:    movq $1, -{{[0-9]+}}(%rsp)
+; HYBRID-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; HYBRID-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; HYBRID-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; HYBRID-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
 ; HYBRID-NEXT:    addl %esi, %esi
-; HYBRID-NEXT:    movb $125, %cl
-; HYBRID-NEXT:    subb %sil, %cl
-; HYBRID-NEXT:    xorl %edi, %edi
-; HYBRID-NEXT:    movl $1, %r9d
-; HYBRID-NEXT:    movl $1, %r8d
-; HYBRID-NEXT:    shrdq %cl, %rdi, %r8
-; HYBRID-NEXT:    testb $64, %cl
-; HYBRID-NEXT:    cmovneq %rdi, %r8
-; HYBRID-NEXT:    leal 3(%rsi), %edx
-; HYBRID-NEXT:    xorl %r11d, %r11d
-; HYBRID-NEXT:    movl %edx, %ecx
-; HYBRID-NEXT:    shldq %cl, %r9, %r11
-; HYBRID-NEXT:    addb $-125, %sil
-; HYBRID-NEXT:    xorl %ebx, %ebx
+; HYBRID-NEXT:    addb $3, %sil
 ; HYBRID-NEXT:    movl %esi, %ecx
-; HYBRID-NEXT:    shldq %cl, %r9, %rbx
-; HYBRID-NEXT:    movl $1, %r10d
-; HYBRID-NEXT:    shlq %cl, %r10
-; HYBRID-NEXT:    testb $64, %sil
-; HYBRID-NEXT:    cmovneq %r10, %rbx
-; HYBRID-NEXT:    cmovneq %rdi, %r10
-; HYBRID-NEXT:    movl %edx, %ecx
-; HYBRID-NEXT:    shlq %cl, %r9
-; HYBRID-NEXT:    testb $64, %dl
-; HYBRID-NEXT:    cmovneq %r9, %r11
-; HYBRID-NEXT:    cmovneq %rdi, %r9
-; HYBRID-NEXT:    testb %dl, %dl
-; HYBRID-NEXT:    cmovsq %rdi, %r11
-; HYBRID-NEXT:    movq %r11, 8(%rax)
-; HYBRID-NEXT:    cmovsq %rdi, %r9
-; HYBRID-NEXT:    movq %r9, (%rax)
-; HYBRID-NEXT:    cmovnsq %rdi, %rbx
-; HYBRID-NEXT:    cmoveq %rdi, %rbx
-; HYBRID-NEXT:    movq %rbx, 24(%rax)
-; HYBRID-NEXT:    cmovnsq %r8, %r10
-; HYBRID-NEXT:    cmoveq %rdi, %r10
-; HYBRID-NEXT:    movq %r10, 16(%rax)
-; HYBRID-NEXT:    popq %rbx
+; HYBRID-NEXT:    andb $7, %cl
+; HYBRID-NEXT:    shrb $3, %sil
+; HYBRID-NEXT:    negb %sil
+; HYBRID-NEXT:    movsbq %sil, %rdx
+; HYBRID-NEXT:    movq -16(%rsp,%rdx), %rsi
+; HYBRID-NEXT:    movq -8(%rsp,%rdx), %rdi
+; HYBRID-NEXT:    shldq %cl, %rsi, %rdi
+; HYBRID-NEXT:    movq %rdi, 24(%rax)
+; HYBRID-NEXT:    movq -32(%rsp,%rdx), %rdi
+; HYBRID-NEXT:    movq -24(%rsp,%rdx), %rdx
+; HYBRID-NEXT:    movq %rdx, %r8
+; HYBRID-NEXT:    shldq %cl, %rdi, %r8
+; HYBRID-NEXT:    movq %r8, 8(%rax)
+; HYBRID-NEXT:    shlq %cl, %rdi
+; HYBRID-NEXT:    movq %rdi, (%rax)
+; HYBRID-NEXT:    shlq %cl, %rsi
+; HYBRID-NEXT:    notb %cl
+; HYBRID-NEXT:    shrq %rdx
+; HYBRID-NEXT:    shrq %cl, %rdx
+; HYBRID-NEXT:    orq %rsi, %rdx
+; HYBRID-NEXT:    movq %rdx, 16(%rax)
 ; HYBRID-NEXT:    retq
 ;
 ; BURR-LABEL: test1:
 ; BURR:       # %bb.0:
-; BURR-NEXT:    pushq %rbx
 ; BURR-NEXT:    movq %rdi, %rax
+; BURR-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; BURR-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; BURR-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; BURR-NEXT:    movq $1, -{{[0-9]+}}(%rsp)
+; BURR-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; BURR-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; BURR-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; BURR-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
 ; BURR-NEXT:    addl %esi, %esi
-; BURR-NEXT:    movb $125, %cl
-; BURR-NEXT:    subb %sil, %cl
-; BURR-NEXT:    xorl %edi, %edi
-; BURR-NEXT:    movl $1, %r9d
-; BURR-NEXT:    movl $1, %r8d
-; BURR-NEXT:    shrdq %cl, %rdi, %r8
-; BURR-NEXT:    testb $64, %cl
-; BURR-NEXT:    cmovneq %rdi, %r8
-; BURR-NEXT:    leal 3(%rsi), %edx
-; BURR-NEXT:    xorl %r11d, %r11d
-; BURR-NEXT:    movl %edx, %ecx
-; BURR-NEXT:    shldq %cl, %r9, %r11
-; BURR-NEXT:    addb $-125, %sil
-; BURR-NEXT:    xorl %ebx, %ebx
+; BURR-NEXT:    addb $3, %sil
 ; BURR-NEXT:    movl %esi, %ecx
-; BURR-NEXT:    shldq %cl, %r9, %rbx
-; BURR-NEXT:    movl $1, %r10d
-; BURR-NEXT:    shlq %cl, %r10
-; BURR-NEXT:    testb $64, %sil
-; BURR-NEXT:    cmovneq %r10, %rbx
-; BURR-NEXT:    cmovneq %rdi, %r10
-; BURR-NEXT:    movl %edx, %ecx
-; BURR-NEXT:    shlq %cl, %r9
-; BURR-NEXT:    testb $64, %dl
-; BURR-NEXT:    cmovneq %r9, %r11
-; BURR-NEXT:    cmovneq %rdi, %r9
-; BURR-NEXT:    testb %dl, %dl
-; BURR-NEXT:    cmovsq %rdi, %r11
-; BURR-NEXT:    movq %r11, 8(%rax)
-; BURR-NEXT:    cmovsq %rdi, %r9
-; BURR-NEXT:    movq %r9, (%rax)
-; BURR-NEXT:    cmovnsq %rdi, %rbx
-; BURR-NEXT:    cmoveq %rdi, %rbx
-; BURR-NEXT:    movq %rbx, 24(%rax)
-; BURR-NEXT:    cmovnsq %r8, %r10
-; BURR-NEXT:    cmoveq %rdi, %r10
-; BURR-NEXT:    movq %r10, 16(%rax)
-; BURR-NEXT:    popq %rbx
+; BURR-NEXT:    andb $7, %cl
+; BURR-NEXT:    shrb $3, %sil
+; BURR-NEXT:    negb %sil
+; BURR-NEXT:    movsbq %sil, %rdx
+; BURR-NEXT:    movq -16(%rsp,%rdx), %rsi
+; BURR-NEXT:    movq -8(%rsp,%rdx), %rdi
+; BURR-NEXT:    shldq %cl, %rsi, %rdi
+; BURR-NEXT:    movq %rdi, 24(%rax)
+; BURR-NEXT:    movq -32(%rsp,%rdx), %rdi
+; BURR-NEXT:    movq -24(%rsp,%rdx), %rdx
+; BURR-NEXT:    movq %rdx, %r8
+; BURR-NEXT:    shldq %cl, %rdi, %r8
+; BURR-NEXT:    movq %r8, 8(%rax)
+; BURR-NEXT:    shlq %cl, %rdi
+; BURR-NEXT:    movq %rdi, (%rax)
+; BURR-NEXT:    shlq %cl, %rsi
+; BURR-NEXT:    notb %cl
+; BURR-NEXT:    shrq %rdx
+; BURR-NEXT:    shrq %cl, %rdx
+; BURR-NEXT:    orq %rsi, %rdx
+; BURR-NEXT:    movq %rdx, 16(%rax)
 ; BURR-NEXT:    retq
 ;
 ; SRC-LABEL: test1:
 ; SRC:       # %bb.0:
-; SRC-NEXT:    pushq %rbx
 ; SRC-NEXT:    movq %rdi, %rax
 ; SRC-NEXT:    addl %esi, %esi
-; SRC-NEXT:    leal 3(%rsi), %edx
-; SRC-NEXT:    movb $125, %cl
-; SRC-NEXT:    subb %sil, %cl
-; SRC-NEXT:    xorl %r8d, %r8d
-; SRC-NEXT:    movl $1, %edi
-; SRC-NEXT:    movl $1, %r10d
-; SRC-NEXT:    shrdq %cl, %r8, %r10
-; SRC-NEXT:    testb $64, %cl
-; SRC-NEXT:    cmovneq %r8, %r10
-; SRC-NEXT:    addb $-125, %sil
-; SRC-NEXT:    xorl %r9d, %r9d
-; SRC-NEXT:    movl %esi, %ecx
-; SRC-NEXT:    shldq %cl, %rdi, %r9
-; SRC-NEXT:    xorl %r11d, %r11d
+; SRC-NEXT:    addb $3, %sil
+; SRC-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; SRC-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; SRC-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; SRC-NEXT:    movq $1, -{{[0-9]+}}(%rsp)
+; SRC-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; SRC-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; SRC-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; SRC-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; SRC-NEXT:    movl %esi, %edx
+; SRC-NEXT:    andb $7, %dl
+; SRC-NEXT:    shrb $3, %sil
+; SRC-NEXT:    negb %sil
+; SRC-NEXT:    movsbq %sil, %rsi
+; SRC-NEXT:    movq -16(%rsp,%rsi), %rdi
+; SRC-NEXT:    movq %rdi, %r8
+; SRC-NEXT:    movl %edx, %ecx
+; SRC-NEXT:    shlq %cl, %r8
+; SRC-NEXT:    notb %cl
+; SRC-NEXT:    movq -32(%rsp,%rsi), %r9
+; SRC-NEXT:    movq -24(%rsp,%rsi), %r10
+; SRC-NEXT:    movq %r10, %r11
+; SRC-NEXT:    shrq %r11
+; SRC-NEXT:    shrq %cl, %r11
+; SRC-NEXT:    orq %r8, %r11
+; SRC-NEXT:    movq -8(%rsp,%rsi), %rsi
 ; SRC-NEXT:    movl %edx, %ecx
-; SRC-NEXT:    shldq %cl, %rdi, %r11
-; SRC-NEXT:    movl $1, %ebx
-; SRC-NEXT:    shlq %cl, %rbx
-; SRC-NEXT:    testb $64, %dl
-; SRC-NEXT:    cmovneq %rbx, %r11
-; SRC-NEXT:    cmovneq %r8, %rbx
-; SRC-NEXT:    movl %esi, %ecx
+; SRC-NEXT:    shldq %cl, %rdi, %rsi
+; SRC-NEXT:    movq %r9, %rdi
 ; SRC-NEXT:    shlq %cl, %rdi
-; SRC-NEXT:    testb $64, %sil
-; SRC-NEXT:    cmovneq %rdi, %r9
-; SRC-NEXT:    cmovneq %r8, %rdi
-; SRC-NEXT:    testb %dl, %dl
-; SRC-NEXT:    cmovnsq %r10, %rdi
-; SRC-NEXT:    cmoveq %r8, %rdi
-; SRC-NEXT:    cmovnsq %r8, %r9
-; SRC-NEXT:    cmoveq %r8, %r9
-; SRC-NEXT:    cmovsq %r8, %r11
-; SRC-NEXT:    cmovsq %r8, %rbx
-; SRC-NEXT:    movq %r11, 8(%rax)
-; SRC-NEXT:    movq %rbx, (%rax)
-; SRC-NEXT:    movq %r9, 24(%rax)
-; SRC-NEXT:    movq %rdi, 16(%rax)
-; SRC-NEXT:    popq %rbx
+; SRC-NEXT:    shldq %cl, %r9, %r10
+; SRC-NEXT:    movq %rsi, 24(%rax)
+; SRC-NEXT:    movq %r10, 8(%rax)
+; SRC-NEXT:    movq %rdi, (%rax)
+; SRC-NEXT:    movq %r11, 16(%rax)
 ; SRC-NEXT:    retq
 ;
 ; LIN-LABEL: test1:
 ; LIN:       # %bb.0:
 ; LIN-NEXT:    movq %rdi, %rax
-; LIN-NEXT:    xorl %edi, %edi
-; LIN-NEXT:    movl $1, %r8d
-; LIN-NEXT:    addl %esi, %esi
-; LIN-NEXT:    leal 3(%rsi), %ecx
-; LIN-NEXT:    movl $1, %edx
-; LIN-NEXT:    shlq %cl, %rdx
-; LIN-NEXT:    testb $64, %cl
-; LIN-NEXT:    movq %rdx, %r9
-; LIN-NEXT:    cmovneq %rdi, %r9
-; LIN-NEXT:    testb %cl, %cl
-; LIN-NEXT:    cmovsq %rdi, %r9
-; LIN-NEXT:    movq %r9, (%rax)
-; LIN-NEXT:    xorl %r9d, %r9d
-; LIN-NEXT:    # kill: def $cl killed $cl killed $ecx
-; LIN-NEXT:    shldq %cl, %r8, %r9
-; LIN-NEXT:    cmovneq %rdx, %r9
-; LIN-NEXT:    cmovsq %rdi, %r9
-; LIN-NEXT:    movq %r9, 8(%rax)
-; LIN-NEXT:    leal -125(%rsi), %edx
-; LIN-NEXT:    movl $1, %r9d
+; LIN-NEXT:    leal (%rsi,%rsi), %edx
+; LIN-NEXT:    addb $3, %dl
+; LIN-NEXT:    movl %edx, %ecx
+; LIN-NEXT:    shrb $3, %cl
+; LIN-NEXT:    negb %cl
+; LIN-NEXT:    movsbq %cl, %rsi
+; LIN-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; LIN-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; LIN-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; LIN-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; LIN-NEXT:    movq $1, -{{[0-9]+}}(%rsp)
+; LIN-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; LIN-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; LIN-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; LIN-NEXT:    movq -32(%rsp,%rsi), %rdi
+; LIN-NEXT:    andb $7, %dl
+; LIN-NEXT:    movq %rdi, %r8
 ; LIN-NEXT:    movl %edx, %ecx
+; LIN-NEXT:    shlq %cl, %r8
+; LIN-NEXT:    movq %r8, (%rax)
+; LIN-NEXT:    movq -24(%rsp,%rsi), %r8
+; LIN-NEXT:    movq %r8, %r9
+; LIN-NEXT:    shldq %cl, %rdi, %r9
+; LIN-NEXT:    movq %r9, 8(%rax)
+; LIN-NEXT:    movq -16(%rsp,%rsi), %rdi
+; LIN-NEXT:    movq %rdi, %r9
 ; LIN-NEXT:    shlq %cl, %r9
-; LIN-NEXT:    testb $64, %dl
-; LIN-NEXT:    movq %r9, %r10
-; LIN-NEXT:    cmovneq %rdi, %r10
-; LIN-NEXT:    movb $125, %cl
-; LIN-NEXT:    subb %sil, %cl
-; LIN-NEXT:    movl $1, %esi
-; LIN-NEXT:    shrdq %cl, %rdi, %rsi
-; LIN-NEXT:    testb $64, %cl
-; LIN-NEXT:    cmovneq %rdi, %rsi
-; LIN-NEXT:    cmovsq %r10, %rsi
-; LIN-NEXT:    cmoveq %rdi, %rsi
-; LIN-NEXT:    movq %rsi, 16(%rax)
-; LIN-NEXT:    xorl %esi, %esi
+; LIN-NEXT:    shrq %r8
+; LIN-NEXT:    notb %cl
+; LIN-NEXT:    shrq %cl, %r8
+; LIN-NEXT:    orq %r9, %r8
+; LIN-NEXT:    movq %r8, 16(%rax)
+; LIN-NEXT:    movq -8(%rsp,%rsi), %rsi
 ; LIN-NEXT:    movl %edx, %ecx
-; LIN-NEXT:    shldq %cl, %r8, %rsi
-; LIN-NEXT:    cmovneq %r9, %rsi
-; LIN-NEXT:    cmovnsq %rdi, %rsi
-; LIN-NEXT:    cmoveq %rdi, %rsi
+; LIN-NEXT:    shldq %cl, %rdi, %rsi
 ; LIN-NEXT:    movq %rsi, 24(%rax)
 ; LIN-NEXT:    retq
   %b = add i256 %a, 1
   %m = shl i256 %b, 1
   %p = add i256 %m, 1
   %v = lshr i256 %b, %p
   %t = trunc i256 %v to i1
   %c = shl i256 1, %p

llvm/test/CodeGen/X86/shift-i128.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-- | FileCheck %s --check-prefixes=ALL,i686
 ; RUN: llc < %s -mtriple=x86_64-- | FileCheck %s --check-prefixes=ALL,x86_64
 
 ;
 ; Scalars
 ;
 
 define void @test_lshr_i128(i128 %x, i128 %a, ptr nocapture %r) nounwind {
 ; i686-LABEL: test_lshr_i128:
 ; i686:       # %bb.0: # %entry
 ; i686-NEXT:    pushl %ebp
 ; i686-NEXT:    pushl %ebx
 ; i686-NEXT:    pushl %edi
 ; i686-NEXT:    pushl %esi
-; i686-NEXT:    subl $20, %esp
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl %ebp, %esi
-; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    shrdl %cl, %edi, %esi
-; i686-NEXT:    shrl %cl, %edx
-; i686-NEXT:    shrl %cl, %edi
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    jne .LBB0_1
-; i686-NEXT:  # %bb.2: # %entry
-; i686-NEXT:    movl %edx, (%esp) # 4-byte Spill
-; i686-NEXT:    jmp .LBB0_3
-; i686-NEXT:  .LBB0_1:
-; i686-NEXT:    movl %edi, %esi
-; i686-NEXT:    movl $0, (%esp) # 4-byte Folded Spill
-; i686-NEXT:    xorl %edi, %edi
-; i686-NEXT:  .LBB0_3: # %entry
-; i686-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %eax, %edx
-; i686-NEXT:    subb $64, %dl
-; i686-NEXT:    jb .LBB0_5
-; i686-NEXT:  # %bb.4: # %entry
-; i686-NEXT:    xorl %edi, %edi
-; i686-NEXT:  .LBB0_5: # %entry
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    negb %dl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    shldl %cl, %ebp, %edi
-; i686-NEXT:    movl %ebp, %esi
-; i686-NEXT:    shll %cl, %esi
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    movl %esi, %ebx
-; i686-NEXT:    jne .LBB0_7
-; i686-NEXT:  # %bb.6: # %entry
-; i686-NEXT:    movl %edi, %ebx
-; i686-NEXT:  .LBB0_7: # %entry
-; i686-NEXT:    movb %al, %ah
-; i686-NEXT:    addb $-64, %ah
+; i686-NEXT:    subl $32, %esp
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movb %ah, %cl
-; i686-NEXT:    shrl %cl, %edi
-; i686-NEXT:    testb $32, %ah
-; i686-NEXT:    movl $0, %ecx
-; i686-NEXT:    jne .LBB0_9
-; i686-NEXT:  # %bb.8: # %entry
-; i686-NEXT:    movl %edi, %ecx
-; i686-NEXT:  .LBB0_9: # %entry
-; i686-NEXT:    cmpb $64, %al
-; i686-NEXT:    jb .LBB0_10
-; i686-NEXT:  # %bb.11: # %entry
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    jmp .LBB0_12
-; i686-NEXT:  .LBB0_10:
-; i686-NEXT:    movl (%esp), %ecx # 4-byte Reload
-; i686-NEXT:    orl %ebx, %ecx
-; i686-NEXT:  .LBB0_12: # %entry
-; i686-NEXT:    movl %ecx, (%esp) # 4-byte Spill
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    jne .LBB0_14
-; i686-NEXT:  # %bb.13: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB0_14: # %entry
-; i686-NEXT:    movl %ebx, %edx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %edi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %eax, (%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ecx, %eax
+; i686-NEXT:    andb $7, %al
+; i686-NEXT:    shrb $3, %cl
+; i686-NEXT:    andb $15, %cl
+; i686-NEXT:    movzbl %cl, %ebp
+; i686-NEXT:    movl 4(%esp,%ebp), %edx
+; i686-NEXT:    movl %edx, %esi
 ; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    shrdl %cl, %esi, %edx
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    jne .LBB0_16
-; i686-NEXT:  # %bb.15: # %entry
-; i686-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB0_16: # %entry
-; i686-NEXT:    movb %ah, %cl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    shrdl %cl, %edx, %ebp
-; i686-NEXT:    testb $32, %ah
-; i686-NEXT:    jne .LBB0_18
-; i686-NEXT:  # %bb.17: # %entry
-; i686-NEXT:    movl %ebp, %edi
-; i686-NEXT:  .LBB0_18: # %entry
-; i686-NEXT:    cmpb $64, %al
-; i686-NEXT:    jae .LBB0_20
-; i686-NEXT:  # %bb.19:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
-; i686-NEXT:  .LBB0_20: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    testb %al, %al
-; i686-NEXT:    je .LBB0_22
-; i686-NEXT:  # %bb.21: # %entry
-; i686-NEXT:    movl %edi, %ebx
-; i686-NEXT:    movl (%esp), %esi # 4-byte Reload
-; i686-NEXT:  .LBB0_22: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl %eax, 12(%ecx)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl %eax, 8(%ecx)
-; i686-NEXT:    movl %esi, 4(%ecx)
-; i686-NEXT:    movl %ebx, (%ecx)
-; i686-NEXT:    addl $20, %esp
+; i686-NEXT:    shrl %cl, %esi
+; i686-NEXT:    notb %cl
+; i686-NEXT:    movl 8(%esp,%ebp), %ebx
+; i686-NEXT:    leal (%ebx,%ebx), %edi
+; i686-NEXT:    shll %cl, %edi
+; i686-NEXT:    orl %esi, %edi
+; i686-NEXT:    movl (%esp,%ebp), %esi
+; i686-NEXT:    movl 12(%esp,%ebp), %ebp
+; i686-NEXT:    movl %eax, %ecx
+; i686-NEXT:    shrdl %cl, %ebp, %ebx
+; i686-NEXT:    shrdl %cl, %edx, %esi
+; i686-NEXT:    shrl %cl, %ebp
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; i686-NEXT:    movl %ebp, 12(%eax)
+; i686-NEXT:    movl %ebx, 8(%eax)
+; i686-NEXT:    movl %esi, (%eax)
+; i686-NEXT:    movl %edi, 4(%eax)
+; i686-NEXT:    addl $32, %esp
 ; i686-NEXT:    popl %esi
 ; i686-NEXT:    popl %edi
 ; i686-NEXT:    popl %ebx
 ; i686-NEXT:    popl %ebp
 ; i686-NEXT:    retl
 ;
 ; x86_64-LABEL: test_lshr_i128:
 ; x86_64:       # %bb.0: # %entry

 
 define void @test_ashr_i128(i128 %x, i128 %a, ptr nocapture %r) nounwind {
 ; i686-LABEL: test_ashr_i128:
 ; i686:       # %bb.0: # %entry
 ; i686-NEXT:    pushl %ebp
 ; i686-NEXT:    pushl %ebx
 ; i686-NEXT:    pushl %edi
 ; i686-NEXT:    pushl %esi
-; i686-NEXT:    subl $24, %esp
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl %ebp, %esi
-; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    shrdl %cl, %ebx, %esi
-; i686-NEXT:    shrl %cl, %edx
-; i686-NEXT:    movl %ebx, %edi
-; i686-NEXT:    sarl %cl, %edi
-; i686-NEXT:    sarl $31, %ebx
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    jne .LBB1_1
-; i686-NEXT:  # %bb.2: # %entry
-; i686-NEXT:    movl %edx, (%esp) # 4-byte Spill
-; i686-NEXT:    jmp .LBB1_3
-; i686-NEXT:  .LBB1_1:
-; i686-NEXT:    movl %edi, %esi
-; i686-NEXT:    movl $0, (%esp) # 4-byte Folded Spill
-; i686-NEXT:    movl %ebx, %edi
-; i686-NEXT:  .LBB1_3: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %eax, %edx
-; i686-NEXT:    subb $64, %dl
-; i686-NEXT:    jb .LBB1_5
-; i686-NEXT:  # %bb.4: # %entry
-; i686-NEXT:    movl %ebx, %edi
-; i686-NEXT:  .LBB1_5: # %entry
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    negb %dl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    shldl %cl, %ebp, %edi
-; i686-NEXT:    movl %ebp, %esi
-; i686-NEXT:    shll %cl, %esi
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    movl %esi, %ecx
-; i686-NEXT:    jne .LBB1_7
-; i686-NEXT:  # %bb.6: # %entry
-; i686-NEXT:    movl %edi, %ecx
-; i686-NEXT:  .LBB1_7: # %entry
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movb %al, %ah
-; i686-NEXT:    addb $-64, %ah
+; i686-NEXT:    subl $32, %esp
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movb %ah, %cl
-; i686-NEXT:    sarl %cl, %edi
-; i686-NEXT:    testb $32, %ah
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:    jne .LBB1_9
-; i686-NEXT:  # %bb.8: # %entry
-; i686-NEXT:    movl %edi, %ecx
-; i686-NEXT:  .LBB1_9: # %entry
-; i686-NEXT:    cmpb $64, %al
-; i686-NEXT:    jb .LBB1_10
-; i686-NEXT:  # %bb.11: # %entry
-; i686-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    jmp .LBB1_12
-; i686-NEXT:  .LBB1_10:
-; i686-NEXT:    movl (%esp), %ecx # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; i686-NEXT:  .LBB1_12: # %entry
-; i686-NEXT:    movl %ecx, (%esp) # 4-byte Spill
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    jne .LBB1_14
-; i686-NEXT:  # %bb.13: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB1_14: # %entry
-; i686-NEXT:    movl %ebx, %edx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %edi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %eax, (%esp)
+; i686-NEXT:    sarl $31, %ebx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ecx, %eax
+; i686-NEXT:    andb $7, %al
+; i686-NEXT:    shrb $3, %cl
+; i686-NEXT:    andb $15, %cl
+; i686-NEXT:    movzbl %cl, %ebp
+; i686-NEXT:    movl 4(%esp,%ebp), %edx
+; i686-NEXT:    movl %edx, %esi
 ; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    shrdl %cl, %esi, %edx
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    jne .LBB1_16
-; i686-NEXT:  # %bb.15: # %entry
-; i686-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB1_16: # %entry
-; i686-NEXT:    movb %ah, %cl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    shrdl %cl, %edx, %ebp
-; i686-NEXT:    testb $32, %ah
-; i686-NEXT:    jne .LBB1_18
-; i686-NEXT:  # %bb.17: # %entry
-; i686-NEXT:    movl %ebp, %edi
-; i686-NEXT:  .LBB1_18: # %entry
-; i686-NEXT:    cmpb $64, %al
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    jae .LBB1_20
-; i686-NEXT:  # %bb.19:
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; i686-NEXT:    movl %ecx, %edi
-; i686-NEXT:  .LBB1_20: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    testb %al, %al
-; i686-NEXT:    je .LBB1_22
-; i686-NEXT:  # %bb.21: # %entry
-; i686-NEXT:    movl %edi, %ebx
-; i686-NEXT:    movl (%esp), %esi # 4-byte Reload
-; i686-NEXT:  .LBB1_22: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl %eax, 12(%ecx)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl %eax, 8(%ecx)
-; i686-NEXT:    movl %esi, 4(%ecx)
-; i686-NEXT:    movl %ebx, (%ecx)
-; i686-NEXT:    addl $24, %esp
+; i686-NEXT:    shrl %cl, %esi
+; i686-NEXT:    notb %cl
+; i686-NEXT:    movl 8(%esp,%ebp), %ebx
+; i686-NEXT:    leal (%ebx,%ebx), %edi
+; i686-NEXT:    shll %cl, %edi
+; i686-NEXT:    orl %esi, %edi
+; i686-NEXT:    movl (%esp,%ebp), %esi
+; i686-NEXT:    movl 12(%esp,%ebp), %ebp
+; i686-NEXT:    movl %eax, %ecx
+; i686-NEXT:    shrdl %cl, %ebp, %ebx
+; i686-NEXT:    shrdl %cl, %edx, %esi
+; i686-NEXT:    sarl %cl, %ebp
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; i686-NEXT:    movl %ebp, 12(%eax)
+; i686-NEXT:    movl %ebx, 8(%eax)
+; i686-NEXT:    movl %esi, (%eax)
+; i686-NEXT:    movl %edi, 4(%eax)
+; i686-NEXT:    addl $32, %esp
 ; i686-NEXT:    popl %esi
 ; i686-NEXT:    popl %edi
 ; i686-NEXT:    popl %ebx
 ; i686-NEXT:    popl %ebp
 ; i686-NEXT:    retl
 ;
 ; x86_64-LABEL: test_ashr_i128:
 ; x86_64:       # %bb.0: # %entry

 
 define void @test_shl_i128(i128 %x, i128 %a, ptr nocapture %r) nounwind {
 ; i686-LABEL: test_shl_i128:
 ; i686:       # %bb.0: # %entry
 ; i686-NEXT:    pushl %ebp
 ; i686-NEXT:    pushl %ebx
 ; i686-NEXT:    pushl %edi
 ; i686-NEXT:    pushl %esi
-; i686-NEXT:    subl $20, %esp
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; i686-NEXT:    subl $32, %esp
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %edi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, (%esp)
+; i686-NEXT:    movl %ecx, %eax
+; i686-NEXT:    andb $7, %al
+; i686-NEXT:    shrb $3, %cl
+; i686-NEXT:    andb $15, %cl
+; i686-NEXT:    negb %cl
+; i686-NEXT:    movsbl %cl, %ebp
+; i686-NEXT:    movl 24(%esp,%ebp), %edx
+; i686-NEXT:    movl %edx, %ebx
 ; i686-NEXT:    movl %eax, %ecx
 ; i686-NEXT:    shll %cl, %ebx
-; i686-NEXT:    movl %ebp, %esi
-; i686-NEXT:    shll %cl, %esi
-; i686-NEXT:    movl %edi, %edx
-; i686-NEXT:    shldl %cl, %ebp, %edx
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    jne .LBB2_1
-; i686-NEXT:  # %bb.2: # %entry
-; i686-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ebx, (%esp) # 4-byte Spill
-; i686-NEXT:    jmp .LBB2_3
-; i686-NEXT:  .LBB2_1:
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl $0, (%esp) # 4-byte Folded Spill
-; i686-NEXT:    xorl %esi, %esi
-; i686-NEXT:  .LBB2_3: # %entry
-; i686-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %eax, %edx
-; i686-NEXT:    subb $64, %dl
-; i686-NEXT:    jb .LBB2_5
-; i686-NEXT:  # %bb.4: # %entry
-; i686-NEXT:    xorl %esi, %esi
-; i686-NEXT:  .LBB2_5: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    negb %dl
+; i686-NEXT:    notb %cl
+; i686-NEXT:    movl 20(%esp,%ebp), %edi
 ; i686-NEXT:    movl %edi, %esi
-; i686-NEXT:    movl %edx, %ecx
+; i686-NEXT:    shrl %esi
 ; i686-NEXT:    shrl %cl, %esi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    shrdl %cl, %edi, %ebx
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    movl %esi, %ebp
-; i686-NEXT:    jne .LBB2_7
-; i686-NEXT:  # %bb.6: # %entry
-; i686-NEXT:    movl %ebx, %ebp
-; i686-NEXT:  .LBB2_7: # %entry
-; i686-NEXT:    movb %al, %ah
-; i686-NEXT:    addb $-64, %ah
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    movb %ah, %cl
-; i686-NEXT:    shll %cl, %ebx
-; i686-NEXT:    testb $32, %ah
-; i686-NEXT:    movl $0, %ecx
-; i686-NEXT:    jne .LBB2_9
-; i686-NEXT:  # %bb.8: # %entry
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:  .LBB2_9: # %entry
-; i686-NEXT:    cmpb $64, %al
-; i686-NEXT:    jb .LBB2_10
-; i686-NEXT:  # %bb.11: # %entry
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    jmp .LBB2_12
-; i686-NEXT:  .LBB2_10:
-; i686-NEXT:    movl (%esp), %ecx # 4-byte Reload
-; i686-NEXT:    orl %ebp, %ecx
-; i686-NEXT:  .LBB2_12: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl %ecx, (%esp) # 4-byte Spill
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    jne .LBB2_14
-; i686-NEXT:  # %bb.13: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB2_14: # %entry
-; i686-NEXT:    movl %edx, %esi
+; i686-NEXT:    orl %ebx, %esi
+; i686-NEXT:    movl 16(%esp,%ebp), %ebx
+; i686-NEXT:    movl 28(%esp,%ebp), %ebp
 ; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    shldl %cl, %ebp, %esi
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    jne .LBB2_16
-; i686-NEXT:  # %bb.15: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB2_16: # %entry
-; i686-NEXT:    movb %ah, %cl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    shldl %cl, %esi, %edi
-; i686-NEXT:    testb $32, %ah
-; i686-NEXT:    jne .LBB2_18
-; i686-NEXT:  # %bb.17: # %entry
-; i686-NEXT:    movl %edi, %ebx
-; i686-NEXT:  .LBB2_18: # %entry
-; i686-NEXT:    cmpb $64, %al
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; i686-NEXT:    jae .LBB2_20
-; i686-NEXT:  # %bb.19:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
-; i686-NEXT:  .LBB2_20: # %entry
+; i686-NEXT:    shldl %cl, %edx, %ebp
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    testb %al, %al
-; i686-NEXT:    je .LBB2_22
-; i686-NEXT:  # %bb.21: # %entry
+; i686-NEXT:    movl %ebp, 12(%ecx)
 ; i686-NEXT:    movl %ebx, %edx
-; i686-NEXT:    movl (%esp), %ebp # 4-byte Reload
-; i686-NEXT:  .LBB2_22: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl %eax, 4(%ecx)
-; i686-NEXT:    movl %esi, (%ecx)
-; i686-NEXT:    movl %edx, 12(%ecx)
-; i686-NEXT:    movl %ebp, 8(%ecx)
-; i686-NEXT:    addl $20, %esp
+; i686-NEXT:    movl %eax, %ecx
+; i686-NEXT:    shll %cl, %edx
+; i686-NEXT:    shldl %cl, %ebx, %edi
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; i686-NEXT:    movl %edi, 4(%eax)
+; i686-NEXT:    movl %edx, (%eax)
+; i686-NEXT:    movl %esi, 8(%eax)
+; i686-NEXT:    addl $32, %esp
 ; i686-NEXT:    popl %esi
 ; i686-NEXT:    popl %edi
 ; i686-NEXT:    popl %ebx
 ; i686-NEXT:    popl %ebp
 ; i686-NEXT:    retl
 ;
 ; x86_64-LABEL: test_shl_i128:
 ; x86_64:       # %bb.0: # %entry

 
 define void @test_lshr_v2i128(<2 x i128> %x, <2 x i128> %a, ptr nocapture %r) nounwind {
 ; i686-LABEL: test_lshr_v2i128:
 ; i686:       # %bb.0: # %entry
 ; i686-NEXT:    pushl %ebp
 ; i686-NEXT:    pushl %ebx
 ; i686-NEXT:    pushl %edi
 ; i686-NEXT:    pushl %esi
-; i686-NEXT:    subl $68, %esp
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; i686-NEXT:    subl $100, %esp
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    movl %ebx, %edi
-; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    shrl %cl, %edi
-; i686-NEXT:    movl %esi, %ebp
-; i686-NEXT:    shrl %cl, %ebp
-; i686-NEXT:    shrdl %cl, %esi, %edx
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    jne .LBB6_1
-; i686-NEXT:  # %bb.2: # %entry
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; i686-NEXT:    movl %ebp, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; i686-NEXT:    movl %ebp, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; i686-NEXT:    movl %ebp, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; i686-NEXT:    movl %ebp, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %edi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %esi, %ecx
+; i686-NEXT:    andl $7, %ecx
+; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    shrl $3, %esi
+; i686-NEXT:    andl $15, %esi
+; i686-NEXT:    movl 40(%esp,%esi), %eax
+; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    shrl %cl, %eax
+; i686-NEXT:    notl %ecx
+; i686-NEXT:    movl 44(%esp,%esi), %edx
 ; i686-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    addl %edx, %edx
+; i686-NEXT:    # kill: def $cl killed $cl killed $ecx
+; i686-NEXT:    shll %cl, %edx
+; i686-NEXT:    orl %eax, %edx
+; i686-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    movl 36(%esp,%esi), %eax
+; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ebx, %edx
+; i686-NEXT:    andl $7, %edx
+; i686-NEXT:    shrl $3, %ebx
+; i686-NEXT:    andl $15, %ebx
+; i686-NEXT:    movl 72(%esp,%ebx), %ebp
 ; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    jmp .LBB6_3
-; i686-NEXT:  .LBB6_1:
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:  .LBB6_3: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    shrdl %cl, %ebx, %esi
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    jne .LBB6_5
-; i686-NEXT:  # %bb.4: # %entry
-; i686-NEXT:    movl %esi, %edi
-; i686-NEXT:  .LBB6_5: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    shrl %cl, %ebx
 ; i686-NEXT:    shrl %cl, %ebp
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    subl $64, %ecx
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    sbbl $0, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    sbbl $0, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    sbbl $0, %ecx
-; i686-NEXT:    setb {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Spill
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    movl $0, %ecx
-; i686-NEXT:    jne .LBB6_7
-; i686-NEXT:  # %bb.6: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:  .LBB6_7: # %entry
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    shrdl %cl, %ebp, %esi
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    jne .LBB6_9
-; i686-NEXT:  # %bb.8: # %entry
-; i686-NEXT:    movl %esi, %ebx
-; i686-NEXT:  .LBB6_9: # %entry
-; i686-NEXT:    movl %edi, %esi
-; i686-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    shrl %cl, %ebp
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    movl $0, %ecx
-; i686-NEXT:    jne .LBB6_11
-; i686-NEXT:  # %bb.10: # %entry
-; i686-NEXT:    movl %ebp, %ecx
-; i686-NEXT:  .LBB6_11: # %entry
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movb $64, %cl
-; i686-NEXT:    subb %dl, %cl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    shldl %cl, %ebx, %edi
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ebx, %edi
+; i686-NEXT:    notl %ecx
+; i686-NEXT:    movl 76(%esp,%ebx), %eax
+; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    leal (%eax,%eax), %edi
+; i686-NEXT:    # kill: def $cl killed $cl killed $ecx
 ; i686-NEXT:    shll %cl, %edi
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    movb $64, %bl
-; i686-NEXT:    jne .LBB6_12
-; i686-NEXT:  # %bb.13: # %entry
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    jmp .LBB6_14
-; i686-NEXT:  .LBB6_12:
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:  .LBB6_14: # %entry
-; i686-NEXT:    movl %esi, %edi
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    shrdl %cl, %ebp, %esi
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    jne .LBB6_16
-; i686-NEXT:  # %bb.15: # %entry
+; i686-NEXT:    orl %ebp, %edi
+; i686-NEXT:    movl 48(%esp,%esi), %esi
 ; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB6_16: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    subb %al, %bl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:    shll %cl, %ebp
-; i686-NEXT:    testb $32, %bl
-; i686-NEXT:    movl $0, %ecx
-; i686-NEXT:    jne .LBB6_18
-; i686-NEXT:  # %bb.17: # %entry
-; i686-NEXT:    movl %ebp, %ecx
-; i686-NEXT:  .LBB6_18: # %entry
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    subl $64, %ecx
-; i686-NEXT:    sbbl $0, %esi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    sbbl $0, %esi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    sbbl $0, %esi
-; i686-NEXT:    setae %bh
-; i686-NEXT:    jb .LBB6_20
-; i686-NEXT:  # %bb.19: # %entry
-; i686-NEXT:    xorl %edi, %edi
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:  .LBB6_20: # %entry
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    shrdl %cl, %esi, %edi
-; i686-NEXT:    shrl %cl, %esi
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    jne .LBB6_22
-; i686-NEXT:  # %bb.21: # %entry
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB6_22: # %entry
-; i686-NEXT:    testb %bh, %bh
-; i686-NEXT:    jne .LBB6_24
-; i686-NEXT:  # %bb.23:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB6_24: # %entry
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    movl $0, %ecx
-; i686-NEXT:    jne .LBB6_26
-; i686-NEXT:  # %bb.25: # %entry
-; i686-NEXT:    movl %esi, %ecx
-; i686-NEXT:  .LBB6_26: # %entry
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    shldl %cl, %edi, %esi
-; i686-NEXT:    testb $32, %bl
-; i686-NEXT:    jne .LBB6_28
-; i686-NEXT:  # %bb.27: # %entry
-; i686-NEXT:    movl %esi, %ebp
-; i686-NEXT:  .LBB6_28: # %entry
-; i686-NEXT:    testb %bh, %bh
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    jne .LBB6_30
-; i686-NEXT:  # %bb.29:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    orl %ebp, %ecx
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB6_30: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    jne .LBB6_32
-; i686-NEXT:  # %bb.31: # %entry
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:  .LBB6_32: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    shrdl %cl, %ebp, %edi
-; i686-NEXT:    movl %edi, %ebp
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; i686-NEXT:    je .LBB6_33
-; i686-NEXT:  # %bb.34: # %entry
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    jne .LBB6_35
-; i686-NEXT:  .LBB6_36: # %entry
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    je .LBB6_38
-; i686-NEXT:  .LBB6_37:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB6_38: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    orl %ecx, %edx
-; i686-NEXT:    je .LBB6_40
-; i686-NEXT:  # %bb.39: # %entry
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    shrdl %cl, %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; i686-NEXT:    movl 68(%esp,%ebx), %ecx
+; i686-NEXT:    movl %ecx, (%esp) # 4-byte Spill
+; i686-NEXT:    movl 80(%esp,%ebx), %esi
+; i686-NEXT:    movl %edx, %ecx
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; i686-NEXT:  .LBB6_40: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    orl %edx, %eax
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    je .LBB6_42
-; i686-NEXT:  # %bb.41: # %entry
+; i686-NEXT:    shrdl %cl, %esi, %ebx
+; i686-NEXT:    movl %eax, %ecx
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
+; i686-NEXT:    shrdl %cl, %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
+; i686-NEXT:    shrl %cl, %ebp
+; i686-NEXT:    movl %edx, %ecx
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; i686-NEXT:  .LBB6_42: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; i686-NEXT:    movl %edx, 28(%ecx)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; i686-NEXT:    movl %edx, 24(%ecx)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; i686-NEXT:    movl %edx, 12(%ecx)
+; i686-NEXT:    shrdl %cl, %eax, (%esp) # 4-byte Folded Spill
+; i686-NEXT:    movl %edx, %ecx
+; i686-NEXT:    shrl %cl, %esi
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; i686-NEXT:    movl %esi, 28(%ecx)
+; i686-NEXT:    movl %ebx, 24(%ecx)
+; i686-NEXT:    movl (%esp), %eax # 4-byte Reload
+; i686-NEXT:    movl %eax, 16(%ecx)
+; i686-NEXT:    movl %ebp, 12(%ecx)
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; i686-NEXT:    movl %edx, 8(%ecx)
-; i686-NEXT:    movl %esi, 20(%ecx)
-; i686-NEXT:    movl %eax, 16(%ecx)
-; i686-NEXT:    movl %ebx, 4(%ecx)
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; i686-NEXT:    movl %edx, (%ecx)
+; i686-NEXT:    movl %edi, 20(%ecx)
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl %eax, (%ecx)
-; i686-NEXT:    addl $68, %esp
+; i686-NEXT:    movl %eax, 4(%ecx)
+; i686-NEXT:    addl $100, %esp
 ; i686-NEXT:    popl %esi
 ; i686-NEXT:    popl %edi
 ; i686-NEXT:    popl %ebx
 ; i686-NEXT:    popl %ebp
 ; i686-NEXT:    retl
-; i686-NEXT:  .LBB6_33: # %entry
-; i686-NEXT:    movl %ebp, %edi
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    je .LBB6_36
-; i686-NEXT:  .LBB6_35:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; i686-NEXT:    movl %ecx, %edi
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    jne .LBB6_37
-; i686-NEXT:    jmp .LBB6_38
 ;
 ; x86_64-LABEL: test_lshr_v2i128:
 ; x86_64:       # %bb.0: # %entry
 ; x86_64-NEXT:    movq %rcx, %rax
 ; x86_64-NEXT:    movq {{[0-9]+}}(%rsp), %r10
 ; x86_64-NEXT:    movzbl {{[0-9]+}}(%rsp), %r9d
 ; x86_64-NEXT:    movl %r9d, %ecx
 ; x86_64-NEXT:    shrdq %cl, %rax, %rdx

 
 define void @test_ashr_v2i128(<2 x i128> %x, <2 x i128> %a, ptr nocapture %r) nounwind {
 ; i686-LABEL: test_ashr_v2i128:
 ; i686:       # %bb.0: # %entry
 ; i686-NEXT:    pushl %ebp
 ; i686-NEXT:    pushl %ebx
 ; i686-NEXT:    pushl %edi
 ; i686-NEXT:    pushl %esi
-; i686-NEXT:    subl $80, %esp
+; i686-NEXT:    subl $92, %esp
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    sarl $31, %ebx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
+; i686-NEXT:    sarl $31, %eax
+; i686-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; i686-NEXT:    movl %ebp, %ebx
-; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    sarl %cl, %ebx
-; i686-NEXT:    movl %esi, %edi
-; i686-NEXT:    shrl %cl, %edi
-; i686-NEXT:    shrdl %cl, %esi, %edx
-; i686-NEXT:    sarl $31, %ebp
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    jne .LBB7_1
-; i686-NEXT:  # %bb.2: # %entry
+; i686-NEXT:    andl $7, %ebx
+; i686-NEXT:    shrl $3, %ebp
+; i686-NEXT:    andl $15, %ebp
+; i686-NEXT:    movl 32(%esp,%ebp), %eax
+; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    movl %ebx, %ecx
+; i686-NEXT:    shrl %cl, %eax
+; i686-NEXT:    movl %ebx, %ecx
+; i686-NEXT:    notl %ecx
+; i686-NEXT:    movl 36(%esp,%ebp), %edx
 ; i686-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    jmp .LBB7_3
-; i686-NEXT:  .LBB7_1:
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB7_3: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    shrdl %cl, %edx, %edi
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    jne .LBB7_5
-; i686-NEXT:  # %bb.4: # %entry
-; i686-NEXT:    movl %edi, %ebx
-; i686-NEXT:  .LBB7_5: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl %ebp, %edi
+; i686-NEXT:    addl %edx, %edx
+; i686-NEXT:    # kill: def $cl killed $cl killed $ecx
+; i686-NEXT:    shll %cl, %edx
+; i686-NEXT:    orl %eax, %edx
+; i686-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    movl %edi, %ecx
+; i686-NEXT:    movl %edi, %edx
+; i686-NEXT:    andl $7, %edx
+; i686-NEXT:    shrl $3, %ecx
+; i686-NEXT:    andl $15, %ecx
+; i686-NEXT:    movl 64(%esp,%ecx), %esi
+; i686-NEXT:    movl %ecx, %edi
+; i686-NEXT:    movl %ecx, (%esp) # 4-byte Spill
+; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    sarl %cl, %edi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; i686-NEXT:    shrl %cl, %esi
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    sarl $31, %ebp
 ; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    subl $64, %ecx
+; i686-NEXT:    notl %ecx
+; i686-NEXT:    movl 68(%esp,%edi), %eax
+; i686-NEXT:    leal (%eax,%eax), %edi
+; i686-NEXT:    # kill: def $cl killed $cl killed $ecx
+; i686-NEXT:    shll %cl, %edi
+; i686-NEXT:    orl %esi, %edi
+; i686-NEXT:    movl 28(%esp,%ebp), %ecx
 ; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    sbbl $0, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    sbbl $0, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    sbbl $0, %ecx
-; i686-NEXT:    setb {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Spill
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    movl $0, %esi
+; i686-NEXT:    movl 40(%esp,%ebp), %esi
+; i686-NEXT:    movl %ebx, %ecx
+; i686-NEXT:    shrdl %cl, %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; i686-NEXT:    movl (%esp), %ecx # 4-byte Reload
+; i686-NEXT:    movl 60(%esp,%ecx), %ebp
 ; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ebp, %ecx
-; i686-NEXT:    jne .LBB7_7
-; i686-NEXT:  # %bb.6: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; i686-NEXT:    movl %edi, %ecx
-; i686-NEXT:  .LBB7_7: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; i686-NEXT:    movl 72(%esp,%ecx), %ebp
 ; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    shrdl %cl, %ebp, %esi
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    jne .LBB7_9
-; i686-NEXT:  # %bb.8: # %entry
-; i686-NEXT:    movl %esi, %edi
-; i686-NEXT:  .LBB7_9: # %entry
-; i686-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; i686-NEXT:    shrdl %cl, %ebp, %eax
+; i686-NEXT:    movl %eax, (%esp) # 4-byte Spill
+; i686-NEXT:    movl %ebx, %ecx
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; i686-NEXT:    shrdl %cl, %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
 ; i686-NEXT:    sarl %cl, %esi
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    jne .LBB7_11
-; i686-NEXT:  # %bb.10: # %entry
-; i686-NEXT:    movl %esi, %ecx
-; i686-NEXT:  .LBB7_11: # %entry
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    movb $64, %cl
-; i686-NEXT:    subb %dl, %cl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    shldl %cl, %ebx, %ebp
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ebx, %ebp
-; i686-NEXT:    shll %cl, %ebp
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    movb $64, %bl
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    je .LBB7_13
-; i686-NEXT:  # %bb.12:
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    xorl %ebp, %ebp
-; i686-NEXT:  .LBB7_13: # %entry
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    shrdl %cl, %edi, %esi
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    jne .LBB7_15
-; i686-NEXT:  # %bb.14: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB7_15: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    subb %al, %bl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:    shll %cl, %ebp
-; i686-NEXT:    testb $32, %bl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    jne .LBB7_17
-; i686-NEXT:  # %bb.16: # %entry
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB7_17: # %entry
-; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    subl $64, %ecx
-; i686-NEXT:    sbbl $0, %esi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    sbbl $0, %esi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    sbbl $0, %esi
-; i686-NEXT:    setae %bh
-; i686-NEXT:    jb .LBB7_19
-; i686-NEXT:  # %bb.18: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB7_19: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    shrdl %cl, %edi, %esi
-; i686-NEXT:    sarl %cl, %edi
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    je .LBB7_20
-; i686-NEXT:  # %bb.21: # %entry
-; i686-NEXT:    testb %bh, %bh
-; i686-NEXT:    je .LBB7_22
-; i686-NEXT:  .LBB7_23: # %entry
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    jne .LBB7_25
-; i686-NEXT:  .LBB7_24: # %entry
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB7_25: # %entry
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    shldl %cl, %esi, %edi
-; i686-NEXT:    testb $32, %bl
-; i686-NEXT:    jne .LBB7_27
-; i686-NEXT:  # %bb.26: # %entry
-; i686-NEXT:    movl %edi, %ebp
-; i686-NEXT:  .LBB7_27: # %entry
-; i686-NEXT:    testb %bh, %bh
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; i686-NEXT:    jne .LBB7_29
-; i686-NEXT:  # %bb.28:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; i686-NEXT:    orl %ebp, %ebx
-; i686-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB7_29: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    jne .LBB7_31
-; i686-NEXT:  # %bb.30: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB7_31: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    shrdl %cl, %ebp, %ebx
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    jne .LBB7_33
-; i686-NEXT:  # %bb.32: # %entry
-; i686-NEXT:    movl %ebx, %esi
-; i686-NEXT:  .LBB7_33: # %entry
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; i686-NEXT:    je .LBB7_35
-; i686-NEXT:  # %bb.34:
+; i686-NEXT:    shrdl %cl, %eax, %ebx
+; i686-NEXT:    movl %edx, %ecx
+; i686-NEXT:    sarl %cl, %ebp
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; i686-NEXT:    movl %ebp, 28(%eax)
+; i686-NEXT:    movl (%esp), %ecx # 4-byte Reload
+; i686-NEXT:    movl %ecx, 24(%eax)
+; i686-NEXT:    movl %ebx, 16(%eax)
+; i686-NEXT:    movl %esi, 12(%eax)
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    orl %ebx, %ecx
-; i686-NEXT:    movl %ecx, %esi
-; i686-NEXT:  .LBB7_35: # %entry
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    je .LBB7_37
-; i686-NEXT:  # %bb.36:
+; i686-NEXT:    movl %ecx, 8(%eax)
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB7_37: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    orl %ecx, %edx
-; i686-NEXT:    je .LBB7_39
-; i686-NEXT:  # %bb.38: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    movl %ecx, (%eax)
+; i686-NEXT:    movl %edi, 20(%eax)
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB7_39: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    orl %edx, %eax
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    je .LBB7_41
-; i686-NEXT:  # %bb.40: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; i686-NEXT:  .LBB7_41: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; i686-NEXT:    movl %edx, 28(%ecx)
-; i686-NEXT:    movl %edi, 24(%ecx)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; i686-NEXT:    movl %edx, 12(%ecx)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; i686-NEXT:    movl %edx, 8(%ecx)
-; i686-NEXT:    movl %esi, 20(%ecx)
-; i686-NEXT:    movl %eax, 16(%ecx)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl %eax, 4(%ecx)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl %eax, (%ecx)
-; i686-NEXT:    addl $80, %esp
+; i686-NEXT:    movl %ecx, 4(%eax)
+; i686-NEXT:    addl $92, %esp
 ; i686-NEXT:    popl %esi
 ; i686-NEXT:    popl %edi
 ; i686-NEXT:    popl %ebx
 ; i686-NEXT:    popl %ebp
 ; i686-NEXT:    retl
-; i686-NEXT:  .LBB7_20: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    testb %bh, %bh
-; i686-NEXT:    jne .LBB7_23
-; i686-NEXT:  .LBB7_22:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    je .LBB7_24
-; i686-NEXT:    jmp .LBB7_25
 ;
 ; x86_64-LABEL: test_ashr_v2i128:
 ; x86_64:       # %bb.0: # %entry
 ; x86_64-NEXT:    movq %rcx, %rax
 ; x86_64-NEXT:    movq {{[0-9]+}}(%rsp), %r10
 ; x86_64-NEXT:    movzbl {{[0-9]+}}(%rsp), %r9d
 ; x86_64-NEXT:    movl %r9d, %ecx
 ; x86_64-NEXT:    shrdq %cl, %rax, %rdx

 
 define void @test_shl_v2i128(<2 x i128> %x, <2 x i128> %a, ptr nocapture %r) nounwind {
 ; i686-LABEL: test_shl_v2i128:
 ; i686:       # %bb.0: # %entry
 ; i686-NEXT:    pushl %ebp
 ; i686-NEXT:    pushl %ebx
 ; i686-NEXT:    pushl %edi
 ; i686-NEXT:    pushl %esi
-; i686-NEXT:    subl $72, %esp
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; i686-NEXT:    subl $100, %esp
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:    shll %cl, %ebp
-; i686-NEXT:    shll %cl, %esi
-; i686-NEXT:    movl %edx, %eax
-; i686-NEXT:    subl $64, %eax
-; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    sbbl $0, %eax
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    sbbl $0, %eax
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    sbbl $0, %eax
-; i686-NEXT:    setb {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Spill
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    testb $32, %bl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl $0, %eax
-; i686-NEXT:    movl $0, %ecx
-; i686-NEXT:    jne .LBB8_2
-; i686-NEXT:  # %bb.1: # %entry
-; i686-NEXT:    movl %esi, %eax
-; i686-NEXT:    movl %ebp, %ecx
-; i686-NEXT:  .LBB8_2: # %entry
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %edi, %eax
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    shldl %cl, %edi, %eax
-; i686-NEXT:    testb $32, %bl
-; i686-NEXT:    jne .LBB8_4
-; i686-NEXT:  # %bb.3: # %entry
-; i686-NEXT:    movl %eax, %esi
-; i686-NEXT:  .LBB8_4: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movb $64, %cl
-; i686-NEXT:    subb %bl, %cl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl %edi, %esi
-; i686-NEXT:    shrl %cl, %esi
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    shrdl %cl, %edi, %eax
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    jne .LBB8_5
-; i686-NEXT:  # %bb.6: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    jmp .LBB8_7
-; i686-NEXT:  .LBB8_5:
-; i686-NEXT:    movl %esi, %eax
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:  .LBB8_7: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    shldl %cl, %esi, %edi
-; i686-NEXT:    testb $32, %bl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    jne .LBB8_9
-; i686-NEXT:  # %bb.8: # %entry
-; i686-NEXT:    movl %edi, %ebp
-; i686-NEXT:  .LBB8_9: # %entry
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ecx, %ebp
-; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    shll %cl, %ebp
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    shll %cl, %esi
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    movl $0, %edi
-; i686-NEXT:    movl $0, %ecx
-; i686-NEXT:    jne .LBB8_11
-; i686-NEXT:  # %bb.10: # %entry
-; i686-NEXT:    movl %esi, %edi
-; i686-NEXT:    movl %ebp, %ecx
-; i686-NEXT:  .LBB8_11: # %entry
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl %edx, %ecx
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    shldl %cl, %ebx, %edi
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    jne .LBB8_13
-; i686-NEXT:  # %bb.12: # %entry
-; i686-NEXT:    movl %edi, %ebp
-; i686-NEXT:  .LBB8_13: # %entry
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movb $64, %cl
-; i686-NEXT:    subb %dl, %cl
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    shrl %cl, %ebx
-; i686-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movb %cl, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    movl $0, %ecx
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    jne .LBB8_15
-; i686-NEXT:  # %bb.14: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:  .LBB8_15: # %entry
+; i686-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %edi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %eax, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl %ebp, %ecx
+; i686-NEXT:    shrl $3, %ebp
+; i686-NEXT:    andl $15, %ebp
+; i686-NEXT:    leal {{[0-9]+}}(%esp), %eax
+; i686-NEXT:    subl %ebp, %eax
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl 8(%eax), %edx
+; i686-NEXT:    movl %edx, (%esp) # 4-byte Spill
+; i686-NEXT:    andl $7, %ecx
 ; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    shll %cl, %edx
+; i686-NEXT:    movl 4(%eax), %esi
+; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    shrl %esi
+; i686-NEXT:    notl %ecx
+; i686-NEXT:    # kill: def $cl killed $cl killed $ecx
+; i686-NEXT:    shrl %cl, %esi
+; i686-NEXT:    orl %edx, %esi
+; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
+; i686-NEXT:    movl (%eax), %eax
+; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    movl %ebx, %edx
+; i686-NEXT:    shrl $3, %edx
+; i686-NEXT:    andl $15, %edx
+; i686-NEXT:    leal {{[0-9]+}}(%esp), %esi
+; i686-NEXT:    subl %edx, %esi
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; i686-NEXT:    andl $7, %ebx
+; i686-NEXT:    movl 8(%esi), %edi
 ; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl %edx, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    shldl %cl, %ebp, %edi
-; i686-NEXT:    testb $32, %dl
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    jne .LBB8_17
-; i686-NEXT:  # %bb.16: # %entry
-; i686-NEXT:    movl %edi, %esi
-; i686-NEXT:  .LBB8_17: # %entry
-; i686-NEXT:    orl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    movl %ebx, %eax
-; i686-NEXT:    subl $64, %eax
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    sbbl $0, %ecx
-; i686-NEXT:    movl %ebp, %ecx
-; i686-NEXT:    sbbl $0, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    sbbl $0, %ecx
-; i686-NEXT:    setb {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Spill
-; i686-NEXT:    jb .LBB8_19
-; i686-NEXT:  # %bb.18: # %entry
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:  .LBB8_19: # %entry
-; i686-NEXT:    jb .LBB8_21
-; i686-NEXT:  # %bb.20: # %entry
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:  .LBB8_21: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; i686-NEXT:    movl %ebp, %ebx
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    shll %cl, %ebx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    shldl %cl, %ebp, %edi
-; i686-NEXT:    testb $32, %cl
 ; i686-NEXT:    movl %ebx, %ecx
-; i686-NEXT:    jne .LBB8_23
-; i686-NEXT:  # %bb.22: # %entry
-; i686-NEXT:    movl %edi, %ecx
-; i686-NEXT:  .LBB8_23: # %entry
+; i686-NEXT:    shll %cl, %edi
+; i686-NEXT:    movl 4(%esi), %eax
+; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; i686-NEXT:    shrl %eax
+; i686-NEXT:    movl %ebx, %ecx
+; i686-NEXT:    notl %ecx
+; i686-NEXT:    # kill: def $cl killed $cl killed $ecx
+; i686-NEXT:    shrl %cl, %eax
+; i686-NEXT:    orl %edi, %eax
+; i686-NEXT:    movl (%esi), %ecx
 ; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl %eax, %ecx
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; i686-NEXT:    movl %esi, %edi
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; i686-NEXT:    shll %cl, %edi
 ; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    movl $0, %edi
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; i686-NEXT:    jne .LBB8_25
-; i686-NEXT:  # %bb.24: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; i686-NEXT:  .LBB8_25: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    jne .LBB8_27
-; i686-NEXT:  # %bb.26: # %entry
-; i686-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB8_27: # %entry
-; i686-NEXT:    movl %eax, %ecx
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; i686-NEXT:    movl %ecx, %edi
+; i686-NEXT:    shldl %cl, %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; i686-NEXT:    negl %ebp
+; i686-NEXT:    movl 64(%esp,%ebp), %esi
+; i686-NEXT:    movl %edi, %ecx
+; i686-NEXT:    # kill: def $cl killed $cl killed $ecx
+; i686-NEXT:    movl (%esp), %edi # 4-byte Reload
 ; i686-NEXT:    shldl %cl, %edi, %esi
-; i686-NEXT:    testb $32, %al
-; i686-NEXT:    jne .LBB8_29
-; i686-NEXT:  # %bb.28: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB8_29: # %entry
-; i686-NEXT:    cmpb $0, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; i686-NEXT:    movzbl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 1-byte Folded Reload
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; i686-NEXT:    jne .LBB8_30
-; i686-NEXT:  # %bb.31: # %entry
-; i686-NEXT:    testb %al, %al
-; i686-NEXT:    je .LBB8_32
-; i686-NEXT:  .LBB8_33: # %entry
-; i686-NEXT:    testb $32, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    jne .LBB8_35
-; i686-NEXT:  .LBB8_34: # %entry
-; i686-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB8_35: # %entry
-; i686-NEXT:    movzbl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 1-byte Folded Reload
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    shrdl %cl, %ebx, %esi
-; i686-NEXT:    testb $32, %cl
-; i686-NEXT:    jne .LBB8_37
-; i686-NEXT:  # %bb.36: # %entry
-; i686-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB8_37: # %entry
-; i686-NEXT:    testb %al, %al
+; i686-NEXT:    movl %esi, (%esp) # 4-byte Spill
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    jne .LBB8_38
-; i686-NEXT:  # %bb.39: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; i686-NEXT:    testb %al, %al
-; i686-NEXT:    jne .LBB8_41
-; i686-NEXT:    jmp .LBB8_42
-; i686-NEXT:  .LBB8_30:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    orl %ebp, %ecx
-; i686-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    testb %al, %al
-; i686-NEXT:    jne .LBB8_33
-; i686-NEXT:  .LBB8_32: # %entry
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; i686-NEXT:    testb $32, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Folded Reload
-; i686-NEXT:    je .LBB8_34
-; i686-NEXT:    jmp .LBB8_35
-; i686-NEXT:  .LBB8_38:
+; i686-NEXT:    movl %esi, %edi
+; i686-NEXT:    movl %ebx, %ecx
+; i686-NEXT:    shll %cl, %edi
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Folded Reload
+; i686-NEXT:    shldl %cl, %esi, %ebp
+; i686-NEXT:    negl %edx
+; i686-NEXT:    movl 96(%esp,%edx), %edx
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; i686-NEXT:    shldl %cl, %ebx, %edx
 ; i686-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; i686-NEXT:    testb %al, %al
-; i686-NEXT:    je .LBB8_42
-; i686-NEXT:  .LBB8_41:
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB8_42: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    orl %eax, %edx
-; i686-NEXT:    je .LBB8_44
-; i686-NEXT:  # %bb.43: # %entry
-; i686-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; i686-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; i686-NEXT:  .LBB8_44: # %entry
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; i686-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %edx
-; i686-NEXT:    orl {{[0-9]+}}(%esp), %ebx
-; i686-NEXT:    orl %edx, %ebx
-; i686-NEXT:    je .LBB8_46
-; i686-NEXT:  # %bb.45: # %entry
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; i686-NEXT:  .LBB8_46: # %entry
-; i686-NEXT:    movl %esi, 20(%eax)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; i686-NEXT:    movl %edx, 16(%eax)
+; i686-NEXT:    movl %edx, 28(%ecx)
+; i686-NEXT:    movl %ebp, 20(%ecx)
+; i686-NEXT:    movl %edi, 16(%ecx)
+; i686-NEXT:    movl (%esp), %edx # 4-byte Reload
+; i686-NEXT:    movl %edx, 12(%ecx)
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; i686-NEXT:    movl %edx, 4(%eax)
+; i686-NEXT:    movl %edx, 4(%ecx)
 ; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; i686-NEXT:    movl %edx, (%eax)
-; i686-NEXT:    movl %edi, 28(%eax)
-; i686-NEXT:    movl %ecx, 24(%eax)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    movl %ecx, 12(%eax)
-; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; i686-NEXT:    movl %ecx, 8(%eax)
-; i686-NEXT:    addl $72, %esp
+; i686-NEXT:    movl %edx, (%ecx)
+; i686-NEXT:    movl %eax, 24(%ecx)
+; i686-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; i686-NEXT:    movl %eax, 8(%ecx)
+; i686-NEXT:    addl $100, %esp
 ; i686-NEXT:    popl %esi
 ; i686-NEXT:    popl %edi
 ; i686-NEXT:    popl %ebx
 ; i686-NEXT:    popl %ebp
 ; i686-NEXT:    retl
 ;
 ; x86_64-LABEL: test_shl_v2i128:
 ; x86_64:       # %bb.0: # %entry

llvm/test/CodeGen/X86/shift-i256.ll

 
 define i256 @shift2(i256 %c) nounwind
 ; CHECK-LABEL: shift2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pushl %ebp
 ; CHECK-NEXT:    pushl %ebx
 ; CHECK-NEXT:    pushl %edi
 ; CHECK-NEXT:    pushl %esi
-; CHECK-NEXT:    subl $32, %esp
-; CHECK-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movb %al, %ah
-; CHECK-NEXT:    addb $64, %ah
-; CHECK-NEXT:    movl $1, %edi
-; CHECK-NEXT:    xorl %edx, %edx
-; CHECK-NEXT:    movb %ah, %cl
-; CHECK-NEXT:    shldl %cl, %edi, %edx
-; CHECK-NEXT:    movl $1, %ebx
-; CHECK-NEXT:    shll %cl, %ebx
-; CHECK-NEXT:    testb $32, %ah
-; CHECK-NEXT:    movl %ebx, %ebp
-; CHECK-NEXT:    jne .LBB1_2
-; CHECK-NEXT:  # %bb.1:
-; CHECK-NEXT:    movl %edx, %ebp
-; CHECK-NEXT:  .LBB1_2:
-; CHECK-NEXT:    movl %eax, %edx
-; CHECK-NEXT:    addb $-128, %dl
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    movl %edx, %ecx
-; CHECK-NEXT:    shldl %cl, %edi, %esi
-; CHECK-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    movl $1, %esi
-; CHECK-NEXT:    shll %cl, %esi
-; CHECK-NEXT:    testb $32, %dl
-; CHECK-NEXT:    je .LBB1_4
-; CHECK-NEXT:  # %bb.3:
-; CHECK-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:  .LBB1_4:
-; CHECK-NEXT:    cmpb $64, %dl
-; CHECK-NEXT:    movl $0, (%esp) # 4-byte Folded Spill
-; CHECK-NEXT:    jb .LBB1_6
-; CHECK-NEXT:  # %bb.5:
-; CHECK-NEXT:    movl %ebp, (%esp) # 4-byte Spill
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:  .LBB1_6:
-; CHECK-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    testb $32, %ah
-; CHECK-NEXT:    movl $0, %ebp
-; CHECK-NEXT:    jne .LBB1_8
-; CHECK-NEXT:  # %bb.7:
-; CHECK-NEXT:    movl %ebx, %ebp
-; CHECK-NEXT:  .LBB1_8:
-; CHECK-NEXT:    movb $-64, %cl
-; CHECK-NEXT:    subb %al, %cl
-; CHECK-NEXT:    movl $1, %esi
-; CHECK-NEXT:    xorl %ebx, %ebx
-; CHECK-NEXT:    shrdl %cl, %ebx, %esi
-; CHECK-NEXT:    testb $32, %cl
-; CHECK-NEXT:    movl $0, %ebx
-; CHECK-NEXT:    jne .LBB1_10
-; CHECK-NEXT:  # %bb.9:
-; CHECK-NEXT:    movl %esi, %ebx
-; CHECK-NEXT:  .LBB1_10:
-; CHECK-NEXT:    cmpb $64, %dl
-; CHECK-NEXT:    jb .LBB1_12
-; CHECK-NEXT:  # %bb.11:
-; CHECK-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; CHECK-NEXT:    movl %ebp, %ebx
-; CHECK-NEXT:  .LBB1_12:
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    movl %eax, %ecx
-; CHECK-NEXT:    shldl %cl, %edi, %esi
-; CHECK-NEXT:    movl $1, %ebp
-; CHECK-NEXT:    shll %cl, %ebp
-; CHECK-NEXT:    testb $32, %al
-; CHECK-NEXT:    je .LBB1_14
-; CHECK-NEXT:  # %bb.13:
-; CHECK-NEXT:    movl %ebp, %esi
-; CHECK-NEXT:    xorl %ebp, %ebp
-; CHECK-NEXT:  .LBB1_14:
+; CHECK-NEXT:    subl $92, %esp
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $1, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    movb %al, %ch
+; CHECK-NEXT:    andb $7, %ch
+; CHECK-NEXT:    shrb $3, %al
+; CHECK-NEXT:    negb %al
+; CHECK-NEXT:    movsbl %al, %eax
+; CHECK-NEXT:    movl 68(%esp,%eax), %edx
+; CHECK-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; CHECK-NEXT:    movb %ch, %cl
+; CHECK-NEXT:    shll %cl, %edx
+; CHECK-NEXT:    notb %cl
+; CHECK-NEXT:    movb %cl, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
+; CHECK-NEXT:    movl 64(%esp,%eax), %edi
+; CHECK-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; CHECK-NEXT:    shrl %edi
+; CHECK-NEXT:    shrl %cl, %edi
+; CHECK-NEXT:    orl %edx, %edi
+; CHECK-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; CHECK-NEXT:    movl 76(%esp,%eax), %edx
+; CHECK-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; CHECK-NEXT:    movb %ch, %cl
+; CHECK-NEXT:    shll %cl, %edx
+; CHECK-NEXT:    movl 72(%esp,%eax), %ebx
 ; CHECK-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; CHECK-NEXT:    shrl %ebx
+; CHECK-NEXT:    movb {{[-0-9]+}}(%e{{[sb]}}p), %cl # 1-byte Reload
+; CHECK-NEXT:    shrl %cl, %ebx
+; CHECK-NEXT:    orl %edx, %ebx
+; CHECK-NEXT:    movl 84(%esp,%eax), %esi
 ; CHECK-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    movl %eax, %ecx
-; CHECK-NEXT:    subb $64, %cl
-; CHECK-NEXT:    jb .LBB1_16
-; CHECK-NEXT:  # %bb.15:
-; CHECK-NEXT:    xorl %ebp, %ebp
-; CHECK-NEXT:  .LBB1_16:
-; CHECK-NEXT:    negb %cl
-; CHECK-NEXT:    movl $1, %esi
-; CHECK-NEXT:    xorl %ebx, %ebx
-; CHECK-NEXT:    shrdl %cl, %ebx, %esi
-; CHECK-NEXT:    testb $32, %cl
-; CHECK-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; CHECK-NEXT:    jne .LBB1_18
-; CHECK-NEXT:  # %bb.17:
-; CHECK-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:  .LBB1_18:
-; CHECK-NEXT:    movl %eax, %ecx
-; CHECK-NEXT:    addb $-64, %cl
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    shldl %cl, %edi, %esi
-; CHECK-NEXT:    movl $1, %ebx
-; CHECK-NEXT:    shll %cl, %ebx
-; CHECK-NEXT:    testb $32, %cl
-; CHECK-NEXT:    je .LBB1_20
-; CHECK-NEXT:  # %bb.19:
-; CHECK-NEXT:    movl %ebx, %esi
-; CHECK-NEXT:    xorl %ebx, %ebx
-; CHECK-NEXT:  .LBB1_20:
-; CHECK-NEXT:    cmpb $64, %al
-; CHECK-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; CHECK-NEXT:    jb .LBB1_22
-; CHECK-NEXT:  # %bb.21:
-; CHECK-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; CHECK-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:  .LBB1_22:
-; CHECK-NEXT:    testb %dl, %dl
-; CHECK-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; CHECK-NEXT:    movl $0, %edx
-; CHECK-NEXT:    je .LBB1_24
-; CHECK-NEXT:  # %bb.23:
-; CHECK-NEXT:    movl (%esp), %ecx # 4-byte Reload
-; CHECK-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; CHECK-NEXT:  .LBB1_24:
-; CHECK-NEXT:    movb $-128, %cl
-; CHECK-NEXT:    subb %al, %cl
-; CHECK-NEXT:    movl $1, %ebx
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    shrdl %cl, %esi, %ebx
-; CHECK-NEXT:    testb $32, %cl
-; CHECK-NEXT:    movl $0, %esi
-; CHECK-NEXT:    jne .LBB1_26
-; CHECK-NEXT:  # %bb.25:
-; CHECK-NEXT:    movl %ebx, %esi
-; CHECK-NEXT:  .LBB1_26:
-; CHECK-NEXT:    cmpb $64, %cl
-; CHECK-NEXT:    jb .LBB1_28
-; CHECK-NEXT:  # %bb.27:
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:  .LBB1_28:
-; CHECK-NEXT:    movl %ebp, (%esp) # 4-byte Spill
-; CHECK-NEXT:    testb %cl, %cl
-; CHECK-NEXT:    je .LBB1_30
-; CHECK-NEXT:  # %bb.29:
-; CHECK-NEXT:    movl %esi, %edi
-; CHECK-NEXT:  .LBB1_30:
-; CHECK-NEXT:    testb %al, %al
-; CHECK-NEXT:    movl $0, %ebp
-; CHECK-NEXT:    jne .LBB1_31
-; CHECK-NEXT:  # %bb.32:
-; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; CHECK-NEXT:    js .LBB1_33
-; CHECK-NEXT:  .LBB1_34:
-; CHECK-NEXT:    movl $0, %eax
-; CHECK-NEXT:    jne .LBB1_35
-; CHECK-NEXT:  .LBB1_36:
-; CHECK-NEXT:    movl $0, %edi
-; CHECK-NEXT:    js .LBB1_37
-; CHECK-NEXT:    jmp .LBB1_39
-; CHECK-NEXT:  .LBB1_31:
-; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
+; CHECK-NEXT:    movb %ch, %cl
+; CHECK-NEXT:    shll %cl, %esi
+; CHECK-NEXT:    movl 80(%esp,%eax), %ebp
+; CHECK-NEXT:    movl %ebp, %edx
+; CHECK-NEXT:    shrl %edx
+; CHECK-NEXT:    movb {{[-0-9]+}}(%e{{[sb]}}p), %cl # 1-byte Reload
+; CHECK-NEXT:    shrl %cl, %edx
+; CHECK-NEXT:    orl %esi, %edx
+; CHECK-NEXT:    movb %ch, %cl
+; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; CHECK-NEXT:    shldl %cl, %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; CHECK-NEXT:    shldl %cl, %esi, %ebp
+; CHECK-NEXT:    movl 60(%esp,%eax), %edi
+; CHECK-NEXT:    movl 88(%esp,%eax), %esi
 ; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; CHECK-NEXT:    jns .LBB1_34
-; CHECK-NEXT:  .LBB1_33:
-; CHECK-NEXT:    movl $0, %ebp
-; CHECK-NEXT:    movl %eax, %edi
-; CHECK-NEXT:    movl $0, %eax
-; CHECK-NEXT:    je .LBB1_36
-; CHECK-NEXT:  .LBB1_35:
-; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    movl $0, %edi
-; CHECK-NEXT:    jns .LBB1_39
-; CHECK-NEXT:  .LBB1_37:
-; CHECK-NEXT:    je .LBB1_39
-; CHECK-NEXT:  # %bb.38:
-; CHECK-NEXT:    movl %edx, %edi
-; CHECK-NEXT:  .LBB1_39:
-; CHECK-NEXT:    movl $0, %edx
-; CHECK-NEXT:    jns .LBB1_42
-; CHECK-NEXT:  # %bb.40:
-; CHECK-NEXT:    je .LBB1_42
-; CHECK-NEXT:  # %bb.41:
-; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; CHECK-NEXT:  .LBB1_42:
-; CHECK-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    movl $0, %ebx
-; CHECK-NEXT:    je .LBB1_44
-; CHECK-NEXT:  # %bb.43:
-; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; CHECK-NEXT:  .LBB1_44:
-; CHECK-NEXT:    movl %eax, %ebp
-; CHECK-NEXT:    movl $0, %ecx
-; CHECK-NEXT:    jns .LBB1_46
-; CHECK-NEXT:  # %bb.45:
-; CHECK-NEXT:    movl $0, %ebx
-; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; CHECK-NEXT:  .LBB1_46:
+; CHECK-NEXT:    shldl %cl, %eax, %esi
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movl $0, %esi
-; CHECK-NEXT:    je .LBB1_48
-; CHECK-NEXT:  # %bb.47:
-; CHECK-NEXT:    movl %ecx, %esi
-; CHECK-NEXT:  .LBB1_48:
-; CHECK-NEXT:    jns .LBB1_50
-; CHECK-NEXT:  # %bb.49:
-; CHECK-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; CHECK-NEXT:    movl $0, (%esp) # 4-byte Folded Spill
-; CHECK-NEXT:  .LBB1_50:
-; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; CHECK-NEXT:    movl %ecx, 4(%eax)
-; CHECK-NEXT:    movl (%esp), %ecx # 4-byte Reload
-; CHECK-NEXT:    movl %ecx, (%eax)
-; CHECK-NEXT:    movl %esi, 20(%eax)
-; CHECK-NEXT:    movl %ebx, 12(%eax)
-; CHECK-NEXT:    movl %edx, 28(%eax)
-; CHECK-NEXT:    movl %edi, 24(%eax)
-; CHECK-NEXT:    movl %ebp, 16(%eax)
+; CHECK-NEXT:    movl %esi, 28(%eax)
+; CHECK-NEXT:    movl %ebp, 20(%eax)
+; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; CHECK-NEXT:    movl %esi, 12(%eax)
+; CHECK-NEXT:    movl %edi, %esi
+; CHECK-NEXT:    shll %cl, %esi
+; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
+; CHECK-NEXT:    shldl %cl, %edi, %ebp
+; CHECK-NEXT:    movl %ebp, 4(%eax)
+; CHECK-NEXT:    movl %esi, (%eax)
+; CHECK-NEXT:    movl %edx, 24(%eax)
+; CHECK-NEXT:    movl %ebx, 16(%eax)
 ; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; CHECK-NEXT:    movl %ecx, 8(%eax)
-; CHECK-NEXT:    addl $32, %esp
+; CHECK-NEXT:    addl $92, %esp
 ; CHECK-NEXT:    popl %esi
 ; CHECK-NEXT:    popl %edi
 ; CHECK-NEXT:    popl %ebx
 ; CHECK-NEXT:    popl %ebp
 ; CHECK-NEXT:    retl $4
 ;
 ; CHECK-X64-O0-LABEL: shift2:
 ; CHECK-X64-O0:       # %bb.0:
-; CHECK-X64-O0-NEXT:    pushq %r14
-; CHECK-X64-O0-NEXT:    pushq %rbx
 ; CHECK-X64-O0-NEXT:    movq %rdi, %rax
-; CHECK-X64-O0-NEXT:    movb %sil, %r11b
-; CHECK-X64-O0-NEXT:    movb $-128, %cl
-; CHECK-X64-O0-NEXT:    subb %r11b, %cl
+; CHECK-X64-O0-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O0-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O0-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O0-NEXT:    movq $1, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O0-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O0-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O0-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O0-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O0-NEXT:    movb %sil, %dl
+; CHECK-X64-O0-NEXT:    movb %dl, %cl
+; CHECK-X64-O0-NEXT:    andb $7, %cl
 ; CHECK-X64-O0-NEXT:    movb %cl, {{[-0-9]+}}(%r{{[sb]}}p) # 1-byte Spill
-; CHECK-X64-O0-NEXT:    xorl %edx, %edx
-; CHECK-X64-O0-NEXT:    movl %edx, %esi
-; CHECK-X64-O0-NEXT:    movl $1, %r14d
-; CHECK-X64-O0-NEXT:    movq %r14, %r8
-; CHECK-X64-O0-NEXT:    shrdq %cl, %rsi, %r8
-; CHECK-X64-O0-NEXT:    movb {{[-0-9]+}}(%r{{[sb]}}p), %cl # 1-byte Reload
-; CHECK-X64-O0-NEXT:    testb $64, %cl
-; CHECK-X64-O0-NEXT:    cmovneq %rsi, %r8
-; CHECK-X64-O0-NEXT:    movb %r11b, %bl
-; CHECK-X64-O0-NEXT:    addb $-128, %bl
-; CHECK-X64-O0-NEXT:    movb %bl, %cl
-; CHECK-X64-O0-NEXT:    movq %rsi, %rdx
-; CHECK-X64-O0-NEXT:    shldq %cl, %r14, %rdx
-; CHECK-X64-O0-NEXT:    movb %r11b, %cl
+; CHECK-X64-O0-NEXT:    shrb $3, %dl
+; CHECK-X64-O0-NEXT:    negb %dl
+; CHECK-X64-O0-NEXT:    movsbq %dl, %rdx
+; CHECK-X64-O0-NEXT:    movq -16(%rsp,%rdx), %rsi
 ; CHECK-X64-O0-NEXT:    movq %rsi, %r10
-; CHECK-X64-O0-NEXT:    shldq %cl, %r14, %r10
-; CHECK-X64-O0-NEXT:    movb %r11b, %cl
-; CHECK-X64-O0-NEXT:    movq %r14, %r9
-; CHECK-X64-O0-NEXT:    shlq %cl, %r9
-; CHECK-X64-O0-NEXT:    testb $64, %r11b
-; CHECK-X64-O0-NEXT:    cmovneq %r9, %r10
-; CHECK-X64-O0-NEXT:    cmovneq %rsi, %r9
-; CHECK-X64-O0-NEXT:    movb %bl, %cl
-; CHECK-X64-O0-NEXT:    shlq %cl, %r14
-; CHECK-X64-O0-NEXT:    movq %r14, %rcx
-; CHECK-X64-O0-NEXT:    testb $64, %bl
-; CHECK-X64-O0-NEXT:    cmovneq %rcx, %rdx
-; CHECK-X64-O0-NEXT:    cmovneq %rsi, %rcx
-; CHECK-X64-O0-NEXT:    testb %r11b, %r11b
-; CHECK-X64-O0-NEXT:    cmovnsq %r8, %rcx
-; CHECK-X64-O0-NEXT:    cmoveq %rsi, %rcx
-; CHECK-X64-O0-NEXT:    cmovnsq %rsi, %rdx
-; CHECK-X64-O0-NEXT:    cmoveq %rsi, %rdx
-; CHECK-X64-O0-NEXT:    movq %rsi, %r8
-; CHECK-X64-O0-NEXT:    cmovnsq %r10, %r8
-; CHECK-X64-O0-NEXT:    cmovnsq %r9, %rsi
+; CHECK-X64-O0-NEXT:    shlq %cl, %r10
+; CHECK-X64-O0-NEXT:    movb {{[-0-9]+}}(%r{{[sb]}}p), %cl # 1-byte Reload
+; CHECK-X64-O0-NEXT:    notb %cl
+; CHECK-X64-O0-NEXT:    movq -32(%rsp,%rdx), %r9
+; CHECK-X64-O0-NEXT:    movq -24(%rsp,%rdx), %r8
+; CHECK-X64-O0-NEXT:    movq %r8, %r11
+; CHECK-X64-O0-NEXT:    shrq %r11
+; CHECK-X64-O0-NEXT:    shrq %cl, %r11
+; CHECK-X64-O0-NEXT:    movb {{[-0-9]+}}(%r{{[sb]}}p), %cl # 1-byte Reload
+; CHECK-X64-O0-NEXT:    orq %r11, %r10
+; CHECK-X64-O0-NEXT:    movq %r10, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
+; CHECK-X64-O0-NEXT:    movq -8(%rsp,%rdx), %rdx
+; CHECK-X64-O0-NEXT:    shldq %cl, %rsi, %rdx
+; CHECK-X64-O0-NEXT:    movb {{[-0-9]+}}(%r{{[sb]}}p), %cl # 1-byte Reload
+; CHECK-X64-O0-NEXT:    movq %r9, %rsi
+; CHECK-X64-O0-NEXT:    shlq %cl, %rsi
+; CHECK-X64-O0-NEXT:    movb {{[-0-9]+}}(%r{{[sb]}}p), %cl # 1-byte Reload
+; CHECK-X64-O0-NEXT:    shldq %cl, %r9, %r8
+; CHECK-X64-O0-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rcx # 8-byte Reload
 ; CHECK-X64-O0-NEXT:    movq %r8, 8(%rdi)
 ; CHECK-X64-O0-NEXT:    movq %rsi, (%rdi)
 ; CHECK-X64-O0-NEXT:    movq %rdx, 24(%rdi)
 ; CHECK-X64-O0-NEXT:    movq %rcx, 16(%rdi)
-; CHECK-X64-O0-NEXT:    popq %rbx
-; CHECK-X64-O0-NEXT:    popq %r14
 ; CHECK-X64-O0-NEXT:    retq
 ;
 ; CHECK-X64-O2-LABEL: shift2:
 ; CHECK-X64-O2:       # %bb.0:
-; CHECK-X64-O2-NEXT:    pushq %rbx
 ; CHECK-X64-O2-NEXT:    movq %rdi, %rax
-; CHECK-X64-O2-NEXT:    movb $-128, %cl
-; CHECK-X64-O2-NEXT:    subb %sil, %cl
-; CHECK-X64-O2-NEXT:    xorl %r8d, %r8d
-; CHECK-X64-O2-NEXT:    movl $1, %edi
-; CHECK-X64-O2-NEXT:    movl $1, %r10d
-; CHECK-X64-O2-NEXT:    shrdq %cl, %r8, %r10
-; CHECK-X64-O2-NEXT:    testb $64, %cl
-; CHECK-X64-O2-NEXT:    cmovneq %r8, %r10
-; CHECK-X64-O2-NEXT:    leal -128(%rsi), %edx
-; CHECK-X64-O2-NEXT:    xorl %r9d, %r9d
+; CHECK-X64-O2-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O2-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O2-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O2-NEXT:    movq $1, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O2-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O2-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O2-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O2-NEXT:    movq $0, -{{[0-9]+}}(%rsp)
+; CHECK-X64-O2-NEXT:    movl %esi, %edx
+; CHECK-X64-O2-NEXT:    andb $7, %dl
+; CHECK-X64-O2-NEXT:    shrb $3, %sil
+; CHECK-X64-O2-NEXT:    negb %sil
+; CHECK-X64-O2-NEXT:    movsbq %sil, %rsi
+; CHECK-X64-O2-NEXT:    movq -16(%rsp,%rsi), %rdi
+; CHECK-X64-O2-NEXT:    movq %rdi, %r8
 ; CHECK-X64-O2-NEXT:    movl %edx, %ecx
-; CHECK-X64-O2-NEXT:    shldq %cl, %rdi, %r9
-; CHECK-X64-O2-NEXT:    xorl %r11d, %r11d
-; CHECK-X64-O2-NEXT:    movl %esi, %ecx
-; CHECK-X64-O2-NEXT:    shldq %cl, %rdi, %r11
-; CHECK-X64-O2-NEXT:    movl $1, %ebx
-; CHECK-X64-O2-NEXT:    shlq %cl, %rbx
-; CHECK-X64-O2-NEXT:    testb $64, %sil
-; CHECK-X64-O2-NEXT:    cmovneq %rbx, %r11
-; CHECK-X64-O2-NEXT:    cmovneq %r8, %rbx
+; CHECK-X64-O2-NEXT:    shlq %cl, %r8
+; CHECK-X64-O2-NEXT:    notb %cl
+; CHECK-X64-O2-NEXT:    movq -32(%rsp,%rsi), %r9
+; CHECK-X64-O2-NEXT:    movq -24(%rsp,%rsi), %r10
+; CHECK-X64-O2-NEXT:    movq %r10, %r11
+; CHECK-X64-O2-NEXT:    shrq %r11
+; CHECK-X64-O2-NEXT:    shrq %cl, %r11
+; CHECK-X64-O2-NEXT:    orq %r8, %r11
+; CHECK-X64-O2-NEXT:    movq -8(%rsp,%rsi), %rsi
 ; CHECK-X64-O2-NEXT:    movl %edx, %ecx
+; CHECK-X64-O2-NEXT:    shldq %cl, %rdi, %rsi
+; CHECK-X64-O2-NEXT:    movq %r9, %rdi
 ; CHECK-X64-O2-NEXT:    shlq %cl, %rdi
-; CHECK-X64-O2-NEXT:    testb $64, %dl
-; CHECK-X64-O2-NEXT:    cmovneq %rdi, %r9
-; CHECK-X64-O2-NEXT:    cmovneq %r8, %rdi
-; CHECK-X64-O2-NEXT:    testb %sil, %sil
-; CHECK-X64-O2-NEXT:    cmovnsq %r10, %rdi
-; CHECK-X64-O2-NEXT:    cmoveq %r8, %rdi
-; CHECK-X64-O2-NEXT:    cmovnsq %r8, %r9
-; CHECK-X64-O2-NEXT:    cmoveq %r8, %r9
-; CHECK-X64-O2-NEXT:    cmovsq %r8, %r11
-; CHECK-X64-O2-NEXT:    cmovsq %r8, %rbx
-; CHECK-X64-O2-NEXT:    movq %r11, 8(%rax)
-; CHECK-X64-O2-NEXT:    movq %rbx, (%rax)
-; CHECK-X64-O2-NEXT:    movq %r9, 24(%rax)
-; CHECK-X64-O2-NEXT:    movq %rdi, 16(%rax)
-; CHECK-X64-O2-NEXT:    popq %rbx
+; CHECK-X64-O2-NEXT:    shldq %cl, %r9, %r10
+; CHECK-X64-O2-NEXT:    movq %rsi, 24(%rax)
+; CHECK-X64-O2-NEXT:    movq %r10, 8(%rax)
+; CHECK-X64-O2-NEXT:    movq %rdi, (%rax)
+; CHECK-X64-O2-NEXT:    movq %r11, 16(%rax)
 ; CHECK-X64-O2-NEXT:    retq
 {
   %b = shl i256 1, %c  ; %c must not be a constant
   ; Special case when %c is 0:
   ret i256 %b
 }
 ;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
 ; CHECK-X64: {{.*}}