[PowerPC] Combine address computation to favour selecting DForm instructions

[nemanjai]

Summary

When selecting a memory operation for which the only available D-Form instructions have alignment requirements for the displacement immediate, we end up having to select X-Form instructions if the immediate isn't a required multiple. However, if the base register is used for multiple memory operations, we don't need to be so pessimistic.

For example,

vector int test(char *Ptr) {
  vector int a = *(vector int *)(Ptr+7);
  vector int b = *(vector int *)(Ptr+23);
  vector int c = *(vector int *)(Ptr+39);
  vector int d = *(vector int *)(Ptr+55);
  return a + b + c + d;
}

We currently materialize the constants 3, 7, 23, 39 in registers and use those as index registers with R3 as the base. However, if we computed the value r3 + 55, we can use DQ-Form instructions with displacements (48, 23, 16, 0).

This patch performs these adjustments and fixes up some of the addressing computation functions so as to avoid redundant code.

[nemanjai]

Please note that essentially all the code in the SelectAddress* functions is somewhat orthogonal to this. Those changes are required to optimize what this patch does, but if the reviewers prefer, I can pull those out into a separate patch. However, I opted to keep them part of this patch at least initially as motivation for those changes on their own would be hard to understand without the context of this patch.

[steven.zhang]

The idea is great! Some comments.

lib/Target/PowerPC/PPCISelDAGToDAG.cpp

5492

I am not sure the intention that you select the vector instead of map. The map make more sense to me.

5507

Maybe, we need to respect the volatile and atomic flag ?

5679

bool getBaseAndRequiredAlignmentFor(SDNode *MemOp, const SDValue *&Base, unsigned &alignment) makes more sense.

5686

There are some helper function to calculate the base and offset. i.e.
BaseIndexOffset::match(Store, DAG)

5714

The intention of addMemOpToDFormTracker is to collect the information and keep it in tracker. However, we are walking the tracker again later to get the uses of current base and discard the tracker. Why not generate the relationship between the base and offset here directly ? i.e.

maintain a map between: Base -> { reqAlign, {BaseIndexOffset1, ... } } and BaseIndexOffset::match() will help us get the base and offset. Maybe, we need also keep the load/store pointer for each BaseIndexOffset as we will update it later.

Inside the updateMemOp..., walk the map one by one and handle it for each Base as what you did.

5739

uint64_t

5746

If we maintain the base map, we don't need to walk the uses as for each load/store, we will get the alignment and compare with the alignment in the map for each base.

5760

Don't need the loop if it is a map.

test/CodeGen/PowerPC/unal-altivec-wint.ll

19

I didn't see the benefit for this change. Can we avoid it ?

[steven.zhang]

We need to double check the base offset we select to avoid exceeding the 16-bit as much as possible

lib/Target/PowerPC/PPCISelDAGToDAG.cpp

5655

If the offset exceed the 16-bit, we should try to scale it into 16-bit.

; Function Attrs: norecurse nounwind readonly
define dso_local signext i32 @foo(i8* nocapture readonly %p) local_unnamed_addr #0 {
entry:
  %add.ptr = getelementptr inbounds i8, i8* %p, i64 69340
  %0 = bitcast i8* %add.ptr to i32*
  %1 = load i32, i32* %0, align 4, !tbaa !2
  %add.ptr1 = getelementptr inbounds i8, i8* %p, i64 69344
  %2 = bitcast i8* %add.ptr1 to i32*
  %3 = load i32, i32* %2, align 4, !tbaa !2
  %cmp = icmp eq i32 %1, %3
  %conv = zext i1 %cmp to i32
  ret i32 %conv
}

lib/Target/PowerPC/PPCISelDAGToDAG.cpp

 STATISTIC(ZeroExtensionsAdded,
           "Number of zero extensions for compare inputs added.");
 STATISTIC(NumLogicOpsOnComparison,
           "Number of logical ops on i1 values calculated in GPR.");
 STATISTIC(OmittedForNonExtendUses,
           "Number of compares not eliminated as they have non-extending uses.");
 STATISTIC(NumP9Setb,
           "Number of compares lowered to setb.");
+STATISTIC(NumMemopsAdaptedToDForm,
+          "Number of memory op offsets adjusted to match D-Form requirements");
 
 // FIXME: Remove this once the bug has been fixed!
 cl::opt<bool> ANDIGlueBug("expose-ppc-andi-glue-bug",
 cl::desc("expose the ANDI glue bug on PPC"), cl::Hidden);
 
 static cl::opt<bool>
     UseBitPermRewriter("ppc-use-bit-perm-rewriter", cl::init(true),
                        cl::desc("use aggressive ppc isel for bit permutations"),
public:
 
     /// SelectAddrIdx - Given the specified address, check to see if it can be
     /// represented as an indexed [r+r] operation.
     /// This is for xform instructions whose associated displacement form is D.
     /// The last parameter \p 0 means associated D form has no requirment for 16
     /// bit signed displacement.
     /// Returns false if it can be represented by [r+imm], which are preferred.
     bool SelectAddrIdx(SDValue N, SDValue &Base, SDValue &Index) {
-      return PPCLowering->SelectAddressRegReg(N, Base, Index, *CurDAG, 0);
+      return PPCLowering->SelectAddressRegReg(N, Base, Index, *CurDAG, 1);
     }
 
     /// SelectAddrIdx4 - Given the specified address, check to see if it can be
     /// represented as an indexed [r+r] operation.
     /// This is for xform instructions whose associated displacement form is DS.
     /// The last parameter \p 4 means associated DS form 16 bit signed
     /// displacement must be a multiple of 4.
     /// Returns false if it can be represented by [r+imm], which are preferred.
public:
     /// Returns false if it can be represented by [r+imm], which are preferred.
     bool SelectAddrIdxX16(SDValue N, SDValue &Base, SDValue &Index) {
       return PPCLowering->SelectAddressRegReg(N, Base, Index, *CurDAG, 16);
     }
 
     /// SelectAddrIdxOnly - Given the specified address, force it to be
     /// represented as an indexed [r+r] operation.
     bool SelectAddrIdxOnly(SDValue N, SDValue &Base, SDValue &Index) {
-      return PPCLowering->SelectAddressRegRegOnly(N, Base, Index, *CurDAG);
+      return PPCLowering->SelectAddressRegRegOnly(N, Base, Index, *CurDAG, 1);
     }
     
     /// SelectAddrImm - Returns true if the address N can be represented by
     /// a base register plus a signed 16-bit displacement [r+imm].
-    /// The last parameter \p 0 means D form has no requirment for 16 bit signed
+    /// The last parameter \p 1 means D form has no requirment for 16 bit signed
     /// displacement.
     bool SelectAddrImm(SDValue N, SDValue &Disp,
                        SDValue &Base) {
-      return PPCLowering->SelectAddressRegImm(N, Disp, Base, *CurDAG, 0);
+      return PPCLowering->SelectAddressRegImm(N, Disp, Base, *CurDAG, 1);
     }
 
     /// SelectAddrImmX4 - Returns true if the address N can be represented by
     /// a base register plus a signed 16-bit displacement that is a multiple of
     /// 4 (last parameter). Suitable for use by STD and friends.
     bool SelectAddrImmX4(SDValue N, SDValue &Disp, SDValue &Base) {
       return PPCLowering->SelectAddressRegImm(N, Disp, Base, *CurDAG, 4);
     }
private:
     SDValue combineToCMPB(SDNode *N);
     void foldBoolExts(SDValue &Res, SDNode *&N);
 
     bool AllUsersSelectZero(SDNode *N);
     void SwapAllSelectUsers(SDNode *N);
 
     bool isOffsetMultipleOf(SDNode *N, unsigned Val) const;
     void transferMemOperands(SDNode *N, SDNode *Result);
+    bool updateMemOpOffsetsForDForm(
+      SmallVectorImpl<std::pair<const SDValue *, unsigned>> &DFormTracker);
+    void addMemOpToDFormTracker(
+      SDNode *MemOp,
+      SmallVectorImpl<std::pair<const SDValue *, unsigned>> &DFormTracker);
+    unsigned getBaseAndRequiredAlignmentFor(SDNode *MemOp,
+                                            const SDValue *&BasePtr) const;
   };
 
 } // end anonymous namespace
 
 /// InsertVRSaveCode - Once the entire function has been instruction selected,
 /// all virtual registers are created and all machine instructions are built,
 /// check to see if we need to save/restore VRSAVE.  If so, do it.
 void PPCDAGToDAGISel::InsertVRSaveCode(MachineFunction &Fn) {
do {
     N = User;
     ConstTrue = TrueRes;
     ConstFalse = FalseRes;
   } while (N->hasOneUse());
 }
 
 void PPCDAGToDAGISel::PreprocessISelDAG() {
   SelectionDAG::allnodes_iterator Position = CurDAG->allnodes_end();
+  SmallVector<std::pair<const SDValue *, unsigned>, 16> DFormTracker;
 
   bool MadeChange = false;
   while (Position != CurDAG->allnodes_begin()) {
     SDNode *N = &*--Position;
     if (N->use_empty())
       continue;
 
     SDValue Res;
     switch (N->getOpcode()) {
     default: break;
     case ISD::OR:
       Res = combineToCMPB(N);
       break;
     }
+    if (ISD::isUNINDEXEDLoad(N) || ISD::isUNINDEXEDStore(N))
+      addMemOpToDFormTracker(N, DFormTracker);
 
     if (!Res)
       foldBoolExts(Res, N);
 
     if (Res) {
       LLVM_DEBUG(dbgs() << "PPC DAG preprocessing replacing:\nOld:    ");
       LLVM_DEBUG(N->dump(CurDAG));
       LLVM_DEBUG(dbgs() << "\nNew: ");
       LLVM_DEBUG(Res.getNode()->dump(CurDAG));
       LLVM_DEBUG(dbgs() << "\n");
 
       CurDAG->ReplaceAllUsesOfValueWith(SDValue(N, 0), Res);
       MadeChange = true;
     }
   }
 
+  MadeChange |= updateMemOpOffsetsForDForm(DFormTracker);
   if (MadeChange)
     CurDAG->RemoveDeadNodes();
 }
 
 /// PostprocessISelDAG - Perform some late peephole optimizations
 /// on the DAG representation.
 void PPCDAGToDAGISel::PostprocessISelDAG() {
   // Skip peepholes at -O0.
   if (TM.getOptLevel() == CodeGenOpt::None)
     return;
 
   PeepholePPC64();
   PeepholeCROps();
   PeepholePPC64ZExt();
 }
 
+/// Look at memory operations in the DAG that have constant offsets from a
+/// common base register. If those constants are such that they cannot be
+/// encoded in a D-Form instruction (i.e. not multiples of 4 for DS-Form or
+/// multiples of 16 for DQ-Form), produce a new base register from which the
+/// offsets can be encoded.
+/// Example:
+/// 1: (store v4i32:$a, (add $x1, 8))
+/// 2: (store v4i32:$b, (add $x1, 24))
+/// Can become:
+/// 1: (store v4i32:$a, (add $x1, 8))
+/// 2: (store v4i32:$b, (add (add $x1, 8), 16))
+/// While the first pair would produce:
+/// li %1, 8
+/// stxvx %a, $x1, %1
+/// li %2, 24
+/// stxvx %b, $x1, %2
+/// The updated form would produce:
+/// addi %1, $x1, 8
+/// stxvx %a, 0, %1
+/// stxv %b, 16(%1)
+bool PPCDAGToDAGISel::updateMemOpOffsetsForDForm(
+    SmallVectorImpl<std::pair<const SDValue *, unsigned>> &DFormTracker) {
+  bool Changed = false;
+  // For each base register, find the entries in the vector that are uses
+  // of that register.
+  while (DFormTracker.size() > 1) {
+    SmallVector<std::pair<const SDValue *, unsigned>, 4> UsersOfCurrBase;
+    auto CurrElement = DFormTracker.pop_back_val();
+    const SDValue *CurrAddr = CurrElement.first;
+    UsersOfCurrBase.push_back(CurrElement);
+    SDValue CurrBase = CurrAddr->getOperand(0);
+    for (int i = DFormTracker.size() - 1; i >= 0; i--) {
+      auto NextElement = DFormTracker[i];
+      if (NextElement.first->getOperand(0) == CurrBase) {
+        UsersOfCurrBase.push_back(NextElement);
+        DFormTracker.erase(DFormTracker.begin() + i);
+      }
+    }
+    // Not profitable for a single memory operation.
+    if (UsersOfCurrBase.size() < 2)
+      continue;
+
+    // Sort the vector in non-increasing order by the alignment requirement.
+    // We try to accomodate the ones with the highest alignment first.
+    std::sort(UsersOfCurrBase.begin(), UsersOfCurrBase.end(),
+              [](const std::pair<const SDValue *, unsigned> &L,
+                 const std::pair<const SDValue *, unsigned> &R) {
+                return R.second < L.second;
+              });
+
+    LLVM_DEBUG(dbgs() << "Tracking address computation that may be used for "
+                         "DForm memory ops\n");
+    // We now have all the address nodes that use the same base register and the
+    // largest alignment required for the memory accesses that use that base
+    // register. We want to pick an anchor value on which we can base as many
+    // other address computations as possible.
+    // For example, if we have something like:
+    // (store v4i32:%a, (add $x1, 3))
+    // (store v4i32:%b, (add $x1, 7))
+    // (store v4i32:%c, (add $x1, 23))
+    // (store v4i32:%d, (add $x1, 39))
+    // We want to choose (add $x1, 7) as the anchor value since the following
+    // two can be based on it whereas selecting (add $x1, 3) would not allow
+    // us to base any of the others off of it.
+    // So we iterate over UsersOfCurrBase and compute a set of anchor values
+    // and indices for nodes we will base on that anchor.
+    // We could also look at the range to see if an optimal anchor would be
+    // some value in between (i.e. some value that we could compute with just
+    // an addis), but the analysis effort does not seem worth the miniscule
+    // gains it would provide.
+    SmallVector<SmallVector<const SDValue *, 4>, 4> AnchorSets;
+    for (auto &Elem : UsersOfCurrBase) {
+      // For each anchor set, check if the address can be computed based on
+      // the first element (anchor) in the set.
+      int CurrOffset =
+          cast<ConstantSDNode>(Elem.first->getOperand(1))->getZExtValue();
+      bool Inserted = false;
+      for (auto &AnchorSet : AnchorSets) {
+        int AnchorOffset =
+            cast<ConstantSDNode>(AnchorSet[0]->getOperand(1))->getZExtValue();
+        if (isInt<16>(CurrOffset - AnchorOffset) &&
+            (abs(CurrOffset - AnchorOffset) % Elem.second == 0)) {
+          AnchorSet.push_back(Elem.first);
+          Inserted = true;
+          LLVM_DEBUG(dbgs() << "Node: ");
+          LLVM_DEBUG(Elem.first->dump());
+          LLVM_DEBUG(dbgs() << "Added to set anchored by: ");
+          LLVM_DEBUG(AnchorSet[0]->dump());
+          break;
+        }
+      }
+      // If we were unable to add Addr to any of the anchor sets, create a new
+      // anchor set and add it.
+      if (!Inserted) {
+        LLVM_DEBUG(dbgs() << "Adding the following node as an anchor: ");
+        LLVM_DEBUG(Elem.first->dump());
+        SmallVector<const SDValue *, 4> Tmp;
+        Tmp.push_back(Elem.first);
+        AnchorSets.push_back(Tmp);
+      }
+    }
+
+    LLVM_DEBUG(dbgs() << "Attempting to scale pointers to favour DForm ops\n");
+    // FIXME: We could actually try merging sets to get the fewest possible add
+    // nodes, but there is currently no indication that this would be worth the
+    // effort based on workloads available.
+    for (auto &AnchorSet : AnchorSets) {
+      if (AnchorSet.size() == 1)
+        continue;
+      const SDValue &Anchor = *AnchorSet[0];
+      for (int i = 1, e = AnchorSet.size(); i < e; i++) {
+        const SDValue *Addr = AnchorSet[i];
+        SDLoc dl(Addr->getNode());
+        int64_t NewOffset =
+            cast<ConstantSDNode>(Addr->getOperand(1))->getSExtValue() -
+            cast<ConstantSDNode>(Anchor.getOperand(1))->getSExtValue();
+        // Rebase all the address computations off of AnchorSet[0].
+        SDValue NewBase =
+            CurDAG->getNode(ISD::ADD, dl, Anchor.getValueType(), Anchor,
+                            CurDAG->getIntPtrConstant(NewOffset, dl));
+        LLVM_DEBUG(dbgs() << "Replaced: ");
+        LLVM_DEBUG(Addr->dump());
+        LLVM_DEBUG(dbgs() << "With: ");
+        LLVM_DEBUG(NewBase.dump());
+        NumMemopsAdaptedToDForm++;
+        CurDAG->ReplaceAllUsesOfValueWith(*Addr, NewBase);
+        Changed = true;
+      }
+    }
+  }
+  return Changed;
+}
+
+// Get the base pointer and required alignment for a memory operation. If there
+// are no D-Form instructions for the type being accessed, the required
+// alignment is meaningless - signal this by returning -1U.
+unsigned
+PPCDAGToDAGISel::getBaseAndRequiredAlignmentFor(SDNode *MemOp,
+                                                const SDValue *&BasePtr) const {
+  assert((ISD::isUNINDEXEDLoad(MemOp) || ISD::isUNINDEXEDStore(MemOp)) &&
+         "Expecting an unidexed memory op");
+  unsigned Opc = MemOp->getOpcode();
+  EVT MemVT;
+  bool IsSExt = false;
+  if (Opc == ISD::LOAD) {
+    LoadSDNode *LD = cast<LoadSDNode>(MemOp);
+    BasePtr = &LD->getBasePtr();
+    MemVT = LD->getMemoryVT();
+    IsSExt = LD->getExtensionType() == ISD::SEXTLOAD;
+  } else {
+    StoreSDNode *ST = cast<StoreSDNode>(MemOp);
+    BasePtr = &ST->getBasePtr();
+    MemVT = ST->getMemoryVT();
+  }
+  if (MemVT == MVT::f64 || MemVT == MVT::f32 || MemVT == MVT::i64 ||
+      (MemVT == MVT::i32 && IsSExt))
+    return 4;
+  if (MemVT.isVector() && PPCSubTarget->hasP9Vector())
+    return 16;
+  // FIXME: We really only need to ignore vector memory operations on CPUs
+  // that don't have DForms if the same base pointer isn't also used for
+  // scalar operations.
+  if (MemVT.isVector())
+    return -1U;
+  return 1;
+}
+
+// Collect all the Base operands of memory operations which are produced
+// by adding a multi-use value (base register) and a constant that is known
+// not to match the requirements of a D-Form instruction for the type.
+void PPCDAGToDAGISel::addMemOpToDFormTracker(
+    SDNode *MemOp,
+    SmallVectorImpl<std::pair<const SDValue *, unsigned>> &DFormTracker) {
+
+  // We only collect load/store nodes fed by adds with constants.
+  assert((ISD::isUNINDEXEDLoad(MemOp) || ISD::isUNINDEXEDStore(MemOp)) &&
+         "Expecting an unidexed memory op");
+  const SDValue *AddrValue;
+  EVT MemVT;
+  unsigned RequiredAlignment = getBaseAndRequiredAlignmentFor(MemOp, AddrValue);
+  if (RequiredAlignment == -1U)
+    return;
+
+  // We only care about addresses that are computed by adding a base register
+  // and a constant offset.
+  if (AddrValue->getOpcode() != ISD::ADD)
+    return;
+  ConstantSDNode *C = dyn_cast<ConstantSDNode>(AddrValue->getOperand(1));
+  if (!C)
+    return;
+
+  // If there is only a single use of the base register, we can't do any better
+  // than actually doing the addition for that one address computation.
+  SDValue BaseReg = AddrValue->getOperand(0);
+  if (BaseReg.hasOneUse())
+    return;
+
+  int Offset = C->getZExtValue();
+  if (Offset % RequiredAlignment == 0 && isInt<16>(Offset))
+    return;
+
+  // If there are multiple uses of AddrValue, find the maximum required
+  // alignment among all of its uses.
+  if (!AddrValue->hasOneUse()) {
+    for (auto *UserOfAddr : AddrValue->getNode()->uses()) {
+      const SDValue *Dummy;
+      if (!ISD::isUNINDEXEDLoad(UserOfAddr) &&
+          !ISD::isUNINDEXEDLoad(UserOfAddr))
+        continue;
+      unsigned UserReqAlign = getBaseAndRequiredAlignmentFor(UserOfAddr, Dummy);
+      RequiredAlignment =
+          UserReqAlign > RequiredAlignment ? UserReqAlign : RequiredAlignment;
+    }
+  }
+
+  // We need to save only a single pointer to any address node, so if we have
+  // multiple memory operations that have the same base pointer, we need to
+  // save just one.
+  for (auto Elem : DFormTracker) {
+    if (*Elem.first == *AddrValue)
+      return;
+  }
+  DFormTracker.push_back(std::make_pair(AddrValue, RequiredAlignment));
+}
+
 // Check if all users of this node will become isel where the second operand
 // is the constant zero. If this is so, and if we can negate the condition,
 // then we can flip the true and false operands. This will allow the zero to
 // be folded with the isel so that we don't need to materialize a register
 // containing zero.
 bool PPCDAGToDAGISel::AllUsersSelectZero(SDNode *N) {
   for (SDNode::use_iterator UI = N->use_begin(), UE = N->use_end();
        UI != UE; ++UI) {

lib/Target/PowerPC/PPCISelLowering.h

public:
     /// requirement, i.e. multiples of 4 for DS form.
     bool SelectAddressRegImm(SDValue N, SDValue &Disp, SDValue &Base,
                              SelectionDAG &DAG,
                              unsigned EncodingAlignment) const;
 
     /// SelectAddressRegRegOnly - Given the specified addressed, force it to be
     /// represented as an indexed [r+r] operation.
     bool SelectAddressRegRegOnly(SDValue N, SDValue &Base, SDValue &Index,
-                                 SelectionDAG &DAG) const;
+                                 SelectionDAG &DAG,
+                                 unsigned EncodingAlignment = 0) const;
 
     Sched::Preference getSchedulingPreference(SDNode *N) const override;
 
     /// LowerOperation - Provide custom lowering hooks for some operations.
     ///
     SDValue LowerOperation(SDValue Op, SelectionDAG &DAG) const override;
 
     /// ReplaceNodeResults - Replace the results of node with an illegal result

lib/Target/PowerPC/PPCISelLowering.cpp

if (N.getOpcode() == ISD::ADD) {
     if (hasSPE() && SelectAddressEVXRegReg(N, Base, Index, DAG))
         return true;
     if (isIntS16Immediate(N.getOperand(1), imm) &&
         (!EncodingAlignment || !(imm % EncodingAlignment)))
       return false; // r+i
     if (N.getOperand(1).getOpcode() == PPCISD::Lo)
       return false;    // r+i
 
+    // If there are other uses of the address computation, we want to be able
+    // to reuse the node, so use ZERO for RA and the computed address as RB.
+    // However, we don't want to do this if we can use a pre-inc operation.
+    // Use the fact that EncodingAlignment defaults to zero for pre-inc
+    // computation but is set on calls at ISEL time.
+    if (!N->hasOneUse() && (DisablePPCPreinc || EncodingAlignment != 0)) {
+      Base = DAG.getRegister(Subtarget.isPPC64() ? PPC::ZERO8 : PPC::ZERO,
+                             N.getValueType());
+      Index = N;
+    }
+    else {
       Base = N.getOperand(0);
       Index = N.getOperand(1);
+    }
     return true;
   } else if (N.getOpcode() == ISD::OR) {
     if (isIntS16Immediate(N.getOperand(1), imm) &&
         (!EncodingAlignment || !(imm % EncodingAlignment)))
       return false; // r+i can fold it if we can.
 
     // If this is an or of disjoint bitfields, we can codegen this as an add
     // (for better address arithmetic) if the LHS and RHS of the OR are provably
if (FrameIndexSDNode *FI = dyn_cast<FrameIndexSDNode>(N)) {
     fixupFuncForFI(DAG, FI->getIndex(), N.getValueType());
   } else
     Base = N;
   return true;      // [r+0]
 }
 
 /// SelectAddressRegRegOnly - Given the specified addressed, force it to be
 /// represented as an indexed [r+r] operation.
-bool PPCTargetLowering::SelectAddressRegRegOnly(SDValue N, SDValue &Base,
-                                                SDValue &Index,
-                                                SelectionDAG &DAG) const {
+bool PPCTargetLowering::SelectAddressRegRegOnly(
+    SDValue N, SDValue &Base, SDValue &Index, SelectionDAG &DAG,
+    unsigned EncodingAlignment) const {
   // Check to see if we can easily represent this as an [r+r] address.  This
   // will fail if it thinks that the address is more profitably represented as
   // reg+imm, e.g. where imm = 0.
-  if (SelectAddressRegReg(N, Base, Index, DAG))
+  if (SelectAddressRegReg(N, Base, Index, DAG, EncodingAlignment))
     return true;
 
   // If the address is the result of an add, we will utilize the fact that the
   // address calculation includes an implicit add.  However, we can reduce
   // register pressure if we do not materialize a constant just for use as the
   // index register.  We only get rid of the add if it is not an add of a
-  // value and a 16-bit signed constant and both have a single use.
+  // value and a 16-bit signed constant and both have a single use. However,
+  // if the result of the add itself has multiple uses, we want to just use
+  // the node unless that will take away a pre-inc opportunity.
+  if (N.getOpcode() == ISD::ADD) {
+    bool NHasOneUse = N.hasOneUse();
+    bool Op0HasOneUse = N.getOperand(0).hasOneUse();
+    bool Op1HasOneUse = N.getOperand(1).hasOneUse();
     int16_t imm = 0;
-  if (N.getOpcode() == ISD::ADD &&
-      (!isIntS16Immediate(N.getOperand(1), imm) ||
-       !N.getOperand(1).hasOneUse() || !N.getOperand(0).hasOneUse())) {
+    if ((NHasOneUse || EncodingAlignment == 0) &&
+        (!isIntS16Immediate(N.getOperand(1), imm) || !Op0HasOneUse ||
+         !Op1HasOneUse)) {
       Base = N.getOperand(0);
       Index = N.getOperand(1);
       return true;
     }
+  }
 
   // Otherwise, do it the hard way, using R0 as the base register.
   Base = DAG.getRegister(Subtarget.isPPC64() ? PPC::ZERO8 : PPC::ZERO,
                          N.getValueType());
   Index = N;
   return true;
 }
 
bool PPCTargetLowering::getPreIndexedAddressParts(SDNode *N, SDValue &Base,
 
   if (SelectAddressRegReg(Ptr, Base, Offset, DAG)) {
     // Common code will reject creating a pre-inc form if the base pointer
     // is a frame index, or if N is a store and the base pointer is either
     // the same as or a predecessor of the value being stored.  Check for
     // those situations here, and try with swapped Base/Offset instead.
     bool Swap = false;
 
-    if (isa<FrameIndexSDNode>(Base) || isa<RegisterSDNode>(Base))
+    RegisterSDNode *RegNodeBase = dyn_cast<RegisterSDNode>(Base);
+    if (RegNodeBase && (RegNodeBase->getReg() == PPC::ZERO ||
+                        RegNodeBase->getReg() == PPC::ZERO8))
+      return false;
+    if (isa<FrameIndexSDNode>(Base) || RegNodeBase)
       Swap = true;
     else if (!isLoad) {
       SDValue Val = cast<StoreSDNode>(N)->getValue();
       if (Val == Base || Base.getNode()->isPredecessorOf(Val.getNode()))
         Swap = true;
     }
 
     if (Swap)

lib/Target/PowerPC/PPCInstr64Bit.td

 def : Pat<(add i64:$in, (PPChi tglobaladdr:$g, 0)),
           (ADDIS8 $in, tglobaladdr:$g)>;
 def : Pat<(add i64:$in, (PPChi tconstpool:$g, 0)),
           (ADDIS8 $in, tconstpool:$g)>;
 def : Pat<(add i64:$in, (PPChi tjumptable:$g, 0)),
           (ADDIS8 $in, tjumptable:$g)>;
 def : Pat<(add i64:$in, (PPChi tblockaddress:$g, 0)),
           (ADDIS8 $in, tblockaddress:$g)>;
+def : Pat<(add i64:$in, imm64SExt32:$imm),
+          (ADDIS8 (ADDI8 $in, (LO16 imm:$imm)), (HI16 imm:$imm))>;
 
 // Patterns to match r+r indexed loads and stores for
 // addresses without at least 4-byte alignment.
 def : Pat<(i64 (unaligned4sextloadi32 xoaddr:$src)),
           (LWAX xoaddr:$src)>;
 def : Pat<(i64 (unaligned4load xoaddr:$src)),
           (LDX xoaddr:$src)>;
 def : Pat<(unaligned4store i64:$rS, xoaddr:$dst),

lib/Target/PowerPC/PPCInstrInfo.td

def imm32SExt16  : Operand<i32>, ImmLeaf<i32, [{
   // sign extended field.  Used by instructions like 'addi'.
   return (int32_t)Imm == (short)Imm;
 }]>;
 def imm64SExt16  : Operand<i64>, ImmLeaf<i64, [{
   // imm64SExt16 predicate - True if the i64 immediate fits in a 16-bit
   // sign extended field.  Used by instructions like 'addi'.
   return (int64_t)Imm == (short)Imm;
 }]>;
+def imm64SExt32  : Operand<i64>, ImmLeaf<i64, [{
+  // imm64SExt32 predicate - True if the i64 immediate fits in a 32-bit
+  // sign extended field.  Used by instruction pairs like 'addis, addi'.
+  // Since addi requires a signed 16-bit value, the high bit for the
+  // low 16-bits must not be set.
+  return isInt<32>(Imm) && ((Imm & ~0x8000) == Imm);
+}]>;
 def immZExt16  : PatLeaf<(imm), [{
   // immZExt16 predicate - True if the immediate fits in a 16-bit zero extended
   // field.  Used by instructions like 'ori'.
   return (uint64_t)N->getZExtValue() == (unsigned short)N->getZExtValue();
 }], LO16>;
 def immAnyExt8 : ImmLeaf<i32, [{ return isInt<8>(Imm) || isUInt<8>(Imm); }]>;
 def immSExt5NonZero : ImmLeaf<i32, [{ return Imm && isInt<5>(Imm); }]>;
 

test/CodeGen/PowerPC/dform-adjust.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
 ; RUN:   -mcpu=pwr9 < %s | FileCheck %s
 define dso_local i64 @test1(i8* nocapture readonly %p, i32 signext %count) local_unnamed_addr #0 {
 ; CHECK-LABEL: test1:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    li 5, -13
-; CHECK-NEXT:    lxvx 0, 3, 5
-; CHECK-NEXT:    li 5, 19
-; CHECK-NEXT:    lxvx 1, 3, 5
-; CHECK-NEXT:    li 5, 3
-; CHECK-NEXT:    li 6, 7
-; CHECK-NEXT:    li 7, 11
-; CHECK-NEXT:    li 8, 15
+; CHECK-NEXT:    addi 3, 3, -13
+; CHECK-NEXT:    lxvx 0, 0, 3
 ; CHECK-NEXT:    mfvsrld 9, 0
-; CHECK-NEXT:    ldx 5, 3, 5
-; CHECK-NEXT:    ldx 6, 3, 6
-; CHECK-NEXT:    ldx 7, 3, 7
-; CHECK-NEXT:    ldx 3, 3, 8
 ; CHECK-NEXT:    mffprd 8, 0
+; CHECK-NEXT:    mulld 8, 9, 8
+; CHECK-NEXT:    ld 5, 16(3)
+; CHECK-NEXT:    lxv 1, 32(3)
 ; CHECK-NEXT:    mfvsrld 10, 1
 ; CHECK-NEXT:    mfvsrd 11, 1
-; CHECK-NEXT:    mulld 8, 9, 8
 ; CHECK-NEXT:    mulld 5, 8, 5
 ; CHECK-NEXT:    mulld 5, 5, 10
 ; CHECK-NEXT:    mulld 5, 5, 11
+; CHECK-NEXT:    ld 6, 20(3)
 ; CHECK-NEXT:    mulld 5, 5, 6
+; CHECK-NEXT:    ld 7, 24(3)
+; CHECK-NEXT:    ld 3, 28(3)
 ; CHECK-NEXT:    mulld 5, 5, 7
 ; CHECK-NEXT:    maddld 3, 5, 3, 4
 ; CHECK-NEXT:    blr
 entry:
   %add.ptr = getelementptr inbounds i8, i8* %p, i64 -13
   %0 = bitcast i8* %add.ptr to <2 x i64>*
   %1 = load <2 x i64>, <2 x i64>* %0, align 16
   %add.ptr1 = getelementptr inbounds i8, i8* %p, i64 19
entry:
   %add19 = add i64 %mul18, %conv
   ret i64 %add19
 }
 
 define dso_local i64 @test2(i8* nocapture readonly %p, i32 signext %count) local_unnamed_addr #0 {
 ; CHECK-LABEL: test2:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    li 5, 0
-; CHECK-NEXT:    ori 6, 5, 40009
-; CHECK-NEXT:    ori 7, 5, 40001
 ; CHECK-NEXT:    ori 5, 5, 40005
-; CHECK-NEXT:    ldx 6, 3, 6
-; CHECK-NEXT:    ldx 7, 3, 7
-; CHECK-NEXT:    ldx 3, 3, 5
-; CHECK-NEXT:    mulld 5, 7, 6
+; CHECK-NEXT:    add 3, 3, 5
+; CHECK-NEXT:    ld 5, 4(3)
+; CHECK-NEXT:    ld 6, -4(3)
+; CHECK-NEXT:    ldx 3, 0, 3
+; CHECK-NEXT:    mulld 5, 6, 5
 ; CHECK-NEXT:    maddld 3, 5, 3, 4
 ; CHECK-NEXT:    blr
 entry:
   %add.ptr = getelementptr inbounds i8, i8* %p, i64 40009
   %0 = bitcast i8* %add.ptr to i64*
   %1 = load i64, i64* %0, align 8
   %add.ptr2 = getelementptr inbounds i8, i8* %p, i64 40001
   %2 = bitcast i8* %add.ptr2 to i64*
   %3 = load i64, i64* %2, align 8
   %add.ptr4 = getelementptr inbounds i8, i8* %p, i64 40005
   %4 = bitcast i8* %add.ptr4 to i64*
   %5 = load i64, i64* %4, align 8
   %mul = mul i64 %3, %1
   %mul5 = mul i64 %mul, %5
   %conv = sext i32 %count to i64
   %add6 = add i64 %mul5, %conv
   ret i64 %add6
 }
 
 define dso_local i64 @test3(i8* nocapture readonly %p, i32 signext %count) local_unnamed_addr {
 ; CHECK-LABEL: test3:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    lis 5, 1
-; CHECK-NEXT:    ori 6, 5, 14497
-; CHECK-NEXT:    ori 7, 5, 14465
-; CHECK-NEXT:    ori 5, 5, 14481
-; CHECK-NEXT:    ldx 6, 3, 6
-; CHECK-NEXT:    ldx 7, 3, 7
-; CHECK-NEXT:    ldx 3, 3, 5
-; CHECK-NEXT:    mulld 5, 7, 6
+; CHECK-NEXT:    addi 3, 3, 14481
+; CHECK-NEXT:    addis 3, 3, 1
+; CHECK-NEXT:    ld 5, 16(3)
+; CHECK-NEXT:    ld 6, -16(3)
+; CHECK-NEXT:    ldx 3, 0, 3
+; CHECK-NEXT:    mulld 5, 6, 5
 ; CHECK-NEXT:    maddld 3, 5, 3, 4
 ; CHECK-NEXT:    blr
 entry:
   %add.ptr = getelementptr inbounds i8, i8* %p, i64 80033
   %0 = bitcast i8* %add.ptr to i64*
   %1 = load i64, i64* %0, align 8
   %add.ptr2 = getelementptr inbounds i8, i8* %p, i64 80001
   %2 = bitcast i8* %add.ptr2 to i64*
   %3 = load i64, i64* %2, align 8
   %add.ptr4 = getelementptr inbounds i8, i8* %p, i64 80017
   %4 = bitcast i8* %add.ptr4 to i64*
   %5 = load i64, i64* %4, align 8
   %mul = mul i64 %3, %1
   %mul5 = mul i64 %mul, %5
   %conv = sext i32 %count to i64
   %add6 = add i64 %mul5, %conv
   ret i64 %add6
 }
 
+define dso_local <4 x i32> @test(i8* nocapture readonly %Ptr) local_unnamed_addr #0 {
+; CHECK-LABEL: test:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    li 4, 3
+; CHECK-NEXT:    lxvx 34, 3, 4
+; CHECK-NEXT:    addi 3, 3, 39
+; CHECK-NEXT:    lxv 35, -32(3)
+; CHECK-NEXT:    lxv 36, -16(3)
+; CHECK-NEXT:    lxvx 37, 0, 3
+; CHECK-NEXT:    vadduwm 2, 3, 2
+; CHECK-NEXT:    vadduwm 2, 2, 4
+; CHECK-NEXT:    vadduwm 2, 2, 5
+; CHECK-NEXT:    blr
+entry:
+  %add.ptr = getelementptr inbounds i8, i8* %Ptr, i64 3
+  %0 = bitcast i8* %add.ptr to <4 x i32>*
+  %1 = load <4 x i32>, <4 x i32>* %0, align 16
+  %add.ptr1 = getelementptr inbounds i8, i8* %Ptr, i64 7
+  %2 = bitcast i8* %add.ptr1 to <4 x i32>*
+  %3 = load <4 x i32>, <4 x i32>* %2, align 16
+  %add.ptr2 = getelementptr inbounds i8, i8* %Ptr, i64 23
+  %4 = bitcast i8* %add.ptr2 to <4 x i32>*
+  %5 = load <4 x i32>, <4 x i32>* %4, align 16
+  %add.ptr3 = getelementptr inbounds i8, i8* %Ptr, i64 39
+  %6 = bitcast i8* %add.ptr3 to <4 x i32>*
+  %7 = load <4 x i32>, <4 x i32>* %6, align 16
+  %add = add <4 x i32> %3, %1
+  %add4 = add <4 x i32> %add, %5
+  %add5 = add <4 x i32> %add4, %7
+  ret <4 x i32> %add5
+}

test/CodeGen/PowerPC/float-load-store-pair.ll

 ; CHECK-NEXT:    lfd 11, a11@toc@l(3)
 ; CHECK-NEXT:    addis 3, 2, a12@toc@ha
 ; CHECK-NEXT:    lfd 12, a12@toc@l(3)
 ; CHECK-NEXT:    addis 3, 2, a13@toc@ha
 ; CHECK-NEXT:    lfd 13, a13@toc@l(3)
 ; CHECK-NEXT:    addis 3, 2, a14@toc@ha
 ; CHECK-NEXT:    lfd 0, a14@toc@l(3)
 ; CHECK-NEXT:    addis 3, 2, a15@toc@ha
-; CHECK-NEXT:    addis 4, 2, a17@toc@ha
-; CHECK-NEXT:    addi 4, 4, a17@toc@l
 ; CHECK-NEXT:    lxsd 2, a15@toc@l(3)
 ; CHECK-NEXT:    addis 3, 2, a16@toc@ha
-; CHECK-NEXT:    addi 3, 3, a16@toc@l
+; CHECK-NEXT:    addis 4, 2, a17@toc@ha
+; CHECK-NEXT:    addi 4, 4, a17@toc@l
 ; CHECK-NEXT:    lxvx 36, 0, 4
+; CHECK-NEXT:    addi 3, 3, a16@toc@l
 ; CHECK-NEXT:    lxvx 35, 0, 3
-; CHECK-NEXT:    li 3, 168
-; CHECK-NEXT:    stxvx 36, 1, 3
-; CHECK-NEXT:    li 3, 152
-; CHECK-NEXT:    stxvx 35, 1, 3
+; CHECK-NEXT:    addi 3, 1, 152
+; CHECK-NEXT:    stxv 36, 16(3)
+; CHECK-NEXT:    stxvx 35, 0, 3
 ; CHECK-NEXT:    stxsd 2, 144(1)
 ; CHECK-NEXT:    stfd 0, 136(1)
 ; CHECK-NEXT:    bl _Z3fooddddddddddddddd
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    li 3, 0
 ; CHECK-NEXT:    addi 1, 1, 192
 ; CHECK-NEXT:    ld 0, 16(1)
 ; CHECK-NEXT:    mtlr 0

test/CodeGen/PowerPC/peephole-align.ll

entry:
   %0 = load i64, i64* getelementptr inbounds (%struct.d2, %struct.d2* @d2v, i32 0, i32 1), align 8
   ret i64 %0
 }
 
 ; Make sure the optimization fails to fire if the symbol is aligned, but the offset is not.
 ; CHECK-LABEL: test_misalign
 ; CHECK: addis [[REGSTRUCT_0:[0-9]+]], 2, misalign_v@toc@ha
 ; CHECK-DAG: addi [[REGSTRUCT:[0-9]+]], [[REGSTRUCT_0]], misalign_v@toc@l
-; CHECK-DAG: li [[OFFSET_REG:[0-9]+]], 1
-; CHECK: ldx [[REG0_0:[0-9]+]], [[REGSTRUCT]], [[OFFSET_REG]]
+; CHECK-DAG: addi [[OFFSET_REG:[0-9]+]], [[REGSTRUCT]], 1
+; CHECK: ldx [[REG0_0:[0-9]+]], 0, [[OFFSET_REG]]
 ; CHECK: addi [[REG0_1:[0-9]+]], [[REG0_0]], 1
-; CHECK: stdx [[REG0_1]], [[REGSTRUCT]], [[OFFSET_REG]]
+; CHECK: stdx [[REG0_1]], 0, [[OFFSET_REG]]
 define void @test_misalign() nounwind {
 entry:
   %0 = load i64, i64* getelementptr inbounds (%struct.misalign, %struct.misalign* @misalign_v, i32 0, i32 1), align 1
   %inc0 = add nsw i64 %0, 1
   store i64 %inc0, i64* getelementptr inbounds (%struct.misalign, %struct.misalign* @misalign_v, i32 0, i32 1), align 1
   ret void
 }

test/CodeGen/PowerPC/pre-inc-disable.ll

 ; CHECK-NEXT:    vperm v1, v5, v3, v4
 ; CHECK-NEXT:    li r6, 0
 ; CHECK-NEXT:    xvnegsp v5, v0
 ; CHECK-NEXT:    xvnegsp v0, v1
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  .LBB0_1: # %for.cond1.preheader
 ; CHECK-NEXT:    #
 ; CHECK-NEXT:    lfd f0, 0(r3)
+; CHECK-NEXT:    add r3, r3, r4
 ; CHECK-NEXT:    xxpermdi v1, f0, f0, 2
 ; CHECK-NEXT:    vperm v6, v1, v3, v4
 ; CHECK-NEXT:    vperm v1, v3, v1, v2
 ; CHECK-NEXT:    xvnegsp v1, v1
 ; CHECK-NEXT:    xvnegsp v6, v6
 ; CHECK-NEXT:    vabsduw v1, v1, v5
 ; CHECK-NEXT:    vabsduw v6, v6, v0
 ; CHECK-NEXT:    vadduwm v1, v6, v1
 ; CHECK-NEXT:    xxswapd v6, v1
 ; CHECK-NEXT:    vadduwm v1, v1, v6
 ; CHECK-NEXT:    xxspltw v6, v1, 2
 ; CHECK-NEXT:    vadduwm v1, v1, v6
 ; CHECK-NEXT:    vextuwrx r7, r5, v1
-; CHECK-NEXT:    lfdx f0, r3, r4
+; CHECK-NEXT:    lfdx f0, 0, r3
+; CHECK-NEXT:    add r3, r3, r4
 ; CHECK-NEXT:    add r6, r7, r6
-; CHECK-NEXT:    add r7, r3, r4
 ; CHECK-NEXT:    xxpermdi v1, f0, f0, 2
-; CHECK-NEXT:    add r3, r7, r4
 ; CHECK-NEXT:    vperm v6, v3, v1, v2
 ; CHECK-NEXT:    vperm v1, v1, v3, v4
 ; CHECK-NEXT:    xvnegsp v6, v6
 ; CHECK-NEXT:    xvnegsp v1, v1
 ; CHECK-NEXT:    vabsduw v6, v6, v5
 ; CHECK-NEXT:    vabsduw v1, v1, v0
 ; CHECK-NEXT:    vadduwm v1, v1, v6
 ; CHECK-NEXT:    xxswapd v6, v1
 ; CHECK-NEXT:    vadduwm v1, v1, v6
 ; CHECK-NEXT:    xxspltw v6, v1, 2
 ; CHECK-NEXT:    vadduwm v1, v1, v6
-; CHECK-NEXT:    vextuwrx r8, r5, v1
-; CHECK-NEXT:    add r6, r8, r6
+; CHECK-NEXT:    vextuwrx r7, r5, v1
+; CHECK-NEXT:    add r6, r7, r6
 ; CHECK-NEXT:    bdnz .LBB0_1
 ; CHECK-NEXT:  # %bb.2: # %for.cond.cleanup
 ; CHECK-NEXT:    extsw r3, r6
 ; CHECK-NEXT:    blr
 ;
 ; P9BE-LABEL: test_pre_inc_disable_1:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 0(r5)

 ; P9BE-NEXT:    vperm v1, v3, v5, v4
 ; P9BE-NEXT:    li r6, 0
 ; P9BE-NEXT:    xvnegsp v5, v0
 ; P9BE-NEXT:    xvnegsp v0, v1
 ; P9BE-NEXT:    .p2align 4
 ; P9BE-NEXT:  .LBB0_1: # %for.cond1.preheader
 ; P9BE-NEXT:    #
 ; P9BE-NEXT:    lfd f0, 0(r3)
+; P9BE-NEXT:    add r3, r3, r4
 ; P9BE-NEXT:    xxlor v1, vs0, vs0
 ; P9BE-NEXT:    vperm v6, v3, v1, v4
 ; P9BE-NEXT:    vperm v1, v3, v1, v2
 ; P9BE-NEXT:    xvnegsp v1, v1
 ; P9BE-NEXT:    xvnegsp v6, v6
 ; P9BE-NEXT:    vabsduw v1, v1, v5
 ; P9BE-NEXT:    vabsduw v6, v6, v0
 ; P9BE-NEXT:    vadduwm v1, v6, v1
 ; P9BE-NEXT:    xxswapd v6, v1
 ; P9BE-NEXT:    vadduwm v1, v1, v6
 ; P9BE-NEXT:    xxspltw v6, v1, 1
 ; P9BE-NEXT:    vadduwm v1, v1, v6
 ; P9BE-NEXT:    vextuwlx r7, r5, v1
-; P9BE-NEXT:    lfdx f0, r3, r4
+; P9BE-NEXT:    lfdx f0, 0, r3
+; P9BE-NEXT:    add r3, r3, r4
 ; P9BE-NEXT:    add r6, r7, r6
-; P9BE-NEXT:    add r7, r3, r4
 ; P9BE-NEXT:    xxlor v1, vs0, vs0
-; P9BE-NEXT:    add r3, r7, r4
 ; P9BE-NEXT:    vperm v6, v3, v1, v2
 ; P9BE-NEXT:    vperm v1, v3, v1, v4
 ; P9BE-NEXT:    xvnegsp v6, v6
 ; P9BE-NEXT:    xvnegsp v1, v1
 ; P9BE-NEXT:    vabsduw v6, v6, v5
 ; P9BE-NEXT:    vabsduw v1, v1, v0
 ; P9BE-NEXT:    vadduwm v1, v1, v6
 ; P9BE-NEXT:    xxswapd v6, v1
 ; P9BE-NEXT:    vadduwm v1, v1, v6
 ; P9BE-NEXT:    xxspltw v6, v1, 1
 ; P9BE-NEXT:    vadduwm v1, v1, v6
-; P9BE-NEXT:    vextuwlx r8, r5, v1
-; P9BE-NEXT:    add r6, r8, r6
+; P9BE-NEXT:    vextuwlx r7, r5, v1
+; P9BE-NEXT:    add r6, r7, r6
 ; P9BE-NEXT:    bdnz .LBB0_1
 ; P9BE-NEXT:  # %bb.2: # %for.cond.cleanup
 ; P9BE-NEXT:    extsw r3, r6
 ; P9BE-NEXT:    blr
 entry:
   %idx.ext = sext i32 %i_stride_pix1 to i64
   %0 = bitcast i8* %pix2 to <8 x i8>*
   %1 = load <8 x i8>, <8 x i8>* %0, align 1

 ;  }
 ;
 ;  return i_sum;
 ;}
 
 define void @test32(i8* nocapture readonly %pix2, i32 signext %i_pix2) {
 ; CHECK-LABEL: test32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    add r5, r3, r4
-; CHECK-NEXT:    lfiwzx f0, r3, r4
-; CHECK-NEXT:    addis r3, r2, .LCPI2_0@toc@ha
-; CHECK-NEXT:    addi r3, r3, .LCPI2_0@toc@l
-; CHECK-NEXT:    lxvx v4, 0, r3
-; CHECK-NEXT:    li r3, 4
+; CHECK-NEXT:    add r3, r3, r4
+; CHECK-NEXT:    addis r4, r2, .LCPI2_0@toc@ha
+; CHECK-NEXT:    lfiwzx f0, 0, r3
+; CHECK-NEXT:    addi r4, r4, .LCPI2_0@toc@l
+; CHECK-NEXT:    lxvx v4, 0, r4
+; CHECK-NEXT:    li r4, 4
 ; CHECK-NEXT:    xxpermdi v2, f0, f0, 2
-; CHECK-NEXT:    lfiwzx f0, r5, r3
+; CHECK-NEXT:    lfiwzx f0, r3, r4
 ; CHECK-NEXT:    xxlxor v3, v3, v3
 ; CHECK-NEXT:    vperm v2, v2, v3, v4
 ; CHECK-NEXT:    xxpermdi v5, f0, f0, 2
 ; CHECK-NEXT:    vperm v3, v5, v3, v4
 ; CHECK-NEXT:    vspltisw v4, 8
 ; CHECK-NEXT:    vnegw v3, v3
 ; CHECK-NEXT:    vadduwm v4, v4, v4
 ; CHECK-NEXT:    vslw v3, v3, v4
 ; CHECK-NEXT:    vsubuwm v2, v3, v2
 ; CHECK-NEXT:    xxswapd vs0, v2
 ; CHECK-NEXT:    stxvx vs0, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; P9BE-LABEL: test32:
 ; P9BE:       # %bb.0: # %entry
-; P9BE-NEXT:    add r5, r3, r4
-; P9BE-NEXT:    lfiwzx f0, r3, r4
-; P9BE-NEXT:    addis r3, r2, .LCPI2_0@toc@ha
-; P9BE-NEXT:    addi r3, r3, .LCPI2_0@toc@l
-; P9BE-NEXT:    lxvx v4, 0, r3
-; P9BE-NEXT:    li r3, 4
+; P9BE-NEXT:    add r3, r3, r4
+; P9BE-NEXT:    addis r4, r2, .LCPI2_0@toc@ha
+; P9BE-NEXT:    lfiwzx f0, 0, r3
+; P9BE-NEXT:    addi r4, r4, .LCPI2_0@toc@l
+; P9BE-NEXT:    lxvx v4, 0, r4
+; P9BE-NEXT:    li r4, 4
 ; P9BE-NEXT:    xxsldwi v2, f0, f0, 1
-; P9BE-NEXT:    lfiwzx f0, r5, r3
+; P9BE-NEXT:    lfiwzx f0, r3, r4
 ; P9BE-NEXT:    xxlxor v3, v3, v3
 ; P9BE-NEXT:    vperm v2, v3, v2, v4
 ; P9BE-NEXT:    xxsldwi v5, f0, f0, 1
 ; P9BE-NEXT:    vperm v3, v3, v5, v4
 ; P9BE-NEXT:    vspltisw v4, 8
 ; P9BE-NEXT:    vnegw v3, v3
 ; P9BE-NEXT:    vadduwm v4, v4, v4
 ; P9BE-NEXT:    vslw v3, v3, v4
entry:
   store <4 x i32> %13, <4 x i32>* undef, align 16
   ret void
 }
 
 define void @test16(i16* nocapture readonly %sums, i32 signext %delta, i32 signext %thresh) {
 ; CHECK-LABEL: test16:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    sldi r4, r4, 1
+; CHECK-NEXT:    add r3, r3, r4
+; CHECK-NEXT:    li r4, 16
 ; CHECK-NEXT:    lxsihzx v2, r3, r4
 ; CHECK-NEXT:    vsplth v2, v2, 3
+; CHECK-NEXT:    lxsihzx v4, 0, r3
+; CHECK-NEXT:    vsplth v4, v4, 3
+; CHECK-NEXT:    addis r3, r2, .LCPI3_0@toc@ha
 ; CHECK-NEXT:    xxlxor v3, v3, v3
 ; CHECK-NEXT:    vmrglh v2, v3, v2
-; CHECK-NEXT:    vsplth v4, v3, 7
-; CHECK-NEXT:    add r6, r3, r4
-; CHECK-NEXT:    li r3, 16
-; CHECK-NEXT:    vmrglw v2, v2, v4
-; CHECK-NEXT:    lxsihzx v4, r6, r3
-; CHECK-NEXT:    addis r3, r2, .LCPI3_0@toc@ha
+; CHECK-NEXT:    vmrglh v4, v3, v4
+; CHECK-NEXT:    vsplth v3, v3, 7
 ; CHECK-NEXT:    addi r3, r3, .LCPI3_0@toc@l
-; CHECK-NEXT:    vsplth v4, v4, 3
-; CHECK-NEXT:    vmrglh v3, v3, v4
+; CHECK-NEXT:    vmrglw v3, v4, v3
 ; CHECK-NEXT:    lxvx v4, 0, r3
 ; CHECK-NEXT:    li r3, 0
-; CHECK-NEXT:    vperm v2, v3, v2, v4
+; CHECK-NEXT:    vperm v2, v2, v3, v4
 ; CHECK-NEXT:    xxspltw v3, v2, 2
 ; CHECK-NEXT:    vadduwm v2, v2, v3
 ; CHECK-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-NEXT:    cmpw cr0, r3, r5
 ; CHECK-NEXT:    bgelr+ cr0
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ;
 ; P9BE-LABEL: test16:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    sldi r4, r4, 1
-; P9BE-NEXT:    add r6, r3, r4
-; P9BE-NEXT:    li r7, 16
-; P9BE-NEXT:    lxsihzx v2, r6, r7
+; P9BE-NEXT:    add r3, r3, r4
+; P9BE-NEXT:    li r4, 16
+; P9BE-NEXT:    lxsihzx v2, r3, r4
 ; P9BE-NEXT:    vsplth v2, v2, 3
-; P9BE-NEXT:    lxsihzx v4, r3, r4
-; P9BE-NEXT:    li r6, 0
-; P9BE-NEXT:    sldi r6, r6, 48
-; P9BE-NEXT:    mtvsrd v3, r6
+; P9BE-NEXT:    lxsihzx v4, 0, r3
+; P9BE-NEXT:    li r4, 0
+; P9BE-NEXT:    sldi r4, r4, 48
+; P9BE-NEXT:    mtvsrd v3, r4
 ; P9BE-NEXT:    addis r3, r2, .LCPI3_0@toc@ha
 ; P9BE-NEXT:    addi r3, r3, .LCPI3_0@toc@l
 ; P9BE-NEXT:    vmrghh v2, v3, v2
 ; P9BE-NEXT:    vsplth v4, v4, 3
 ; P9BE-NEXT:    vmrghh v4, v3, v4
 ; P9BE-NEXT:    vsplth v3, v3, 0
 ; P9BE-NEXT:    vmrghw v3, v3, v4
 ; P9BE-NEXT:    lxvx v4, 0, r3

 
 if.end:                                           ; preds = %for.body
   ret void
 }
 
 define void @test8(i8* nocapture readonly %sums, i32 signext %delta, i32 signext %thresh) {
 ; CHECK-LABEL: test8:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    lxsibzx v2, r3, r4
-; CHECK-NEXT:    add r6, r3, r4
-; CHECK-NEXT:    li r3, 0
-; CHECK-NEXT:    mtvsrd f0, r3
-; CHECK-NEXT:    li r3, 8
+; CHECK-NEXT:    add r3, r3, r4
+; CHECK-NEXT:    li r4, 0
+; CHECK-NEXT:    lxsibzx v2, 0, r3
+; CHECK-NEXT:    mtvsrd f0, r4
+; CHECK-NEXT:    li r4, 8
 ; CHECK-NEXT:    xxswapd v3, vs0
 ; CHECK-NEXT:    vspltb v2, v2, 7
-; CHECK-NEXT:    lxsibzx v5, r6, r3
+; CHECK-NEXT:    lxsibzx v5, r3, r4
 ; CHECK-NEXT:    vspltb v5, v5, 7
 ; CHECK-NEXT:    vmrglb v2, v3, v2
 ; CHECK-NEXT:    vspltb v4, v3, 15
 ; CHECK-NEXT:    vmrglb v3, v3, v5
 ; CHECK-NEXT:    addis r3, r2, .LCPI4_0@toc@ha
 ; CHECK-NEXT:    vmrglh v2, v2, v4
 ; CHECK-NEXT:    addi r3, r3, .LCPI4_0@toc@l
 ; CHECK-NEXT:    vmrglw v2, v2, v4
 ; CHECK-NEXT:    vmrglh v3, v3, v4
 ; CHECK-NEXT:    vmrglw v3, v4, v3
 ; CHECK-NEXT:    lxvx v4, 0, r3
 ; CHECK-NEXT:    li r3, 0
 ; CHECK-NEXT:    vperm v2, v3, v2, v4
 ; CHECK-NEXT:    xxspltw v3, v2, 2
 ; CHECK-NEXT:    vadduwm v2, v2, v3
 ; CHECK-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-NEXT:    cmpw cr0, r3, r5
 ; CHECK-NEXT:    bgelr+ cr0
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ;
 ; P9BE-LABEL: test8:
 ; P9BE:       # %bb.0: # %entry
-; P9BE-NEXT:    add r6, r3, r4
-; P9BE-NEXT:    li r7, 8
-; P9BE-NEXT:    lxsibzx v2, r6, r7
+; P9BE-NEXT:    add r3, r3, r4
+; P9BE-NEXT:    li r4, 8
+; P9BE-NEXT:    lxsibzx v2, r3, r4
 ; P9BE-NEXT:    vspltb v2, v2, 7
-; P9BE-NEXT:    lxsibzx v4, r3, r4
-; P9BE-NEXT:    li r6, 0
-; P9BE-NEXT:    sldi r6, r6, 56
-; P9BE-NEXT:    mtvsrd v3, r6
+; P9BE-NEXT:    lxsibzx v4, 0, r3
+; P9BE-NEXT:    li r4, 0
+; P9BE-NEXT:    sldi r4, r4, 56
+; P9BE-NEXT:    mtvsrd v3, r4
 ; P9BE-NEXT:    vmrghb v2, v3, v2
 ; P9BE-NEXT:    addis r3, r2, .LCPI4_0@toc@ha
 ; P9BE-NEXT:    addi r3, r3, .LCPI4_0@toc@l
 ; P9BE-NEXT:    vspltb v4, v4, 7
 ; P9BE-NEXT:    vmrghb v4, v3, v4
 ; P9BE-NEXT:    vspltb v3, v3, 0
 ; P9BE-NEXT:    vmrghh v4, v4, v3
 ; P9BE-NEXT:    xxspltw v3, v3, 0

test/CodeGen/PowerPC/qpx-unal-cons-lds.ll

 ; RUN: llc -verify-machineinstrs < %s | FileCheck %s
 target datalayout = "E-m:e-i64:64-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 ; Function Attrs: nounwind
 define void @foo(double* noalias nocapture %a, double* noalias nocapture readonly %b) #0 {
 entry:
   br label %vector.body
 
 ; CHECK-LABEL: @foo
 ; Make sure that the offset constants we use are all even (only the last should be odd).
-; CHECK-DAG: li {{[0-9]+}}, 1056
-; CHECK-DAG: li {{[0-9]+}}, 1088
-; CHECK-DAG: li {{[0-9]+}}, 1152
-; CHECK-DAG: li {{[0-9]+}}, 1216
-; CHECK-DAG: li {{[0-9]+}}, 1280
-; CHECK-DAG: li {{[0-9]+}}, 1344
-; CHECK-DAG: li {{[0-9]+}}, 1408
-; CHECK-DAG: li {{[0-9]+}}, 1472
-; CHECK-DAG: li {{[0-9]+}}, 1536
-; CHECK-DAG: li {{[0-9]+}}, 1600
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1056
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1088
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1152
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1216
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1280
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1344
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1408
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1472
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1536
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1600
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1664
+; CHECK-DAG: addi {{[0-9]+}}, {{[0-9]+}}, 1024
 ; CHECK-DAG: li {{[0-9]+}}, 1568
-; CHECK-DAG: li {{[0-9]+}}, 1664
 ; CHECK-DAG: li {{[0-9]+}}, 1632
 ; CHECK-DAG: li {{[0-9]+}}, 1728
 ; CHECK-DAG: li {{[0-9]+}}, 1696
 ; CHECK-DAG: li {{[0-9]+}}, 1792
 ; CHECK-DAG: li {{[0-9]+}}, 1760
 ; CHECK-DAG: li {{[0-9]+}}, 1856
 ; CHECK-DAG: li {{[0-9]+}}, 1824
 ; CHECK-DAG: li {{[0-9]+}}, 1920
 ; CHECK-DAG: li {{[0-9]+}}, 1888
 ; CHECK-DAG: li {{[0-9]+}}, 1984
 ; CHECK-DAG: li {{[0-9]+}}, 1952
 ; CHECK-DAG: li {{[0-9]+}}, 2016
-; CHECK-DAG: li {{[0-9]+}}, 1024
 ; CHECK-DAG: li {{[0-9]+}}, 1120
 ; CHECK-DAG: li {{[0-9]+}}, 1184
 ; CHECK-DAG: li {{[0-9]+}}, 1248
 ; CHECK-DAG: li {{[0-9]+}}, 1312
 ; CHECK-DAG: li {{[0-9]+}}, 1376
 ; CHECK-DAG: li {{[0-9]+}}, 1440
 ; CHECK-DAG: li {{[0-9]+}}, 1504
 ; CHECK-DAG: li {{[0-9]+}}, 2047

test/CodeGen/PowerPC/unal-altivec-wint.ll

entry:
   %vl = call <4 x i32> @llvm.ppc.altivec.lvx(i8* %hv)
 
   %v0 = load <4 x i32>, <4 x i32>* %h, align 8
 
   %a = add <4 x i32> %v0, %vl
   ret <4 x i32> %a
 
 ; CHECK-LABEL: @test1
-; CHECK: li [[REG:[0-9]+]], 16
+; CHECK: addi [[REG:[0-9]+]], 3, 16
 ; CHECK-NOT: li {{[0-9]+}}, 15
 ; CHECK-DAG: lvx {{[0-9]+}}, 0, 3
-; CHECK-DAG: lvx {{[0-9]+}}, 3, [[REG]]
+; CHECK-DAG: lvx {{[0-9]+}}, 0, [[REG]]
 ; CHECK: blr
 }
 
 declare void @llvm.ppc.altivec.stvx(<4 x i32>, i8*) #0
 
 define <4 x i32> @test2(<4 x i32>* %h, <4 x i32> %d) #0 {
 entry:
   %h1 = getelementptr <4 x i32>, <4 x i32>* %h, i64 1
   %hv = bitcast <4 x i32>* %h1 to i8*
   call void @llvm.ppc.altivec.stvx(<4 x i32> %d, i8* %hv)
 
   %v0 = load <4 x i32>, <4 x i32>* %h, align 8
 
   ret <4 x i32> %v0
 
 ; CHECK-LABEL: @test2
-; CHECK: li [[REG:[0-9]+]], 16
+; CHECK: addi [[REG:[0-9]+]], 3, 16
 ; CHECK-NOT: li {{[0-9]+}}, 15
 ; CHECK-DAG: lvx {{[0-9]+}}, 0, 3
-; CHECK-DAG: lvx {{[0-9]+}}, 3, [[REG]]
+; CHECK-DAG: lvx {{[0-9]+}}, 0, [[REG]]
 ; CHECK: blr
 }
 
 attributes #0 = { nounwind }
 attributes #1 = { nounwind readonly }
 

test/CodeGen/PowerPC/unal-altivec.ll

vector.body:                                      ; preds = %vector.body, %vector.ph
   %8 = getelementptr float, float* %a, i64 %.sum12
   %9 = bitcast float* %8 to <4 x float>*
   store <4 x float> %5, <4 x float>* %9, align 4
   %index.next = add i64 %index, 8
   %10 = icmp eq i64 %index.next, 16000
   br i1 %10, label %for.end, label %vector.body
 
 ; CHECK: @foo
-; CHECK-DAG: li [[C16:[0-9]+]], 16
 ; CHECK-DAG: lvx [[CNST:[0-9]+]],
 ; CHECK: .LBB0_1:
+; CHECK-DAG: addi [[C16:[0-9]+]], 4, 16
 ; CHECK-DAG: lvx [[LD1:[0-9]+]], 0, [[C0:[0-9]+]]
-; CHECK-DAG: lvx [[LD2:[0-9]+]], [[C0]], [[C16]]
+; CHECK-DAG: lvx [[LD2:[0-9]+]], 0, [[C16]]
 ; CHECK-DAG: lvsl [[MASK1:[0-9]+]], 0, [[C0]]
 ; CHECK-DAG: vperm [[VR1:[0-9]+]], [[LD1]], [[LD2]], [[MASK1]]
 ; CHECK-DAG: vaddfp {{[0-9]+}}, [[VR1]], [[CNST]]
 ; CHECK: blr
 
 for.end:                                          ; preds = %vector.body
   ret void
 }
 
 attributes #0 = { nounwind }

test/CodeGen/PowerPC/unal-vec-ldst.ll

 
 define <32 x i8> @test_l_v32i8(<32 x i8>* %p) #0 {
 entry:
   %r = load <32 x i8>, <32 x i8>* %p, align 1
   ret <32 x i8> %r
 
 ; CHECK-LABEL: @test_l_v32i8
 ; CHECK-DAG: li [[REG1:[0-9]+]], 31
-; CHECK-DAG: li [[REG2:[0-9]+]], 16
+; CHECK-DAG: addi [[REG2:[0-9]+]], 3, 16
 ; CHECK-DAG: lvsl [[REG3:[0-9]+]], 0, 3
 ; CHECK-DAG: lvx [[REG4:[0-9]+]], 3, [[REG1]]
-; CHECK-DAG: lvx [[REG5:[0-9]+]], 3, [[REG2]]
+; CHECK-DAG: lvx [[REG5:[0-9]+]], 0, [[REG2]]
 ; CHECK-DAG: lvx [[REG6:[0-9]+]], 0, 3
 ; CHECK-DAG: vperm 3, {{[0-9]+}}, {{[0-9]+}}, [[REG3]]
 ; CHECK-DAG: vperm 2, {{[0-9]+}}, {{[0-9]+}}, [[REG3]]
 ; CHECK: blr
 }
 
 define <8 x i16> @test_l_v8i16(<8 x i16>* %p) #0 {
 entry:

 
 define <16 x i16> @test_l_v16i16(<16 x i16>* %p) #0 {
 entry:
   %r = load <16 x i16>, <16 x i16>* %p, align 2
   ret <16 x i16> %r
 
 ; CHECK-LABEL: @test_l_v16i16
 ; CHECK-DAG: li [[REG1:[0-9]+]], 31
-; CHECK-DAG: li [[REG2:[0-9]+]], 16
+; CHECK-DAG: addi [[REG2:[0-9]+]], 3, 16
 ; CHECK-DAG: lvsl [[REG3:[0-9]+]], 0, 3
 ; CHECK-DAG: lvx [[REG4:[0-9]+]], 3, [[REG1]]
-; CHECK-DAG: lvx [[REG5:[0-9]+]], 3, [[REG2]]
+; CHECK-DAG: lvx [[REG5:[0-9]+]], 0, [[REG2]]
 ; CHECK-DAG: lvx [[REG6:[0-9]+]], 0, 3
 ; CHECK-DAG: vperm 3, {{[0-9]+}}, {{[0-9]+}}, [[REG3]]
 ; CHECK-DAG: vperm 2, {{[0-9]+}}, {{[0-9]+}}, [[REG3]]
 ; CHECK: blr
 }
 
 define <4 x i32> @test_l_v4i32(<4 x i32>* %p) #0 {
 entry:

 
 define <8 x i32> @test_l_v8i32(<8 x i32>* %p) #0 {
 entry:
   %r = load <8 x i32>, <8 x i32>* %p, align 4
   ret <8 x i32> %r
 
 ; CHECK-LABEL: @test_l_v8i32
 ; CHECK-DAG: li [[REG1:[0-9]+]], 31
-; CHECK-DAG: li [[REG2:[0-9]+]], 16
+; CHECK-DAG: addi [[REG2:[0-9]+]], 3, 16
 ; CHECK-DAG: lvsl [[REG3:[0-9]+]], 0, 3
 ; CHECK-DAG: lvx [[REG4:[0-9]+]], 3, [[REG1]]
-; CHECK-DAG: lvx [[REG5:[0-9]+]], 3, [[REG2]]
+; CHECK-DAG: lvx [[REG5:[0-9]+]], 0, [[REG2]]
 ; CHECK-DAG: lvx [[REG6:[0-9]+]], 0, 3
 ; CHECK-DAG: vperm 3, {{[0-9]+}}, {{[0-9]+}}, [[REG3]]
 ; CHECK-DAG: vperm 2, {{[0-9]+}}, {{[0-9]+}}, [[REG3]]
 ; CHECK: blr
 }
 
 define <2 x i64> @test_l_v2i64(<2 x i64>* %p) #0 {
 entry:

 
 define <8 x float> @test_l_v8float(<8 x float>* %p) #0 {
 entry:
   %r = load <8 x float>, <8 x float>* %p, align 4
   ret <8 x float> %r
 
 ; CHECK-LABEL: @test_l_v8float
 ; CHECK-DAG: li [[REG1:[0-9]+]], 31
-; CHECK-DAG: li [[REG2:[0-9]+]], 16
+; CHECK-DAG: addi [[REG2:[0-9]+]], 3, 16
 ; CHECK-DAG: lvsl [[REG3:[0-9]+]], 0, 3
 ; CHECK-DAG: lvx [[REG4:[0-9]+]], 3, [[REG1]]
-; CHECK-DAG: lvx [[REG5:[0-9]+]], 3, [[REG2]]
+; CHECK-DAG: lvx [[REG5:[0-9]+]], 0, [[REG2]]
 ; CHECK-DAG: lvx [[REG6:[0-9]+]], 0, 3
 ; CHECK-DAG: vperm 3, {{[0-9]+}}, {{[0-9]+}}, [[REG3]]
 ; CHECK-DAG: vperm 2, {{[0-9]+}}, {{[0-9]+}}, [[REG3]]
 ; CHECK: blr
 }
 
 define <2 x double> @test_l_v2double(<2 x double>* %p) #0 {
 entry:

 
 define <8 x float> @test_l_qv8float(<8 x float>* %p) #1 {
 entry:
   %r = load <8 x float>, <8 x float>* %p, align 4
   ret <8 x float> %r
 
 ; CHECK-LABEL: @test_l_qv8float
 ; CHECK-DAG: li [[REG1:[0-9]+]], 31
-; CHECK-DAG: li [[REG2:[0-9]+]], 16
+; CHECK-DAG: addi [[REG2:[0-9]+]], 3, 16
 ; CHECK-DAG: qvlfsx [[REG3:[0-9]+]], 3, [[REG1]]
-; CHECK-DAG: qvlfsx [[REG4:[0-9]+]], 3, [[REG2]]
+; CHECK-DAG: qvlfsx [[REG4:[0-9]+]], 0, [[REG2]]
 ; CHECK-DAG: qvlpclsx [[REG5:[0-5]+]], 0, 3
 ; CHECK-DAG: qvlfsx [[REG6:[0-9]+]], 0, 3
 ; CHECK-DAG: qvfperm 2, {{[0-9]+}}, {{[0-9]+}}, [[REG5]]
 ; CHECK-DAG: qvfperm 1, {{[0-9]+}}, {{[0-9]+}}, [[REG5]]
 ; CHECK: blr
 }
 
 define <4 x double> @test_l_qv4double(<4 x double>* %p) #1 {

 
 define <8 x double> @test_l_qv8double(<8 x double>* %p) #1 {
 entry:
   %r = load <8 x double>, <8 x double>* %p, align 8
   ret <8 x double> %r
 
 ; CHECK-LABEL: @test_l_qv8double
 ; CHECK-DAG: li [[REG1:[0-9]+]], 63
-; CHECK-DAG: li [[REG2:[0-9]+]], 32
+; CHECK-DAG: addi [[REG2:[0-9]+]], 3, 32
 ; CHECK-DAG: qvlfdx [[REG3:[0-9]+]], 3, [[REG1]]
-; CHECK-DAG: qvlfdx [[REG4:[0-9]+]], 3, [[REG2]]
+; CHECK-DAG: qvlfdx [[REG4:[0-9]+]], 0, [[REG2]]
 ; CHECK-DAG: qvlpcldx [[REG5:[0-5]+]], 0, 3
 ; CHECK-DAG: qvlfdx [[REG6:[0-9]+]], 0, 3
 ; CHECK-DAG: qvfperm 2, {{[0-9]+}}, {{[0-9]+}}, [[REG5]]
 ; CHECK-DAG: qvfperm 1, {{[0-9]+}}, {{[0-9]+}}, [[REG5]]
 ; CHECK: blr
 }
 
 define void @test_s_v16i8(<16 x i8>* %p, <16 x i8> %v) #0 {