[DAGCombine] Improve Load-Store Forwarding

[niravd]

Summary

Extend analysis forwarding loads from preceeding stores to work with
extended loads and truncated stores to the same address so long as the
load is fully subsumed by the store.

Hexagon's swp-epilog-phis.ll and swp-memrefs-epilog1.ll test are
deleted as they've no longer seem to be relevant.

[niravd]

Ping?

[rnk]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

12377–12382 ↗

(On Diff #155047)

This whole if looks like it should be its own helper, and even then it could be broken up more.

12382–12385 ↗

(On Diff #155047)

This use of auto is not consistent with LLVM's coding standards:
https://llvm.org/docs/CodingStandards.html#use-auto-type-deduction-to-make-code-more-readable

12417 ↗

(On Diff #155047)

TODO" seems like a typo

12421–12424 ↗

(On Diff #155047)

These conditionals look like they can be simplified.

12449–12450 ↗

(On Diff #155047)

This switch could be its own helper function, extendingLoadToExtension, or something like that. Putting switches into helpers is nice since it can save you a local variable and saves the break; line in each case.

[niravd]

Resolve comments.

[niravd]

Ping

[niravd]

Ping.

[RKSimon]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

12622 ↗

(On Diff #162252)

Not sure if this is that useful (especially as similar helpers typically return 1 for scalars)?

12626 ↗

(On Diff #162252)

Should you update Val even if you return false?

12661 ↗

(On Diff #162252)

I think you need breaks here, else add LLVM_FALLTHROUGHs

[niravd]

Resolve comments

[niravd]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

12622 ↗

(On Diff #162252)

I have no strong opinions on this. The only reason for the zero is 0 is to distinguish a singleton vector and it's element type. It's saves having to check if compared types are both isVector or not isVector.

12626 ↗

(On Diff #162252)

We can either assign Val when we consider the store, or before when we pass it in. I have a mild preference for this way as it avoid having to check that Val is ST->getValue(). We don't need to use Val afterwards if it's false; we can replace the continue on 12724 with a "return SDValue();", but I think it's a bit cleaner looking this way.

12661 ↗

(On Diff #162252)

Right. Thanks.

[niravd]

Ping.

[niravd]

Ping.

[niravd]

Ping again.

[RKSimon]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

12622 ↗

(On Diff #162252)

How about numVectorEltsOrZero ?

[niravd]

Rebase to tip and address rksimon's renaming suggestion.

[rnk]

I think it looks good. I think this got lost in the gulf of responsibility between multiple reviewers.

[evandro]

This comment was removed by evandro.

[peter.smith]

This commit (found via git bisect) looks like it is the cause of https://bugs.llvm.org/show_bug.cgi?id=39571 which is derived from the Linux kernel and is a regression from LLVM 7.0. We get an internal fault in instruction selection for ARM state for pre ARM-v7 architectures (test uses armv4t but it is reproducible on v5t and v6).

A test case and details of how to reproduce can be found in the PR. It is possible that this has exposed a limitation in the Arm backend or the optimisation may not be suitable for it. Would you be able to take a look?

llvm/trunk/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

private:
     bool SimplifyDemandedVectorElts(SDValue Op, const APInt &Demanded,
                                     bool AssumeSingleUse = false);
 
     bool CombineToPreIndexedLoadStore(SDNode *N);
     bool CombineToPostIndexedLoadStore(SDNode *N);
     SDValue SplitIndexingFromLoad(LoadSDNode *LD);
     bool SliceUpLoad(SDNode *N);
 
+    // Scalars have size 0 to distinguish from singleton vectors.
+    SDValue ForwardStoreValueToDirectLoad(LoadSDNode *LD);
+    bool getTruncatedStoreValue(StoreSDNode *ST, SDValue &Val);
+    bool extendLoadedValueToExtension(LoadSDNode *LD, SDValue &Val);
+
     /// Replace an ISD::EXTRACT_VECTOR_ELT of a load with a narrowed
     ///   load.
     ///
     /// \param EVE ISD::EXTRACT_VECTOR_ELT to be replaced.
     /// \param InVecVT type of the input vector to EVE with bitcasts resolved.
     /// \param EltNo index of the vector element to load.
     /// \param OriginalLoad load that EVE came from to be replaced.
     /// \returns EVE on success SDValue() on failure.
Inc = DAG.getConstant(*ConstInc->getConstantIntValue(), SDLoc(Inc),
                           ConstInc->getValueType(0));
   }
 
   unsigned Opc =
       (AM == ISD::PRE_INC || AM == ISD::POST_INC ? ISD::ADD : ISD::SUB);
   return DAG.getNode(Opc, SDLoc(LD), BP.getSimpleValueType(), BP, Inc);
 }
 
+static inline int numVectorEltsOrZero(EVT T) {
+  return T.isVector() ? T.getVectorNumElements() : 0;
+}
+
+bool DAGCombiner::getTruncatedStoreValue(StoreSDNode *ST, SDValue &Val) {
+  Val = ST->getValue();
+  EVT STType = Val.getValueType();
+  EVT STMemType = ST->getMemoryVT();
+  if (STType == STMemType)
+    return true;
+  if (isTypeLegal(STMemType))
+    return false; // fail.
+  if (STType.isFloatingPoint() && STMemType.isFloatingPoint() &&
+      TLI.isOperationLegal(ISD::FTRUNC, STMemType)) {
+    Val = DAG.getNode(ISD::FTRUNC, SDLoc(ST), STMemType, Val);
+    return true;
+  }
+  if (numVectorEltsOrZero(STType) == numVectorEltsOrZero(STMemType) &&
+      STType.isInteger() && STMemType.isInteger()) {
+    Val = DAG.getNode(ISD::TRUNCATE, SDLoc(ST), STMemType, Val);
+    return true;
+  }
+  if (STType.getSizeInBits() == STMemType.getSizeInBits()) {
+    Val = DAG.getBitcast(STMemType, Val);
+    return true;
+  }
+  return false; // fail.
+}
+
+bool DAGCombiner::extendLoadedValueToExtension(LoadSDNode *LD, SDValue &Val) {
+  EVT LDMemType = LD->getMemoryVT();
+  EVT LDType = LD->getValueType(0);
+  assert(Val.getValueType() == LDMemType &&
+         "Attempting to extend value of non-matching type");
+  if (LDType == LDMemType)
+    return true;
+  if (LDMemType.isInteger() && LDType.isInteger()) {
+    switch (LD->getExtensionType()) {
+    case ISD::NON_EXTLOAD:
+      Val = DAG.getBitcast(LDType, Val);
+      return true;
+    case ISD::EXTLOAD:
+      Val = DAG.getNode(ISD::ANY_EXTEND, SDLoc(LD), LDType, Val);
+      return true;
+    case ISD::SEXTLOAD:
+      Val = DAG.getNode(ISD::SIGN_EXTEND, SDLoc(LD), LDType, Val);
+      return true;
+    case ISD::ZEXTLOAD:
+      Val = DAG.getNode(ISD::ZERO_EXTEND, SDLoc(LD), LDType, Val);
+      return true;
+    }
+  }
+  return false;
+}
+
+SDValue DAGCombiner::ForwardStoreValueToDirectLoad(LoadSDNode *LD) {
+  if (OptLevel == CodeGenOpt::None || LD->isVolatile())
+    return SDValue();
+  SDValue Chain = LD->getOperand(0);
+  StoreSDNode *ST = dyn_cast<StoreSDNode>(Chain.getNode());
+  if (!ST || ST->isVolatile())
+    return SDValue();
+
+  EVT LDType = LD->getValueType(0);
+  EVT LDMemType = LD->getMemoryVT();
+  EVT STMemType = ST->getMemoryVT();
+  EVT STType = ST->getValue().getValueType();
+
+  BaseIndexOffset BasePtrLD = BaseIndexOffset::match(LD, DAG);
+  BaseIndexOffset BasePtrST = BaseIndexOffset::match(ST, DAG);
+  int64_t Offset;
+
+  bool STCoversLD =
+      BasePtrST.equalBaseIndex(BasePtrLD, DAG, Offset) && (Offset >= 0) &&
+      (Offset * 8 <= LDMemType.getSizeInBits()) &&
+      (Offset * 8 + LDMemType.getSizeInBits() <= STMemType.getSizeInBits());
+
+  if (!STCoversLD)
+    return SDValue();
+
+  // Memory as copy space (potentially masked).
+  if (Offset == 0 && LDType == STType && STMemType == LDMemType) {
+    // Simple case: Direct non-truncating forwarding
+    if (LDType.getSizeInBits() == LDMemType.getSizeInBits())
+      return CombineTo(LD, ST->getValue(), Chain);
+    // Can we model the truncate and extension with an and mask?
+    if (STType.isInteger() && LDMemType.isInteger() && !STType.isVector() &&
+        !LDMemType.isVector() && LD->getExtensionType() != ISD::SEXTLOAD) {
+      // Mask to size of LDMemType
+      auto Mask =
+          DAG.getConstant(APInt::getLowBitsSet(STType.getSizeInBits(),
+                                               STMemType.getSizeInBits()),
+                          SDLoc(ST), STType);
+      auto Val = DAG.getNode(ISD::AND, SDLoc(LD), LDType, ST->getValue(), Mask);
+      return CombineTo(LD, Val, Chain);
+    }
+  }
+
+  // TODO: Deal with nonzero offset.
+  if (LD->getBasePtr().isUndef() || Offset != 0)
+    return SDValue();
+  // Model necessary truncations / extenstions.
+  SDValue Val;
+  // Truncate Value To Stored Memory Size.
+  do {
+    if (!getTruncatedStoreValue(ST, Val))
+      continue;
+    if (!isTypeLegal(LDMemType))
+      continue;
+    if (STMemType != LDMemType) {
+      if (numVectorEltsOrZero(STMemType) == numVectorEltsOrZero(LDMemType) &&
+          STMemType.isInteger() && LDMemType.isInteger())
+        Val = DAG.getNode(ISD::TRUNCATE, SDLoc(LD), LDMemType, Val);
+      else
+        continue;
+    }
+    if (!extendLoadedValueToExtension(LD, Val))
+      continue;
+    return CombineTo(LD, Val, Chain);
+  } while (false);
+
+  // On failure, cleanup dead nodes we may have created.
+  if (Val->use_empty())
+    deleteAndRecombine(Val.getNode());
+  return SDValue();
+}
+
 SDValue DAGCombiner::visitLOAD(SDNode *N) {
   LoadSDNode *LD  = cast<LoadSDNode>(N);
   SDValue Chain = LD->getChain();
   SDValue Ptr   = LD->getBasePtr();
 
   // If load is not volatile and there are no uses of the loaded value (and
   // the updated indexed value in case of indexed loads), change uses of the
   // chain value into uses of the chain input (i.e. delete the dead load).
if (N->getValueType(1) == MVT::Other) {
         deleteAndRecombine(N);
         return SDValue(N, 0);   // Return N so it doesn't get rechecked!
       }
     }
   }
 
   // If this load is directly stored, replace the load value with the stored
   // value.
-  // TODO: Handle store large -> read small portion.
-  // TODO: Handle TRUNCSTORE/LOADEXT
-  if (OptLevel != CodeGenOpt::None &&
-      ISD::isNormalLoad(N) && !LD->isVolatile()) {
-    if (ISD::isNON_TRUNCStore(Chain.getNode())) {
-      StoreSDNode *PrevST = cast<StoreSDNode>(Chain);
-      if (PrevST->getBasePtr() == Ptr &&
-          PrevST->getValue().getValueType() == N->getValueType(0))
-        return CombineTo(N, PrevST->getOperand(1), Chain);
-    }
-  }
+  if (auto V = ForwardStoreValueToDirectLoad(LD))
+    return V;
 
   // Try to infer better alignment information than the load already has.
   if (OptLevel != CodeGenOpt::None && LD->isUnindexed()) {
     if (unsigned Align = DAG.InferPtrAlignment(Ptr)) {
       if (Align > LD->getAlignment() && LD->getSrcValueOffset() % Align == 0) {
         SDValue NewLoad = DAG.getExtLoad(
             LD->getExtensionType(), SDLoc(N), LD->getValueType(0), Chain, Ptr,
             LD->getPointerInfo(), LD->getMemoryVT(), Align,

llvm/trunk/test/CodeGen/AArch64/arm64-ld-from-st.ll

 ; RUN: llc < %s -mtriple aarch64--none-eabi -verify-machineinstrs | FileCheck %s
 
 ; CHECK-LABEL: Str64Ldr64
 ; CHECK: mov x0, x1
 define i64 @Str64Ldr64(i64* nocapture %P, i64 %v, i64 %n) {
 entry:
   %0 = bitcast i64* %P to i64*
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i64, i64* %0, i64 1
   %1 = load i64, i64* %arrayidx1
   ret i64 %1
 }
 
 ; CHECK-LABEL: Str64Ldr32_0
-; CHECK: and x0, x1, #0xffffffff
+; CHECK: mov w0, w1
 define i32 @Str64Ldr32_0(i64* nocapture %P, i64 %v, i64 %n) {
 entry:
   %0 = bitcast i64* %P to i32*
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i32, i32* %0, i64 2
   %1 = load i32, i32* %arrayidx1
   ret i32 %1
 }
 
 ; CHECK-LABEL: Str64Ldr32_1
 ; CHECK: lsr x0, x1, #32
 define i32 @Str64Ldr32_1(i64* nocapture %P, i64 %v, i64 %n) {
 entry:
   %0 = bitcast i64* %P to i32*
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i32, i32* %0, i64 3
   %1 = load i32, i32* %arrayidx1
   ret i32 %1
 }
 
 ; CHECK-LABEL: Str64Ldr16_0
-; CHECK: and x0, x1, #0xffff
+; CHECK: mov w0, w1
 define i16 @Str64Ldr16_0(i64* nocapture %P, i64 %v, i64 %n) {
 entry:
   %0 = bitcast i64* %P to i16*
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 4
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
entry:
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 7
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
 }
 
 ; CHECK-LABEL: Str64Ldr8_0
-; CHECK: and x0, x1, #0xff
+; CHECK: mov w0, w1
 define i8 @Str64Ldr8_0(i64* nocapture %P, i64 %v, i64 %n) {
 entry:
   %0 = bitcast i64* %P to i8*
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i8, i8* %0, i64 8
   %1 = load i8, i8* %arrayidx1
   ret i8 %1
entry:
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i32, i32* %0, i64 1
   %1 = load i32, i32* %arrayidx1
   ret i32 %1
 }
 
 ; CHECK-LABEL: Str32Ldr16_0
-; CHECK: and w0, w1, #0xffff
+; CHECK: mov w0, w1
 define i16 @Str32Ldr16_0(i32* nocapture %P, i32 %v, i64 %n) {
 entry:
   %0 = bitcast i32* %P to i16*
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 2
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
 }
 
 ; CHECK-LABEL: Str32Ldr16_1
 ; CHECK: lsr  w0, w1, #16
 define i16 @Str32Ldr16_1(i32* nocapture %P, i32 %v, i64 %n) {
 entry:
   %0 = bitcast i32* %P to i16*
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 3
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
 }
 
 ; CHECK-LABEL: Str32Ldr8_0
-; CHECK: and w0, w1, #0xff
+; CHECK: mov w0, w1
 define i8 @Str32Ldr8_0(i32* nocapture %P, i32 %v, i64 %n) {
 entry:
   %0 = bitcast i32* %P to i8*
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i8, i8* %0, i64 4
   %1 = load i8, i8* %arrayidx1
   ret i8 %1
entry:
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i8, i8* %0, i64 7
   %1 = load i8, i8* %arrayidx1
   ret i8 %1
 }
 
 ; CHECK-LABEL: Str16Ldr16
-; CHECK: and w0, w1, #0xffff
+; CHECK: mov w0, w1
 define i16 @Str16Ldr16(i16* nocapture %P, i16 %v, i64 %n) {
 entry:
   %0 = bitcast i16* %P to i16*
   %arrayidx0 = getelementptr inbounds i16, i16* %P, i64 1
   store i16 %v, i16* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 1
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
 }
 
 ; CHECK-LABEL: Str16Ldr8_0
-; CHECK: and w0, w1, #0xff
+; CHECK: mov w0, w1
 define i8 @Str16Ldr8_0(i16* nocapture %P, i16 %v, i64 %n) {
 entry:
   %0 = bitcast i16* %P to i8*
   %arrayidx0 = getelementptr inbounds i16, i16* %P, i64 1
   store i16 %v, i16* %arrayidx0
   %arrayidx1 = getelementptr inbounds i8, i8* %0, i64 2
   %1 = load i8, i8* %arrayidx1
   ret i8 %1
entry:
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 -1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i64, i64* %0, i64 -1
   %1 = load i64, i64* %arrayidx1
   ret i64 %1
 }
 
 ; CHECK-LABEL: Unscaled_Str64Ldr32_0
-; CHECK: and x0, x1, #0xffffffff
+; CHECK: mov w0, w1
 define i32 @Unscaled_Str64Ldr32_0(i64* nocapture %P, i64 %v, i64 %n) {
 entry:
   %0 = bitcast i64* %P to i32*
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 -1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i32, i32* %0, i64 -2
   %1 = load i32, i32* %arrayidx1
   ret i32 %1
 }
 
 ; CHECK-LABEL: Unscaled_Str64Ldr32_1
 ; CHECK: lsr x0, x1, #32
 define i32 @Unscaled_Str64Ldr32_1(i64* nocapture %P, i64 %v, i64 %n) {
 entry:
   %0 = bitcast i64* %P to i32*
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 -1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i32, i32* %0, i64 -1
   %1 = load i32, i32* %arrayidx1
   ret i32 %1
 }
 
 ; CHECK-LABEL: Unscaled_Str64Ldr16_0
-; CHECK: and x0, x1, #0xffff
+; CHECK: mov w0, w1
 define i16 @Unscaled_Str64Ldr16_0(i64* nocapture %P, i64 %v, i64 %n) {
 entry:
   %0 = bitcast i64* %P to i16*
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 -1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 -4
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
entry:
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 -1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 -1
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
 }
 
 ; CHECK-LABEL: Unscaled_Str64Ldr8_0
-; CHECK: and x0, x1, #0xff
+; CHECK: mov w0, w1
 define i8 @Unscaled_Str64Ldr8_0(i64* nocapture %P, i64 %v, i64 %n) {
 entry:
   %0 = bitcast i64* %P to i8*
   %arrayidx0 = getelementptr inbounds i64, i64* %P, i64 -1
   store i64 %v, i64* %arrayidx0
   %arrayidx1 = getelementptr inbounds i8, i8* %0, i64 -8
   %1 = load i8, i8* %arrayidx1
   ret i8 %1
entry:
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 -1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i32, i32* %0, i64 -1
   %1 = load i32, i32* %arrayidx1
   ret i32 %1
 }
 
 ; CHECK-LABEL: Unscaled_Str32Ldr16_0
-; CHECK: and w0, w1, #0xffff
+; CHECK: mov w0, w1
 define i16 @Unscaled_Str32Ldr16_0(i32* nocapture %P, i32 %v, i64 %n) {
 entry:
   %0 = bitcast i32* %P to i16*
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 -1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 -2
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
 }
 
 ; CHECK-LABEL: Unscaled_Str32Ldr16_1
 ; CHECK: lsr  w0, w1, #16
 define i16 @Unscaled_Str32Ldr16_1(i32* nocapture %P, i32 %v, i64 %n) {
 entry:
   %0 = bitcast i32* %P to i16*
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 -1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 -1
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
 }
 
 ; CHECK-LABEL: Unscaled_Str32Ldr8_0
-; CHECK: and w0, w1, #0xff
+; CHECK: mov w0, w1
 define i8 @Unscaled_Str32Ldr8_0(i32* nocapture %P, i32 %v, i64 %n) {
 entry:
   %0 = bitcast i32* %P to i8*
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 -1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i8, i8* %0, i64 -4
   %1 = load i8, i8* %arrayidx1
   ret i8 %1
entry:
   %arrayidx0 = getelementptr inbounds i32, i32* %P, i64 -1
   store i32 %v, i32* %arrayidx0
   %arrayidx1 = getelementptr inbounds i8, i8* %0, i64 -1
   %1 = load i8, i8* %arrayidx1
   ret i8 %1
 }
 
 ; CHECK-LABEL: Unscaled_Str16Ldr16
-; CHECK: and w0, w1, #0xffff
+; CHECK: mov w0, w1
 define i16 @Unscaled_Str16Ldr16(i16* nocapture %P, i16 %v, i64 %n) {
 entry:
   %0 = bitcast i16* %P to i16*
   %arrayidx0 = getelementptr inbounds i16, i16* %P, i64 -1
   store i16 %v, i16* %arrayidx0
   %arrayidx1 = getelementptr inbounds i16, i16* %0, i64 -1
   %1 = load i16, i16* %arrayidx1
   ret i16 %1
 }
 
 ; CHECK-LABEL: Unscaled_Str16Ldr8_0
-; CHECK: and w0, w1, #0xff
+; CHECK: mov w0, w1
 define i8 @Unscaled_Str16Ldr8_0(i16* nocapture %P, i16 %v, i64 %n) {
 entry:
   %0 = bitcast i16* %P to i8*
   %arrayidx0 = getelementptr inbounds i16, i16* %P, i64 -1
   store i16 %v, i16* %arrayidx0
   %arrayidx1 = getelementptr inbounds i8, i8* %0, i64 -2
   %1 = load i8, i8* %arrayidx1
   ret i8 %1

llvm/trunk/test/CodeGen/AArch64/regress-tblgen-chains.ll

; CHECK: bl {{_?bar}}
 
   %inc.1 = load i8, i8* %locvar
   %inc.2 = zext i8 %inc.1 to i64
   %inc.3 = add i64 %inc.2, 1
   %inc.4 = trunc i64 %inc.3 to i8
   store i8 %inc.4, i8* %locvar
 
 ; CHECK: ldurb {{w[0-9]+}}, [x29, [[LOCADDR:#-?[0-9]+]]]
-; CHECK: add {{w[0-9]+}}, {{w[0-9]+}}, #1
+; CHECK: add {{x[0-9]+}}, {{x[0-9]+}}, #1
 ; CHECK: sturb w[[STRVAL:[0-9]+]], [x29, [[LOCADDR]]]
-; CHECK: and w0, w[[STRVAL]], #0xff
+; CHECK: and x0, x[[STRVAL]], #0xff
 
   %ret.1 = load i8, i8* %locvar
   %ret.2 = zext i8 %ret.1 to i64
   ret i64 %ret.2
 ; CHECK: ret
 }

llvm/trunk/test/CodeGen/Hexagon/clr_set_toggle.ll

; CHECK: r{{[0-9]+}} = clrbit(r{{[0-9]+}},#27)
   %0 = load i64, i64* %x.addr, align 8
   %and = and i64 %0, -576460752303423489
   ret i64 %and
 }
 
 define zeroext i16 @my_setbit(i16 zeroext %crc) nounwind {
 entry:
 ; CHECK-LABEL: my_setbit
-; CHECK: memh(r{{[0-9]+}}+#{{[0-9]+}}) = setbit(#15)
+; CHECK: r{{[0-9]+}} = setbit(r{{[0-9]+}},#15)
   %crc.addr = alloca i16, align 2
   store i16 %crc, i16* %crc.addr, align 2
   %0 = load i16, i16* %crc.addr, align 2
   %conv = zext i16 %0 to i32
   %or = or i32 %conv, 32768
   %conv1 = trunc i32 %or to i16
   store i16 %conv1, i16* %crc.addr, align 2
   %1 = load i16, i16* %crc.addr, align 2

llvm/trunk/test/CodeGen/Hexagon/swp-epilog-phis.ll

-; RUN: llc -march=hexagon -enable-pipeliner -pipeliner-max-stages=2 \
-; RUN:     -pipeliner-ignore-recmii -disable-hexagon-nv-schedule \
-; RUN:     -hexagon-initial-cfg-cleanup=0 -stats -o /dev/null \
-; RUN:     -enable-aa-sched-mi < %s 2>&1 | FileCheck %s --check-prefix=STATS
-; REQUIRES: asserts
-;
-; Test that we generate the correct phis in the last epilog block when
-; allowing multiple stages.
-;
-; STATS: 1 pipeliner        - Number of loops software pipelined
-
-; Function Attrs: nounwind
-define void @f0() #0 {
-b0:
-  br i1 undef, label %b6, label %b1
-
-b1:                                               ; preds = %b0
-  br i1 undef, label %b6, label %b2
-
-b2:                                               ; preds = %b1
-  br label %b4
-
-b3:                                               ; preds = %b4, %b3
-  %v0 = add nsw i32 0, 57344
-  %v1 = trunc i32 %v0 to i16
-  store i16 %v1, i16* null, align 2, !tbaa !0
-  %v2 = getelementptr inbounds i8, i8* null, i32 undef
-  %v3 = load i8, i8* %v2, align 1, !tbaa !4
-  %v4 = zext i8 %v3 to i32
-  %v5 = shl nuw nsw i32 %v4, 6
-  %v6 = add nsw i32 %v5, 57344
-  %v7 = trunc i32 %v6 to i16
-  store i16 %v7, i16* undef, align 2, !tbaa !0
-  br i1 undef, label %b5, label %b3
-
-b4:                                               ; preds = %b5, %b2
-  %v8 = phi i32 [ 0, %b2 ], [ %v9, %b5 ]
-  br label %b3
-
-b5:                                               ; preds = %b3
-  %v9 = add i32 %v8, 1
-  %v10 = icmp eq i32 %v9, undef
-  br i1 %v10, label %b6, label %b4
-
-b6:                                               ; preds = %b5, %b1, %b0
-  ret void
-}
-
-attributes #0 = { nounwind "target-cpu"="hexagonv55" }
-
-!0 = !{!1, !1, i64 0}
-!1 = !{!"short", !2}
-!2 = !{!"omnipotent char", !3}
-!3 = !{!"Simple C/C++ TBAA"}
-!4 = !{!2, !2, i64 0}

llvm/trunk/test/CodeGen/Hexagon/swp-memrefs-epilog1.ll

-; RUN: llc -march=hexagon -enable-pipeliner < %s | FileCheck %s
-
-; Test that a store and load, that alias, are not put in the same packet. The
-; pipeliner altered the size of the memrefs for these instructions, which
-; resulted in no order dependence between the instructions in the DAG. No order
-; dependence was added since the size was set to UINT_MAX, but there is a
-; computation using the size that overflowed.
-
-; CHECK: endloop0
-; CHECK: memh([[REG:r([0-9]+)]]+#0) =
-; CHECK: = memh([[REG]]++#2)
-
-; Function Attrs: nounwind
-define signext i16 @f0(i16* nocapture readonly %a0, i16* nocapture readonly %a1) local_unnamed_addr #0 {
-b0:
-  %v0 = alloca [40 x i16], align 8
-  %v1 = bitcast [40 x i16]* %v0 to i8*
-  call void @llvm.lifetime.start.p0i8(i64 80, i8* nonnull %v1) #2
-  %v2 = getelementptr inbounds [40 x i16], [40 x i16]* %v0, i32 0, i32 0
-  br label %b1
-
-b1:                                               ; preds = %b1, %b0
-  %v3 = phi i16* [ %a1, %b0 ], [ %v24, %b1 ]
-  %v4 = phi i16* [ %v2, %b0 ], [ %v25, %b1 ]
-  %v5 = phi i32 [ 0, %b0 ], [ %v14, %b1 ]
-  %v6 = phi i32 [ 1, %b0 ], [ %v22, %b1 ]
-  %v7 = phi i32 [ 0, %b0 ], [ %v23, %b1 ]
-  %v8 = load i16, i16* %v3, align 2
-  %v9 = sext i16 %v8 to i32
-  %v10 = tail call i32 @llvm.hexagon.A2.aslh(i32 %v9)
-  %v11 = tail call i32 @llvm.hexagon.S2.asr.r.r.sat(i32 %v10, i32 1)
-  %v12 = tail call i32 @llvm.hexagon.A2.asrh(i32 %v11)
-  %v13 = trunc i32 %v12 to i16
-  store i16 %v13, i16* %v4, align 2
-  %v14 = add nuw nsw i32 %v5, 1
-  %v15 = icmp eq i32 %v14, 40
-  %v16 = getelementptr inbounds i16, i16* %a0, i32 %v7
-  %v17 = load i16, i16* %v16, align 2
-  %v18 = sext i16 %v17 to i32
-  %v19 = getelementptr inbounds [40 x i16], [40 x i16]* %v0, i32 0, i32 %v7
-  %v20 = load i16, i16* %v19, align 2
-  %v21 = sext i16 %v20 to i32
-  %v22 = tail call i32 @llvm.hexagon.M2.mpy.acc.sat.ll.s1(i32 %v6, i32 %v18, i32 %v21)
-  %v23 = add nuw nsw i32 %v7, 1
-  %v24 = getelementptr i16, i16* %v3, i32 1
-  %v25 = getelementptr i16, i16* %v4, i32 1
-  br i1 %v15, label %b2, label %b1
-
-b2:                                               ; preds = %b1
-  %v26 = tail call signext i16 @f1(i32 %v22) #0
-  %v27 = sext i16 %v26 to i32
-  %v28 = tail call i32 @llvm.hexagon.S2.asl.r.r.sat(i32 %v22, i32 %v27)
-  %v29 = tail call i32 @llvm.hexagon.A2.asrh(i32 %v28)
-  %v30 = shl i32 %v29, 16
-  %v31 = ashr exact i32 %v30, 16
-  %v32 = icmp slt i32 %v30, 65536
-  br label %b3
-
-b3:                                               ; preds = %b2
-  call void @llvm.lifetime.end.p0i8(i64 80, i8* nonnull %v1) #2
-  ret i16 0
-}
-
-; Function Attrs: argmemonly nounwind
-declare void @llvm.lifetime.start.p0i8(i64, i8* nocapture) #1
-
-; Function Attrs: nounwind readnone
-declare i32 @llvm.hexagon.S2.asr.r.r.sat(i32, i32) #2
-
-; Function Attrs: nounwind readnone
-declare i32 @llvm.hexagon.A2.aslh(i32) #2
-
-; Function Attrs: nounwind readnone
-declare i32 @llvm.hexagon.A2.asrh(i32) #2
-
-; Function Attrs: nounwind readnone
-declare i32 @llvm.hexagon.M2.mpy.acc.sat.ll.s1(i32, i32, i32) #2
-
-; Function Attrs: nounwind
-declare signext i16 @f1(i32) local_unnamed_addr #0
-
-; Function Attrs: nounwind readnone
-declare i32 @llvm.hexagon.S2.asl.r.r.sat(i32, i32) #2
-
-; Function Attrs: argmemonly nounwind
-declare void @llvm.lifetime.end.p0i8(i64, i8* nocapture) #1
-
-attributes #0 = { nounwind }
-attributes #1 = { argmemonly nounwind }
-attributes #2 = { nounwind readnone }

llvm/trunk/test/CodeGen/Mips/cconv/vector.ll

 ; MIPS64R5-LABEL: i32_2:
 ; MIPS64R5:       # %bb.0:
 ; MIPS64R5-NEXT:    daddiu $sp, $sp, -32
 ; MIPS64R5-NEXT:    .cfi_def_cfa_offset 32
 ; MIPS64R5-NEXT:    sd $5, 16($sp)
 ; MIPS64R5-NEXT:    sd $4, 24($sp)
 ; MIPS64R5-NEXT:    ldi.b $w0, 0
 ; MIPS64R5-NEXT:    lw $1, 20($sp)
-; MIPS64R5-NEXT:    lw $2, 16($sp)
 ; MIPS64R5-NEXT:    move.v $w1, $w0
-; MIPS64R5-NEXT:    insert.d $w1[0], $2
+; MIPS64R5-NEXT:    insert.d $w1[0], $5
 ; MIPS64R5-NEXT:    insert.d $w1[1], $1
-; MIPS64R5-NEXT:    lw $1, 24($sp)
-; MIPS64R5-NEXT:    insert.d $w0[0], $1
+; MIPS64R5-NEXT:    insert.d $w0[0], $4
 ; MIPS64R5-NEXT:    lw $1, 28($sp)
 ; MIPS64R5-NEXT:    insert.d $w0[1], $1
 ; MIPS64R5-NEXT:    addv.d $w0, $w0, $w1
 ; MIPS64R5-NEXT:    copy_s.d $1, $w0[0]
 ; MIPS64R5-NEXT:    copy_s.d $2, $w0[1]
 ; MIPS64R5-NEXT:    sw $2, 12($sp)
 ; MIPS64R5-NEXT:    sw $1, 8($sp)
 ; MIPS64R5-NEXT:    ld $2, 8($sp)

 ; MIPS64EB-NEXT:    nop
 ;
 ; MIPS32R5EB-LABEL: call_i8_2:
 ; MIPS32R5EB:       # %bb.0: # %entry
 ; MIPS32R5EB-NEXT:    addiu $sp, $sp, -32
 ; MIPS32R5EB-NEXT:    .cfi_def_cfa_offset 32
 ; MIPS32R5EB-NEXT:    sw $ra, 28($sp) # 4-byte Folded Spill
 ; MIPS32R5EB-NEXT:    .cfi_offset 31, -4
-; MIPS32R5EB-NEXT:    addiu $1, $zero, 1543
-; MIPS32R5EB-NEXT:    sh $1, 20($sp)
-; MIPS32R5EB-NEXT:    addiu $1, $zero, 3080
-; MIPS32R5EB-NEXT:    sh $1, 24($sp)
-; MIPS32R5EB-NEXT:    lhu $4, 20($sp)
-; MIPS32R5EB-NEXT:    lhu $5, 24($sp)
+; MIPS32R5EB-NEXT:    addiu $4, $zero, 1543
+; MIPS32R5EB-NEXT:    addiu $5, $zero, 3080
 ; MIPS32R5EB-NEXT:    jal i8_2
 ; MIPS32R5EB-NEXT:    nop
 ; MIPS32R5EB-NEXT:    sw $2, 16($sp)
 ; MIPS32R5EB-NEXT:    lui $1, %hi(gv2i8)
 ; MIPS32R5EB-NEXT:    lhu $2, 16($sp)
 ; MIPS32R5EB-NEXT:    sh $2, %lo(gv2i8)($1)
 ; MIPS32R5EB-NEXT:    lw $ra, 28($sp) # 4-byte Folded Reload
 ; MIPS32R5EB-NEXT:    addiu $sp, $sp, 32

 ; MIPS64EL-NEXT:    nop
 ;
 ; MIPS32R5EL-LABEL: call_i8_2:
 ; MIPS32R5EL:       # %bb.0: # %entry
 ; MIPS32R5EL-NEXT:    addiu $sp, $sp, -32
 ; MIPS32R5EL-NEXT:    .cfi_def_cfa_offset 32
 ; MIPS32R5EL-NEXT:    sw $ra, 28($sp) # 4-byte Folded Spill
 ; MIPS32R5EL-NEXT:    .cfi_offset 31, -4
-; MIPS32R5EL-NEXT:    addiu $1, $zero, 1798
-; MIPS32R5EL-NEXT:    sh $1, 20($sp)
-; MIPS32R5EL-NEXT:    addiu $1, $zero, 2060
-; MIPS32R5EL-NEXT:    sh $1, 24($sp)
-; MIPS32R5EL-NEXT:    lhu $4, 20($sp)
-; MIPS32R5EL-NEXT:    lhu $5, 24($sp)
+; MIPS32R5EL-NEXT:    addiu $4, $zero, 1798
+; MIPS32R5EL-NEXT:    addiu $5, $zero, 2060
 ; MIPS32R5EL-NEXT:    jal i8_2
 ; MIPS32R5EL-NEXT:    nop
 ; MIPS32R5EL-NEXT:    sw $2, 16($sp)
 ; MIPS32R5EL-NEXT:    lui $1, %hi(gv2i8)
 ; MIPS32R5EL-NEXT:    lhu $2, 16($sp)
 ; MIPS32R5EL-NEXT:    sh $2, %lo(gv2i8)($1)
 ; MIPS32R5EL-NEXT:    lw $ra, 28($sp) # 4-byte Folded Reload
 ; MIPS32R5EL-NEXT:    addiu $sp, $sp, 32

llvm/trunk/test/CodeGen/Mips/indirect-jump-hazard/jumptables.ll

 ; MIPS32R6-NEXT:    lw $2, 8($sp)
 ; MIPS32R6-NEXT:    jr $ra
 ; MIPS32R6-NEXT:    addiu $sp, $sp, 16
 ;
 ; MIPS64R2-LABEL: _Z3fooi:
 ; MIPS64R2:       # %bb.0: # %entry
 ; MIPS64R2-NEXT:    daddiu $sp, $sp, -16
 ; MIPS64R2-NEXT:    .cfi_def_cfa_offset 16
-; MIPS64R2-NEXT:    sw $4, 4($sp)
-; MIPS64R2-NEXT:    lwu $2, 4($sp)
+; MIPS64R2-NEXT:    dext $2, $4, 0, 32
 ; MIPS64R2-NEXT:    sltiu $1, $2, 7
 ; MIPS64R2-NEXT:    beqz $1, .LBB0_3
-; MIPS64R2-NEXT:    nop
+; MIPS64R2-NEXT:    sw $4, 4($sp)
 ; MIPS64R2-NEXT:  .LBB0_1: # %entry
 ; MIPS64R2-NEXT:    dsll $1, $2, 3
 ; MIPS64R2-NEXT:    lui $2, %highest(.LJTI0_0)
 ; MIPS64R2-NEXT:    daddiu $2, $2, %higher(.LJTI0_0)
 ; MIPS64R2-NEXT:    dsll $2, $2, 16
 ; MIPS64R2-NEXT:    daddiu $2, $2, %hi(.LJTI0_0)
 ; MIPS64R2-NEXT:    dsll $2, $2, 16
 ; MIPS64R2-NEXT:    daddu $1, $1, $2

 ; MIPS64R2-NEXT:    ld $2, 8($sp)
 ; MIPS64R2-NEXT:    jr $ra
 ; MIPS64R2-NEXT:    daddiu $sp, $sp, 16
 ;
 ; MIPS64R6-LABEL: _Z3fooi:
 ; MIPS64R6:       # %bb.0: # %entry
 ; MIPS64R6-NEXT:    daddiu $sp, $sp, -16
 ; MIPS64R6-NEXT:    .cfi_def_cfa_offset 16
-; MIPS64R6-NEXT:    sw $4, 4($sp)
-; MIPS64R6-NEXT:    lwu $2, 4($sp)
+; MIPS64R6-NEXT:    dext $2, $4, 0, 32
 ; MIPS64R6-NEXT:    sltiu $1, $2, 7
-; MIPS64R6-NEXT:    beqzc $1, .LBB0_3
+; MIPS64R6-NEXT:    beqz $1, .LBB0_3
+; MIPS64R6-NEXT:    sw $4, 4($sp)
 ; MIPS64R6-NEXT:  .LBB0_1: # %entry
 ; MIPS64R6-NEXT:    dsll $1, $2, 3
 ; MIPS64R6-NEXT:    lui $2, %highest(.LJTI0_0)
 ; MIPS64R6-NEXT:    daddiu $2, $2, %higher(.LJTI0_0)
 ; MIPS64R6-NEXT:    dsll $2, $2, 16
 ; MIPS64R6-NEXT:    daddiu $2, $2, %hi(.LJTI0_0)
 ; MIPS64R6-NEXT:    dsll $2, $2, 16
 ; MIPS64R6-NEXT:    daddu $1, $1, $2

 ;
 ; PIC-MIPS64R2-LABEL: _Z3fooi:
 ; PIC-MIPS64R2:       # %bb.0: # %entry
 ; PIC-MIPS64R2-NEXT:    daddiu $sp, $sp, -16
 ; PIC-MIPS64R2-NEXT:    .cfi_def_cfa_offset 16
 ; PIC-MIPS64R2-NEXT:    lui $1, %hi(%neg(%gp_rel(_Z3fooi)))
 ; PIC-MIPS64R2-NEXT:    daddu $1, $1, $25
 ; PIC-MIPS64R2-NEXT:    daddiu $2, $1, %lo(%neg(%gp_rel(_Z3fooi)))
-; PIC-MIPS64R2-NEXT:    sw $4, 4($sp)
-; PIC-MIPS64R2-NEXT:    lwu $3, 4($sp)
+; PIC-MIPS64R2-NEXT:    dext $3, $4, 0, 32
 ; PIC-MIPS64R2-NEXT:    sltiu $1, $3, 7
 ; PIC-MIPS64R2-NEXT:    beqz $1, .LBB0_3
-; PIC-MIPS64R2-NEXT:    nop
+; PIC-MIPS64R2-NEXT:    sw $4, 4($sp)
 ; PIC-MIPS64R2-NEXT:  .LBB0_1: # %entry
 ; PIC-MIPS64R2-NEXT:    dsll $1, $3, 3
 ; PIC-MIPS64R2-NEXT:    ld $3, %got_page(.LJTI0_0)($2)
 ; PIC-MIPS64R2-NEXT:    daddu $1, $1, $3
 ; PIC-MIPS64R2-NEXT:    ld $1, %got_ofst(.LJTI0_0)($1)
 ; PIC-MIPS64R2-NEXT:    daddu $1, $1, $2
 ; PIC-MIPS64R2-NEXT:    jr.hb $1
 ; PIC-MIPS64R2-NEXT:    nop

 ;
 ; PIC-MIPS64R6-LABEL: _Z3fooi:
 ; PIC-MIPS64R6:       # %bb.0: # %entry
 ; PIC-MIPS64R6-NEXT:    daddiu $sp, $sp, -16
 ; PIC-MIPS64R6-NEXT:    .cfi_def_cfa_offset 16
 ; PIC-MIPS64R6-NEXT:    lui $1, %hi(%neg(%gp_rel(_Z3fooi)))
 ; PIC-MIPS64R6-NEXT:    daddu $1, $1, $25
 ; PIC-MIPS64R6-NEXT:    daddiu $2, $1, %lo(%neg(%gp_rel(_Z3fooi)))
-; PIC-MIPS64R6-NEXT:    sw $4, 4($sp)
-; PIC-MIPS64R6-NEXT:    lwu $3, 4($sp)
+; PIC-MIPS64R6-NEXT:    dext $3, $4, 0, 32
 ; PIC-MIPS64R6-NEXT:    sltiu $1, $3, 7
-; PIC-MIPS64R6-NEXT:    beqzc $1, .LBB0_3
+; PIC-MIPS64R6-NEXT:    beqz $1, .LBB0_3
+; PIC-MIPS64R6-NEXT:    sw $4, 4($sp)
 ; PIC-MIPS64R6-NEXT:  .LBB0_1: # %entry
 ; PIC-MIPS64R6-NEXT:    dsll $1, $3, 3
 ; PIC-MIPS64R6-NEXT:    ld $3, %got_page(.LJTI0_0)($2)
 ; PIC-MIPS64R6-NEXT:    daddu $1, $1, $3
 ; PIC-MIPS64R6-NEXT:    ld $1, %got_ofst(.LJTI0_0)($1)
 ; PIC-MIPS64R6-NEXT:    daddu $1, $1, $2
 ; PIC-MIPS64R6-NEXT:    jr.hb $1
 ; PIC-MIPS64R6-NEXT:    nop

llvm/trunk/test/CodeGen/Mips/o32_cc_byval.ll

 ; CHECK-NEXT:    addu $gp, $2, $25
 ; CHECK-NEXT:    sw $6, 56($sp)
 ; CHECK-NEXT:    sw $7, 60($sp)
 ; CHECK-NEXT:    lw $4, 80($sp)
 ; CHECK-NEXT:    ldc1 $f0, 72($sp)
 ; CHECK-NEXT:    lw $1, 64($sp)
 ; CHECK-NEXT:    lw $2, 68($sp)
 ; CHECK-NEXT:    lh $3, 58($sp)
-; CHECK-NEXT:    lb $5, 56($sp)
+; CHECK-NEXT:    sll $5, $6, 24
+; CHECK-NEXT:    sra $5, $5, 24
 ; CHECK-NEXT:    swc1 $f12, 36($sp)
 ; CHECK-NEXT:    sw $5, 32($sp)
 ; CHECK-NEXT:    sw $3, 28($sp)
 ; CHECK-NEXT:    sw $2, 20($sp)
 ; CHECK-NEXT:    sw $1, 16($sp)
 ; CHECK-NEXT:    sw $7, 24($sp)
 ; CHECK-NEXT:    mfc1 $6, $f0
 ; CHECK-NEXT:    lw $25, %call16(callee4)($gp)

 ; CHECK-LABEL: f4:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lui $2, %hi(_gp_disp)
 ; CHECK-NEXT:    addiu $2, $2, %lo(_gp_disp)
 ; CHECK-NEXT:    addiu $sp, $sp, -48
 ; CHECK-NEXT:    sw $ra, 44($sp) # 4-byte Folded Spill
 ; CHECK-NEXT:    addu $gp, $2, $25
 ; CHECK-NEXT:    move $4, $7
-; CHECK-NEXT:    sw $5, 52($sp)
 ; CHECK-NEXT:    sw $6, 56($sp)
+; CHECK-NEXT:    sw $5, 52($sp)
 ; CHECK-NEXT:    sw $7, 60($sp)
 ; CHECK-NEXT:    lw $1, 80($sp)
-; CHECK-NEXT:    lb $2, 52($sp)
+; CHECK-NEXT:    sll $2, $5, 24
+; CHECK-NEXT:    sra $2, $2, 24
 ; CHECK-NEXT:    addiu $3, $zero, 4
 ; CHECK-NEXT:    lui $5, 16576
 ; CHECK-NEXT:    sw $5, 36($sp)
 ; CHECK-NEXT:    sw $2, 32($sp)
 ; CHECK-NEXT:    sw $3, 28($sp)
 ; CHECK-NEXT:    sw $1, 24($sp)
 ; CHECK-NEXT:    addiu $1, $zero, 3
 ; CHECK-NEXT:    sw $1, 16($sp)

llvm/trunk/test/CodeGen/Mips/o32_cc_vararg.ll

entry:
   store i32 %0, i32* %b, align 4
   %ap2 = bitcast i8** %ap to i8*
   call void @llvm.va_end(i8* %ap2)
   %tmp = load i32, i32* %b, align 4
   ret i32 %tmp
 
 ; CHECK-LABEL: va1:
 ; CHECK: addiu   $sp, $sp, -16
-; CHECK: sw      $5, 20($sp)
 ; CHECK: sw      $7, 28($sp)
 ; CHECK: sw      $6, 24($sp)
-; CHECK: lw      $2, 20($sp)
+; CHECK: sw      $5, 20($sp)
+; CHECK: move    $2, $5
 }
 
 ; check whether the variable double argument will be accessed from the 8-byte
 ; aligned location (i.e. whether the address is computed by adding 7 and
 ; clearing lower 3 bits)
 define double @va2(i32 %a, ...) nounwind {
 entry:
   %a.addr = alloca i32, align 4
entry:
   store i32 %0, i32* %b, align 4
   %ap2 = bitcast i8** %ap to i8*
   call void @llvm.va_end(i8* %ap2)
   %tmp = load i32, i32* %b, align 4
   ret i32 %tmp
 
 ; CHECK-LABEL: va3:
 ; CHECK: addiu   $sp, $sp, -16
-; CHECK: sw      $6, 24($sp)
 ; CHECK: sw      $7, 28($sp)
-; CHECK: lw      $2, 24($sp)
+; CHECK: sw      $6, 24($sp)
+; CHECK: move    $2, $6
 }
 
 ; double
 define double @va4(double %a, ...) nounwind {
 entry:
   %a.addr = alloca double, align 8
   %ap = alloca i8*, align 4
   %b = alloca double, align 8
entry:
   %ap2 = bitcast i8** %ap to i8*
   call void @llvm.va_end(i8* %ap2)
   %tmp = load i32, i32* %d, align 4
   ret i32 %tmp
 
 ; CHECK-LABEL: va5:
 ; CHECK: addiu   $sp, $sp, -24
 ; CHECK: sw      $7, 36($sp)
-; CHECK: lw      $2, 36($sp)
+; CHECK: move    $2, $7
 }
 
 ; double
 define double @va6(i32 %a, i32 %b, i32 %c, ...) nounwind {
 entry:
   %a.addr = alloca i32, align 4
   %b.addr = alloca i32, align 4
   %c.addr = alloca i32, align 4

llvm/trunk/test/CodeGen/PowerPC/addi-offset-fold.ll

entry:
   %a.4.a.4..sroa_cast = bitcast [5 x i8]* %a.4.a.4..sroa_idx to i40*
   %a.4.a.4.bf.load = load i40, i40* %a.4.a.4..sroa_cast, align 4
   %bf.lshr = lshr i40 %a.4.a.4.bf.load, 31
   %bf.lshr.tr = trunc i40 %bf.lshr to i32
   %bf.cast = and i32 %bf.lshr.tr, 127
   ret i32 %bf.cast
 
 ; CHECK-LABEL: @foo
-; FIXME: We don't need to do these stores/loads at all.
+; FIXME: We don't need to do these stores at all.
 ; CHECK-DAG: std 3, -24(1)
 ; CHECK-DAG: stb 4, -16(1)
-; CHECK-DAG: lbz [[REG1:[0-9]+]], -16(1)
+; CHECK-DAG: sldi [[REG3:[0-9]+]], 4, 32
 ; CHECK-DAG: lwz [[REG2:[0-9]+]], -20(1)
-; CHECK-DAG: sldi [[REG3:[0-9]+]], [[REG1]], 32
 ; CHECK-DAG: or [[REG4:[0-9]+]], [[REG2]], [[REG3]]
 ; CHECK: rldicl 3, [[REG4]], 33, 57
 ; CHECK: blr
 }
 
 attributes #0 = { nounwind "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "target-cpu"="ppc64le" }
 

llvm/trunk/test/CodeGen/SystemZ/store_nonbytesized_vecs.ll

 ; CHECK-NEXT:    vlgvb %r1, %v24, 12
 ; CHECK-NEXT:    rosbg %r0, %r1, 60, 60, 3
 ; CHECK-NEXT:    vlgvb %r1, %v24, 13
 ; CHECK-NEXT:    rosbg %r0, %r1, 61, 61, 2
 ; CHECK-NEXT:    vlgvb %r1, %v24, 14
 ; CHECK-NEXT:    rosbg %r0, %r1, 62, 62, 1
 ; CHECK-NEXT:    vlgvb %r1, %v24, 15
 ; CHECK-NEXT:    rosbg %r0, %r1, 63, 63, 0
-; CHECK-NEXT:    sth %r0, 160(%r15)
-; CHECK-NEXT:    lh %r2, 160(%r15)
+; CHECK-NEXT:    llhr %r2, %r0
 ; CHECK-NEXT:    aghi %r15, 168
 ; CHECK-NEXT:    br %r14
 {
   %res = bitcast <16 x i1> %src to i16
   ret i16 %res
 }
 
 ; Truncate a <8 x i32> vector to <8 x i31> and store it (test splitting).

llvm/trunk/test/CodeGen/X86/i386-shrink-wrapping.ll

 ;
 ; ENABLE-NEXT: pushl
 ; ENABLE-NEXT: pushl
 ; We must not use sub here otherwise we will clobber the eflags.
 ; ENABLE-NEXT: leal -20(%esp), %esp
 ;
 ; CHECK-NEXT: L_e$non_lazy_ptr, [[E:%[a-z]+]]
 ; CHECK-NEXT: movb %dl, ([[E]])
-; CHECK-NEXT: movsbl ([[E]]), [[CONV:%[a-z]+]]
+; CHECK-NEXT: movzbl %dl, [[CONV:%[a-z]+]]
 ; CHECK-NEXT: movl $6, [[CONV:%[a-z]+]]
 ; The eflags is used in the next instruction.
 ; If that instruction disappear, we are not exercising the bug
 ; anymore.
 ; CHECK-NEXT: cmovnel {{%[a-z]+}}, [[CONV]]
 ;
 ; Skip all the crust of vaarg lowering.
 ; CHECK: calll _varfunc

llvm/trunk/test/CodeGen/X86/pr32108.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s
 
 define void @pr32108() {
 ; CHECK-LABEL: pr32108:
 ; CHECK:       # %bb.0: # %BB
-; CHECK-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_1: # %CF244
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    jmp .LBB0_1
 BB:
   %Cmp45 = icmp slt <4 x i32> undef, undef
   br label %CF243
 

llvm/trunk/test/CodeGen/X86/pr38533.ll

 
 ; Similarly this makes sure that the opposite bitcast of the above is also legalized without crashing.
 define void @pr38533_2(half %x) {
 ; SSE-LABEL: pr38533_2:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    pushq %rax
 ; SSE-NEXT:    .cfi_def_cfa_offset 16
 ; SSE-NEXT:    callq __gnu_f2h_ieee
-; SSE-NEXT:    movw %ax, {{[0-9]+}}(%rsp)
-; SSE-NEXT:    movzwl {{[0-9]+}}(%rsp), %eax
 ; SSE-NEXT:    movw %ax, (%rax)
 ; SSE-NEXT:    popq %rax
 ; SSE-NEXT:    .cfi_def_cfa_offset 8
 ; SSE-NEXT:    retq
 ;
 ; AVX512-LABEL: pr38533_2:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vcvtps2ph $4, %xmm0, %xmm0
 ; AVX512-NEXT:    vmovd %xmm0, %eax
-; AVX512-NEXT:    movw %ax, -{{[0-9]+}}(%rsp)
-; AVX512-NEXT:    movzwl -{{[0-9]+}}(%rsp), %eax
 ; AVX512-NEXT:    movw %ax, (%rax)
 ; AVX512-NEXT:    retq
   %a = bitcast half %x to <4 x i4>
   store volatile <4 x i4> %a, <4 x i4>* undef
   ret void
 }
 
 ; This case is a bitcast from fp16 to a 16-bit wide legal vector type. In this case the result type is legal when the bitcast gets type legalized.
 define void @pr38533_3(half %x) {
 ; SSE-LABEL: pr38533_3:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    pushq %rax
 ; SSE-NEXT:    .cfi_def_cfa_offset 16
 ; SSE-NEXT:    callq __gnu_f2h_ieee
-; SSE-NEXT:    movw %ax, (%rsp)
-; SSE-NEXT:    movzwl (%rsp), %eax
 ; SSE-NEXT:    movw %ax, (%rax)
 ; SSE-NEXT:    popq %rax
 ; SSE-NEXT:    .cfi_def_cfa_offset 8
 ; SSE-NEXT:    retq
 ;
 ; AVX512-LABEL: pr38533_3:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vcvtps2ph $4, %xmm0, %xmm0
 ; AVX512-NEXT:    vmovd %xmm0, %eax
 ; AVX512-NEXT:    movw %ax, (%rax)
 ; AVX512-NEXT:    retq
   %a = bitcast half %x to <16 x i1>
   store volatile <16 x i1> %a, <16 x i1>* undef
   ret void
 }

llvm/trunk/test/CodeGen/X86/win64_vararg.ll

entry:
   store i32 %tmp, i32* %agg.result
   ret void
 }
 ; CHECK-LABEL: sret_arg:
 ; CHECK: pushq
 ; CHECK: movq %rcx, %rax
 ; CHECK-DAG: movq %r9, 40(%rsp)
 ; CHECK-DAG: movq %r8, 32(%rsp)
-; CHECK: movl 32(%rsp), %[[tmp:[^ ]*]]
-; CHECK: movl %[[tmp]], (%rax)
+; CHECK-DAG: leaq 36(%rsp), %[[sret:[^ ]*]]
+; CHECK-DAG: movl %r8d, (%rax)
+; CHECK-DAG: movq %[[sret]], (%rsp)
 ; CHECK: popq
 ; CHECK: retq