[DAG] Improve Aliasing of operations to static alloca

[niravd]

Summary

Memory accesses offset from frame indices may alias, e.g., we may merge write from function arguments passed on the stack when they are contiguous. As a result, when checking aliasing, we consider the underlying frame index's offset from the stack pointer.

Static allocs are realized as stack objects in SelectionDAG, but its offset is not set until post-DAG causing DAGCombiner's alias check to consider access to static allocas to frequently alias. Modify isAlias to consider access between static allocas and access from other frame objects to be considered aliasing.

Many test changes are included here. Most are fixes for tests which indirectly relied on our aliasing ability and needed to be modified to preserve their original intent.

The remaining tests have minor improvements due to relaxed ordering. The exception is CodeGen/X86/2011-10-19-widen_vselect.ll which has a minor degradation dispite though the pre-legalized DAG is improved.

[rnk]

lib/CodeGen/SelectionDAG/FunctionLoweringInfo.cpp

160 ↗

(On Diff #99516)

I don't like setting SPOffset once and then setting it again later during PEI. Passes will get different results over time, and that seems bad.

Why does TCO cause frame index aliasing? That seems like a problem we might be able to address in TCO.

I'd much rather let alias analysis assume that loads and stores based on different non-fixed frame indices can't alias. Fixed frame indices can probably alias in things like va_arg handling, but in that case we should have a reliable fixed offset here. It should hold that fixed and un-fixed stack objects don't overlap.

These offsets here also don't account for alignment.

[niravd]

New patch

[niravd]

lib/CodeGen/SelectionDAG/FunctionLoweringInfo.cpp

160 ↗

(On Diff #99516)

These concerns should all be resolved (explanation in new patch summary).

[rnk]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16712–16713

I think we could strengthen this to !MFI.isFixedObjectIndex(A->getIndex()). Fixed objects have offsets, and all non-fixed objects will be allocated in non-overlapping memory during PEI regardless of whether they have allocas or not.

[niravd]

Use IsFixedObjectIndex.

[niravd]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16712–16713

Seems reasonable. Changing this allows some reordering of statepoint updates.

[niravd]

rebase past extracted tests.

[rnk]

lgtm

[MatzeB]

Hi, this change leads to llvm-test-suite tramp3d-v4 failing (at least on macOS systems).

Example: http://lab.llvm.org:8080/green/job/perf_darwin_x86_O3flto/11000/

This should reproduce the problem:

$ cd test-suite
$ mkdir build
$ cd build
$ cmake -DCMAKE_C_COMPILER=/Users/mbraun/dev/public_llvm/assertbuild/bin/clang -GNinja ..
$ ninja tramp3d-v4
$ llvm-lit -v MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4.test

I will revert this in the meantime.

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

if (Chain != BetterChain) {
                                   BetterChain, Ptr, LD->getMemoryVT(),
                                   LD->getMemOperand());
       }
 
       // Create token factor to keep old chain connected.
       SDValue Token = DAG.getNode(ISD::TokenFactor, SDLoc(N),
                                   MVT::Other, Chain, ReplLoad.getValue(1));
 
-      // Make sure the new and old chains are cleaned up.
-      AddToWorklist(Token.getNode());
-
-      // Replace uses with load result and token factor. Don't add users
-      // to work list.
-      return CombineTo(N, ReplLoad.getValue(0), Token, false);
+      // Replace uses with load result and token factor
+      return CombineTo(N, ReplLoad.getValue(0), Token);
     }
   }
 
   // Try transforming N to an indexed load.
   if (CombineToPreIndexedLoadStore(N) || CombineToPostIndexedLoadStore(N))
     return SDValue(N, 0);
 
   // Try to slice up N to more direct loads if the slices are mapped to
bool DAGCombiner::isAlias(LSBaseSDNode *Op0, LSBaseSDNode *Op1) const {
 
   // Check for BaseIndexOffset matching.
   BaseIndexOffset BasePtr0 = BaseIndexOffset::match(Op0->getBasePtr(), DAG);
   BaseIndexOffset BasePtr1 = BaseIndexOffset::match(Op1->getBasePtr(), DAG);
   int64_t PtrDiff;
   if (BasePtr0.equalBaseIndex(BasePtr1, DAG, PtrDiff))
     return !((NumBytes0 <= PtrDiff) || (PtrDiff + NumBytes1 <= 0));
 
+  // If both BasePtr0 and BasePtr1 are FrameIndexes, we will not be
+  // able to calculate their relative offset if at least one arises
+  // from an alloca. However, these allocas cannot overlap and we
+  // can infer there is no alias.
+  if (auto *A = dyn_cast<FrameIndexSDNode>(BasePtr0.getBase()))
+    if (auto *B = dyn_cast<FrameIndexSDNode>(BasePtr1.getBase())) {
+      MachineFrameInfo &MFI = DAG.getMachineFunction().getFrameInfo();
+      if (!MFI.isFixedObjectIndex(A->getIndex()) ||
+          !MFI.isFixedObjectIndex(B->getIndex()))
+        return false;
+    }
+
   // FIXME: findBaseOffset and ConstantValue/GlobalValue/FrameIndex analysis
   // modified to use BaseIndexOffset.
 
   // Gather base node and offset information.
   SDValue Base0, Base1;
   int64_t Offset0, Offset1;
   const GlobalValue *GV0, *GV1;
   const void *CV0, *CV1;

test/CodeGen/AArch64/arm64-abi-varargs.ll

 ; RUN: llc < %s -mtriple=arm64-apple-ios7.0.0 -mcpu=cyclone -enable-misched=false | FileCheck %s
 
 ; rdar://13625505
 ; Here we have 9 fixed integer arguments the 9th argument in on stack, the
 ; varargs start right after at 8-byte alignment.
 define void @fn9(i32* %a1, i32 %a2, i32 %a3, i32 %a4, i32 %a5, i32 %a6, i32 %a7, i32 %a8, i32 %a9, ...) nounwind noinline ssp {
 ; CHECK-LABEL: fn9:
 ; 9th fixed argument
 ; CHECK: ldr {{w[0-9]+}}, [sp, #64]
 ; CHECK: add [[ARGS:x[0-9]+]], sp, #72
 ; CHECK: add {{x[0-9]+}}, [[ARGS]], #8
 ; First vararg
 ; CHECK: ldr {{w[0-9]+}}, [sp, #72]
-; CHECK: add {{x[0-9]+}}, {{x[0-9]+}}, #8
 ; Second vararg
-; CHECK: ldr {{w[0-9]+}}, [{{x[0-9]+}}]
+; CHECK: ldr {{w[0-9]+}}, [{{x[0-9]+}}], #8
 ; CHECK: add {{x[0-9]+}}, {{x[0-9]+}}, #8
 ; Third vararg
 ; CHECK: ldr {{w[0-9]+}}, [{{x[0-9]+}}]
   %1 = alloca i32, align 4
   %2 = alloca i32, align 4
   %3 = alloca i32, align 4
   %4 = alloca i32, align 4
   %5 = alloca i32, align 4

test/CodeGen/AArch64/arm64-abi_align.ll

; FAST: add {{w[0-9]+}}, w[[C]], w[[B]]
   %add7 = add i32 %add4, %conv6
   ret i32 %add7
 }
 
 ; For s1, we allocate a 22-byte space, pass its address via x1
 define i32 @caller42() #3 {
 entry:
 ; CHECK-LABEL: caller42
-; CHECK: str {{x[0-9]+}}, [sp, #48]
-; CHECK: str {{q[0-9]+}}, [sp, #32]
-; CHECK: str {{x[0-9]+}}, [sp, #16]
-; CHECK: str {{q[0-9]+}}, [sp]
+; CHECK-DAG: str {{x[0-9]+}}, [sp, #48]
+; CHECK-DAG: str {{q[0-9]+}}, [sp, #32]
+; CHECK-DAG: str {{x[0-9]+}}, [sp, #16]
+; CHECK-DAG: str {{q[0-9]+}}, [sp]
 ; CHECK: add x1, sp, #32
 ; CHECK: mov x2, sp
 ; Space for s1 is allocated at sp+32
 ; Space for s2 is allocated at sp
 
 ; FAST-LABEL: caller42
 ; FAST: sub sp, sp, #112
 ; Space for s1 is allocated at fp-24 = sp+72
declare i32 @f42_stack(i32 %i, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6,
                        i32 %i7, i32 %i8, i32 %i9, %struct.s42* nocapture %s1,
                        %struct.s42* nocapture %s2) #2
 
 define i32 @caller42_stack() #3 {
 entry:
 ; CHECK-LABEL: caller42_stack
 ; CHECK: sub sp, sp, #112
 ; CHECK: add x29, sp, #96
-; CHECK: stur {{x[0-9]+}}, [x29, #-16]
-; CHECK: stur {{q[0-9]+}}, [x29, #-32]
-; CHECK: str {{x[0-9]+}}, [sp, #48]
-; CHECK: str {{q[0-9]+}}, [sp, #32]
+; CHECK-DAG: stur {{x[0-9]+}}, [x29, #-16]
+; CHECK-DAG: stur {{q[0-9]+}}, [x29, #-32]
+; CHECK-DAG: str {{x[0-9]+}}, [sp, #48]
+; CHECK-DAG: str {{q[0-9]+}}, [sp, #32]
 ; Space for s1 is allocated at x29-32 = sp+64
 ; Space for s2 is allocated at sp+32
 ; CHECK: add x[[B:[0-9]+]], sp, #32
 ; CHECK: str x[[B]], [sp, #16]
 ; CHECK: sub x[[A:[0-9]+]], x29, #32
 ; Address of s1 is passed on stack at sp+8
 ; CHECK: str x[[A]], [sp, #8]
 ; CHECK: mov w[[C:[0-9]+]], #9
; FAST: add {{w[0-9]+}}, w[[C]], w[[B]]
   %add4 = add i32 %add3, %conv
   %add7 = add i32 %add4, %conv6
   ret i32 %add7
 }
 
 define i32 @caller43() #3 {
 entry:
 ; CHECK-LABEL: caller43
-; CHECK: str {{q[0-9]+}}, [sp, #48]
-; CHECK: str {{q[0-9]+}}, [sp, #32]
-; CHECK: str {{q[0-9]+}}, [sp, #16]
-; CHECK: str {{q[0-9]+}}, [sp]
+; CHECK-DAG: str {{q[0-9]+}}, [sp, #48]
+; CHECK-DAG: str {{q[0-9]+}}, [sp, #32]
+; CHECK-DAG: str {{q[0-9]+}}, [sp, #16]
+; CHECK-DAG: str {{q[0-9]+}}, [sp]
 ; CHECK: add x1, sp, #32
 ; CHECK: mov x2, sp
 ; Space for s1 is allocated at sp+32
 ; Space for s2 is allocated at sp
 
 ; FAST-LABEL: caller43
 ; FAST: add x29, sp, #64
 ; Space for s1 is allocated at sp+32
declare i32 @f43_stack(i32 %i, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6,
                        i32 %i7, i32 %i8, i32 %i9, %struct.s43* nocapture %s1,
                        %struct.s43* nocapture %s2) #2
 
 define i32 @caller43_stack() #3 {
 entry:
 ; CHECK-LABEL: caller43_stack
 ; CHECK: sub sp, sp, #112
 ; CHECK: add x29, sp, #96
-; CHECK: stur {{q[0-9]+}}, [x29, #-16]
-; CHECK: stur {{q[0-9]+}}, [x29, #-32]
-; CHECK: str {{q[0-9]+}}, [sp, #48]
-; CHECK: str {{q[0-9]+}}, [sp, #32]
+; CHECK-DAG: stur {{q[0-9]+}}, [x29, #-16]
+; CHECK-DAG: stur {{q[0-9]+}}, [x29, #-32]
+; CHECK-DAG: str {{q[0-9]+}}, [sp, #48]
+; CHECK-DAG: str {{q[0-9]+}}, [sp, #32]
 ; Space for s1 is allocated at x29-32 = sp+64
 ; Space for s2 is allocated at sp+32
 ; CHECK: add x[[B:[0-9]+]], sp, #32
 ; CHECK: str x[[B]], [sp, #16]
 ; CHECK: sub x[[A:[0-9]+]], x29, #32
 ; Address of s1 is passed on stack at sp+8
 ; CHECK: str x[[A]], [sp, #8]
 ; CHECK: mov w[[C:[0-9]+]], #9

test/CodeGen/AArch64/arm64-alloca-frame-pointer-offset.ll

 ; RUN: llc -mtriple=arm64-eabi -mcpu=cyclone < %s | FileCheck %s
 
 ; CHECK: foo
-; CHECK: str w[[REG0:[0-9]+]], [x19, #264]
-; CHECK: mov w[[REG1:[0-9]+]], w[[REG0]]
-; CHECK: str w[[REG1]], [x19, #132]
-
+; CHECK-DAG: str w[[REG0:[0-9]+]], [x19, #132]
+; CHECK-DAG: str w[[REG0]], [x19, #264]
 define i32 @foo(i32 %a) nounwind {
   %retval = alloca i32, align 4
   %a.addr = alloca i32, align 4
   %arr = alloca [32 x i32], align 4
   %i = alloca i32, align 4
   %arr2 = alloca [32 x i32], align 4
   %j = alloca i32, align 4
   store i32 %a, i32* %a.addr, align 4

test/CodeGen/AArch64/arm64-vext.ll

define void @test_vext_p16() nounwind ssp {
   %tmp7 = bitcast <8 x i8> %tmp5 to <4 x i16>
   %vext = shufflevector <4 x i16> %tmp6, <4 x i16> %tmp7, <4 x i32> <i32 3, i32 4, i32 5, i32 6>
   store <4 x i16> %vext, <4 x i16>* %xP16x4, align 8
   ret void
 }
 
 define void @test_vext_s32() nounwind ssp {
   ; CHECK-LABEL: test_vext_s32:
-  ; CHECK: {{ext.8.*#4}}
+  ; CHECK: {{rev64.2s.*}}
   %xS32x2 = alloca <2 x i32>, align 8
   %__a = alloca <2 x i32>, align 8
   %__b = alloca <2 x i32>, align 8
   %tmp = load <2 x i32>, <2 x i32>* %xS32x2, align 8
   store <2 x i32> %tmp, <2 x i32>* %__a, align 8
   %tmp1 = load <2 x i32>, <2 x i32>* %xS32x2, align 8
   store <2 x i32> %tmp1, <2 x i32>* %__b, align 8
   %tmp2 = load <2 x i32>, <2 x i32>* %__a, align 8
   %tmp3 = bitcast <2 x i32> %tmp2 to <8 x i8>
   %tmp4 = load <2 x i32>, <2 x i32>* %__b, align 8
   %tmp5 = bitcast <2 x i32> %tmp4 to <8 x i8>
   %tmp6 = bitcast <8 x i8> %tmp3 to <2 x i32>
   %tmp7 = bitcast <8 x i8> %tmp5 to <2 x i32>
   %vext = shufflevector <2 x i32> %tmp6, <2 x i32> %tmp7, <2 x i32> <i32 1, i32 2>
   store <2 x i32> %vext, <2 x i32>* %xS32x2, align 8
   ret void
 }
 
 define void @test_vext_u32() nounwind ssp {
   ; CHECK-LABEL: test_vext_u32:
-  ; CHECK: {{ext.8.*#4}}
+  ; CHECK: {{rev64.2s.*}}
   %xU32x2 = alloca <2 x i32>, align 8
   %__a = alloca <2 x i32>, align 8
   %__b = alloca <2 x i32>, align 8
   %tmp = load <2 x i32>, <2 x i32>* %xU32x2, align 8
   store <2 x i32> %tmp, <2 x i32>* %__a, align 8
   %tmp1 = load <2 x i32>, <2 x i32>* %xU32x2, align 8
   store <2 x i32> %tmp1, <2 x i32>* %__b, align 8
   %tmp2 = load <2 x i32>, <2 x i32>* %__a, align 8
   %tmp3 = bitcast <2 x i32> %tmp2 to <8 x i8>
   %tmp4 = load <2 x i32>, <2 x i32>* %__b, align 8
   %tmp5 = bitcast <2 x i32> %tmp4 to <8 x i8>
   %tmp6 = bitcast <8 x i8> %tmp3 to <2 x i32>
   %tmp7 = bitcast <8 x i8> %tmp5 to <2 x i32>
   %vext = shufflevector <2 x i32> %tmp6, <2 x i32> %tmp7, <2 x i32> <i32 1, i32 2>
   store <2 x i32> %vext, <2 x i32>* %xU32x2, align 8
   ret void
 }
 
 define void @test_vext_f32() nounwind ssp {
   ; CHECK-LABEL: test_vext_f32:
-  ; CHECK: {{ext.8.*#4}}
+  ; CHECK: {{rev64.2s.*}}
   %xF32x2 = alloca <2 x float>, align 8
   %__a = alloca <2 x float>, align 8
   %__b = alloca <2 x float>, align 8
   %tmp = load <2 x float>, <2 x float>* %xF32x2, align 8
   store <2 x float> %tmp, <2 x float>* %__a, align 8
   %tmp1 = load <2 x float>, <2 x float>* %xF32x2, align 8
   store <2 x float> %tmp1, <2 x float>* %__b, align 8
   %tmp2 = load <2 x float>, <2 x float>* %__a, align 8
   %tmp3 = bitcast <2 x float> %tmp2 to <8 x i8>
   %tmp4 = load <2 x float>, <2 x float>* %__b, align 8
   %tmp5 = bitcast <2 x float> %tmp4 to <8 x i8>
   %tmp6 = bitcast <8 x i8> %tmp3 to <2 x float>
   %tmp7 = bitcast <8 x i8> %tmp5 to <2 x float>
   %vext = shufflevector <2 x float> %tmp6, <2 x float> %tmp7, <2 x i32> <i32 1, i32 2>
   store <2 x float> %vext, <2 x float>* %xF32x2, align 8
   ret void
 }
 
 define void @test_vext_s64() nounwind ssp {
   ; CHECK-LABEL: test_vext_s64:
-  ; CHECK_FIXME: {{ext.8.*#1}}
+  ; CHECK_FIXME: {{rev64.2s.*}}
   ; this just turns into a load of the second element
   %xS64x1 = alloca <1 x i64>, align 8
   %__a = alloca <1 x i64>, align 8
   %__b = alloca <1 x i64>, align 8
   %tmp = load <1 x i64>, <1 x i64>* %xS64x1, align 8
   store <1 x i64> %tmp, <1 x i64>* %__a, align 8
   %tmp1 = load <1 x i64>, <1 x i64>* %xS64x1, align 8
   store <1 x i64> %tmp1, <1 x i64>* %__b, align 8

test/CodeGen/AArch64/dag-combine-invaraints.ll

 ; RUN: llc -mtriple=arm64-apple-darwin8.0 -relocation-model=pic -O1 < %s | FileCheck %s
 
 @.str2 = private unnamed_addr constant [9 x i8] c"_%d____\0A\00", align 1
 
 ; Function Attrs: nounwind ssp
 define i32 @main(i32 %argc, i8** %argv) #0 {
 main_:
   %tmp = alloca i32, align 4
   %i32T = alloca i32, align 4
   %i32F = alloca i32, align 4
   %i32X = alloca i32, align 4
-  store i32 0, i32* %tmp
+  store i32 %argc, i32* %tmp
   store i32 15, i32* %i32T, align 4
   store i32 5, i32* %i32F, align 4
   %tmp6 = load i32, i32* %tmp, align 4
   %tmp7 = icmp ne i32 %tmp6, 0
   %tmp8 = xor i1 %tmp7, true
   %tmp9 = load i32, i32* %i32T, align 4
   %tmp10 = load i32, i32* %i32F, align 4
   %DHSelect = select i1 %tmp8, i32 %tmp9, i32 %tmp10

test/CodeGen/AArch64/swifterror.ll

 
 ; "foo_vararg" is a function that takes a swifterror parameter, it also has
 ; variable number of arguments.
 declare void @llvm.va_start(i8*) nounwind
 define float @foo_vararg(%swift_error** swifterror %error_ptr_ref, ...) {
 ; CHECK-APPLE-LABEL: foo_vararg:
 ; CHECK-APPLE: orr w0, wzr, #0x10
 ; CHECK-APPLE: malloc
-; CHECK-APPLE: orr [[ID:w[0-9]+]], wzr, #0x1
-; CHECK-APPLE: add [[ARGS:x[0-9]+]], [[TMP:x[0-9]+]], #16
-; CHECK-APPLE: strb [[ID]], [x0, #8]
+; CHECK-APPLE-DAG: orr [[ID:w[0-9]+]], wzr, #0x1
+; CHECK-APPLE-DAG: add [[ARGS:x[0-9]+]], [[TMP:x[0-9]+]], #16
+; CHECK-APPLE-DAG: strb [[ID]], [x0, #8]
 
 ; First vararg
 ; CHECK-APPLE-DAG: orr {{x[0-9]+}}, [[ARGS]], #0x8
 ; CHECK-APPLE-DAG: ldr {{w[0-9]+}}, [{{.*}}[[TMP]], #16]
-; CHECK-APPLE: add {{x[0-9]+}}, {{x[0-9]+}}, #8
+; CHECK-APPLE-DAG: add {{x[0-9]+}}, {{x[0-9]+}}, #8
 ; Second vararg
-; CHECK-APPLE: ldr {{w[0-9]+}}, [{{x[0-9]+}}]
-; CHECK-APPLE: add {{x[0-9]+}}, {{x[0-9]+}}, #8
+; CHECK-APPLE-DAG: ldr {{w[0-9]+}}, [{{x[0-9]+}}], #8
+; CHECK-APPLE-DAG: add {{x[0-9]+}}, {{x[0-9]+}}, #16
 ; Third vararg
 ; CHECK-APPLE: ldr {{w[0-9]+}}, [{{x[0-9]+}}]
 
 ; CHECK-APPLE: mov x21, x0
 ; CHECK-APPLE-NOT: x21
 entry:
   %call = call i8* @malloc(i64 16)
   %call.0 = bitcast i8* %call to %swift_error*

test/CodeGen/ARM/atomic-op.ll

entry:
   store i8** %argv, i8*** %argv.addr
   store i32 0, i32* %val1
   store i32 31, i32* %val2
   store i32 3855, i32* %andt
   store i32 3855, i32* %ort
   store i32 3855, i32* %xort
   store i32 4, i32* %temp
   %tmp = load i32, i32* %temp
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: add
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_add_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_add_4
   ; CHECK-BAREMETAL: add
   ; CHECK-BAREMETAL-NOT: __sync
   %0 = atomicrmw add i32* %val1, i32 %tmp monotonic
   store i32 %0, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: sub
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_sub_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_sub_4
   ; CHECK-BAREMETAL: sub
   ; CHECK-BAREMETAL-NOT: __sync
   %1 = atomicrmw sub i32* %val2, i32 30 monotonic
   store i32 %1, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: add
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_add_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_add_4
   ; CHECK-BAREMETAL: add
   ; CHECK-BAREMETAL-NOT: __sync
   %2 = atomicrmw add i32* %val2, i32 1 monotonic
   store i32 %2, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: sub
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_sub_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_sub_4
   ; CHECK-BAREMETAL: sub
   ; CHECK-BAREMETAL-NOT: __sync
   %3 = atomicrmw sub i32* %val2, i32 1 monotonic
   store i32 %3, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: and
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_and_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_and_4
   ; CHECK-BAREMETAL: and
   ; CHECK-BAREMETAL-NOT: __sync
   %4 = atomicrmw and i32* %andt, i32 4080 monotonic
   store i32 %4, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: or
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_or_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_or_4
   ; CHECK-BAREMETAL: or
   ; CHECK-BAREMETAL-NOT: __sync
   %5 = atomicrmw or i32* %ort, i32 4080 monotonic
   store i32 %5, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: eor
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_xor_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_xor_4
   ; CHECK-BAREMETAL: eor
   ; CHECK-BAREMETAL-NOT: __sync
   %6 = atomicrmw xor i32* %xort, i32 4080 monotonic
   store i32 %6, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: cmp
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_min_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_min_4
   ; CHECK-BAREMETAL: cmp
   ; CHECK-BAREMETAL-NOT: __sync
   %7 = atomicrmw min i32* %val2, i32 16 monotonic
   store i32 %7, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   %neg = sub i32 0, 1
   ; CHECK: ldrex
   ; CHECK: cmp
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_min_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_min_4
   ; CHECK-BAREMETAL: cmp
   ; CHECK-BAREMETAL-NOT: __sync
   %8 = atomicrmw min i32* %val2, i32 %neg monotonic
   store i32 %8, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: cmp
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_max_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_max_4
   ; CHECK-BAREMETAL: cmp
   ; CHECK-BAREMETAL-NOT: __sync
   %9 = atomicrmw max i32* %val2, i32 1 monotonic
   store i32 %9, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: cmp
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_max_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_max_4
   ; CHECK-BAREMETAL: cmp
   ; CHECK-BAREMETAL-NOT: __sync
   %10 = atomicrmw max i32* %val2, i32 0 monotonic
   store i32 %10, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: cmp
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_umin_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_umin_4
   ; CHECK-BAREMETAL: cmp
   ; CHECK-BAREMETAL-NOT: __sync
   %11 = atomicrmw umin i32* %val2, i32 16 monotonic
   store i32 %11, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   %uneg = sub i32 0, 1
   ; CHECK: ldrex
   ; CHECK: cmp
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_umin_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_umin_4
   ; CHECK-BAREMETAL: cmp
   ; CHECK-BAREMETAL-NOT: __sync
   %12 = atomicrmw umin i32* %val2, i32 %uneg monotonic
   store i32 %12, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: cmp
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_umax_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_umax_4
   ; CHECK-BAREMETAL: cmp
   ; CHECK-BAREMETAL-NOT: __sync
   %13 = atomicrmw umax i32* %val2, i32 1 monotonic
   store i32 %13, i32* %old
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: ldrex
   ; CHECK: cmp
   ; CHECK: strex
   ; CHECK-T1: bl ___sync_fetch_and_umax_4
   ; CHECK-T1-M0: bl ___sync_fetch_and_umax_4
   ; CHECK-BAREMETAL: cmp
   ; CHECK-BAREMETAL-NOT: __sync
   %14 = atomicrmw umax i32* %val2, i32 0 monotonic

test/CodeGen/MSP430/vararg.ll

; CHECK-NEXT: mov.w 0(r12), r12
   ret i16 %0
 }
 
 define void @va_copy(i8* %vl) nounwind {
 entry:
 ; CHECK-LABEL: va_copy:
   %vl.addr = alloca i8*, align 2
   %vl2 = alloca i8*, align 2
-; CHECK: mov.w r12, 2(r1)
+; CHECK-DAG: mov.w r12, 2(r1)
   store i8* %vl, i8** %vl.addr, align 2
   %0 = bitcast i8** %vl2 to i8*
   %1 = bitcast i8** %vl.addr to i8*
-; CHECK-NEXT: mov.w r12, 0(r1)
+; CHECK-DAG: mov.w r12, 0(r1)
   call void @llvm.va_copy(i8* %0, i8* %1)
   ret void
 }

test/CodeGen/Mips/dins.ll

entry:
   %bufptr.sroa.0.0.bufptr.sroa.0.0.bufptr.sroa.0.0.bf.load17 = load volatile i64, i64* %bufptr.sroa.0, align 8
   %bf.lshr18 = lshr i64 %bufptr.sroa.0.0.bufptr.sroa.0.0.bufptr.sroa.0.0.bf.load17, 27
   ret i64 %bf.lshr18
 }
 
 ; CHECK-LABEL: f123:
 ; MIPS64R2: daddiu  $[[R0:[0-9]+]], $zero, 123
 ; MIPS64R2: dins    $[[R0:[0-9]+]], $[[R1:[0-9]+]], 27, 37
-; MIPS64R2: daddiu  $[[R0:[0-9]+]], $zero, 5
 ; MIPS64R2: daddiu  $[[R0:[0-9]+]], $zero, 4
 ; MIPS64R2: dins    $[[R0:[0-9]+]], $[[R1:[0-9]+]], 28, 6
+; MIPS64R2: daddiu  $[[R0:[0-9]+]], $zero, 5
 ; MIPS64R2: dins    $[[R0:[0-9]+]], $[[R1:[0-9]+]], 50, 14
 ; MIPS64R2: dsrl    $[[R0:[0-9]+]], $[[R1:[0-9]+]], 50
 ; MIPS64R2: dins    $[[R0:[0-9]+]], $[[R1:[0-9]+]], 34, 16
 ; MIPS32R2: ins     $[[R0:[0-9]+]], $[[R1:[0-9]+]], 2, 16
 ; MIPS32R2-NOT: ins $[[R0:[0-9]+]], $[[R1:[0-9]+]], 18, 46
 ; MIPS16-NOT: ins{{[[:space:]].*}}
 
 

 }
 
 ; CHECK-LABEL: foo:
 ; MIPS64R2:        ori  $[[R0:[0-9]+]], $[[R0:[0-9]+]], 8
 ; MIPS64R2-NOT:    ins {{[[:space:]].*}}
 ; MIPS32R2:        ori  $[[R0:[0-9]+]], $[[R0:[0-9]+]], 8
 ; MIPS32R2-NOT:    ins {{[[:space:]].*}}
 ; MIPS64R2N32:     ori  $[[R0:[0-9]+]], $[[R0:[0-9]+]], 8
-; MIPS64R2N32-NOT: ins {{[[:space:]].*}}
+; MIPS64R2N32-NOT: ins {{[[:space:]].*}}
 

test/CodeGen/X86/2011-10-19-widen_vselect.ll

 ; X32-NEXT:    cmpltps %xmm2, %xmm0
 ; X32-NEXT:    movaps {{.*#+}} xmm3 = <1,1,u,u>
 ; X32-NEXT:    addps %xmm1, %xmm3
 ; X32-NEXT:    movaps %xmm1, %xmm4
 ; X32-NEXT:    blendvps %xmm0, %xmm3, %xmm4
 ; X32-NEXT:    cmpeqps %xmm2, %xmm1
 ; X32-NEXT:    movaps %xmm1, %xmm0
 ; X32-NEXT:    blendvps %xmm0, %xmm2, %xmm4
-; X32-NEXT:    extractps $1, %xmm4, {{[0-9]+}}(%esp)
 ; X32-NEXT:    movss %xmm4, {{[0-9]+}}(%esp)
-; X32-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; X32-NEXT:    movsd %xmm0, {{[0-9]+}}(%esp)
+; X32-NEXT:    movshdup {{.*#+}} xmm0 = xmm4[1,1,3,3]
+; X32-NEXT:    movss %xmm0, {{[0-9]+}}(%esp)
+; X32-NEXT:    movss %xmm4, {{[0-9]+}}(%esp)
+; X32-NEXT:    movss %xmm0, {{[0-9]+}}(%esp)
 ; X32-NEXT:    addl $60, %esp
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: full_test:
 ; X64:       # BB#0: # %entry
 ; X64-NEXT:    movsd {{.*#+}} xmm2 = mem[0],zero
 ; X64-NEXT:    cvttps2dq %xmm2, %xmm0
 ; X64-NEXT:    cvtdq2ps %xmm0, %xmm1

test/CodeGen/X86/alias-static-alloca.ll

+; RUN: llc -o - -mtriple=x86_64-linux-gnu %s | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+; We should be able to bypass the load values to their corresponding
+; stores here.
+
+; CHECK-LABEL: foo
+; CHECK-DAG: movl %esi, -8(%rsp)
+; CHECK-DAG: movl %ecx, -16(%rsp)
+; CHECK-DAG: movl %edi, -4(%rsp)
+; CHECK-DAG: movl %edx, -12(%rsp)
+; CHECK: leal
+; CHECK: addl
+; CHECK: addl
+; CHECK: retq
+
+define i32 @foo(i32 %a, i32 %b, i32 %c, i32 %d) {
+entry:
+  %a0 = alloca i32
+  %a1 = alloca i32
+  %a2 = alloca i32
+  %a3 = alloca i32
+  store i32 %b, i32* %a1
+  store i32 %d, i32* %a3
+  store i32 %a, i32* %a0
+  store i32 %c, i32* %a2
+  %l0 = load i32, i32* %a0
+  %l1 = load i32, i32* %a1
+  %l2 = load i32, i32* %a2
+  %l3 = load i32, i32* %a3
+  %add0 = add nsw i32 %l0, %l1
+  %add1 = add nsw i32 %add0, %l2
+  %add2 = add nsw i32 %add1, %l3
+  ret i32 %add2
+}

test/CodeGen/X86/clobber-fi0.ll

 ; CHECK: main
 ; CHECK: subq{{.*}}rsp
 ; CHECK: ret
 define i32 @main(i32 %arg, i8** %arg1) nounwind {
 bb:
   %tmp = alloca i32, align 4                      ; [#uses=3 type=i32*]
   %tmp2 = alloca i32, align 4                     ; [#uses=3 type=i32*]
   %tmp3 = alloca i32                              ; [#uses=1 type=i32*]
-  store i32 1, i32* %tmp, align 4
-  store i32 1, i32* %tmp2, align 4
+  store volatile i32 1, i32* %tmp, align 4
+  store volatile i32 1, i32* %tmp2, align 4
   br label %bb4
 
 bb4:                                              ; preds = %bb4, %bb
-  %tmp6 = load i32, i32* %tmp2, align 4                ; [#uses=1 type=i32]
+  %tmp6 = load volatile i32, i32* %tmp2, align 4                ; [#uses=1 type=i32]
   %tmp7 = add i32 %tmp6, -1                       ; [#uses=2 type=i32]
-  store i32 %tmp7, i32* %tmp2, align 4
+  store volatile i32 %tmp7, i32* %tmp2, align 4
   %tmp8 = icmp eq i32 %tmp7, 0                    ; [#uses=1 type=i1]
-  %tmp9 = load i32, i32* %tmp                          ; [#uses=1 type=i32]
+  %tmp9 = load volatile i32, i32* %tmp                          ; [#uses=1 type=i32]
   %tmp10 = add i32 %tmp9, -1              ; [#uses=1 type=i32]
-  store i32 %tmp10, i32* %tmp3
+  store volatile i32 %tmp10, i32* %tmp3
   br i1 %tmp8, label %bb11, label %bb4
 
 bb11:                                             ; preds = %bb4
-  %tmp12 = load i32, i32* %tmp, align 4                ; [#uses=1 type=i32]
+  %tmp12 = load volatile i32, i32* %tmp, align 4                ; [#uses=1 type=i32]
   ret i32 %tmp12
 }
 
 

test/CodeGen/X86/hipe-cc.ll

entry:
   %arg0_var = alloca i32
   %arg1_var = alloca i32
   %arg2_var = alloca i32
   store i32 %hp, i32* %hp_var
   store i32 %p, i32* %p_var
   store i32 %arg0, i32* %arg0_var
   store i32 %arg1, i32* %arg1_var
   store i32 %arg2, i32* %arg2_var
-
-  ; CHECK:      movl  16(%esp), %esi
-  ; CHECK-NEXT: movl  12(%esp), %ebp
-  ; CHECK-NEXT: movl   8(%esp), %eax
-  ; CHECK-NEXT: movl   4(%esp), %edx
+  ; These loads are loading the values from their previous stores and are optimized away.
   %0 = load i32, i32* %hp_var
   %1 = load i32, i32* %p_var
   %2 = load i32, i32* %arg0_var
   %3 = load i32, i32* %arg1_var
   %4 = load i32, i32* %arg2_var
   ; CHECK:      jmp bar
   tail call cc 11 void @bar(i32 %0, i32 %1, i32 %2, i32 %3, i32 %4) nounwind
   ret void

test/CodeGen/X86/hipe-cc64.ll

entry:
   %arg3_var = alloca i64
   store i64 %hp, i64* %hp_var
   store i64 %p, i64* %p_var
   store i64 %arg0, i64* %arg0_var
   store i64 %arg1, i64* %arg1_var
   store i64 %arg2, i64* %arg2_var
   store i64 %arg3, i64* %arg3_var
 
-  ; CHECK:      movq  40(%rsp), %r15
-  ; CHECK-NEXT: movq  32(%rsp), %rbp
-  ; CHECK-NEXT: movq  24(%rsp), %rsi
-  ; CHECK-NEXT: movq  16(%rsp), %rdx
-  ; CHECK-NEXT: movq  8(%rsp), %rcx
+  ; Loads are reading values just writen from corresponding register and are therefore noops. 
   %0 = load i64, i64* %hp_var
   %1 = load i64, i64* %p_var
   %2 = load i64, i64* %arg0_var
   %3 = load i64, i64* %arg1_var
   %4 = load i64, i64* %arg2_var
   %5 = load i64, i64* %arg3_var
   ; CHECK:      jmp bar
   tail call cc 11 void @bar(i64 %0, i64 %1, i64 %2, i64 %3, i64 %4, i64 %5) nounwind

test/CodeGen/X86/legalize-shift-64.ll

 ; CHECK-NEXT:  .Lcfi9:
 ; CHECK-NEXT:    .cfi_offset %ebp, -8
 ; CHECK-NEXT:    movl %esp, %ebp
 ; CHECK-NEXT:  .Lcfi10:
 ; CHECK-NEXT:    .cfi_def_cfa_register %ebp
 ; CHECK-NEXT:    andl $-8, %esp
 ; CHECK-NEXT:    subl $16, %esp
 ; CHECK-NEXT:    movl $1, {{[0-9]+}}(%esp)
-; CHECK-NEXT:    movl $0, {{[0-9]+}}(%esp)
-; CHECK-NEXT:    movl $1, (%esp)
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl $1, %eax
 ; CHECK-NEXT:    xorl %ecx, %ecx
 ; CHECK-NEXT:    shldl $32, %eax, %ecx
 ; CHECK-NEXT:    movb $32, %dl
 ; CHECK-NEXT:    testb %dl, %dl
 ; CHECK-NEXT:    jne .LBB5_2
 ; CHECK-NEXT:  # BB#1:
 ; CHECK-NEXT:    movl %ecx, %eax

 ; CHECK-NEXT:  .LBB5_5: # %if.end
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:  .LBB5_4: # %if.then
 ; CHECK-NEXT:    movl %ebp, %esp
 ; CHECK-NEXT:    popl %ebp
 ; CHECK-NEXT:    retl
   %x = alloca i32, align 4
   %t = alloca i64, align 8
-  store i32 1, i32* %x, align 4
-  store i64 1, i64* %t, align 8  ;; DEAD
-  %load = load i32, i32* %x, align 4
+  store volatile i32 1, i32* %x, align 4
+  %load = load volatile i32, i32* %x, align 4
   %shl = shl i32 %load, 8
   %add = add i32 %shl, -224
   %sh_prom = zext i32 %add to i64
   %shl1 = shl i64 1, %sh_prom
   %cmp = icmp ne i64 %shl1, 4294967296
   br i1 %cmp, label %if.then, label %if.end
 
 if.then:                                          ; preds = %entry
   ret i32 1
 
 if.end:                                           ; preds = %entry
   ret i32 0
 
 }

test/CodeGen/X86/machine-outliner-debuginfo.ll

define i32 @main() #0 !dbg !11 {
   ; There is a debug value in the middle of this section, make sure debug values are ignored.
   ; CHECK: callq l_OUTLINED_FUNCTION_0
   store i32 1, i32* %2, align 4
   store i32 2, i32* %3, align 4
   store i32 3, i32* %4, align 4
   call void @llvm.dbg.value(metadata i32 10, i64 0, metadata !15, metadata !16), !dbg !17
   store i32 4, i32* %5, align 4
   store i32 0, i32* @x, align 4, !dbg !24
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; This is the same sequence of instructions without a debug value. It should be outlined
   ; in the same way.
   ; CHECK: callq l_OUTLINED_FUNCTION_0
   store i32 1, i32* %2, align 4
   store i32 2, i32* %3, align 4
   store i32 3, i32* %4, align 4
   store i32 4, i32* %5, align 4
   store i32 1, i32* @x, align 4, !dbg !14

test/CodeGen/X86/machine-outliner.ll

define i32 @main() #0 {
   store i32 0, i32* %1, align 4
   store i32 0, i32* @x, align 4
   ; CHECK: callq [[OFUNC2:l_OUTLINED_FUNCTION_[0-9]+]]
   store i32 1, i32* %2, align 4
   store i32 2, i32* %3, align 4
   store i32 3, i32* %4, align 4
   store i32 4, i32* %5, align 4
   store i32 1, i32* @x, align 4
+  call void asm sideeffect "", "~{memory},~{dirflag},~{fpsr},~{flags}"()
   ; CHECK: callq [[OFUNC2]]
   store i32 1, i32* %2, align 4
   store i32 2, i32* %3, align 4
   store i32 3, i32* %4, align 4
   store i32 4, i32* %5, align 4
   ret i32 0
 }
 

test/CodeGen/X86/statepoint-invoke.ll

left.relocs:
   ; CHECK: movq (%rsp),
   ; CHECK: movq 8(%rsp), [[REGVAL2:%[a-z]+]]
   %val1.relocated = call coldcc i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(token %sp1, i32 13, i32 13)
   %val2.relocated_left = call coldcc i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(token %sp1, i32 14, i32 14)
   br label %normal_return
 
 right:
   ; CHECK-LABEL: %right
-  ; CHECK: movq
   ; CHECK: movq %rdx, (%rsp)
+  ; CHECK: movq
   ; CHECK: callq some_call
   %sp2 = invoke token (i64, i32, void (i64 addrspace(1)*)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidp1i64f(i64 0, i32 0, void (i64 addrspace(1)*)* @some_call, i32 1, i32 0, i64 addrspace(1)* %val1, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i64 addrspace(1)* %val2, i64 addrspace(1)* %val3)
            to label %right.relocs unwind label %exceptional_return.right
 
 right.relocs:
   ; CHECK: movq (%rsp), [[REGVAL2]]
   ; CHECK: movq
   %val2.relocated_right = call coldcc i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(token %sp2, i32 13, i32 13)

test/CodeGen/X86/statepoint-stack-usage.ll

 ; RUN: llc -verify-machineinstrs -stack-symbol-ordering=0 < %s | FileCheck %s
 
 target datalayout = "e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-pc-linux-gnu"
 
 ; This test is checking to make sure that we reuse the same stack slots
 ; for GC values spilled over two different call sites.  Since the order
 ; of GC arguments differ, niave lowering code would insert loads and 
 ; stores to rearrange items on the stack.  We need to make sure (for
 ; performance) that this doesn't happen.
 define i32 @back_to_back_calls(i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c) #1 gc "statepoint-example" {
 ; CHECK-LABEL: back_to_back_calls
 ; The exact stores don't matter, but there need to be three stack slots created
-; CHECK: movq %rdi, 16(%rsp)
-; CHECK: movq %rdx, 8(%rsp)
-; CHECK: movq %rsi, (%rsp)
+; CHECK-DAG: movq %rdi, 16(%rsp)
+; CHECK-DAG: movq %rdx, 8(%rsp)
+; CHECK-DAG: movq %rsi, (%rsp)
 ; There should be no more than three moves
 ; CHECK-NOT: movq
   %safepoint_token = tail call token (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c)
   %a1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token, i32 12, i32 12)
   %b1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token, i32 12, i32 13)
   %c1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token, i32 12, i32 14)
 ; CHECK: callq
 ; This is the key check.  There should NOT be any memory moves here
 ; CHECK-NOT: movq
   %safepoint_token2 = tail call token (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %c1, i32 addrspace(1)* %b1, i32 addrspace(1)* %a1)
   %a2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token2, i32 12, i32 14)
   %b2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token2, i32 12, i32 13)
   %c2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token2, i32 12, i32 12)
 ; CHECK: callq
   ret i32 1
 }
 
 ; This test simply checks that minor changes in vm state don't prevent slots
 ; being reused for gc values.  
 define i32 @reserve_first(i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c) #1 gc "statepoint-example" {
 ; CHECK-LABEL: reserve_first
 ; The exact stores don't matter, but there need to be three stack slots created
-; CHECK: movq %rdi, 16(%rsp)
-; CHECK: movq %rdx, 8(%rsp)
-; CHECK: movq %rsi, (%rsp)
+; CHECK-DAG: movq %rdi, 16(%rsp)
+; CHECK-DAG: movq %rdx, 8(%rsp)
+; CHECK-DAG: movq %rsi, (%rsp)
   %safepoint_token = tail call token (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c)
   %a1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token, i32 12, i32 12)
   %b1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token, i32 12, i32 13)
   %c1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token, i32 12, i32 14)
 ; CHECK: callq
 ; This is the key check.  There should NOT be any memory moves here
 ; CHECK-NOT: movq
   %safepoint_token2 = tail call token (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 addrspace(1)* %a1, i32 0, i32 addrspace(1)* %c1, i32 0, i32 0, i32 addrspace(1)* %c1, i32 addrspace(1)* %b1, i32 addrspace(1)* %a1)
   %a2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token2, i32 12, i32 14)
   %b2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token2, i32 12, i32 13)
   %c2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token2, i32 12, i32 12)
 ; CHECK: callq
   ret i32 1
 }
 
 ; Check that we reuse the same stack slot across multiple calls.  The use of
 ; more than two calls here is critical.  We've had a bug which allowed reuse
 ; exactly once which went undetected for a long time.
 define i32 @back_to_back_deopt(i32 %a, i32 %b, i32 %c) #1 
   gc "statepoint-example" {
 ; CHECK-LABEL: back_to_back_deopt
 ; The exact stores don't matter, but there need to be three stack slots created
-; CHECK: movl %ebx, 12(%rsp)
-; CHECK: movl %ebp, 8(%rsp)
-; CHECK: movl %r14d, 4(%rsp)
-; CHECK: callq
-; CHECK: movl %ebx, 12(%rsp)
-; CHECK: movl %ebp, 8(%rsp)
-; CHECK: movl %r14d, 4(%rsp)
-; CHECK: callq
-; CHECK: movl %ebx, 12(%rsp)
-; CHECK: movl %ebp, 8(%rsp)
-; CHECK: movl %r14d, 4(%rsp)
-; CHECK: callq
-; CHECK: movl %ebx, 12(%rsp)
-; CHECK: movl %ebp, 8(%rsp)
-; CHECK: movl %r14d, 4(%rsp)
+; CHECK-DAG: movl %ebx, 12(%rsp)
+; CHECK-DAG: movl %ebp, 8(%rsp)
+; CHECK-DAG: movl %r14d, 4(%rsp)
+; CHECK: callq
+; CHECK-DAG: movl %ebx, 12(%rsp)
+; CHECK-DAG: movl %ebp, 8(%rsp)
+; CHECK-DAG: movl %r14d, 4(%rsp)
+; CHECK: callq
+; CHECK-DAG: movl %ebx, 12(%rsp)
+; CHECK-DAG: movl %ebp, 8(%rsp)
+; CHECK-DAG: movl %r14d, 4(%rsp)
+; CHECK: callq
+; CHECK-DAG: movl %ebx, 12(%rsp)
+; CHECK-DAG: movl %ebp, 8(%rsp)
+; CHECK-DAG: movl %r14d, 4(%rsp)
 ; CHECK: callq
   call token (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* undef, i32 0, i32 0, i32 0, i32 3, i32 %a, i32 %b, i32 %c)
 call token (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* undef, i32 0, i32 0, i32 0, i32 3, i32 %a, i32 %b, i32 %c)
 call token (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* undef, i32 0, i32 0, i32 0, i32 3, i32 %a, i32 %b, i32 %c)
 call token (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* undef, i32 0, i32 0, i32 0, i32 3, i32 %a, i32 %b, i32 %c)
   ret i32 1
 }
 
 ; Test that stack slots are reused for invokes
 define i32 @back_to_back_invokes(i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c) #1 gc "statepoint-example" personality i32 ()* @"personality_function" {
 ; CHECK-LABEL: back_to_back_invokes
 entry:
   ; The exact stores don't matter, but there need to be three stack slots created
-  ; CHECK: movq %rdi, 16(%rsp)
-  ; CHECK: movq %rdx, 8(%rsp)
-  ; CHECK: movq %rsi, (%rsp)
+  ; CHECK-DAG: movq %rdi, 16(%rsp)
+  ; CHECK-DAG: movq %rdx, 8(%rsp)
+  ; CHECK-DAG: movq %rsi, (%rsp)
   ; CHECK: callq
   %safepoint_token = invoke token (i64, i32, void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(i64 0, i32 0, void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c)
                    to label %normal_return unwind label %exceptional_return
 
 normal_return:
   %a1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token, i32 12, i32 12)
   %b1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token, i32 12, i32 13)
   %c1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(token %safepoint_token, i32 12, i32 14)

test/CodeGen/X86/statepoint-vector.ll

 ; - We could in principal reuse the argument memory (%rsi) and do away
 ;   with stack slots entirely.
 define <2 x i64 addrspace(1)*> @test3(i1 %cnd, <2 x i64 addrspace(1)*>* %ptr) gc "statepoint-example" {
 entry:
 ; CHECK-LABEL: @test3
 ; CHECK: subq $40, %rsp
 ; CHECK: testb  $1, %dil
 ; CHECK: movaps (%rsi), %xmm0
-; CHECK: movaps %xmm0, 16(%rsp)
-; CHECK: movaps %xmm0, (%rsp)
+; CHECK-DAG: movaps %xmm0, (%rsp)
+; CHECK-DAG: movaps %xmm0, 16(%rsp)
 ; CHECK: callq  do_safepoint
 ; CHECK: movaps (%rsp), %xmm0
 ; CHECK: addq $40, %rsp
   br i1 %cnd, label %taken, label %untaken
 
 taken:                                            ; preds = %entry
   %obja = load <2 x i64 addrspace(1)*>, <2 x i64 addrspace(1)*>* %ptr
   br label %merge

test/CodeGen/X86/widen_arith-2.ll

 ; CHECK-NEXT:    pcmpeqd %xmm0, %xmm0
 ; CHECK-NEXT:    movdqa {{.*#+}} xmm1 = [4,4,4,4,4,4,4,4]
 ; CHECK-NEXT:    movdqa {{.*#+}} xmm2 = <0,2,4,6,8,10,12,14,u,u,u,u,u,u,u,u>
 ; CHECK-NEXT:    jmp .LBB0_1
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_2: # %forbody
 ; CHECK-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; CHECK-NEXT:    movl (%esp), %eax
-; CHECK-NEXT:    shll $3, %eax
-; CHECK-NEXT:    addl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movl %eax, {{[0-9]+}}(%esp)
-; CHECK-NEXT:    movl (%esp), %eax
-; CHECK-NEXT:    shll $3, %eax
-; CHECK-NEXT:    addl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movl %eax, {{[0-9]+}}(%esp)
-; CHECK-NEXT:    movl (%esp), %ecx
+; CHECK-NEXT:    leal (,%eax,8), %ecx
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; CHECK-NEXT:    addl %ecx, %edx
+; CHECK-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; CHECK-NEXT:    addl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
 ; CHECK-NEXT:    pmovzxbw {{.*#+}} xmm3 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero,mem[4],zero,mem[5],zero,mem[6],zero,mem[7],zero
 ; CHECK-NEXT:    psubw %xmm0, %xmm3
 ; CHECK-NEXT:    pand %xmm1, %xmm3
 ; CHECK-NEXT:    pshufb %xmm2, %xmm3
-; CHECK-NEXT:    movq %xmm3, (%edx,%ecx,8)
+; CHECK-NEXT:    movq %xmm3, (%edx,%eax,8)
 ; CHECK-NEXT:    incl (%esp)
 ; CHECK-NEXT:  .LBB0_1: # %forcond
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    movl (%esp), %eax
 ; CHECK-NEXT:    cmpl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    jl .LBB0_2
 ; CHECK-NEXT:  # BB#3: # %afterfor
 ; CHECK-NEXT:    addl $12, %esp

test/CodeGen/X86/widen_cast-4.ll

 ; NARROW-NEXT:    movl $0, (%esp)
 ; NARROW-NEXT:    pcmpeqd %xmm0, %xmm0
 ; NARROW-NEXT:    movdqa {{.*#+}} xmm1 = <0,2,4,6,8,10,12,14,u,u,u,u,u,u,u,u>
 ; NARROW-NEXT:    jmp .LBB0_1
 ; NARROW-NEXT:    .p2align 4, 0x90
 ; NARROW-NEXT:  .LBB0_2: # %forbody
 ; NARROW-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; NARROW-NEXT:    movl (%esp), %eax
-; NARROW-NEXT:    shll $3, %eax
-; NARROW-NEXT:    addl {{[0-9]+}}(%esp), %eax
-; NARROW-NEXT:    movl %eax, {{[0-9]+}}(%esp)
-; NARROW-NEXT:    movl (%esp), %eax
-; NARROW-NEXT:    shll $3, %eax
-; NARROW-NEXT:    addl {{[0-9]+}}(%esp), %eax
-; NARROW-NEXT:    movl %eax, {{[0-9]+}}(%esp)
-; NARROW-NEXT:    movl (%esp), %ecx
+; NARROW-NEXT:    leal (,%eax,8), %ecx
 ; NARROW-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; NARROW-NEXT:    addl %ecx, %edx
+; NARROW-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; NARROW-NEXT:    addl {{[0-9]+}}(%esp), %ecx
+; NARROW-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
 ; NARROW-NEXT:    pmovzxbw {{.*#+}} xmm2 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero,mem[4],zero,mem[5],zero,mem[6],zero,mem[7],zero
 ; NARROW-NEXT:    psubw %xmm0, %xmm2
 ; NARROW-NEXT:    psllw $8, %xmm2
 ; NARROW-NEXT:    psraw $8, %xmm2
 ; NARROW-NEXT:    psraw $2, %xmm2
 ; NARROW-NEXT:    pshufb %xmm1, %xmm2
-; NARROW-NEXT:    movq %xmm2, (%edx,%ecx,8)
+; NARROW-NEXT:    movq %xmm2, (%edx,%eax,8)
 ; NARROW-NEXT:    incl (%esp)
 ; NARROW-NEXT:  .LBB0_1: # %forcond
 ; NARROW-NEXT:    # =>This Inner Loop Header: Depth=1
 ; NARROW-NEXT:    movl (%esp), %eax
 ; NARROW-NEXT:    cmpl {{[0-9]+}}(%esp), %eax
 ; NARROW-NEXT:    jl .LBB0_2
 ; NARROW-NEXT:  # BB#3: # %afterfor
 ; NARROW-NEXT:    addl $12, %esp
 ; NARROW-NEXT:    retl
 ;
 ; WIDE-LABEL: update:
 ; WIDE:       # BB#0: # %entry
 ; WIDE-NEXT:    subl $12, %esp
 ; WIDE-NEXT:    movl $0, (%esp)
 ; WIDE-NEXT:    pcmpeqd %xmm0, %xmm0
 ; WIDE-NEXT:    movdqa {{.*#+}} xmm1 = [63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63]
 ; WIDE-NEXT:    movdqa {{.*#+}} xmm2 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32]
 ; WIDE-NEXT:    jmp .LBB0_1
 ; WIDE-NEXT:    .p2align 4, 0x90
 ; WIDE-NEXT:  .LBB0_2: # %forbody
 ; WIDE-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; WIDE-NEXT:    movl (%esp), %eax
-; WIDE-NEXT:    shll $3, %eax
-; WIDE-NEXT:    addl {{[0-9]+}}(%esp), %eax
-; WIDE-NEXT:    movl %eax, {{[0-9]+}}(%esp)
-; WIDE-NEXT:    movl (%esp), %eax
-; WIDE-NEXT:    shll $3, %eax
-; WIDE-NEXT:    addl {{[0-9]+}}(%esp), %eax
-; WIDE-NEXT:    movl %eax, {{[0-9]+}}(%esp)
-; WIDE-NEXT:    movl (%esp), %ecx
+; WIDE-NEXT:    leal (,%eax,8), %ecx
 ; WIDE-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; WIDE-NEXT:    addl %ecx, %edx
+; WIDE-NEXT:    movl %edx, {{[0-9]+}}(%esp)
+; WIDE-NEXT:    addl {{[0-9]+}}(%esp), %ecx
+; WIDE-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
 ; WIDE-NEXT:    movd {{.*#+}} xmm3 = mem[0],zero,zero,zero
-; WIDE-NEXT:    pinsrd $1, 4(%eax,%ecx,8), %xmm3
+; WIDE-NEXT:    pinsrd $1, 4(%ecx,%eax,8), %xmm3
 ; WIDE-NEXT:    psubb %xmm0, %xmm3
 ; WIDE-NEXT:    psrlw $2, %xmm3
 ; WIDE-NEXT:    pand %xmm1, %xmm3
 ; WIDE-NEXT:    pxor %xmm2, %xmm3
 ; WIDE-NEXT:    psubb %xmm2, %xmm3
-; WIDE-NEXT:    pextrd $1, %xmm3, 4(%edx,%ecx,8)
-; WIDE-NEXT:    movd %xmm3, (%edx,%ecx,8)
+; WIDE-NEXT:    pextrd $1, %xmm3, 4(%edx,%eax,8)
+; WIDE-NEXT:    movd %xmm3, (%edx,%eax,8)
 ; WIDE-NEXT:    incl (%esp)
 ; WIDE-NEXT:  .LBB0_1: # %forcond
 ; WIDE-NEXT:    # =>This Inner Loop Header: Depth=1
 ; WIDE-NEXT:    movl (%esp), %eax
 ; WIDE-NEXT:    cmpl {{[0-9]+}}(%esp), %eax
 ; WIDE-NEXT:    jl .LBB0_2
 ; WIDE-NEXT:  # BB#3: # %afterfor
 ; WIDE-NEXT:    addl $12, %esp

test/CodeGen/XCore/varargs.ll

 declare void @llvm.va_start(i8*) nounwind
 declare void @llvm.va_end(i8*) nounwind
 declare void @f(i32) nounwind
 define void @test_vararg(...) nounwind {
 entry:
 ; CHECK-LABEL: test_vararg
 ; CHECK: extsp 6
 ; CHECK: stw lr, sp[1]
-; CHECK: stw r3, sp[6]
-; CHECK: stw r0, sp[3]
-; CHECK: stw r1, sp[4]
-; CHECK: stw r2, sp[5]
+; CHECK-DAG: stw r3, sp[6]
+; CHECK-DAG: stw r0, sp[3]
+; CHECK-DAG: stw r1, sp[4]
+; CHECK-DAG: stw r2, sp[5]
 ; CHECK: ldaw r0, sp[3]
 ; CHECK: stw r0, sp[2]
   %list = alloca i8*, align 4
   %list1 = bitcast i8** %list to i8*
   call void @llvm.va_start(i8* %list1)
   br label %for.cond
 
 ; CHECK-LABEL: .LBB1_1