Set hasSideEffects=0 for TargetOpcode::BUNDLE

[asb]

Summary

This patch builds on top of D37065.

The test changes definitely need someone else to look over them - I'm not familiar with AMDGPU assembly. I need to return to call-argument-types.ll, I just couldn't seem to get it to pass despite the fact the test patterns seemed fine in relation to the generated output (I must be missing something obvious, will look again).

[kparzysz]

As a side note, in case anybody is wondering: the differences in the generated code come from the post-RA scheduler, which can do more reordering of instructions relative to bundles.

[asb]

I'm trying to update this patch to reflect the latest upstream changes in the test files, and am having a terrible time updating the AMDGPU tests. I'd really appreciate a second pair of eyes here.

Command line:
./bin/llc -mtriple=amdgcn-amd-amdhsa -mcpu=fiji -verify-machineinstrs < ../test/CodeGen/AMDGPU/byval-frame-setup.ll | ./bin/FileCheck -enable-var-scope -check-prefixes=GCN,VI ../test/CodeGen/AMDGPU/byval-frame-setup.ll

This complains:

../test/CodeGen/AMDGPU/byval-frame-setup.ll:83:8: error: expected string not found in input
; GCN: buffer_load_dword [[LOAD0:v[0-9]+]], off, s[0:3], s5 offset:8
       ^
<stdin>:111:2: note: scanning from here
 buffer_load_dword v2, off, s[0:3], s5 offset:16
 ^
<stdin>:124:2: note: possible intended match here
 buffer_load_dword v0, off, s[0:3], s5 offset:24
 ^

The relevant part of the test is:

; GCN-LABEL: {{^}}call_void_func_byval_struct_func:
; GCN: s_mov_b32 s5, s32
; GCN-DAG: s_add_u32 s32, s32, 0xc00{{$}}
; GCN-DAG: v_writelane_b32

; GCN-DAG: v_mov_b32_e32 [[NINE:v[0-9]+]], 9
; GCN-DAG: v_mov_b32_e32 [[THIRTEEN:v[0-9]+]], 13

; GCN-DAG: buffer_store_dword [[NINE]], off, s[0:3], s5 offset:8
; GCN-DAG: buffer_store_dword [[THIRTEEN]], off, s[0:3], s5 offset:24

; GCN: buffer_load_dword [[LOAD0:v[0-9]+]], off, s[0:3], s5 offset:8
; GCN: buffer_load_dword [[LOAD1:v[0-9]+]], off, s[0:3], s5 offset:12
; GCN: buffer_load_dword [[LOAD2:v[0-9]+]], off, s[0:3], s5 offset:16
; GCN: buffer_load_dword [[LOAD3:v[0-9]+]], off, s[0:3], s5 offset:20

; GCN-NOT: s_add_u32 s32, s32, 0x800

; GCN-DAG: buffer_store_dword [[LOAD0]], off, s[0:3], s32 offset:4{{$}}
; GCN-DAG: buffer_store_dword [[LOAD1]], off, s[0:3], s32 offset:8
; GCN-DAG: buffer_store_dword [[LOAD2]], off, s[0:3], s32 offset:12
; GCN-DAG: buffer_store_dword [[LOAD3]], off, s[0:3], s32 offset:16

The function in question:

call_void_func_byval_struct_func:       ; @call_void_func_byval_struct_func
; BB#0:                                 ; %entry
	s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
	s_mov_b32 s5, s32
	buffer_store_dword v32, off, s[0:3], s5 offset:40
	s_waitcnt expcnt(0)
	v_writelane_b32 v32, s33, 0
	v_writelane_b32 v32, s34, 1
	v_mov_b32_e32 v0, 9
	v_mov_b32_e32 v1, 13
	v_writelane_b32 v32, s35, 2
	buffer_store_dword v0, off, s[0:3], s5 offset:8
	buffer_store_dword v1, off, s[0:3], s5 offset:24
	s_waitcnt expcnt(1)
	buffer_load_dword v0, off, s[0:3], s5 offset:8
	s_waitcnt expcnt(0)
	buffer_load_dword v1, off, s[0:3], s5 offset:12
	s_add_u32 s32, s32, 0xc00
	buffer_load_dword v2, off, s[0:3], s5 offset:16
	buffer_load_dword v3, off, s[0:3], s5 offset:20
	s_getpc_b64 s[6:7]
	s_add_u32 s6, s6, void_func_byval_struct@rel32@lo+4
	s_addc_u32 s7, s7, void_func_byval_struct@rel32@hi+4
	s_mov_b64 s[34:35], s[30:31]
	s_mov_b32 s33, s5
	s_waitcnt vmcnt(0)
	buffer_store_dword v3, off, s[0:3], s32 offset:16
	buffer_store_dword v2, off, s[0:3], s32 offset:12
	buffer_store_dword v1, off, s[0:3], s32 offset:8
	buffer_store_dword v0, off, s[0:3], s32 offset:4
	s_waitcnt expcnt(0)
	buffer_load_dword v0, off, s[0:3], s5 offset:24
	buffer_load_dword v1, off, s[0:3], s5 offset:28
	buffer_load_dword v2, off, s[0:3], s5 offset:32
	buffer_load_dword v3, off, s[0:3], s5 offset:36
	s_waitcnt vmcnt(0)
	buffer_store_dword v3, off, s[0:3], s32 offset:32
	buffer_store_dword v2, off, s[0:3], s32 offset:28
	buffer_store_dword v1, off, s[0:3], s32 offset:24
	buffer_store_dword v0, off, s[0:3], s32 offset:20
	s_swappc_b64 s[30:31], s[6:7]
	s_mov_b32 s5, s33
	s_mov_b64 s[30:31], s[34:35]
	v_readlane_b32 s35, v32, 2
	v_readlane_b32 s34, v32, 1
	v_readlane_b32 s33, v32, 0
	buffer_load_dword v32, off, s[0:3], s5 offset:40
	s_sub_u32 s32, s32, 0xc00
	s_waitcnt vmcnt(0) expcnt(0)
	s_setpc_b64 s[30:31]
.Lfunc_end2:
	.size	call_void_func_byval_struct_func, .Lfunc_end2-call_void_func_byval_struct_func
                                        ; -- End function

Now unless I'm going crazy here, I really can't see the problem matching buffer_load_dword v0, off, s[0:3], s5 offset:8. Am I missing something obvious, or misunderstanding -DAG in FileCheck?

[tstellar]

I think this is failing because the s_add_u32 s32, s32, 0xc00{{$}} is moved after the loads.

[asb]

As mentioned on the mailing list I don't have time to fix up and audit the test case changes. If anyone wants to see BUNDLE with hasSideEffects=0, please do commandeer the revision.

[bjope]

A BUNDLE instruction is supposed to describe the combined properties of the bundled instructions (at least when it comes to machine operands). Properties like hasSideEffects/mayLoad/mayStore are static, so we can't update those properties for the BUNDLE instruction depending on the properties of the bundled instructions. So maybe it is a good idea to have the defensive approach of letting the BUNDLE instruction have hasSideEffects=1.

Afaik some machine intruction analyses/passes does not look inside the bundle (e.g. iterating using a MachineInstrBundleIterator).
If we set hasSideEffects=0 on the BUNDLE instruction, then that information can't be trusted anyway in case one of the bundled instructions has hasSideEffects=1. Or otherwise I think we need to avoid bundling of instructions with hasSideEffects=1.
If a pass/analysis is smart enough to look inside the BUNDLE, then I guess it can check if all of the bundled instructions has hasSideEffects=0 and ignore the hasSideEffects=1 on the BUNDLE instruction.

Maybe we need to update documentation related to the BUNDLE instruction to describe this better.

include/llvm/Target/Target.td

def COPY : Instruction {
   let hasSideEffects = 0;
   let isAsCheapAsAMove = 1;
   let hasNoSchedulingInfo = 0;
 }
 def BUNDLE : Instruction {
   let OutOperandList = (outs);
   let InOperandList = (ins variable_ops);
   let AsmString = "BUNDLE";
-  let hasSideEffects = 1;
+  let hasSideEffects = 0;
 }
 def LIFETIME_START : Instruction {
   let OutOperandList = (outs);
   let InOperandList = (ins i32imm:$id);
   let AsmString = "LIFETIME_START";
   let hasSideEffects = 0;
 }
 def LIFETIME_END : Instruction {

test/CodeGen/AMDGPU/byval-frame-setup.ll

entry:
   ret void
 }
 
 ; GCN-LABEL: {{^}}call_void_func_byval_struct_func:
 ; GCN: s_mov_b32 s5, s32
 ; GCN: s_add_u32 s32, s32, 0xc00{{$}}
 ; GCN: v_writelane_b32
 
-; GCN-DAG: s_add_u32 s32, s32, 0x800{{$}}
 ; GCN-DAG: v_mov_b32_e32 [[NINE:v[0-9]+]], 9
 ; GCN-DAG: v_mov_b32_e32 [[THIRTEEN:v[0-9]+]], 13
 
 ; GCN-DAG: buffer_store_dword [[NINE]], off, s[0:3], s5 offset:8
 ; GCN-DAG: buffer_store_dword [[THIRTEEN]], off, s[0:3], s5 offset:24
 
 ; GCN: buffer_load_dword [[LOAD0:v[0-9]+]], off, s[0:3], s5 offset:8
 ; GCN: buffer_load_dword [[LOAD1:v[0-9]+]], off, s[0:3], s5 offset:12
 ; GCN: buffer_load_dword [[LOAD2:v[0-9]+]], off, s[0:3], s5 offset:16
 ; GCN: buffer_load_dword [[LOAD3:v[0-9]+]], off, s[0:3], s5 offset:20
 
+; GCN: s_add_u32 s32, s32, 0x800{{$}}
 
 ; GCN-DAG: buffer_store_dword [[LOAD0]], off, s[0:3], s32 offset:4{{$}}
 ; GCN-DAG: buffer_store_dword [[LOAD1]], off, s[0:3], s32 offset:8
 ; GCN-DAG: buffer_store_dword [[LOAD2]], off, s[0:3], s32 offset:12
 ; GCN-DAG: buffer_store_dword [[LOAD3]], off, s[0:3], s32 offset:16
 
 ; GCN: buffer_load_dword [[LOAD4:v[0-9]+]], off, s[0:3], s5 offset:24
 ; GCN: buffer_load_dword [[LOAD5:v[0-9]+]], off, s[0:3], s5 offset:28
entry:
   call void @void_func_byval_struct(%struct.ByValStruct* byval nonnull align 4 %arg0, %struct.ByValStruct* byval nonnull align 4 %arg1)
   call void @llvm.lifetime.end.p0i8(i64 32, i8* %tmp1)
   call void @llvm.lifetime.end.p0i8(i64 32, i8* %tmp)
   ret void
 }
 
 ; GCN-LABEL: {{^}}call_void_func_byval_struct_kernel:
 ; GCN: s_mov_b32 s33, s7
-; GCN: s_add_u32 s32, s33, 0xa00{{$}}
 
 ; GCN-DAG: v_mov_b32_e32 [[NINE:v[0-9]+]], 9
 ; GCN-DAG: v_mov_b32_e32 [[THIRTEEN:v[0-9]+]], 13
 ; GCN-DAG: buffer_store_dword [[NINE]], off, s[0:3], s33 offset:8
 ; GCN: buffer_store_dword [[THIRTEEN]], off, s[0:3], s33 offset:24
 
-; GCN-DAG: s_add_u32 s32, s32, 0x800{{$}}
-
 ; GCN-DAG: buffer_load_dword [[LOAD0:v[0-9]+]], off, s[0:3], s33 offset:8
 ; GCN-DAG: buffer_load_dword [[LOAD1:v[0-9]+]], off, s[0:3], s33 offset:12
 ; GCN-DAG: buffer_load_dword [[LOAD2:v[0-9]+]], off, s[0:3], s33 offset:16
 ; GCN-DAG: buffer_load_dword [[LOAD3:v[0-9]+]], off, s[0:3], s33 offset:20
 
+; GCN: s_add_u32 s32, s33, 0xa00{{$}}
+; GCN: s_add_u32 s32, s32, 0x800{{$}}
+
 ; GCN-DAG: buffer_store_dword [[LOAD0]], off, s[0:3], s32 offset:4{{$}}
 ; GCN-DAG: buffer_store_dword [[LOAD1]], off, s[0:3], s32 offset:8
 ; GCN-DAG: buffer_store_dword [[LOAD2]], off, s[0:3], s32 offset:12
 ; GCN-DAG: buffer_store_dword [[LOAD3]], off, s[0:3], s32 offset:16
 
 ; GCN-DAG: buffer_load_dword [[LOAD4:v[0-9]+]], off, s[0:3], s33 offset:24
 ; GCN-DAG: buffer_load_dword [[LOAD5:v[0-9]+]], off, s[0:3], s33 offset:28
 ; GCN-DAG: buffer_load_dword [[LOAD6:v[0-9]+]], off, s[0:3], s33 offset:32

test/CodeGen/AMDGPU/call-argument-types.ll

define amdgpu_kernel void @test_call_external_void_func_i1_imm() #0 {
   call void @external_void_func_i1(i1 true)
   ret void
 }
 
 ; GCN-LABEL: {{^}}test_call_external_void_func_i1_signext:
 ; MESA: s_mov_b32 s33, s3{{$}}
 ; HSA: s_mov_b32 s33, s9{{$}}
 
+; GCN: buffer_load_ubyte [[VAR:v[0-9]+]]
 ; GCN: s_getpc_b64 s{{\[}}[[PC_LO:[0-9]+]]:[[PC_HI:[0-9]+]]{{\]}}
 ; GCN-NEXT: s_add_u32 s[[PC_LO]], s[[PC_LO]], external_void_func_i1_signext@rel32@lo+4
 ; GCN-NEXT: s_addc_u32 s[[PC_HI]], s[[PC_HI]], external_void_func_i1_signext@rel32@hi+4
-; GCN-NEXT: buffer_load_ubyte [[VAR:v[0-9]+]]
 ; HSA-NEXT: s_mov_b32 s4, s33
 ; HSA-NEXT: s_mov_b32 s32, s33
 
 ; MESA-DAG: s_mov_b32 s4, s33{{$}}
 ; MESA-DAG: s_mov_b32 s32, s33{{$}}
 
 ; GCN: s_waitcnt vmcnt(0)
 ; GCN-NEXT: v_bfe_i32 v0, v0, 0, 1
 ; GCN-NEXT: s_swappc_b64 s[30:31], s{{\[}}[[PC_LO]]:[[PC_HI]]{{\]}}
 ; GCN-NEXT: s_endpgm
 define amdgpu_kernel void @test_call_external_void_func_i1_signext(i32) #0 {
   %var = load volatile i1, i1 addrspace(1)* undef
   call void @external_void_func_i1_signext(i1 %var)
   ret void
 }
 
 ; FIXME: load should be scheduled before getpc
 ; GCN-LABEL: {{^}}test_call_external_void_func_i1_zeroext:
 ; MESA: s_mov_b32 s33, s3{{$}}
 
+; GCN: buffer_load_ubyte v0
 ; GCN: s_getpc_b64 s{{\[}}[[PC_LO:[0-9]+]]:[[PC_HI:[0-9]+]]{{\]}}
 ; GCN-NEXT: s_add_u32 s[[PC_LO]], s[[PC_LO]], external_void_func_i1_zeroext@rel32@lo+4
 ; GCN-NEXT: s_addc_u32 s[[PC_HI]], s[[PC_HI]], external_void_func_i1_zeroext@rel32@hi+4
-; GCN-NEXT: buffer_load_ubyte v0
 
 ; GCN-DAG: s_mov_b32 s4, s33{{$}}
 ; GCN-DAG: s_mov_b32 s32, s33{{$}}
 
 ; GCN: s_waitcnt vmcnt(0)
 ; GCN-NEXT: v_and_b32_e32 v0, 1, v0
 ; GCN-NEXT: s_swappc_b64 s[30:31], s{{\[}}[[PC_LO]]:[[PC_HI]]{{\]}}
 ; GCN-NEXT: s_endpgm

 ; GCN-DAG: buffer_load_dwordx4 v[4:7], off
 ; GCN-DAG: buffer_load_dwordx4 v[8:11], off
 ; GCN-DAG: buffer_load_dwordx4 v[12:15], off
 ; GCN-DAG: buffer_load_dwordx4 v[16:19], off
 ; GCN-DAG: buffer_load_dwordx4 v[20:23], off
 ; GCN-DAG: buffer_load_dwordx4 v[24:27], off
 ; GCN-DAG: buffer_load_dwordx4 v[28:31], off
 
-; GCN: buffer_store_dword [[VAL1]], off, s[{{[0-9]+}}:{{[0-9]+}}], [[SP_REG]] offset:4{{$}}
 ; GCN: s_waitcnt
+; GCN: buffer_store_dword [[VAL1]], off, s[{{[0-9]+}}:{{[0-9]+}}], [[SP_REG]] offset:4{{$}}
 ; GCN-NEXT: s_swappc_b64
 ; GCN-NEXT: s_endpgm
 define amdgpu_kernel void @test_call_external_void_func_v32i32_i32(i32) #0 {
   %ptr0 = load <32 x i32> addrspace(1)*, <32 x i32> addrspace(1)* addrspace(2)* undef
   %val0 = load <32 x i32>, <32 x i32> addrspace(1)* %ptr0
   %val1 = load i32, i32 addrspace(1)* undef
   call void @external_void_func_v32i32_i32(<32 x i32> %val0, i32 %val1)
   ret void

 ; GCN-DAG: v_mov_b32_e32 [[VAL0:v[0-9]+]], 3
 ; GCN-DAG: v_mov_b32_e32 [[VAL1:v[0-9]+]], 8
 ; MESA-DAG: buffer_store_byte [[VAL0]], off, s[36:39], s33 offset:8
 ; MESA-DAG: buffer_store_dword [[VAL1]], off, s[36:39], s33 offset:12
 
 ; HSA-DAG: buffer_store_byte [[VAL0]], off, s[0:3], s33 offset:8
 ; HSA-DAG: buffer_store_dword [[VAL1]], off, s[0:3], s33 offset:12
 
-; GCN: s_add_u32 [[SP]], [[SP]], 0x200
-
-; HSA: buffer_load_dword [[RELOAD_VAL0:v[0-9]+]], off, s[0:3], s33 offset:8
-; HSA: buffer_load_dword [[RELOAD_VAL1:v[0-9]+]], off, s[0:3], s33 offset:12
-
-; HSA: buffer_store_dword [[RELOAD_VAL1]], off, s[0:3], [[SP]] offset:8
-; HSA: buffer_store_dword [[RELOAD_VAL0]], off, s[0:3], [[SP]] offset:4
-
-
-; MESA: buffer_load_dword [[RELOAD_VAL0:v[0-9]+]], off, s[36:39], s33 offset:8
-; MESA: buffer_load_dword [[RELOAD_VAL1:v[0-9]+]], off, s[36:39], s33 offset:12
-
-; MESA: buffer_store_dword [[RELOAD_VAL1]], off, s[36:39], [[SP]] offset:8
-; MESA: buffer_store_dword [[RELOAD_VAL0]], off, s[36:39], [[SP]] offset:4
-
-; GCN-NEXT: s_swappc_b64
-; GCN-NEXT: s_sub_u32 [[SP]], [[SP]], 0x200
+; GCN: s_swappc_b64
+; GCN: s_sub_u32 [[SP]], [[SP]], 0x200
 define amdgpu_kernel void @test_call_external_void_func_byval_struct_i8_i32() #0 {
   %val = alloca { i8, i32 }, align 4
   %gep0 = getelementptr inbounds { i8, i32 }, { i8, i32 }* %val, i32 0, i32 0
   %gep1 = getelementptr inbounds { i8, i32 }, { i8, i32 }* %val, i32 0, i32 1
   store i8 3, i8* %gep0
   store i32 8, i32* %gep1
   call void @external_void_func_byval_struct_i8_i32({ i8, i32 }* %val)
   ret void

test/CodeGen/AMDGPU/call-preserved-registers.ll

define amdgpu_kernel void @test_call_void_func_void_mayclobber_v31(i32 addrspace(1)* %out) #0 {
   %v31 = call i32 asm sideeffect "; def $0", "={v31}"()
   call void @external_void_func_void()
   call void asm sideeffect "; use $0", "{v31}"(i32 %v31)
   ret void
 }
 
 ; GCN-LABEL: {{^}}test_call_void_func_void_preserves_s33:
 ; GCN: s_mov_b32 s34, s9
-; GCN: ; def s33
-; GCN-NEXT: #ASMEND
 ; GCN: s_getpc_b64 s[6:7]
 ; GCN-NEXT: s_add_u32 s6, s6, external_void_func_void@rel32@lo+4
 ; GCN-NEXT: s_addc_u32 s7, s7, external_void_func_void@rel32@hi+4
 ; GCN-NEXT: s_mov_b32 s4, s34
 ; GCN-NEXT: s_mov_b32 s32, s34
+; GCN-NEXT: ;;#ASMSTART
+; GCN-NEXT: ; def s33
+; GCN-NEXT: #ASMEND
 ; GCN-NEXT: s_swappc_b64 s[30:31], s[6:7]
 ; GCN-NEXT: ;;#ASMSTART
 ; GCN-NEXT: ; use s33
 ; GCN-NEXT: ;;#ASMEND
 ; GCN-NEXT: s_endpgm
 define amdgpu_kernel void @test_call_void_func_void_preserves_s33(i32 addrspace(1)* %out) #0 {
   %s33 = call i32 asm sideeffect "; def $0", "={s33}"()
   call void @external_void_func_void()
   call void asm sideeffect "; use $0", "{s33}"(i32 %s33)
   ret void
 }
 
 ; GCN-LABEL: {{^}}test_call_void_func_void_preserves_v32:
 ; GCN: s_mov_b32 s33, s9
-; GCN: ; def v32
-; GCN-NEXT: #ASMEND
 ; GCN: s_getpc_b64 s[6:7]
 ; GCN-NEXT: s_add_u32 s6, s6, external_void_func_void@rel32@lo+4
 ; GCN-NEXT: s_addc_u32 s7, s7, external_void_func_void@rel32@hi+4
 ; GCN-NEXT: s_mov_b32 s4, s33
 ; GCN-NEXT: s_mov_b32 s32, s33
+; GCN-NEXT: ;;#ASMSTART
+; GCN-NEXT: ; def v32
+; GCN-NEXT: #ASMEND
 ; GCN-NEXT: s_swappc_b64 s[30:31], s[6:7]
 ; GCN-NEXT: ;;#ASMSTART
 ; GCN-NEXT: ; use v32
 ; GCN-NEXT: ;;#ASMEND
 ; GCN-NEXT: s_endpgm
 define amdgpu_kernel void @test_call_void_func_void_preserves_v32(i32 addrspace(1)* %out) #0 {
   %v32 = call i32 asm sideeffect "; def $0", "={v32}"()
   call void @external_void_func_void()

test/CodeGen/AMDGPU/callee-frame-setup.ll

 ; GCN: s_mov_b32 s5, s32
 ; GCN: buffer_store_dword v32, off, s[0:3], s5 offset:8
 
 ; GCN-DAG: v_writelane_b32 v32, s33,
 ; GCN-DAG: v_writelane_b32 v32, s34,
 ; GCN-DAG: v_writelane_b32 v32, s35,
 ; GCN-DAG: s_add_u32 s32, s32, 0x300{{$}}
 ; GCN-DAG: v_mov_b32_e32 v0, 0{{$}}
-; GCN: buffer_store_dword v0, off, s[0:3], s5 offset:4{{$}}
+; GCN-DAG: buffer_store_dword v0, off, s[0:3], s5 offset:4{{$}}
 ; GCN-DAG: s_mov_b32 s33, s5
 
 
 ; GCN: s_swappc_b64
 ; GCN: s_mov_b32 s5, s33
 ; GCN-DAG: v_readlane_b32 s35,
 ; GCN-DAG: v_readlane_b32 s34,
 ; GCN-DAG: v_readlane_b32 s33,

test/CodeGen/AMDGPU/sibling-call.ll

 entry:
   %ret = tail call fastcc i32 @i32_fastcc_i32_i32_a32i32(i32 %a, i32 %b, [32 x i32] zeroinitializer)
   ret i32 %ret
 }
 
 ; Have another non-tail in the function
 ; GCN-LABEL: {{^}}sibling_call_i32_fastcc_i32_i32_other_call:
 ; GCN: s_mov_b32 s5, s32
+; GCN: s_add_u32 s32, s32, 0x400
 ; GCN: buffer_store_dword v34, off, s[0:3], s5 offset:12
 ; GCN: buffer_store_dword v32, off, s[0:3], s5 offset:8 ; 4-byte Folded Spill
 ; GCN: buffer_store_dword v33, off, s[0:3], s5 offset:4 ; 4-byte Folded Spill
 ; GCN-DAG: v_writelane_b32 v34, s33, 0
 ; GCN-DAG: v_writelane_b32 v34, s34, 1
 ; GCN-DAG: v_writelane_b32 v34, s35, 2
-; GCN-DAG: s_add_u32 s32, s32, 0x400
 
-; GCN: s_getpc_b64
 ; GCN: s_swappc_b64
 
-; GCN: s_getpc_b64 s[6:7]
-; GCN: s_add_u32 s6, s6, sibling_call_i32_fastcc_i32_i32@rel32@lo+4
-; GCN: s_addc_u32 s7, s7, sibling_call_i32_fastcc_i32_i32@rel32@hi+4
-
+; GCN-DAG: s_getpc_b64
 ; GCN-DAG: v_readlane_b32 s33, v34, 0
 ; GCN-DAG: v_readlane_b32 s34, v34, 1
 ; GCN-DAG: v_readlane_b32 s35, v34, 2
 
-; GCN: buffer_load_dword v33, off, s[0:3], s5 offset:4
-; GCN: buffer_load_dword v32, off, s[0:3], s5 offset:8
-; GCN: buffer_load_dword v34, off, s[0:3], s5 offset:12
+; GCN-DAG: buffer_load_dword v33, off, s[0:3], s5 offset:4
+; GCN-DAG: buffer_load_dword v32, off, s[0:3], s5 offset:8
+; GCN-DAG: buffer_load_dword v34, off, s[0:3], s5 offset:12
+; GCN-DAG: s_getpc_b64 s[6:7]
+; GCN: s_add_u32 s6, s6, sibling_call_i32_fastcc_i32_i32@rel32@lo+4
+; GCN: s_addc_u32 s7, s7, sibling_call_i32_fastcc_i32_i32@rel32@hi+4
 ; GCN: s_sub_u32 s32, s32, 0x400
 ; GCN: s_setpc_b64 s[6:7]
 define fastcc i32 @sibling_call_i32_fastcc_i32_i32_other_call(i32 %a, i32 %b, i32 %c) #1 {
 entry:
   %other.call = tail call fastcc i32 @i32_fastcc_i32_i32(i32 %a, i32 %b)
   %ret = tail call fastcc i32 @sibling_call_i32_fastcc_i32_i32(i32 %a, i32 %b, i32 %other.call)
   ret i32 %ret
 }

test/CodeGen/ARM/Windows/tls.ll

 
 define i32 @f() {
   %1 = load i32, i32* @i
   ret i32 %1
 }
 
 ; CHECK:      mrc p15, #0, [[TEB:r[0-9]]], c13, c0, #2
 
-; CHECK:      ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK:      movw [[TLS_INDEX:r[0-9]]], :lower16:_tls_index
 ; CHECK-NEXT: movt [[TLS_INDEX]], :upper16:_tls_index
+; CHECK-NEXT: ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK-NEXT: ldr [[INDEX:r[0-9]]], {{\[}}[[TLS_INDEX]]]
 
 ; CHECK-NEXT: ldr{{.w}} [[TLS:r[0-9]]], {{\[}}[[TLS_POINTER]], [[INDEX]], lsl #2]
 
 ; CHECK-NEXT: ldr [[SLOT:r[0-9]]], [[CPI:\.LCPI[0-9]+_[0-9]+]]
 
 ; CHECK-NEXT: ldr r0, {{\[}}[[TLS]], [[SLOT]]]
 
 ; CHECK: [[CPI]]:
 ; CHECK-NEXT: .long i(SECREL32)
 
 define i32 @e() {
   %1 = load i32, i32* @j
   ret i32 %1
 }
 
 ; CHECK:      mrc p15, #0, [[TEB:r[0-9]]], c13, c0, #2
 
-; CHECK:      ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK:      movw [[TLS_INDEX:r[0-9]]], :lower16:_tls_index
 ; CHECK-NEXT: movt [[TLS_INDEX]], :upper16:_tls_index
+; CHECK-NEXT: ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK-NEXT: ldr [[INDEX:r[0-9]]], {{\[}}[[TLS_INDEX]]]
 
 ; CHECK-NEXT: ldr{{.w}} [[TLS:r[0-9]]], {{\[}}[[TLS_POINTER]], [[INDEX]], lsl #2]
 
 ; CHECK-NEXT: ldr [[SLOT:r[0-9]]], [[CPI:\.LCPI[0-9]+_[0-9]+]]
 
 ; CHECK-NEXT: ldr r0, {{\[}}[[TLS]], [[SLOT]]]
 
 ; CHECK: [[CPI]]:
 ; CHECK-NEXT: .long j(SECREL32)
 
 define i32 @d() {
   %1 = load i32, i32* @k
   ret i32 %1
 }
 
 ; CHECK:      mrc p15, #0, [[TEB:r[0-9]]], c13, c0, #2
 
-; CHECK:      ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK:      movw [[TLS_INDEX:r[0-9]]], :lower16:_tls_index
 ; CHECK-NEXT: movt [[TLS_INDEX]], :upper16:_tls_index
+; CHECK-NEXT: ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK-NEXT: ldr [[INDEX:r[0-9]]], {{\[}}[[TLS_INDEX]]]
 
 ; CHECK-NEXT: ldr{{.w}} [[TLS:r[0-9]]], {{\[}}[[TLS_POINTER]], [[INDEX]], lsl #2]
 
 ; CHECK-NEXT: ldr [[SLOT:r[0-9]]], [[CPI:\.LCPI[0-9]+_[0-9]+]]
 
 ; CHECK-NEXT: ldr r0, {{\[}}[[TLS]], [[SLOT]]]
 
 ; CHECK: [[CPI]]:
 ; CHECK-NEXT: .long k(SECREL32)
 
 define i32 @c() {
   %1 = load i32, i32* @l
   ret i32 %1
 }
 
 ; CHECK:      mrc p15, #0, [[TEB:r[0-9]]], c13, c0, #2
 
-; CHECK:      ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK:      movw [[TLS_INDEX:r[0-9]]], :lower16:_tls_index
 ; CHECK-NEXT: movt [[TLS_INDEX]], :upper16:_tls_index
+; CHECK-NEXT: ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK-NEXT: ldr [[INDEX:r[0-9]]], {{\[}}[[TLS_INDEX]]]
 
 ; CHECK-NEXT: ldr{{.w}} [[TLS:r[0-9]]], {{\[}}[[TLS_POINTER]], [[INDEX]], lsl #2]
 
 ; CHECK-NEXT: ldr [[SLOT:r[0-9]]], [[CPI:\.LCPI[0-9]+_[0-9]+]]
 
 ; CHECK-NEXT: ldr r0, {{\[}}[[TLS]], [[SLOT]]]
 
 ; CHECK: [[CPI]]:
 ; CHECK-NEXT: .long l(SECREL32)
 
 define i32 @b() {
   %1 = load i32, i32* @m
   ret i32 %1
 }
 
 ; CHECK:      mrc p15, #0, [[TEB:r[0-9]]], c13, c0, #2
 
-; CHECK:      ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK:      movw [[TLS_INDEX:r[0-9]]], :lower16:_tls_index
 ; CHECK-NEXT: movt [[TLS_INDEX]], :upper16:_tls_index
+; CHECK-NEXT: ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK-NEXT: ldr [[INDEX:r[0-9]]], {{\[}}[[TLS_INDEX]]]
 
 ; CHECK-NEXT: ldr{{.w}} [[TLS:r[0-9]]], {{\[}}[[TLS_POINTER]], [[INDEX]], lsl #2]
 
 ; CHECK-NEXT: ldr [[SLOT:r[0-9]]], [[CPI:\.LCPI[0-9]+_[0-9]+]]
 
 ; CHECK-NEXT: ldr r0, {{\[}}[[TLS]], [[SLOT]]]
 
 ; CHECK: [[CPI]]:
 ; CHECK: .long m(SECREL32)
 
 define i16 @a() {
   %1 = load i16, i16* @n
   ret i16 %1
 }
 
 ; CHECK:      mrc p15, #0, [[TEB:r[0-9]]], c13, c0, #2
 
-; CHECK:      ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK:      movw [[TLS_INDEX:r[0-9]]], :lower16:_tls_index
 ; CHECK-NEXT: movt [[TLS_INDEX]], :upper16:_tls_index
+; CHECK-NEXT: ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK-NEXT: ldr [[INDEX:r[0-9]]], {{\[}}[[TLS_INDEX]]]
 
 ; CHECK-NEXT: ldr{{.w}} [[TLS:r[0-9]]], {{\[}}[[TLS_POINTER]], [[INDEX]], lsl #2]
 
 ; CHECK-NEXT: ldr [[SLOT:r[0-9]]], [[CPI:\.LCPI[0-9]+_[0-9]+]]
 
 ; CHECK-NEXT: ldrh r0, {{\[}}[[TLS]], [[SLOT]]]
 
 ; CHECK: [[CPI]]:
 ; CHECK: .long n(SECREL32)
 
 define i8 @Z() {
   %1 = load i8, i8* @o
   ret i8 %1
 }
 
 ; CHECK:      mrc p15, #0, [[TEB:r[0-9]]], c13, c0, #2
 
-; CHECK:      ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK:      movw [[TLS_INDEX:r[0-9]]], :lower16:_tls_index
 ; CHECK-NEXT: movt [[TLS_INDEX]], :upper16:_tls_index
+; CHECK-NEXT: ldr [[TLS_POINTER:r[0-9]]], {{\[}}[[TEB]], #44]
 ; CHECK-NEXT: ldr [[INDEX:r[0-9]]], {{\[}}[[TLS_INDEX]]]
 
 ; CHECK-NEXT: ldr{{.w}} [[TLS:r[0-9]]], {{\[}}[[TLS_POINTER]], [[INDEX]], lsl #2]
 
 ; CHECK-NEXT: ldr [[SLOT:r[0-9]]], [[CPI:\.LCPI[0-9]+_[0-9]+]]
 
 ; CHECK-NEXT: ldrb r0, {{\[}}[[TLS]], [[SLOT]]]
 
 ; CHECK: [[CPI]]:
 ; CHECK-NEXT: .long o(SECREL32)