[DAGCombiner] Support {a|s}ext, {a|z|s}ext load nodes in load combine

[apilipenko]

Summary

Support {a|s}ext, {a|z|s}ext load nodes as a part of load combine patters.

[filcab]

Thanks Artur.
Can you commit the tests before, so we can easily show that there's a difference? Like in the other patch.

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

4459

ANY_EXTEND mentions that the bits are undefined. Maybe we can include it too (probably there's no difference in most cases, though)?

test/CodeGen/AArch64/load-combine-big-endian.ll

355

Should we be doing anything to addrspace !=0 pointers?

test/CodeGen/ARM/fp16-promote.ll

858

This looks sketchy. Does pop change the PC on ARM? Did it appear but wasn't there?
If so, shouldn't bx lr still be there, to go back to the caller? (I'm not very familiar with ARM asm)

[apilipenko]

• Land tests separately and rebase the patch against it.
• Remove addrspace(1) from the test cases.
• Change CHECK pattern matching in fp16-promote.ll test_extractelement slightly.

[apilipenko]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

4459

calculateByteProvider can return either memory location (ByteProvider with memory location specified), zero constant (zero constant ByteProvider) or unknown (None). We can only express the origin of extend bytes for ZERO_EXTEND. In other cases including ANY_EXTEND it's unknown.

Or you meant something different?

test/CodeGen/AArch64/load-combine-big-endian.ll

355

No, there is nothing special we should do. I removed addrspace qualifiers from the tests.

test/CodeGen/ARM/fp16-promote.ll

858

Here is the codegen for this test case:

test_extractelement:
	.fnstart
@ BB#0:
	.save	{r4, r5, r11, lr}
	push	{r4, r5, r11, lr}
	.pad	#8
	sub	sp, sp, #8
	ldrd	r4, r5, [r1]
	and	r1, r2, #3
	mov	r2, sp
	orr	r1, r2, r1, lsl #1
	stm	sp, {r4, r5}
	ldrh	r1, [r1]
	strh	r1, [r0]
	add	sp, sp, #8
	pop	{r4, r5, r11, pc}

So, in fact we push lr and then pop it to pc. I changed the pattern matching slightly to make it more clear.

[filcab]

No problems on my side, but let's see if Renato has comments on the ARM codegen.

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

4459

I was aiming for an "a read of undef can take any value" approach. But I'm not sure if "the value is undefined", in SDAG, has the same semantics. I don't think this is expected to be much of a benefit (or at all), so I think we might as well drop this question.

test/CodeGen/ARM/fp16-promote.ll

858

Ah. That makes sense, yes. I didn't remember you could pop multiple registers in that way.
What doesn't make sense is that code.
@rengolin Can you take a look at the post-patch test_extractelement?
It's loading stuff onto registers only to spill it again and reload the actual data we want. It's also saving r11 for no reason.

[RKSimon]

test/CodeGen/ARM/fp16-promote.ll

858

@rengolin @jmolloy Please can you advise on the ARM codegen in test_extractelement? Is it something to worry about?

[efriedma]

test/CodeGen/ARM/fp16-promote.ll

858

The loads and stores are not ideal, but they're expected: we're scalarizing the <4 x half>, and the default expansion of extractelement involves a stack temporary. The reason the prologue and epilogue are changing is that load merging allows the backend to form ldrd: ldrd requires an even/odd register pair, so the register allocator chooses to use use r4 and r5, which therefore requires spilling r4/r5.

The end result looks fine.

[filcab]

LGTM since ARM codegen seems to be ok

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

if (BitShift % 8 != 0)
       return None;
     uint64_t ByteShift = BitShift / 8;
 
     return Index < ByteShift
                ? ByteProvider::getConstantZero()
                : calculateByteProvider(Op->getOperand(0), Index - ByteShift,
                                        Depth + 1);
   }
+  case ISD::ANY_EXTEND:
+  case ISD::SIGN_EXTEND:
   case ISD::ZERO_EXTEND: {
     SDValue NarrowOp = Op->getOperand(0);
     unsigned NarrowBitWidth = NarrowOp.getScalarValueSizeInBits();
     if (NarrowBitWidth % 8 != 0)
       return None;
     uint64_t NarrowByteWidth = NarrowBitWidth / 8;
 
-    return Index >= NarrowByteWidth
-               ? ByteProvider::getConstantZero()
-               : calculateByteProvider(NarrowOp, Index, Depth + 1);
+    if (Index >= NarrowByteWidth)
+      return Op.getOpcode() == ISD::ZERO_EXTEND
+                 ? Optional<ByteProvider>(ByteProvider::getConstantZero())
+                 : None;
+    else
+      return calculateByteProvider(NarrowOp, Index, Depth + 1);
   }
   case ISD::BSWAP:
     return calculateByteProvider(Op->getOperand(0), ByteWidth - Index - 1,
                                  Depth + 1);
   case ISD::LOAD: {
     auto L = cast<LoadSDNode>(Op.getNode());
+    if (L->isVolatile() || L->isIndexed())
+      return None;
 
-    // TODO: support ext loads
-    if (L->isVolatile() || L->isIndexed() ||
-        L->getExtensionType() != ISD::NON_EXTLOAD)
+    unsigned NarrowBitWidth = L->getMemoryVT().getSizeInBits();
+    if (NarrowBitWidth % 8 != 0)
       return None;
+    uint64_t NarrowByteWidth = NarrowBitWidth / 8;
 
+    if (Index >= NarrowByteWidth)
+      return L->getExtensionType() == ISD::ZEXTLOAD
+                 ? Optional<ByteProvider>(ByteProvider::getConstantZero())
+                 : None;
+    else
       return ByteProvider::getMemory(L, Index);
   }
   }
 
   return None;
 }
 } // namespace
 
 /// Match a pattern where a wide type scalar value is loaded by several narrow
SDValue DAGCombiner::MatchLoadCombine(SDNode *N) {
   SmallVector<int64_t, 4> ByteOffsets(ByteWidth);
   for (unsigned i = 0; i < ByteWidth; i++) {
     auto P = calculateByteProvider(SDValue(N, 0), i, 0, /*Root=*/true);
     if (!P || !P->isMemory()) // All the bytes must be loaded from memory
       return SDValue();
 
     LoadSDNode *L = P->Load;
     assert(L->hasNUsesOfValue(1, 0) && !L->isVolatile() && !L->isIndexed() &&
-           (L->getExtensionType() == ISD::NON_EXTLOAD) &&
            "Must be enforced by calculateByteProvider");
     assert(L->getOffset().isUndef() && "Unindexed load must have undef offset");
 
     // All loads must share the same chain
     SDValue LChain = L->getChain();
     if (!Chain)
       Chain = LChain;
     else if (Chain != LChain)

test/CodeGen/AArch64/load-combine-big-endian.ll

; CHECK-NEXT: ret
   %tmp7 = or i32 %tmp6, %tmp2
   ret i32 %tmp7
 }
 
 ; i16* p; // p is 4 byte aligned
 ; (i32) p[1] | (sext(p[0] << 16) to i32)
 define i32 @load_i32_by_sext_i16(i32* %arg) {
 ; CHECK-LABEL: load_i32_by_sext_i16:
-; CHECK: ldrh    w8, [x0]
-; CHECK-NEXT: ldrh  w0, [x0, #2]
-; CHECK-NEXT: bfi w0, w8, #16, #16
+; CHECK: ldr   w0, [x0]
 ; CHECK-NEXT: ret
-
   %tmp = bitcast i32* %arg to i16*
   %tmp1 = load i16, i16* %tmp, align 4
   %tmp2 = sext i16 %tmp1 to i32
   %tmp3 = getelementptr inbounds i16, i16* %tmp, i32 1
   %tmp4 = load i16, i16* %tmp3, align 1
   %tmp5 = zext i16 %tmp4 to i32
   %tmp6 = shl nuw nsw i32 %tmp2, 16
   %tmp7 = or i32 %tmp6, %tmp5
   ret i32 %tmp7
 }
 
 ; i8* arg; i32 i;
 ; p = arg + 12;
 ; (i32) p[i] | ((i32) p[i + 1] << 8) | ((i32) p[i + 2] << 16) | ((i32) p[i + 3] << 24)
 define i32 @load_i32_by_i8_base_offset_index(i8* %arg, i32 %i) {
 ; CHECK-LABEL: load_i32_by_i8_base_offset_index:
 ; CHECK: add  x8, x0, w1, uxtw
 ; CHECK-NEXT: ldr w8, [x8, #12]
 ; CHECK-NEXT: rev w0, w8
 ; CHECK-NEXT: ret
   %tmp = add nuw nsw i32 %i, 3
   %tmp2 = add nuw nsw i32 %i, 2
   %tmp3 = add nuw nsw i32 %i, 1
; CHECK-NEXT: ret
   ret i32 %tmp26
 }
 
 ; i8* arg; i32 i;
 ; p = arg + 12;
 ; (i32) p[i + 1] | ((i32) p[i + 2] << 8) | ((i32) p[i + 3] << 16) | ((i32) p[i + 4] << 24)
 define i32 @load_i32_by_i8_base_offset_index_2(i8* %arg, i32 %i) {
 ; CHECK-LABEL: load_i32_by_i8_base_offset_index_2:
-; CHECK: add x8, x0, w1, uxtw
-; CHECK-NEXT: ldrb  w0, [x8, #13]
-; CHECK-NEXT: ldrb  w9, [x8, #14]
-; CHECK-NEXT: ldrb  w10, [x8, #15]
-; CHECK-NEXT: ldrb  w8, [x8, #16]
-; CHECK-NEXT: bfi w0, w9, #8, #8
-; CHECK-NEXT: bfi w0, w10, #16, #8
-; CHECK-NEXT: bfi w0, w8, #24, #8
+; CHECK: add  x8, x0, w1, uxtw
+; CHECK-NEXT: ldur w8, [x8, #13]
+; CHECK-NEXT: rev w0, w8
 ; CHECK-NEXT: ret
-
   %tmp = add nuw nsw i32 %i, 4
   %tmp2 = add nuw nsw i32 %i, 3
   %tmp3 = add nuw nsw i32 %i, 2
   %tmp4 = getelementptr inbounds i8, i8* %arg, i64 12
   %tmp5 = add nuw nsw i32 %i, 1
   %tmp27 = zext i32 %tmp5 to i64
   %tmp28 = getelementptr inbounds i8, i8* %tmp4, i64 %tmp27
   %tmp29 = load i8, i8* %tmp28, align 4

test/CodeGen/AArch64/load-combine.ll

; CHECK-NEXT: ret
   %tmp7 = or i32 %tmp6, %tmp5
   ret i32 %tmp7
 }
 
 ; i16* p; // p is 4 byte aligned
 ; (i32) p[0] | (sext(p[1] << 16) to i32)
 define i32 @load_i32_by_sext_i16(i32* %arg) {
 ; CHECK-LABEL: load_i32_by_sext_i16:
-; CHECK: ldrh    w8, [x0]
-; CHECK-NEXT: ldrh  w9, [x0, #2]
-; CHECK-NEXT: bfi w8, w9, #16, #16
-; CHECK-NEXT: mov  w0, w8
+; CHECK: ldr   w0, [x0]
 ; CHECK-NEXT: ret
-
   %tmp = bitcast i32* %arg to i16*
   %tmp1 = load i16, i16* %tmp, align 4
   %tmp2 = zext i16 %tmp1 to i32
   %tmp3 = getelementptr inbounds i16, i16* %tmp, i32 1
   %tmp4 = load i16, i16* %tmp3, align 1
   %tmp5 = sext i16 %tmp4 to i32
   %tmp6 = shl nuw nsw i32 %tmp5, 16
   %tmp7 = or i32 %tmp6, %tmp2

 }
 
 ; i8* arg; i32 i;
 ; p = arg + 12;
 ; (i32) p[i + 1] | ((i32) p[i + 2] << 8) | ((i32) p[i + 3] << 16) | ((i32) p[i + 4] << 24)
 define i32 @load_i32_by_i8_base_offset_index_2(i8* %arg, i32 %i) {
 ; CHECK-LABEL: load_i32_by_i8_base_offset_index_2:
 ; CHECK: add x8, x0, w1, uxtw
-; CHECK-NEXT: ldrb  w0, [x8, #13]
-; CHECK-NEXT: ldrb  w9, [x8, #14]
-; CHECK-NEXT: ldrb  w10, [x8, #15]
-; CHECK-NEXT: ldrb  w8, [x8, #16]
-; CHECK-NEXT: bfi w0, w9, #8, #8
-; CHECK-NEXT: bfi w0, w10, #16, #8
-; CHECK-NEXT: bfi w0, w8, #24, #8
+; CHECK-NEXT: ldur w0, [x8, #13]
 ; CHECK-NEXT: ret
-
   %tmp = add nuw nsw i32 %i, 4
   %tmp2 = add nuw nsw i32 %i, 3
   %tmp3 = add nuw nsw i32 %i, 2
   %tmp4 = getelementptr inbounds i8, i8* %arg, i64 12
   %tmp5 = add nuw nsw i32 %i, 1
   %tmp27 = zext i32 %tmp5 to i64
   %tmp28 = getelementptr inbounds i8, i8* %tmp4, i64 %tmp27
   %tmp29 = load i8, i8* %tmp28, align 4
; CHECK-NEXT: ret
   %tmp43 = zext i32 %tmp to i64
   %tmp44 = getelementptr inbounds i8, i8* %tmp4, i64 %tmp43
   %tmp45 = load i8, i8* %tmp44, align 1
   %tmp46 = zext i8 %tmp45 to i32
   %tmp47 = shl nuw i32 %tmp46, 24
   %tmp48 = or i32 %tmp42, %tmp47
   ret i32 %tmp48
 }
 No newline at end of file

test/CodeGen/ARM/fp16-promote.ll

define void @test_insertelement(half* %p, <4 x half>* %q, i32 %i) #0 {
   %a = load half, half* %p, align 2
   %b = load <4 x half>, <4 x half>* %q, align 8
   %c = insertelement <4 x half> %b, half %a, i32 %i
   store <4 x half> %c, <4 x half>* %q
   ret void
 }
 
 ; CHECK-ALL-LABEL: test_extractelement:
+; CHECK-VFP: push {{{.*}}, lr}
 ; CHECK-VFP: sub sp, sp, #8
-; CHECK-VFP: ldrh
-; CHECK-VFP: ldrh
-; CHECK-VFP: orr
-; CHECK-VFP: str
-; CHECK-VFP: ldrh
-; CHECK-VFP: ldrh
-; CHECK-VFP: orr
-; CHECK-VFP: str
+; CHECK-VFP: ldrd
 ; CHECK-VFP: mov
 ; CHECK-VFP: orr
 ; CHECK-VFP: ldrh
 ; CHECK-VFP: strh
 ; CHECK-VFP: add sp, sp, #8
-; CHECK-VFP: bx lr
+; CHECK-VFP: pop {{{.*}}, pc}
 ; CHECK-NOVFP: ldrh
 ; CHECK-NOVFP: strh
 ; CHECK-NOVFP: ldrh
 ; CHECK-NOVFP: strh
 ; CHECK-NOVFP: ldrh
 ; CHECK-NOVFP: strh
 ; CHECK-NOVFP: ldrh
 ; CHECK-NOVFP: strh

test/CodeGen/ARM/load-combine-big-endian.ll

; CHECK-ARMv6-NEXT: bx  lr
   %tmp7 = or i32 %tmp6, %tmp2
   ret i32 %tmp7
 }
 
 ; i16* p; // p is 4 byte aligned
 ; (i32) p[1] | (sext(p[0] << 16) to i32)
 define i32 @load_i32_by_sext_i16(i32* %arg) {
 ; CHECK-LABEL: load_i32_by_sext_i16:
-; CHECK: ldrh  r1, [r0]
-; CHECK-NEXT: ldrh  r0, [r0, #2]
-; CHECK-NEXT: orr r0, r0, r1, lsl #16
+; CHECK: ldr  r0, [r0]
 ; CHECK-NEXT: mov pc, lr
-
-; CHECK-ARMv6-LABEL: load_i32_by_sext_i16:
-; CHECK-ARMv6: ldrh  r1, [r0]
-; CHECK-ARMv6-NEXT: ldrh  r0, [r0, #2]
-; CHECK-ARMv6-NEXT: orr r0, r0, r1, lsl #16
+;
+; CHECK-ARMv6-LABEL: 
+; CHECK-ARMv6: ldr r0, [r0]
 ; CHECK-ARMv6-NEXT: bx  lr
-  
   %tmp = bitcast i32* %arg to i16*
   %tmp1 = load i16, i16* %tmp, align 4
   %tmp2 = sext i16 %tmp1 to i32
   %tmp3 = getelementptr inbounds i16, i16* %tmp, i32 1
   %tmp4 = load i16, i16* %tmp3, align 1
   %tmp5 = zext i16 %tmp4 to i32
   %tmp6 = shl nuw nsw i32 %tmp2, 16
   %tmp7 = or i32 %tmp6, %tmp5

 }
 
 ; i8* arg; i32 i;
 ; p = arg + 12;
 ; (i32) p[i + 1] | ((i32) p[i + 2] << 8) | ((i32) p[i + 3] << 16) | ((i32) p[i + 4] << 24)
 define i32 @load_i32_by_i8_base_offset_index_2(i8* %arg, i32 %i) {
 ; CHECK-LABEL: load_i32_by_i8_base_offset_index_2:
 ; CHECK: add r0, r0, r1
-; CHECK-NEXT: ldrb  r1, [r0, #13]
-; CHECK-NEXT: ldrb  r2, [r0, #14]
-; CHECK-NEXT: ldrb  r3, [r0, #15]
-; CHECK-NEXT: ldrb  r0, [r0, #16]
-; CHECK-NEXT: orr r1, r1, r2, lsl #8
-; CHECK-NEXT: orr r1, r1, r3, lsl #16
-; CHECK-NEXT: orr r0, r1, r0, lsl #24
+; CHECK-NEXT: mov r1, #65280
+; CHECK-NEXT: mov r2, #16711680
+; CHECK-NEXT: ldr r0, [r0, #13]
+; CHECK-NEXT: and r1, r1, r0, lsr #8
+; CHECK-NEXT: and r2, r2, r0, lsl #8
+; CHECK-NEXT: orr r1, r1, r0, lsr #24
+; CHECK-NEXT: orr r0, r2, r0, lsl #24
+; CHECK-NEXT: orr r0, r0, r1
 ; CHECK-NEXT: mov pc, lr
 ;
 ; CHECK-ARMv6-LABEL: load_i32_by_i8_base_offset_index_2:
 ; CHECK-ARMv6: add r0, r0, r1
-; CHECK-ARMv6-NEXT: ldrb  r1, [r0, #13]
-; CHECK-ARMv6-NEXT: ldrb  r2, [r0, #14]
-; CHECK-ARMv6-NEXT: ldrb  r3, [r0, #15]
-; CHECK-ARMv6-NEXT: ldrb  r0, [r0, #16]
-; CHECK-ARMv6-NEXT: orr r1, r1, r2, lsl #8
-; CHECK-ARMv6-NEXT: orr r1, r1, r3, lsl #16
-; CHECK-ARMv6-NEXT: orr r0, r1, r0, lsl #24
+; CHECK-ARMv6-NEXT: ldr r0, [r0, #13]
+; CHECK-ARMv6-NEXT: rev r0, r0
 ; CHECK-ARMv6-NEXT: bx  lr
 
   %tmp = add nuw nsw i32 %i, 4
   %tmp2 = add nuw nsw i32 %i, 3
   %tmp3 = add nuw nsw i32 %i, 2
   %tmp4 = getelementptr inbounds i8, i8* %arg, i64 12
   %tmp5 = add nuw nsw i32 %i, 1
   %tmp27 = zext i32 %tmp5 to i64
; CHECK-ARMv6-NEXT: bx  lr
   %tmp43 = zext i32 %tmp to i64
   %tmp44 = getelementptr inbounds i8, i8* %tmp4, i64 %tmp43
   %tmp45 = load i8, i8* %tmp44, align 1
   %tmp46 = zext i8 %tmp45 to i32
   %tmp47 = shl nuw i32 %tmp46, 24
   %tmp48 = or i32 %tmp42, %tmp47
   ret i32 %tmp48
 }
 No newline at end of file

test/CodeGen/ARM/load-combine.ll

; CHECK-ARMv6-NEXT: bx  lr
   %tmp7 = or i32 %tmp6, %tmp5
   ret i32 %tmp7
 }
 
 ; i16* p;
 ; (i32) p[0] | (sext(p[1] << 16) to i32)
 define i32 @load_i32_by_sext_i16(i32* %arg) {
 ; CHECK-LABEL: load_i32_by_sext_i16:
-; CHECK: ldrh  r1, [r0, #2]
-; CHECK-NEXT: ldrh  r0, [r0]
-; CHECK-NEXT: orr r0, r0, r1, lsl #16
+; CHECK: ldr  r0, [r0]
 ; CHECK-NEXT: mov pc, lr
 ;
 ; CHECK-ARMv6-LABEL: load_i32_by_sext_i16:
-; CHECK-ARMv6: ldrh  r1, [r0, #2]
-; CHECK-ARMv6-NEXT: ldrh  r0, [r0]
-; CHECK-ARMv6-NEXT: orr r0, r0, r1, lsl #16
-; CHECK-ARMv6-NEXT: bx  lr
-  
+; CHECK-ARMv6: ldr  r0, [r0]
+; CHECK-ARMv6-NEXT: bx lr
   %tmp = bitcast i32* %arg to i16*
   %tmp1 = load i16, i16* %tmp, align 4
   %tmp2 = zext i16 %tmp1 to i32
   %tmp3 = getelementptr inbounds i16, i16* %tmp, i32 1
   %tmp4 = load i16, i16* %tmp3, align 1
   %tmp5 = sext i16 %tmp4 to i32
   %tmp6 = shl nuw nsw i32 %tmp5, 16
   %tmp7 = or i32 %tmp6, %tmp2

 }
 
 ; i8* arg; i32 i;
 ; p = arg + 12;
 ; (i32) p[i + 1] | ((i32) p[i + 2] << 8) | ((i32) p[i + 3] << 16) | ((i32) p[i + 4] << 24)
 define i32 @load_i32_by_i8_base_offset_index_2(i8* %arg, i32 %i) {
 ; CHECK-LABEL: load_i32_by_i8_base_offset_index_2:
 ; CHECK: add r0, r0, r1
-; CHECK-NEXT: ldrb  r1, [r0, #13]
-; CHECK-NEXT: ldrb  r2, [r0, #14]
-; CHECK-NEXT: ldrb  r3, [r0, #15]
-; CHECK-NEXT: ldrb  r0, [r0, #16]
-; CHECK-NEXT: orr r1, r1, r2, lsl #8
-; CHECK-NEXT: orr r1, r1, r3, lsl #16
-; CHECK-NEXT: orr r0, r1, r0, lsl #24
+; CHECK-NEXT: ldr r0, [r0, #13]
 ; CHECK-NEXT: mov pc, lr
 ;
 ; CHECK-ARMv6-LABEL: load_i32_by_i8_base_offset_index_2:
 ; CHECK-ARMv6: add r0, r0, r1
-; CHECK-ARMv6-NEXT: ldrb  r1, [r0, #13]
-; CHECK-ARMv6-NEXT: ldrb  r2, [r0, #14]
-; CHECK-ARMv6-NEXT: ldrb  r3, [r0, #15]
-; CHECK-ARMv6-NEXT: ldrb  r0, [r0, #16]
-; CHECK-ARMv6-NEXT: orr r1, r1, r2, lsl #8
-; CHECK-ARMv6-NEXT: orr r1, r1, r3, lsl #16
-; CHECK-ARMv6-NEXT: orr r0, r1, r0, lsl #24
+; CHECK-ARMv6-NEXT: ldr r0, [r0, #13]
 ; CHECK-ARMv6-NEXT: bx  lr
-
   %tmp = add nuw nsw i32 %i, 4
   %tmp2 = add nuw nsw i32 %i, 3
   %tmp3 = add nuw nsw i32 %i, 2
   %tmp4 = getelementptr inbounds i8, i8* %arg, i64 12
   %tmp5 = add nuw nsw i32 %i, 1
   %tmp27 = zext i32 %tmp5 to i64
   %tmp28 = getelementptr inbounds i8, i8* %tmp4, i64 %tmp27
   %tmp29 = load i8, i8* %tmp28, align 4

test/CodeGen/X86/load-combine.ll

 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; CHECK-NEXT:    movl (%ecx,%eax), %eax
 ; CHECK-NEXT:    bswapl %eax
 ; CHECK-NEXT:    retl
 ;
 ; CHECK64-LABEL: load_i32_by_i8_bswap_base_index_offset:
 ; CHECK64:       # BB#0:
 ; CHECK64-NEXT:    movslq %esi, %rax
-; CHECK64-NEXT:    movzbl (%rdi,%rax), %ecx
-; CHECK64-NEXT:    shll $24, %ecx
-; CHECK64-NEXT:    movzbl 1(%rdi,%rax), %edx
-; CHECK64-NEXT:    shll $16, %edx
-; CHECK64-NEXT:    orl %ecx, %edx
-; CHECK64-NEXT:    movzbl 2(%rdi,%rax), %ecx
-; CHECK64-NEXT:    shll $8, %ecx
-; CHECK64-NEXT:    orl %edx, %ecx
-; CHECK64-NEXT:    movzbl 3(%rdi,%rax), %eax
-; CHECK64-NEXT:    orl %ecx, %eax
+; CHECK64-NEXT:    movl (%rdi,%rax), %eax
+; CHECK64-NEXT:    bswapl %eax
 ; CHECK64-NEXT:    retq
   %tmp = bitcast i32* %arg to i8*
   %tmp2 = getelementptr inbounds i8, i8* %tmp, i32 %arg1
   %tmp3 = load i8, i8* %tmp2, align 1
   %tmp4 = zext i8 %tmp3 to i32
   %tmp5 = shl nuw nsw i32 %tmp4, 24
   %tmp6 = add nuw nsw i32 %arg1, 1
   %tmp7 = getelementptr inbounds i8, i8* %tmp, i32 %tmp6

 }
 
 ; i16* p;
 ; (i32) p[0] | (sext(p[1] << 16) to i32)
 define i32 @load_i32_by_sext_i16(i32* %arg) {
 ; CHECK-LABEL: load_i32_by_sext_i16:
 ; CHECK:       # BB#0:
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movzwl (%eax), %ecx
-; CHECK-NEXT:    movzwl 2(%eax), %eax
-; CHECK-NEXT:    shll $16, %eax
-; CHECK-NEXT:    orl %ecx, %eax
+; CHECK-NEXT:    movl (%eax), %eax
 ; CHECK-NEXT:    retl
 ;
 ; CHECK64-LABEL: load_i32_by_sext_i16:
 ; CHECK64:       # BB#0:
-; CHECK64-NEXT:    movzwl (%rdi), %ecx
-; CHECK64-NEXT:    movzwl 2(%rdi), %eax
-; CHECK64-NEXT:    shll $16, %eax
-; CHECK64-NEXT:    orl %ecx, %eax
+; CHECK64-NEXT:    movl (%rdi), %eax
 ; CHECK64-NEXT:    retq
   %tmp = bitcast i32* %arg to i16*
   %tmp1 = load i16, i16* %tmp, align 1
   %tmp2 = zext i16 %tmp1 to i32
   %tmp3 = getelementptr inbounds i16, i16* %tmp, i32 1
   %tmp4 = load i16, i16* %tmp3, align 1
   %tmp5 = sext i16 %tmp4 to i32
   %tmp6 = shl nuw nsw i32 %tmp5, 16
   %tmp7 = or i32 %tmp6, %tmp2
   ret i32 %tmp7
 }
 
 ; i8* arg; i32 i;
 ; p = arg + 12;
 ; (i32) p[i] | ((i32) p[i + 1] << 8) | ((i32) p[i + 2] << 16) | ((i32) p[i + 3] << 24)
 define i32 @load_i32_by_i8_base_offset_index(i8* %arg, i32 %i) {
 ; CHECK-LABEL: load_i32_by_i8_base_offset_index:
 ; CHECK:       # BB#0:
-; CHECK-NEXT:    pushl %esi
-; CHECK-NEXT:  .Lcfi4:
-; CHECK-NEXT:    .cfi_def_cfa_offset 8
-; CHECK-NEXT:  .Lcfi5:
-; CHECK-NEXT:    .cfi_offset %esi, -8
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    movzbl 12(%eax,%ecx), %edx
-; CHECK-NEXT:    movzbl 13(%eax,%ecx), %esi
-; CHECK-NEXT:    shll $8, %esi
-; CHECK-NEXT:    orl %edx, %esi
-; CHECK-NEXT:    movzbl 14(%eax,%ecx), %edx
-; CHECK-NEXT:    shll $16, %edx
-; CHECK-NEXT:    orl %esi, %edx
-; CHECK-NEXT:    movzbl 15(%eax,%ecx), %eax
-; CHECK-NEXT:    shll $24, %eax
-; CHECK-NEXT:    orl %edx, %eax
-; CHECK-NEXT:    popl %esi
+; CHECK-NEXT:    movl 12(%eax,%ecx), %eax
 ; CHECK-NEXT:    retl
 ;
 ; CHECK64-LABEL: load_i32_by_i8_base_offset_index:
 ; CHECK64:       # BB#0:
 ; CHECK64-NEXT:    movl %esi, %eax
 ; CHECK64-NEXT:    movl 12(%rdi,%rax), %eax
 ; CHECK64-NEXT:    retq
   %tmp = add nuw nsw i32 %i, 3

 }
 
 ; i8* arg; i32 i;
 ; p = arg + 12;
 ; (i32) p[i + 1] | ((i32) p[i + 2] << 8) | ((i32) p[i + 3] << 16) | ((i32) p[i + 4] << 24)
 define i32 @load_i32_by_i8_base_offset_index_2(i8* %arg, i32 %i) {
 ; CHECK-LABEL: load_i32_by_i8_base_offset_index_2:
 ; CHECK:       # BB#0:
-; CHECK-NEXT:    pushl %esi
-; CHECK-NEXT:  .Lcfi6:
-; CHECK-NEXT:    .cfi_def_cfa_offset 8
-; CHECK-NEXT:  .Lcfi7:
-; CHECK-NEXT:    .cfi_offset %esi, -8
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    movzbl 13(%eax,%ecx), %edx
-; CHECK-NEXT:    movzbl 14(%eax,%ecx), %esi
-; CHECK-NEXT:    shll $8, %esi
-; CHECK-NEXT:    orl %edx, %esi
-; CHECK-NEXT:    movzbl 15(%eax,%ecx), %edx
-; CHECK-NEXT:    shll $16, %edx
-; CHECK-NEXT:    orl %esi, %edx
-; CHECK-NEXT:    movzbl 16(%eax,%ecx), %eax
-; CHECK-NEXT:    shll $24, %eax
-; CHECK-NEXT:    orl %edx, %eax
-; CHECK-NEXT:    popl %esi
+; CHECK-NEXT:    movl 13(%eax,%ecx), %eax
 ; CHECK-NEXT:    retl
 ;
 ; CHECK64-LABEL: load_i32_by_i8_base_offset_index_2:
 ; CHECK64:       # BB#0:
 ; CHECK64-NEXT:    movl %esi, %eax
-; CHECK64-NEXT:    movzbl 13(%rdi,%rax), %ecx
-; CHECK64-NEXT:    movzbl 14(%rdi,%rax), %edx
-; CHECK64-NEXT:    shll $8, %edx
-; CHECK64-NEXT:    orl %ecx, %edx
-; CHECK64-NEXT:    movzbl 15(%rdi,%rax), %ecx
-; CHECK64-NEXT:    shll $16, %ecx
-; CHECK64-NEXT:    orl %edx, %ecx
-; CHECK64-NEXT:    movzbl 16(%rdi,%rax), %eax
-; CHECK64-NEXT:    shll $24, %eax
-; CHECK64-NEXT:    orl %ecx, %eax
+; CHECK64-NEXT:    movl 13(%rdi,%rax), %eax
 ; CHECK64-NEXT:    retq
   %tmp = add nuw nsw i32 %i, 4
   %tmp2 = add nuw nsw i32 %i, 3
   %tmp3 = add nuw nsw i32 %i, 2
   %tmp4 = getelementptr inbounds i8, i8* %arg, i64 12
   %tmp5 = add nuw nsw i32 %i, 1
   %tmp27 = zext i32 %tmp5 to i64
   %tmp28 = getelementptr inbounds i8, i8* %tmp4, i64 %tmp27

 ;
 ; This test excercises zero and any extend loads as a part of load combine pattern.
 ; In order to fold the pattern above we need to reassociate the address computation
 ; first. By the time the address computation is reassociated loads are combined to
 ; to zext and aext loads.
 define i32 @load_i32_by_i8_zaext_loads(i8* %arg, i32 %arg1) {
 ; CHECK-LABEL: load_i32_by_i8_zaext_loads:
 ; CHECK:       # BB#0:
-; CHECK-NEXT:    pushl %esi
-; CHECK-NEXT:  .Lcfi8:
-; CHECK-NEXT:    .cfi_def_cfa_offset 8
-; CHECK-NEXT:  .Lcfi9:
-; CHECK-NEXT:    .cfi_offset %esi, -8
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    movzbl 12(%eax,%ecx), %edx
-; CHECK-NEXT:    movzbl 13(%eax,%ecx), %esi
-; CHECK-NEXT:    shll $8, %esi
-; CHECK-NEXT:    orl %edx, %esi
-; CHECK-NEXT:    movzbl 14(%eax,%ecx), %edx
-; CHECK-NEXT:    shll $16, %edx
-; CHECK-NEXT:    orl %esi, %edx
-; CHECK-NEXT:    movzbl 15(%eax,%ecx), %eax
-; CHECK-NEXT:    shll $24, %eax
-; CHECK-NEXT:    orl %edx, %eax
-; CHECK-NEXT:    popl %esi
+; CHECK-NEXT:    movl 12(%eax,%ecx), %eax
 ; CHECK-NEXT:    retl
 ;
 ; CHECK64-LABEL: load_i32_by_i8_zaext_loads:
 ; CHECK64:       # BB#0:
 ; CHECK64-NEXT:    movl %esi, %eax
-; CHECK64-NEXT:    movzbl 12(%rdi,%rax), %ecx
-; CHECK64-NEXT:    movzbl 13(%rdi,%rax), %edx
-; CHECK64-NEXT:    shll $8, %edx
-; CHECK64-NEXT:    orl %ecx, %edx
-; CHECK64-NEXT:    movzbl 14(%rdi,%rax), %ecx
-; CHECK64-NEXT:    shll $16, %ecx
-; CHECK64-NEXT:    orl %edx, %ecx
-; CHECK64-NEXT:    movzbl 15(%rdi,%rax), %eax
-; CHECK64-NEXT:    shll $24, %eax
-; CHECK64-NEXT:    orl %ecx, %eax
+; CHECK64-NEXT:    movl 12(%rdi,%rax), %eax
 ; CHECK64-NEXT:    retq
   %tmp = add nuw nsw i32 %arg1, 3
   %tmp2 = add nuw nsw i32 %arg1, 2
   %tmp3 = add nuw nsw i32 %arg1, 1
   %tmp4 = zext i32 %tmp to i64
   %tmp5 = zext i32 %tmp2 to i64
   %tmp6 = zext i32 %tmp3 to i64
   %tmp24 = getelementptr inbounds i8, i8* %arg, i64 %tmp4

 ; p1 = arg + i + 1;
 ; p2 = arg + i + 2;
 ; p3 = arg + i + 3;
 ;
 ; (i32) p0[12] | ((i32) p1[12] << 8) | ((i32) p2[12] << 16) | ((i32) p3[12] << 24)
 define i32 @load_i32_by_i8_zsext_loads(i8* %arg, i32 %arg1) {
 ; CHECK-LABEL: load_i32_by_i8_zsext_loads:
 ; CHECK:       # BB#0:
-; CHECK-NEXT:    pushl %esi
-; CHECK-NEXT:  .Lcfi10:
-; CHECK-NEXT:    .cfi_def_cfa_offset 8
-; CHECK-NEXT:  .Lcfi11:
-; CHECK-NEXT:    .cfi_offset %esi, -8
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    movzbl 12(%eax,%ecx), %edx
-; CHECK-NEXT:    movzbl 13(%eax,%ecx), %esi
-; CHECK-NEXT:    shll $8, %esi
-; CHECK-NEXT:    orl %edx, %esi
-; CHECK-NEXT:    movzbl 14(%eax,%ecx), %edx
-; CHECK-NEXT:    shll $16, %edx
-; CHECK-NEXT:    orl %esi, %edx
-; CHECK-NEXT:    movsbl 15(%eax,%ecx), %eax
-; CHECK-NEXT:    shll $24, %eax
-; CHECK-NEXT:    orl %edx, %eax
-; CHECK-NEXT:    popl %esi
+; CHECK-NEXT:    movl 12(%eax,%ecx), %eax
 ; CHECK-NEXT:    retl
 ;
 ; CHECK64-LABEL: load_i32_by_i8_zsext_loads:
 ; CHECK64:       # BB#0:
 ; CHECK64-NEXT:    movl %esi, %eax
-; CHECK64-NEXT:    movzbl 12(%rdi,%rax), %ecx
-; CHECK64-NEXT:    movzbl 13(%rdi,%rax), %edx
-; CHECK64-NEXT:    shll $8, %edx
-; CHECK64-NEXT:    orl %ecx, %edx
-; CHECK64-NEXT:    movzbl 14(%rdi,%rax), %ecx
-; CHECK64-NEXT:    shll $16, %ecx
-; CHECK64-NEXT:    orl %edx, %ecx
-; CHECK64-NEXT:    movsbl 15(%rdi,%rax), %eax
-; CHECK64-NEXT:    shll $24, %eax
-; CHECK64-NEXT:    orl %ecx, %eax
+; CHECK64-NEXT:    movl 12(%rdi,%rax), %eax
 ; CHECK64-NEXT:    retq
   %tmp = add nuw nsw i32 %arg1, 3
   %tmp2 = add nuw nsw i32 %arg1, 2
   %tmp3 = add nuw nsw i32 %arg1, 1
   %tmp4 = zext i32 %tmp to i64
   %tmp5 = zext i32 %tmp2 to i64
   %tmp6 = zext i32 %tmp3 to i64
   %tmp24 = getelementptr inbounds i8, i8* %arg, i64 %tmp4