[X86] Remove dead nodes left after ReplaceAllUsesWith calls during address matching

[craig.topper]

Summary

ReplaceAllUsesWith doesn't remove the node that was replaced. So its left around in the graph messing up use counts on other nodes.

One thing to note, is that this isn't valid if the node being deleted is the root node of an LEA match that gets rejected. In that case the node needs to stay alive because the isel table walking code would still have a reference to it that its going to try to match next. I don't think that's the case here though because the nodes being deleted here should be "and", "srl", and "zero_extend" none of which can be the root node of an LEA match.

[craig.topper]

llvm/test/CodeGen/X86/fold-and-shift.ll

9 ↗

(On Diff #196360)

Looks like we were previously selecting movzx because the load folding was being suppressed by an artificial extra use on the load. Now we favor folding the load over folding the immediate like we normally would. We can probably hack IsProfitableToFold to restore the old behavior. The best option would be to narrow the load and use movzx from memory. But that's only valid if the load isn't volatile. I suppose we probably want the previous behavior in the case of a volatile load?

26 ↗

(On Diff #196360)

Similar to above

[niravd]

LGTM.

DAGCombiner now has infrastructure to strip away nodes that were created but unused in during a visit. It probably makes sense to do something similar in ISel

[hctim]

Looks like this change introduced a use-after-poison (aka use-after-d'tor) (asan buildbot).

PTAL :)

==6559==ERROR: AddressSanitizer: use-after-poison on address 0x62100003244a at pc 0x00000305b716 bp 0x7ffeac52afb0 sp 0x7ffeac52afa8
READ of size 2 at 0x62100003244a thread T0
    #0 0x305b715 in getValueType /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/CodeGen/SelectionDAGNodes.h:989:5
    #1 0x305b715 in getValueType /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/CodeGen/SelectionDAGNodes.h:1137
    #2 0x305b715 in getSimpleValueType /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/CodeGen/SelectionDAGNodes.h:169
    #3 0x305b715 in (anonymous namespace)::X86DAGToDAGISel::selectAddr(llvm::SDNode*, llvm::SDValue, llvm::SDValue&, llvm::SDValue&, llvm::SDValue&, llvm::SDValue&, llvm::SDValue&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp:2063
    #4 0x3047a07 in (anonymous namespace)::X86DAGToDAGISel::CheckComplexPattern(llvm::SDNode*, llvm::SDNode*, llvm::SDValue, unsigned int, llvm::SmallVectorImpl<std::__1::pair<llvm::SDValue, llvm::SDNode*> >&) /b/sanitizer-x86_64-linux-fast/build/llvm_build_asan/lib/Target/X86/X86GenDAGISel.inc:275229:10
    #5 0x5b56325 in llvm::SelectionDAGISel::SelectCodeCommon(llvm::SDNode*, unsigned char const*, unsigned int) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:3275:12
    #6 0x302eafe in (anonymous namespace)::X86DAGToDAGISel::Select(llvm::SDNode*) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
    #7 0x5b3bed8 in llvm::SelectionDAGISel::DoInstructionSelection() /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1139:7
    #8 0x5b37154 in llvm::SelectionDAGISel::CodeGenAndEmitDAG() /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:937:5
    #9 0x5b2dd26 in llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1814:7
    #10 0x5b22d25 in llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:496:3
    #11 0x301ac1c in (anonymous namespace)::X86DAGToDAGISel::runOnMachineFunction(llvm::MachineFunction&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp:191:25
    #12 0x41a85aa in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/MachineFunctionPass.cpp:73:13
    #13 0x4c312d3 in llvm::FPPassManager::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:1648:27
    #14 0x4c31b52 in llvm::FPPassManager::runOnModule(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:1685:16
    #15 0x4c32a1c in runOnModule /b/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:1752:27
    #16 0x4c32a1c in llvm::legacy::PassManagerImpl::run(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:1865
    #17 0xa6a209 in compileModule(char**, llvm::LLVMContext&) /b/sanitizer-x86_64-linux-fast/build/llvm/tools/llc/llc.cpp:609:8
    #18 0xa63d50 in main /b/sanitizer-x86_64-linux-fast/build/llvm/tools/llc/llc.cpp:363:22
    #19 0x7f585706f2e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)
    #20 0x9565d9 in _start (/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan/bin/llc+0x9565d9)

0x62100003244a is located 842 bytes inside of 4096-byte region [0x621000032100,0x621000033100)
allocated by thread T0 here:
    #0 0xa227ff in malloc /b/sanitizer-x86_64-linux-fast/build/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:146
    #1 0xaa1143 in safe_malloc /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/Support/MemAlloc.h:26:18
    #2 0xaa1143 in Allocate /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/Support/Allocator.h:99
    #3 0xaa1143 in llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096ul, 4096ul>::StartNewSlab() /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/Support/Allocator.h:400
    #4 0xaa0e79 in llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096ul, 4096ul>::Allocate(unsigned long, unsigned long) /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/Support/Allocator.h:260:5
    #5 0x5a44138 in Allocate<llvm::FrameIndexSDNode, llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096, 4096> > /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/Support/Recycler.h:89:57
    #6 0x5a44138 in Allocate<llvm::FrameIndexSDNode> /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/Support/RecyclingAllocator.h:43
    #7 0x5a44138 in newSDNode<llvm::FrameIndexSDNode, int &, llvm::EVT &, bool &> /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/CodeGen/SelectionDAG.h:341
    #8 0x5a44138 in llvm::SelectionDAG::getFrameIndex(int, llvm::EVT, bool) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:1389
    #9 0x30986e0 in llvm::X86TargetLowering::LowerMemArgument(llvm::SDValue, unsigned int, llvm::SmallVectorImpl<llvm::ISD::InputArg> const&, llvm::SDLoc const&, llvm::SelectionDAG&, llvm::CCValAssign const&, llvm::MachineFrameInfo&, unsigned int) const /b/sanitizer-x86_64-linux-fast/build/llvm/lib/Target/X86/X86ISelLowering.cpp:3068:21
    #10 0x3099f9f in llvm::X86TargetLowering::LowerFormalArguments(llvm::SDValue, unsigned int, bool, llvm::SmallVectorImpl<llvm::ISD::InputArg> const&, llvm::SDLoc const&, llvm::SelectionDAG&, llvm::SmallVectorImpl<llvm::SDValue>&) const /b/sanitizer-x86_64-linux-fast/build/llvm/lib/Target/X86/X86ISelLowering.cpp:3266:11
    #11 0x59d919a in llvm::SelectionDAGISel::LowerArguments(llvm::Function const&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:9503:26
    #12 0x5b29480 in llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1590:5
    #13 0x5b22d25 in llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:496:3
    #14 0x301ac1c in (anonymous namespace)::X86DAGToDAGISel::runOnMachineFunction(llvm::MachineFunction&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp:191:25
    #15 0x41a85aa in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/CodeGen/MachineFunctionPass.cpp:73:13
    #16 0x4c312d3 in llvm::FPPassManager::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:1648:27
    #17 0x4c31b52 in llvm::FPPassManager::runOnModule(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:1685:16
    #18 0x4c32a1c in runOnModule /b/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:1752:27
    #19 0x4c32a1c in llvm::legacy::PassManagerImpl::run(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm/lib/IR/LegacyPassManager.cpp:1865
    #20 0xa6a209 in compileModule(char**, llvm::LLVMContext&) /b/sanitizer-x86_64-linux-fast/build/llvm/tools/llc/llc.cpp:609:8
    #21 0xa63d50 in main /b/sanitizer-x86_64-linux-fast/build/llvm/tools/llc/llc.cpp:363:22
    #22 0x7f585706f2e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)

SUMMARY: AddressSanitizer: use-after-poison /b/sanitizer-x86_64-linux-fast/build/llvm/include/llvm/CodeGen/SelectionDAGNodes.h:989:5 in getValueType
Shadow bytes around the buggy address:
  0x0c427fffe430: 00 00 00 00 00 00 00 00 00 00 00 f7 00 00 00 00
  0x0c427fffe440: 00 00 00 00 00 00 00 00 00 f7 00 00 00 00 00 00
  0x0c427fffe450: 00 00 00 00 00 00 00 f7 f7 f7 f7 02 f7 f7 f7 f7
  0x0c427fffe460: f7 f7 f7 f7 f7 f7 00 00 00 00 00 00 00 00 00 00
  0x0c427fffe470: 00 00 00 f7 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c427fffe480: 00 f7 f7 f7 f7 02 f7 f7 f7[f7]f7 f7 f7 f7 f7 f7
  0x0c427fffe490: f7 f7 f7 02 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
  0x0c427fffe4a0: f7 02 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 00 00 00 00
  0x0c427fffe4b0: 00 00 00 00 00 00 00 00 00 f7 00 00 00 00 00 00
  0x0c427fffe4c0: 00 00 00 00 00 00 00 f7 00 00 00 00 00 00 00 00
  0x0c427fffe4d0: 00 00 00 00 00 f7 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==6559==ABORTING

llvm/trunk/lib/Target/X86/X86ISelDAGToDAG.cpp

static bool foldMaskAndShiftToExtract(SelectionDAG &DAG, SDValue N,
   // hierarchy left to express.
   insertDAGNode(DAG, N, Eight);
   insertDAGNode(DAG, N, Srl);
   insertDAGNode(DAG, N, NewMask);
   insertDAGNode(DAG, N, And);
   insertDAGNode(DAG, N, ShlCount);
   insertDAGNode(DAG, N, Shl);
   DAG.ReplaceAllUsesWith(N, Shl);
+  DAG.RemoveDeadNode(N.getNode());
   AM.IndexReg = And;
   AM.Scale = (1 << ScaleLog);
   return false;
 }
 
 // Transforms "(X << C1) & C2" to "(X & (C2>>C1)) << C1" if safe and if this
 // allows us to fold the shift into this addressing mode. Returns false if the
 // transform succeeded.
static bool foldMaskedShiftToScaledMask(SelectionDAG &DAG, SDValue N,
   // a valid topological ordering as nothing is going to go back and re-sort
   // these nodes. We continually insert before 'N' in sequence as this is
   // essentially a pre-flattened and pre-sorted sequence of nodes. There is no
   // hierarchy left to express.
   insertDAGNode(DAG, N, NewMask);
   insertDAGNode(DAG, N, NewAnd);
   insertDAGNode(DAG, N, NewShift);
   DAG.ReplaceAllUsesWith(N, NewShift);
+  DAG.RemoveDeadNode(N.getNode());
 
   AM.Scale = 1 << ShiftAmt;
   AM.IndexReg = NewAnd;
   return false;
 }
 
 // Implement some heroics to detect shifts of masked values where the mask can
 // be replaced by extending the shift and undoing that in the addressing mode
static bool foldMaskAndShiftToScale(SelectionDAG &DAG, SDValue N,
   // these nodes. We continually insert before 'N' in sequence as this is
   // essentially a pre-flattened and pre-sorted sequence of nodes. There is no
   // hierarchy left to express.
   insertDAGNode(DAG, N, NewSRLAmt);
   insertDAGNode(DAG, N, NewSRL);
   insertDAGNode(DAG, N, NewSHLAmt);
   insertDAGNode(DAG, N, NewSHL);
   DAG.ReplaceAllUsesWith(N, NewSHL);
+  DAG.RemoveDeadNode(N.getNode());
 
   AM.Scale = 1 << AMShiftAmt;
   AM.IndexReg = NewSRL;
   return false;
 }
 
 // Transform "(X >> SHIFT) & (MASK << C1)" to
 // "((X >> (SHIFT + C1)) & (MASK)) << C1". Everything before the SHL will be
static bool foldMaskedShiftToBEXTR(SelectionDAG &DAG, SDValue N,
   // hierarchy left to express.
   insertDAGNode(DAG, N, NewSRLAmt);
   insertDAGNode(DAG, N, NewSRL);
   insertDAGNode(DAG, N, NewMask);
   insertDAGNode(DAG, N, NewAnd);
   insertDAGNode(DAG, N, NewSHLAmt);
   insertDAGNode(DAG, N, NewSHL);
   DAG.ReplaceAllUsesWith(N, NewSHL);
+  DAG.RemoveDeadNode(N.getNode());
 
   AM.Scale = 1 << AMShiftAmt;
   AM.IndexReg = NewAnd;
   return false;
 }
 
 bool X86DAGToDAGISel::matchAddressRecursively(SDValue N, X86ISelAddressMode &AM,
                                               unsigned Depth) {
case ISD::ZERO_EXTEND: {
 
     // Convert the shift to scale factor.
     AM.Scale = 1 << ShAmtC->getZExtValue();
     AM.IndexReg = Zext;
 
     insertDAGNode(*CurDAG, N, Zext);
     insertDAGNode(*CurDAG, N, NewShl);
     CurDAG->ReplaceAllUsesWith(N, NewShl);
+    CurDAG->RemoveDeadNode(N.getNode());
     return false;
   }
   }
 
   return matchAddressBase(N, AM);
 }
 
 /// Helper for MatchAddress. Add the specified node to the

llvm/trunk/test/CodeGen/X86/fold-and-shift.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-- | FileCheck %s
 
 define i32 @t1(i8* %X, i32 %i) {
 ; CHECK-LABEL: t1:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    movzbl %cl, %ecx
+; CHECK-NEXT:    movl $255, %ecx
+; CHECK-NEXT:    andl {{[0-9]+}}(%esp), %ecx
 ; CHECK-NEXT:    movl (%eax,%ecx,4), %eax
 ; CHECK-NEXT:    retl
 
 entry:
   %tmp2 = shl i32 %i, 2
   %tmp4 = and i32 %tmp2, 1020
   %tmp7 = getelementptr i8, i8* %X, i32 %tmp4
   %tmp78 = bitcast i8* %tmp7 to i32*
   %tmp9 = load i32, i32* %tmp78
   ret i32 %tmp9
 }
 
 define i32 @t2(i16* %X, i32 %i) {
 ; CHECK-LABEL: t2:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    movzwl %cx, %ecx
+; CHECK-NEXT:    movl $65535, %ecx # imm = 0xFFFF
+; CHECK-NEXT:    andl {{[0-9]+}}(%esp), %ecx
 ; CHECK-NEXT:    movl (%eax,%ecx,4), %eax
 ; CHECK-NEXT:    retl
 
 entry:
   %tmp2 = shl i32 %i, 1
   %tmp4 = and i32 %tmp2, 131070
   %tmp7 = getelementptr i16, i16* %X, i32 %tmp4
   %tmp78 = bitcast i16* %tmp7 to i32*
entry:
   %tmp9 = load i8, i8* %tmp7
   ret i8 %tmp9
 }
 
 define i8 @t6(i8* %X, i32 %i) {
 ; CHECK-LABEL: t6:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    andl $-255, %ecx
+; CHECK-NEXT:    movl $-255, %ecx
+; CHECK-NEXT:    andl {{[0-9]+}}(%esp), %ecx
 ; CHECK-NEXT:    movb (%eax,%ecx,4), %al
 ; CHECK-NEXT:    retl
 
 entry:
   %tmp2 = shl i32 %i, 2
   %tmp4 = and i32 %tmp2, -1020
   %tmp7 = getelementptr i8, i8* %X, i32 %tmp4
   %tmp9 = load i8, i8* %tmp7
   ret i8 %tmp9
 }

llvm/trunk/test/CodeGen/X86/pr32329.ll

 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    .cfi_def_cfa_offset 16
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 20
 ; X86-NEXT:    .cfi_offset %esi, -20
 ; X86-NEXT:    .cfi_offset %edi, -16
 ; X86-NEXT:    .cfi_offset %ebx, -12
 ; X86-NEXT:    .cfi_offset %ebp, -8
-; X86-NEXT:    movl obj, %edx
 ; X86-NEXT:    movsbl var_27, %eax
 ; X86-NEXT:    movzwl var_2, %esi
 ; X86-NEXT:    movl var_310, %ecx
 ; X86-NEXT:    imull %eax, %ecx
 ; X86-NEXT:    addl var_24, %ecx
-; X86-NEXT:    andl $4194303, %edx # imm = 0x3FFFFF
-; X86-NEXT:    leal (%edx,%edx), %ebx
-; X86-NEXT:    subl %eax, %ebx
-; X86-NEXT:    movl %ebx, %edi
-; X86-NEXT:    subl %esi, %edi
-; X86-NEXT:    imull %edi, %ecx
+; X86-NEXT:    movl $4194303, %edi # imm = 0x3FFFFF
+; X86-NEXT:    andl obj, %edi
+; X86-NEXT:    leal (%edi,%edi), %edx
+; X86-NEXT:    subl %eax, %edx
+; X86-NEXT:    movl %edx, %ebx
+; X86-NEXT:    subl %esi, %ebx
+; X86-NEXT:    imull %ebx, %ecx
 ; X86-NEXT:    addb $113, %cl
 ; X86-NEXT:    movl $9, %esi
 ; X86-NEXT:    xorl %ebp, %ebp
 ; X86-NEXT:    shldl %cl, %esi, %ebp
 ; X86-NEXT:    shll %cl, %esi
 ; X86-NEXT:    testb $32, %cl
 ; X86-NEXT:    cmovnel %esi, %ebp
 ; X86-NEXT:    movl $0, %ecx
 ; X86-NEXT:    cmovnel %ecx, %esi
-; X86-NEXT:    cmpl %edx, %edi
+; X86-NEXT:    cmpl %edi, %ebx
 ; X86-NEXT:    movl %ebp, var_50+4
 ; X86-NEXT:    movl %esi, var_50
 ; X86-NEXT:    setge var_205
-; X86-NEXT:    imull %eax, %ebx
-; X86-NEXT:    movb %bl, var_218
+; X86-NEXT:    imull %eax, %edx
+; X86-NEXT:    movb %dl, var_218
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 16
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    .cfi_def_cfa_offset 12
 ; X86-NEXT:    popl %ebx
 ; X86-NEXT:    .cfi_def_cfa_offset 8
 ; X86-NEXT:    popl %ebp
 ; X86-NEXT:    .cfi_def_cfa_offset 4
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: foo:
 ; X64:       # %bb.0: # %entry
-; X64-NEXT:    movl {{.*}}(%rip), %eax
 ; X64-NEXT:    movsbl {{.*}}(%rip), %r9d
 ; X64-NEXT:    movzwl {{.*}}(%rip), %r8d
 ; X64-NEXT:    movl {{.*}}(%rip), %ecx
 ; X64-NEXT:    imull %r9d, %ecx
 ; X64-NEXT:    addl {{.*}}(%rip), %ecx
-; X64-NEXT:    andl $4194303, %eax # imm = 0x3FFFFF
-; X64-NEXT:    leal (%rax,%rax), %edi
+; X64-NEXT:    movl $4194303, %esi # imm = 0x3FFFFF
+; X64-NEXT:    andl {{.*}}(%rip), %esi
+; X64-NEXT:    leal (%rsi,%rsi), %edi
 ; X64-NEXT:    subl %r9d, %edi
-; X64-NEXT:    movl %edi, %esi
-; X64-NEXT:    subl %r8d, %esi
-; X64-NEXT:    imull %esi, %ecx
+; X64-NEXT:    movl %edi, %edx
+; X64-NEXT:    subl %r8d, %edx
+; X64-NEXT:    imull %edx, %ecx
 ; X64-NEXT:    addb $113, %cl
-; X64-NEXT:    movl $9, %edx
+; X64-NEXT:    movl $9, %eax
 ; X64-NEXT:    # kill: def $cl killed $cl killed $ecx
-; X64-NEXT:    shlq %cl, %rdx
-; X64-NEXT:    movq %rdx, {{.*}}(%rip)
-; X64-NEXT:    cmpl %eax, %esi
+; X64-NEXT:    shlq %cl, %rax
+; X64-NEXT:    movq %rax, {{.*}}(%rip)
+; X64-NEXT:    cmpl %esi, %edx
 ; X64-NEXT:    setge {{.*}}(%rip)
 ; X64-NEXT:    imull %r9d, %edi
 ; X64-NEXT:    movb %dil, {{.*}}(%rip)
 ; X64-NEXT:    retq
   entry:
   %bf.load = load i32, i32* bitcast (%struct.AA* @obj to i32*), align 8
   %bf.clear = shl i32 %bf.load, 1
   %add = and i32 %bf.clear, 8388606