RegAllocFast: Add heuristic to detect values not live-out of a block

[arsenm]

Summary

Add an improved/new heuristic to catch more cases when values are not live out of a basic block.

[MatzeB]

This is part of my rewrite regallocfast series. See also D52010

[arsenm]

I see regressions with this one after some possibly recent commits? In the first test case in test/CodeGen/AMDGPU/basic-branch.ll, it seems to not detect the use of the pointer value and omits the spill. The restore is still present

[arsenm]

lib/CodeGen/RegAllocFast.cpp

272

I think this approach doesn't work because once the virtual register is rewritten in one block, it no longer appears in the vreg's use list in the next block

[arsenm]

lib/CodeGen/RegAllocFast.cpp

106

Typo, extra c in Across

276–277

This is broken because it's skipping the first use. It can also useMRI->reg_nodbg_instructions.

This is also broken for the case where there is a phi/copy when there is a loop back to the same block. This needs to check for whether the use instruction happens before the spilled value def

[arsenm]

Fix use scan skipping first use. Assume live-out if block is its own successor.

Update tests. Not sure what's going on with test/DebugInfo/X86/pieces-1.ll, but I doubt it's testing what it's supposed to

[arsenm]

Rebase

[arsenm]

ping

[arsenm]

ping

[qcolombet]

LGTM. Nitpicks below.

lib/CodeGen/RegAllocFast.cpp

106

The typo is in the comment is still here :).

282

Just a thought, given that we already traversed all the instructions of MBB at this point, we could avoid this loop all together by checking if we saw as many uses as the total number of uses for each virtual register.
That would be valuable only if checking the total number of the uses is O(1), which I am not sure it is right now. Basically, although we have a limit of 8 in that patch, I am worried this loop might have an impact on compile time.

Bonus, if we were to do that, we would know what the status for the back-edges case.

Anyhow, nothing to do here!

1057

For clarity
/*OnlyLiveOut=*/false

1167

Ditto

[arsenm]

r359906

lib/CodeGen/RegAllocFast.cpp

private:
 
     using LiveRegMap = SparseSet<LiveReg>;
     /// This map contains entries for each virtual register that is currently
     /// available in a physical register.
     LiveRegMap LiveVirtRegs;
 
     DenseMap<unsigned, SmallVector<MachineInstr *, 2>> LiveDbgValueMap;
 
+    /// Has a bit set for every virtual register for which it was determined
+    /// that it is alive accross blocks.
+    BitVector MayLiveAcrossBlocks;
+
     /// State of a physical register.
     enum RegState {
       /// A disabled register is not available for allocation, but an alias may
       /// be in use. A register can only be moved out of the disabled state if
       /// all aliases are disabled.
       regDisabled,
 
       /// A free register is not currently in use and can be allocated
private:
     }
 
     void allocVirtReg(MachineInstr &MI, LiveReg &LR, unsigned Hint);
     void allocVirtRegUndef(MachineOperand &MO);
     MCPhysReg defineVirtReg(MachineInstr &MI, unsigned OpNum, unsigned VirtReg,
                             unsigned Hint);
     LiveReg &reloadVirtReg(MachineInstr &MI, unsigned OpNum, unsigned VirtReg,
                            unsigned Hint);
-    void spillAll(MachineBasicBlock::iterator MI);
+    void spillAll(MachineBasicBlock::iterator MI, bool OnlyLiveOut);
     bool setPhysReg(MachineInstr &MI, MachineOperand &MO, MCPhysReg PhysReg);
 
     int getStackSpaceFor(unsigned VirtReg);
     void spill(MachineBasicBlock::iterator Before, unsigned VirtReg,
                MCPhysReg AssignedReg, bool Kill);
     void reload(MachineBasicBlock::iterator Before, unsigned VirtReg,
                 MCPhysReg PhysReg);
 
+    bool mayLiveOut(unsigned VirtReg);
+
     void dumpState();
   };
 
 } // end anonymous namespace
 
 char RegAllocFast::ID = 0;
 
 INITIALIZE_PASS(RegAllocFast, "regallocfast", "Fast Register Allocator", false,
int RegAllocFast::getStackSpaceFor(unsigned VirtReg) {
   unsigned Align = TRI->getSpillAlignment(RC);
   int FrameIdx = MFI->CreateSpillStackObject(Size, Align);
 
   // Assign the slot.
   StackSlotForVirtReg[VirtReg] = FrameIdx;
   return FrameIdx;
 }
 
+/// Returns false if \p VirtReg is known to not live out of the current block.
+bool RegAllocFast::mayLiveOut(unsigned VirtReg) {
+  if (MayLiveAcrossBlocks.test(TargetRegisterInfo::virtReg2Index(VirtReg))) {
+    // Cannot be live-out if there are no successors.
+    return !MBB->succ_empty();
+  }
+
+  // If this block loops back to itself, it would be necessary to check whether
+  // the use comes after the def.
+  if (MBB->isSuccessor(MBB))
+    return true;
+
+  // See if the first \p Limit uses of the register are all in the current
+  // block.
+  static const unsigned Limit = 8;
+  unsigned C = 0;
+  for (const MachineInstr &UseInst : MRI->reg_nodbg_instructions(VirtReg)) {
+    if (UseInst.getParent() != MBB || ++C >= Limit) {
+      MayLiveAcrossBlocks.set(TargetRegisterInfo::virtReg2Index(VirtReg));
+      // Cannot be live-out if there are no successors.
+      return !MBB->succ_empty();
+    }
+  }
+
+  return false;
+}
+
 /// Insert spill instruction for \p AssignedReg before \p Before. Update
 /// DBG_VALUEs with \p VirtReg operands with the stack slot.
 void RegAllocFast::spill(MachineBasicBlock::iterator Before, unsigned VirtReg,
                          MCPhysReg AssignedReg, bool Kill) {
   LLVM_DEBUG(dbgs() << "Spilling " << printReg(VirtReg, TRI)
                     << " in " << printReg(AssignedReg, TRI));
   int FI = getStackSpaceFor(VirtReg);
   LLVM_DEBUG(dbgs() << " to stack slot #" << FI << '\n');
if (LR.Dirty) {
 
     if (SpillKill)
       LR.LastUse = nullptr; // Don't kill register again
   }
   killVirtReg(LR);
 }
 
 /// Spill all dirty virtregs without killing them.
-void RegAllocFast::spillAll(MachineBasicBlock::iterator MI) {
+void RegAllocFast::spillAll(MachineBasicBlock::iterator MI, bool OnlyLiveOut) {
   if (LiveVirtRegs.empty())
     return;
   // The LiveRegMap is keyed by an unsigned (the virtreg number), so the order
   // of spilling here is deterministic, if arbitrary.
   for (LiveReg &LR : LiveVirtRegs) {
     if (!LR.PhysReg)
       continue;
+    if (OnlyLiveOut && !mayLiveOut(LR.VirtReg))
+      continue;
     spillVirtReg(MI, LR);
   }
   LiveVirtRegs.clear();
 }
 
 /// Handle the direct use of a physical register.  Check that the register is
 /// not used by a virtreg. Kill the physreg, marking it free. This may add
 /// implicit kills to MO->getParent() and invalidate MO.
if (MI.isCall()) {
     // Spill all virtregs before a call. This serves one purpose: If an
     // exception is thrown, the landing pad is going to expect to find
     // registers in their spill slots.
     // Note: although this is appealing to just consider all definitions
     // as call-clobbered, this is not correct because some of those
     // definitions may be used later on and we do not want to reuse
     // those for virtual registers in between.
     LLVM_DEBUG(dbgs() << "  Spilling remaining registers before call.\n");
-    spillAll(MI);
+    spillAll(MI, false);
   }
 
   // Third scan.
   // Allocate defs and collect dead defs.
   for (unsigned I = 0; I != DefOpEnd; ++I) {
     const MachineOperand &MO = MI.getOperand(I);
     if (!MO.isReg() || !MO.isDef() || !MO.getReg() || MO.isEarlyClobber())
       continue;
if (MI.isDebugValue()) {
       continue;
     }
 
     allocateInstruction(MI);
   }
 
   // Spill all physical registers holding virtual registers now.
   LLVM_DEBUG(dbgs() << "Spilling live registers at end of block.\n");
-  spillAll(MBB.getFirstTerminator());
+  spillAll(MBB.getFirstTerminator(), true);
 
   // Erase all the coalesced copies. We are delaying it until now because
   // LiveVirtRegs might refer to the instrs.
   for (MachineInstr *MI : Coalesced)
     MBB.erase(MI);
   NumCoalesced += Coalesced.size();
 
   LLVM_DEBUG(MBB.dump());
bool RegAllocFast::runOnMachineFunction(MachineFunction &MF) {
   UsedInInstr.clear();
   UsedInInstr.setUniverse(TRI->getNumRegUnits());
 
   // initialize the virtual->physical register map to have a 'null'
   // mapping for all virtual registers
   unsigned NumVirtRegs = MRI->getNumVirtRegs();
   StackSlotForVirtReg.resize(NumVirtRegs);
   LiveVirtRegs.setUniverse(NumVirtRegs);
+  MayLiveAcrossBlocks.clear();
+  MayLiveAcrossBlocks.resize(NumVirtRegs);
 
   // Loop over all of the basic blocks, eliminating virtual register references
   for (MachineBasicBlock &MBB : MF)
     allocateBasicBlock(MBB);
 
   // All machine operands and other references to virtual registers have been
   // replaced. Remove the virtual registers.
   MRI->clearVirtRegs();

test/CodeGen/AArch64/swifterror.ll

 ; CHECK-O0: malloc
 ; CHECK-O0: mov [[ID:x[0-9]+]], x0
 ; CHECK-O0: strb w{{.*}}, [{{.*}}[[ID]], #8]
 ; spill x0
 ; CHECK-O0: str x0, [sp, [[SLOT2]]]
 ; CHECK-O0:[[BB2]]:
 ; CHECK-O0: ldr     x0, [sp, [[SLOT2]]]
 ; CHECK-O0: fcmp
-; CHECK-O0: str     x0, [sp, [[SLOT3:#[0-9]+]]
+; CHECK-O0: str     x0, [sp]
 ; CHECK-O0: b.le [[BB1]]
 ; reload from stack
-; CHECK-O0: ldr [[ID3:x[0-9]+]], [sp, [[SLOT3]]]
+; CHECK-O0: ldr [[ID3:x[0-9]+]], [sp]
 ; CHECK-O0: mov x21, [[ID3]]
 ; CHECK-O0: ret
 entry:
   br label %bb_loop
 
 bb_loop:
   %cond = icmp ne i32 %cc, 0
   br i1 %cond, label %gen_error, label %bb_cont

test/CodeGen/AMDGPU/indirect-addressing-term.ll

define amdgpu_kernel void @extract_w_offset_vgpr(i32 addrspace(1)* %out) {
   ; GCN:   renamable $sgpr6_sgpr7 = COPY renamable $sgpr4_sgpr5
   ; GCN:   SI_SPILL_S64_SAVE killed $sgpr6_sgpr7, %stack.5, implicit $exec, implicit $sgpr96_sgpr97_sgpr98_sgpr99, implicit $sgpr3, implicit-def dead $m0 :: (store 8 into %stack.5, align 4, addrspace 5)
   ; GCN:   SI_SPILL_V32_SAVE killed $vgpr19, %stack.4, $sgpr96_sgpr97_sgpr98_sgpr99, $sgpr3, 0, implicit $exec :: (store 4 into %stack.4, addrspace 5)
   ; GCN:   SI_SPILL_V32_SAVE killed $vgpr18, %stack.8, $sgpr96_sgpr97_sgpr98_sgpr99, $sgpr3, 0, implicit $exec :: (store 4 into %stack.8, addrspace 5)
   ; GCN:   $exec = S_XOR_B64_term $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
   ; GCN:   S_CBRANCH_EXECNZ %bb.1, implicit $exec
   ; GCN: bb.2:
   ; GCN:   $sgpr0_sgpr1 = SI_SPILL_S64_RESTORE %stack.3, implicit $exec, implicit $sgpr96_sgpr97_sgpr98_sgpr99, implicit $sgpr3, implicit-def dead $m0 :: (load 8 from %stack.3, align 4, addrspace 5)
-  ; GCN:   $exec = S_MOV_B64 killed renamable $sgpr0_sgpr1
+  ; GCN:   $exec = S_MOV_B64 renamable $sgpr0_sgpr1
   ; GCN:   $vgpr0 = SI_SPILL_V32_RESTORE %stack.8, $sgpr96_sgpr97_sgpr98_sgpr99, $sgpr3, 0, implicit $exec :: (load 4 from %stack.8, addrspace 5)
   ; GCN:   $sgpr4_sgpr5_sgpr6_sgpr7 = SI_SPILL_S128_RESTORE %stack.1, implicit $exec, implicit $sgpr96_sgpr97_sgpr98_sgpr99, implicit $sgpr3, implicit-def dead $m0 :: (load 16 from %stack.1, align 4, addrspace 5)
-  ; GCN:   BUFFER_STORE_DWORD_OFFSET killed renamable $vgpr0, killed renamable $sgpr4_sgpr5_sgpr6_sgpr7, 0, 0, 0, 0, 0, implicit $exec :: (store 4 into %ir.out.load, addrspace 1)
-  ; GCN:   S_ENDPGM
+  ; GCN:   BUFFER_STORE_DWORD_OFFSET renamable $vgpr0, renamable $sgpr4_sgpr5_sgpr6_sgpr7, 0, 0, 0, 0, 0, implicit $exec :: (store 4 into %ir.out.load, addrspace 1)
+  ; GCN:   S_ENDPGM 0
 entry:
   %id = call i32 @llvm.amdgcn.workitem.id.x() #1
   %index = add i32 %id, 1
   %value = extractelement <16 x i32> <i32 0, i32 1, i32 2, i32 3, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11, i32 12, i32 13, i32 14, i32 15, i32 16>, i32 %index
   store i32 %value, i32 addrspace(1)* %out
   ret void
 }

test/CodeGen/Mips/atomic.ll

 ; MIPS32O0-NEXT:    move $6, $2
 ; MIPS32O0-NEXT:    sc $6, 0($1)
 ; MIPS32O0-NEXT:    beqz $6, $BB7_1
 ; MIPS32O0-NEXT:    nop
 ; MIPS32O0-NEXT:  $BB7_3: # %entry
 ; MIPS32O0-NEXT:    xor $1, $5, $4
 ; MIPS32O0-NEXT:    sltiu $1, $1, 1
 ; MIPS32O0-NEXT:    move $2, $5
-; MIPS32O0-NEXT:    sw $1, 0($sp) # 4-byte Folded Spill
 ; MIPS32O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32O0-NEXT:    jr $ra
 ; MIPS32O0-NEXT:    nop
 ;
 ; MIPS32R2-LABEL: AtomicCmpSwap32:
 ; MIPS32R2:       # %bb.0: # %entry
 ; MIPS32R2-NEXT:    lui $2, %hi(_gp_disp)
 ; MIPS32R2-NEXT:    addiu $2, $2, %lo(_gp_disp)

 ; MIPS32R6-NEXT:  $BB7_3: # %entry
 ; MIPS32R6-NEXT:    jr $ra
 ; MIPS32R6-NEXT:    addiu $sp, $sp, 8
 ;
 ; MIPS32R6O0-LABEL: AtomicCmpSwap32:
 ; MIPS32R6O0:       # %bb.0: # %entry
 ; MIPS32R6O0-NEXT:    lui $2, %hi(_gp_disp)
 ; MIPS32R6O0-NEXT:    addiu $2, $2, %lo(_gp_disp)
-; MIPS32R6O0-NEXT:    addiu $sp, $sp, -16
+; MIPS32R6O0-NEXT:    addiu $sp, $sp, -8
 ; MIPS32R6O0-NEXT:    addu $1, $2, $25
 ; MIPS32R6O0-NEXT:    move $2, $5
 ; MIPS32R6O0-NEXT:    move $3, $4
-; MIPS32R6O0-NEXT:    sw $5, 12($sp)
-; MIPS32R6O0-NEXT:    lw $5, 12($sp)
+; MIPS32R6O0-NEXT:    sw $5, 4($sp)
+; MIPS32R6O0-NEXT:    lw $5, 4($sp)
 ; MIPS32R6O0-NEXT:    lw $1, %got(x)($1)
 ; MIPS32R6O0-NEXT:  $BB7_1: # %entry
 ; MIPS32R6O0-NEXT:    # =>This Inner Loop Header: Depth=1
 ; MIPS32R6O0-NEXT:    ll $6, 0($1)
 ; MIPS32R6O0-NEXT:    bnec $6, $4, $BB7_3
 ; MIPS32R6O0-NEXT:  # %bb.2: # %entry
 ; MIPS32R6O0-NEXT:    # in Loop: Header=BB7_1 Depth=1
 ; MIPS32R6O0-NEXT:    move $7, $5
 ; MIPS32R6O0-NEXT:    sc $7, 0($1)
 ; MIPS32R6O0-NEXT:    beqzc $7, $BB7_1
 ; MIPS32R6O0-NEXT:  $BB7_3: # %entry
-; MIPS32R6O0-NEXT:    sw $2, 8($sp) # 4-byte Folded Spill
+; MIPS32R6O0-NEXT:    sw $2, 0($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:    move $2, $6
-; MIPS32R6O0-NEXT:    sw $3, 4($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    addiu $sp, $sp, 16
+; MIPS32R6O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32R6O0-NEXT:    jrc $ra
 ;
 ; MIPS4-LABEL: AtomicCmpSwap32:
 ; MIPS4:       # %bb.0: # %entry
 ; MIPS4-NEXT:    daddiu $sp, $sp, -16
 ; MIPS4-NEXT:    lui $1, %hi(%neg(%gp_rel(AtomicCmpSwap32)))
 ; MIPS4-NEXT:    daddu $1, $1, $25
 ; MIPS4-NEXT:    daddiu $1, $1, %lo(%neg(%gp_rel(AtomicCmpSwap32)))

 ; MIPS32R6O0-NEXT:    or $10, $10, $9
 ; MIPS32R6O0-NEXT:    sc $10, 0($3)
 ; MIPS32R6O0-NEXT:    beqzc $10, $BB8_1
 ; MIPS32R6O0-NEXT:  # %bb.2: # %entry
 ; MIPS32R6O0-NEXT:    and $7, $8, $5
 ; MIPS32R6O0-NEXT:    srlv $7, $7, $1
 ; MIPS32R6O0-NEXT:    seb $7, $7
 ; MIPS32R6O0-NEXT:  # %bb.3: # %entry
-; MIPS32R6O0-NEXT:    sw $2, 4($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $7, 0($sp) # 4-byte Folded Spill
+; MIPS32R6O0-NEXT:    sw $7, 4($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:  # %bb.4: # %entry
-; MIPS32R6O0-NEXT:    lw $1, 0($sp) # 4-byte Folded Reload
+; MIPS32R6O0-NEXT:    lw $1, 4($sp) # 4-byte Folded Reload
 ; MIPS32R6O0-NEXT:    seb $2, $1
 ; MIPS32R6O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32R6O0-NEXT:    jrc $ra
 ;
 ; MIPS4-LABEL: AtomicLoadAdd8:
 ; MIPS4:       # %bb.0: # %entry
 ; MIPS4-NEXT:    lui $1, %hi(%neg(%gp_rel(AtomicLoadAdd8)))
 ; MIPS4-NEXT:    daddu $1, $1, $25

 ; MIPS32R6O0-NEXT:    or $10, $10, $9
 ; MIPS32R6O0-NEXT:    sc $10, 0($3)
 ; MIPS32R6O0-NEXT:    beqzc $10, $BB9_1
 ; MIPS32R6O0-NEXT:  # %bb.2: # %entry
 ; MIPS32R6O0-NEXT:    and $7, $8, $5
 ; MIPS32R6O0-NEXT:    srlv $7, $7, $1
 ; MIPS32R6O0-NEXT:    seb $7, $7
 ; MIPS32R6O0-NEXT:  # %bb.3: # %entry
-; MIPS32R6O0-NEXT:    sw $2, 4($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $7, 0($sp) # 4-byte Folded Spill
+; MIPS32R6O0-NEXT:    sw $7, 4($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:  # %bb.4: # %entry
-; MIPS32R6O0-NEXT:    lw $1, 0($sp) # 4-byte Folded Reload
+; MIPS32R6O0-NEXT:    lw $1, 4($sp) # 4-byte Folded Reload
 ; MIPS32R6O0-NEXT:    seb $2, $1
 ; MIPS32R6O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32R6O0-NEXT:    jrc $ra
 ;
 ; MIPS4-LABEL: AtomicLoadSub8:
 ; MIPS4:       # %bb.0: # %entry
 ; MIPS4-NEXT:    lui $1, %hi(%neg(%gp_rel(AtomicLoadSub8)))
 ; MIPS4-NEXT:    daddu $1, $1, $25

 ; MIPS32R6O0-NEXT:    or $10, $10, $9
 ; MIPS32R6O0-NEXT:    sc $10, 0($3)
 ; MIPS32R6O0-NEXT:    beqzc $10, $BB10_1
 ; MIPS32R6O0-NEXT:  # %bb.2: # %entry
 ; MIPS32R6O0-NEXT:    and $7, $8, $5
 ; MIPS32R6O0-NEXT:    srlv $7, $7, $1
 ; MIPS32R6O0-NEXT:    seb $7, $7
 ; MIPS32R6O0-NEXT:  # %bb.3: # %entry
-; MIPS32R6O0-NEXT:    sw $2, 4($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $7, 0($sp) # 4-byte Folded Spill
+; MIPS32R6O0-NEXT:    sw $7, 4($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:  # %bb.4: # %entry
-; MIPS32R6O0-NEXT:    lw $1, 0($sp) # 4-byte Folded Reload
+; MIPS32R6O0-NEXT:    lw $1, 4($sp) # 4-byte Folded Reload
 ; MIPS32R6O0-NEXT:    seb $2, $1
 ; MIPS32R6O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32R6O0-NEXT:    jrc $ra
 ;
 ; MIPS4-LABEL: AtomicLoadNand8:
 ; MIPS4:       # %bb.0: # %entry
 ; MIPS4-NEXT:    lui $1, %hi(%neg(%gp_rel(AtomicLoadNand8)))
 ; MIPS4-NEXT:    daddu $1, $1, $25

 ; MIPS32R6O0-NEXT:    or $10, $10, $9
 ; MIPS32R6O0-NEXT:    sc $10, 0($3)
 ; MIPS32R6O0-NEXT:    beqzc $10, $BB11_1
 ; MIPS32R6O0-NEXT:  # %bb.2: # %entry
 ; MIPS32R6O0-NEXT:    and $7, $8, $5
 ; MIPS32R6O0-NEXT:    srlv $7, $7, $1
 ; MIPS32R6O0-NEXT:    seb $7, $7
 ; MIPS32R6O0-NEXT:  # %bb.3: # %entry
-; MIPS32R6O0-NEXT:    sw $2, 4($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $7, 0($sp) # 4-byte Folded Spill
+; MIPS32R6O0-NEXT:    sw $7, 4($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:  # %bb.4: # %entry
-; MIPS32R6O0-NEXT:    lw $1, 0($sp) # 4-byte Folded Reload
+; MIPS32R6O0-NEXT:    lw $1, 4($sp) # 4-byte Folded Reload
 ; MIPS32R6O0-NEXT:    seb $2, $1
 ; MIPS32R6O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32R6O0-NEXT:    jrc $ra
 ;
 ; MIPS4-LABEL: AtomicSwap8:
 ; MIPS4:       # %bb.0: # %entry
 ; MIPS4-NEXT:    lui $1, %hi(%neg(%gp_rel(AtomicSwap8)))
 ; MIPS4-NEXT:    daddu $1, $1, $25

 ; MIPS32R6-NEXT:    seb $2, $2
 ; MIPS32R6-NEXT:  # %bb.4: # %entry
 ; MIPS32R6-NEXT:    jrc $ra
 ;
 ; MIPS32R6O0-LABEL: AtomicCmpSwap8:
 ; MIPS32R6O0:       # %bb.0: # %entry
 ; MIPS32R6O0-NEXT:    lui $2, %hi(_gp_disp)
 ; MIPS32R6O0-NEXT:    addiu $2, $2, %lo(_gp_disp)
-; MIPS32R6O0-NEXT:    addiu $sp, $sp, -16
+; MIPS32R6O0-NEXT:    addiu $sp, $sp, -8
 ; MIPS32R6O0-NEXT:    addu $1, $2, $25
 ; MIPS32R6O0-NEXT:    move $2, $5
 ; MIPS32R6O0-NEXT:    move $3, $4
 ; MIPS32R6O0-NEXT:    lw $1, %got(y)($1)
 ; MIPS32R6O0-NEXT:    addiu $6, $zero, -4
 ; MIPS32R6O0-NEXT:    and $6, $1, $6
 ; MIPS32R6O0-NEXT:    andi $1, $1, 3
 ; MIPS32R6O0-NEXT:    sll $1, $1, 3

 ; MIPS32R6O0-NEXT:    and $10, $10, $8
 ; MIPS32R6O0-NEXT:    or $10, $10, $5
 ; MIPS32R6O0-NEXT:    sc $10, 0($6)
 ; MIPS32R6O0-NEXT:    beqzc $10, $BB12_1
 ; MIPS32R6O0-NEXT:  $BB12_3: # %entry
 ; MIPS32R6O0-NEXT:    srlv $9, $11, $1
 ; MIPS32R6O0-NEXT:    seb $9, $9
 ; MIPS32R6O0-NEXT:  # %bb.4: # %entry
-; MIPS32R6O0-NEXT:    sw $2, 12($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $3, 8($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:    sw $9, 4($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:  # %bb.5: # %entry
 ; MIPS32R6O0-NEXT:    lw $2, 4($sp) # 4-byte Folded Reload
-; MIPS32R6O0-NEXT:    addiu $sp, $sp, 16
+; MIPS32R6O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32R6O0-NEXT:    jrc $ra
 ;
 ; MIPS4-LABEL: AtomicCmpSwap8:
 ; MIPS4:       # %bb.0: # %entry
 ; MIPS4-NEXT:    lui $1, %hi(%neg(%gp_rel(AtomicCmpSwap8)))
 ; MIPS4-NEXT:    daddu $1, $1, $25
 ; MIPS4-NEXT:    daddiu $1, $1, %lo(%neg(%gp_rel(AtomicCmpSwap8)))
 ; MIPS4-NEXT:    ld $1, %got_disp(y)($1)

 ; MIPS32R6-NEXT:    seb $1, $1
 ; MIPS32R6-NEXT:  # %bb.4: # %entry
 ; MIPS32R6-NEXT:    xor $1, $1, $5
 ; MIPS32R6-NEXT:    jr $ra
 ; MIPS32R6-NEXT:    sltiu $2, $1, 1
 ;
 ; MIPS32R6O0-LABEL: AtomicCmpSwapRes8:
 ; MIPS32R6O0:       # %bb.0: # %entry
-; MIPS32R6O0-NEXT:    addiu $sp, $sp, -24
+; MIPS32R6O0-NEXT:    addiu $sp, $sp, -8
 ; MIPS32R6O0-NEXT:    move $1, $6
 ; MIPS32R6O0-NEXT:    move $2, $5
 ; MIPS32R6O0-NEXT:    move $3, $4
 ; MIPS32R6O0-NEXT:    addiu $7, $zero, -4
 ; MIPS32R6O0-NEXT:    and $7, $4, $7
 ; MIPS32R6O0-NEXT:    andi $4, $4, 3
 ; MIPS32R6O0-NEXT:    sll $4, $4, 3
 ; MIPS32R6O0-NEXT:    ori $8, $zero, 255

 ; MIPS32R6O0-NEXT:    and $12, $12, $9
 ; MIPS32R6O0-NEXT:    or $12, $12, $6
 ; MIPS32R6O0-NEXT:    sc $12, 0($7)
 ; MIPS32R6O0-NEXT:    beqzc $12, $BB13_1
 ; MIPS32R6O0-NEXT:  $BB13_3: # %entry
 ; MIPS32R6O0-NEXT:    srlv $11, $13, $4
 ; MIPS32R6O0-NEXT:    seb $11, $11
 ; MIPS32R6O0-NEXT:  # %bb.4: # %entry
-; MIPS32R6O0-NEXT:    sw $5, 20($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $1, 16($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $2, 12($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $3, 8($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $11, 4($sp) # 4-byte Folded Spill
+; MIPS32R6O0-NEXT:    sw $5, 4($sp) # 4-byte Folded Spill
+; MIPS32R6O0-NEXT:    sw $11, 0($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:  # %bb.5: # %entry
-; MIPS32R6O0-NEXT:    lw $1, 4($sp) # 4-byte Folded Reload
-; MIPS32R6O0-NEXT:    lw $2, 20($sp) # 4-byte Folded Reload
+; MIPS32R6O0-NEXT:    lw $1, 0($sp) # 4-byte Folded Reload
+; MIPS32R6O0-NEXT:    lw $2, 4($sp) # 4-byte Folded Reload
 ; MIPS32R6O0-NEXT:    xor $1, $1, $2
 ; MIPS32R6O0-NEXT:    sltiu $2, $1, 1
-; MIPS32R6O0-NEXT:    addiu $sp, $sp, 24
+; MIPS32R6O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32R6O0-NEXT:    jrc $ra
 ;
 ; MIPS4-LABEL: AtomicCmpSwapRes8:
 ; MIPS4:       # %bb.0: # %entry
 ; MIPS4-NEXT:    daddiu $1, $zero, -4
 ; MIPS4-NEXT:    and $2, $4, $1
 ; MIPS4-NEXT:    andi $1, $4, 3
 ; MIPS4-NEXT:    sll $3, $1, 3

 ; MIPS64R6-NEXT:    seb $1, $1
 ; MIPS64R6-NEXT:  # %bb.4: # %entry
 ; MIPS64R6-NEXT:    xor $1, $1, $5
 ; MIPS64R6-NEXT:    jr $ra
 ; MIPS64R6-NEXT:    sltiu $2, $1, 1
 ;
 ; MIPS64R6O0-LABEL: AtomicCmpSwapRes8:
 ; MIPS64R6O0:       # %bb.0: # %entry
-; MIPS64R6O0-NEXT:    daddiu $sp, $sp, -32
+; MIPS64R6O0-NEXT:    daddiu $sp, $sp, -16
 ; MIPS64R6O0-NEXT:    move $1, $6
 ; MIPS64R6O0-NEXT:    move $2, $5
 ; MIPS64R6O0-NEXT:    move $3, $4
 ; MIPS64R6O0-NEXT:    daddiu $5, $zero, -4
 ; MIPS64R6O0-NEXT:    and $5, $4, $5
 ; MIPS64R6O0-NEXT:    andi $7, $4, 3
 ; MIPS64R6O0-NEXT:    xori $7, $7, 3
 ; MIPS64R6O0-NEXT:    sll $7, $7, 3

 ; MIPS64R6O0-NEXT:    and $12, $12, $9
 ; MIPS64R6O0-NEXT:    or $12, $12, $1
 ; MIPS64R6O0-NEXT:    sc $12, 0($5)
 ; MIPS64R6O0-NEXT:    beqzc $12, .LBB13_1
 ; MIPS64R6O0-NEXT:  .LBB13_3: # %entry
 ; MIPS64R6O0-NEXT:    srlv $11, $13, $7
 ; MIPS64R6O0-NEXT:    seb $11, $11
 ; MIPS64R6O0-NEXT:  # %bb.4: # %entry
-; MIPS64R6O0-NEXT:    sw $2, 28($sp) # 4-byte Folded Spill
-; MIPS64R6O0-NEXT:    sd $3, 16($sp) # 8-byte Folded Spill
-; MIPS64R6O0-NEXT:    sw $11, 12($sp) # 4-byte Folded Spill
+; MIPS64R6O0-NEXT:    sw $2, 12($sp) # 4-byte Folded Spill
+; MIPS64R6O0-NEXT:    sw $11, 8($sp) # 4-byte Folded Spill
 ; MIPS64R6O0-NEXT:  # %bb.5: # %entry
-; MIPS64R6O0-NEXT:    lw $1, 12($sp) # 4-byte Folded Reload
-; MIPS64R6O0-NEXT:    lw $2, 28($sp) # 4-byte Folded Reload
+; MIPS64R6O0-NEXT:    lw $1, 8($sp) # 4-byte Folded Reload
+; MIPS64R6O0-NEXT:    lw $2, 12($sp) # 4-byte Folded Reload
 ; MIPS64R6O0-NEXT:    xor $1, $1, $2
 ; MIPS64R6O0-NEXT:    sltiu $2, $1, 1
-; MIPS64R6O0-NEXT:    daddiu $sp, $sp, 32
+; MIPS64R6O0-NEXT:    daddiu $sp, $sp, 16
 ; MIPS64R6O0-NEXT:    jrc $ra
 ;
 ; MM32-LABEL: AtomicCmpSwapRes8:
 ; MM32:       # %bb.0: # %entry
 ; MM32-NEXT:    addiu $1, $zero, -4
 ; MM32-NEXT:    and $2, $4, $1
 ; MM32-NEXT:    andi $1, $4, 3
 ; MM32-NEXT:    sll $3, $1, 3

 ; MIPS32R6O0-NEXT:    or $10, $10, $9
 ; MIPS32R6O0-NEXT:    sc $10, 0($3)
 ; MIPS32R6O0-NEXT:    beqzc $10, $BB14_1
 ; MIPS32R6O0-NEXT:  # %bb.2: # %entry
 ; MIPS32R6O0-NEXT:    and $7, $8, $5
 ; MIPS32R6O0-NEXT:    srlv $7, $7, $1
 ; MIPS32R6O0-NEXT:    seh $7, $7
 ; MIPS32R6O0-NEXT:  # %bb.3: # %entry
-; MIPS32R6O0-NEXT:    sw $2, 4($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $7, 0($sp) # 4-byte Folded Spill
+; MIPS32R6O0-NEXT:    sw $7, 4($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:  # %bb.4: # %entry
-; MIPS32R6O0-NEXT:    lw $1, 0($sp) # 4-byte Folded Reload
+; MIPS32R6O0-NEXT:    lw $1, 4($sp) # 4-byte Folded Reload
 ; MIPS32R6O0-NEXT:    seh $2, $1
 ; MIPS32R6O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32R6O0-NEXT:    jrc $ra
 ;
 ; MIPS4-LABEL: AtomicLoadAdd16:
 ; MIPS4:       # %bb.0: # %entry
 ; MIPS4-NEXT:    lui $1, %hi(%neg(%gp_rel(AtomicLoadAdd16)))
 ; MIPS4-NEXT:    daddu $1, $1, $25

 ; MIPS32R6-NEXT:    seh $1, $1
 ; MIPS32R6-NEXT:    xor $1, $2, $1
 ; MIPS32R6-NEXT:    sltiu $3, $1, 1
 ; MIPS32R6-NEXT:    sync
 ; MIPS32R6-NEXT:    jrc $ra
 ;
 ; MIPS32R6O0-LABEL: foo:
 ; MIPS32R6O0:       # %bb.0:
-; MIPS32R6O0-NEXT:    addiu $sp, $sp, -24
-; MIPS32R6O0-NEXT:    .cfi_def_cfa_offset 24
+; MIPS32R6O0-NEXT:    addiu $sp, $sp, -8
+; MIPS32R6O0-NEXT:    .cfi_def_cfa_offset 8
 ; MIPS32R6O0-NEXT:    move $1, $7
 ; MIPS32R6O0-NEXT:    move $2, $6
 ; MIPS32R6O0-NEXT:    move $3, $5
 ; MIPS32R6O0-NEXT:    move $8, $4
 ; MIPS32R6O0-NEXT:    addu $5, $5, $6
 ; MIPS32R6O0-NEXT:    sync
 ; MIPS32R6O0-NEXT:    addiu $6, $zero, -4
 ; MIPS32R6O0-NEXT:    and $6, $4, $6

 ; MIPS32R6O0-NEXT:    and $13, $13, $10
 ; MIPS32R6O0-NEXT:    or $13, $13, $7
 ; MIPS32R6O0-NEXT:    sc $13, 0($6)
 ; MIPS32R6O0-NEXT:    beqzc $13, $BB15_1
 ; MIPS32R6O0-NEXT:  $BB15_3:
 ; MIPS32R6O0-NEXT:    srlv $12, $14, $4
 ; MIPS32R6O0-NEXT:    seh $12, $12
 ; MIPS32R6O0-NEXT:  # %bb.4:
-; MIPS32R6O0-NEXT:    sw $1, 20($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $2, 16($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $3, 12($sp) # 4-byte Folded Spill
-; MIPS32R6O0-NEXT:    sw $8, 8($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:    sw $5, 4($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:    sw $12, 0($sp) # 4-byte Folded Spill
 ; MIPS32R6O0-NEXT:  # %bb.5:
 ; MIPS32R6O0-NEXT:    lw $1, 4($sp) # 4-byte Folded Reload
 ; MIPS32R6O0-NEXT:    seh $2, $1
 ; MIPS32R6O0-NEXT:    lw $3, 0($sp) # 4-byte Folded Reload
 ; MIPS32R6O0-NEXT:    xor $2, $3, $2
 ; MIPS32R6O0-NEXT:    sltiu $3, $2, 1
 ; MIPS32R6O0-NEXT:    sync
 ; MIPS32R6O0-NEXT:    lw $2, 0($sp) # 4-byte Folded Reload
-; MIPS32R6O0-NEXT:    addiu $sp, $sp, 24
+; MIPS32R6O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32R6O0-NEXT:    jrc $ra
 ;
 ; MIPS4-LABEL: foo:
 ; MIPS4:       # %bb.0:
 ; MIPS4-NEXT:    sll $1, $6, 0
 ; MIPS4-NEXT:    sll $2, $5, 0
 ; MIPS4-NEXT:    addu $1, $2, $1
 ; MIPS4-NEXT:    sync

 ; MIPS64R6O0-NEXT:    and $12, $12, $9
 ; MIPS64R6O0-NEXT:    or $12, $12, $1
 ; MIPS64R6O0-NEXT:    sc $12, 0($6)
 ; MIPS64R6O0-NEXT:    beqzc $12, .LBB15_1
 ; MIPS64R6O0-NEXT:  .LBB15_3:
 ; MIPS64R6O0-NEXT:    srlv $11, $13, $3
 ; MIPS64R6O0-NEXT:    seh $11, $11
 ; MIPS64R6O0-NEXT:  # %bb.4:
-; MIPS64R6O0-NEXT:    sd $5, 8($sp) # 8-byte Folded Spill
-; MIPS64R6O0-NEXT:    sw $2, 4($sp) # 4-byte Folded Spill
-; MIPS64R6O0-NEXT:    sw $11, 0($sp) # 4-byte Folded Spill
+; MIPS64R6O0-NEXT:    sw $2, 12($sp) # 4-byte Folded Spill
+; MIPS64R6O0-NEXT:    sw $11, 8($sp) # 4-byte Folded Spill
 ; MIPS64R6O0-NEXT:  # %bb.5:
-; MIPS64R6O0-NEXT:    lw $1, 4($sp) # 4-byte Folded Reload
+; MIPS64R6O0-NEXT:    lw $1, 12($sp) # 4-byte Folded Reload
 ; MIPS64R6O0-NEXT:    seh $2, $1
-; MIPS64R6O0-NEXT:    lw $3, 0($sp) # 4-byte Folded Reload
+; MIPS64R6O0-NEXT:    lw $3, 8($sp) # 4-byte Folded Reload
 ; MIPS64R6O0-NEXT:    xor $2, $3, $2
 ; MIPS64R6O0-NEXT:    sltiu $3, $2, 1
 ; MIPS64R6O0-NEXT:    sync
-; MIPS64R6O0-NEXT:    lw $2, 0($sp) # 4-byte Folded Reload
+; MIPS64R6O0-NEXT:    lw $2, 8($sp) # 4-byte Folded Reload
 ; MIPS64R6O0-NEXT:    daddiu $sp, $sp, 16
 ; MIPS64R6O0-NEXT:    jrc $ra
 ;
 ; MM32-LABEL: foo:
 ; MM32:       # %bb.0:
 ; MM32-NEXT:    addu16 $3, $5, $6
 ; MM32-NEXT:    sync
 ; MM32-NEXT:    addiu $1, $zero, -4

 ; MIPS32-NEXT:    sync
 ; MIPS32-NEXT:    jr $ra
 ; MIPS32-NEXT:    nop
 ;
 ; MIPS32O0-LABEL: zeroreg:
 ; MIPS32O0:       # %bb.0: # %entry
 ; MIPS32O0-NEXT:    lui $2, %hi(_gp_disp)
 ; MIPS32O0-NEXT:    addiu $2, $2, %lo(_gp_disp)
-; MIPS32O0-NEXT:    addiu $sp, $sp, -8
 ; MIPS32O0-NEXT:    addu $1, $2, $25
 ; MIPS32O0-NEXT:    sync
 ; MIPS32O0-NEXT:    lw $1, %got(a)($1)
 ; MIPS32O0-NEXT:    addiu $2, $zero, 0
 ; MIPS32O0-NEXT:    addiu $3, $zero, 1
 ; MIPS32O0-NEXT:    move $4, $3
 ; MIPS32O0-NEXT:  $BB17_1: # %entry
 ; MIPS32O0-NEXT:    # =>This Inner Loop Header: Depth=1

 ; MIPS32O0-NEXT:  $BB17_3: # %entry
 ; MIPS32O0-NEXT:    xor $1, $5, $3
 ; MIPS32O0-NEXT:    sltiu $1, $1, 1
 ; MIPS32O0-NEXT:    sync
 ; MIPS32O0-NEXT:    addiu $2, $zero, 1
 ; MIPS32O0-NEXT:    xor $2, $5, $2
 ; MIPS32O0-NEXT:    sltiu $2, $2, 1
 ; MIPS32O0-NEXT:    andi $2, $2, 1
-; MIPS32O0-NEXT:    sw $1, 4($sp) # 4-byte Folded Spill
-; MIPS32O0-NEXT:    addiu $sp, $sp, 8
 ; MIPS32O0-NEXT:    jr $ra
 ; MIPS32O0-NEXT:    nop
 ;
 ; MIPS32R2-LABEL: zeroreg:
 ; MIPS32R2:       # %bb.0: # %entry
 ; MIPS32R2-NEXT:    lui $2, %hi(_gp_disp)
 ; MIPS32R2-NEXT:    addiu $2, $2, %lo(_gp_disp)
 ; MIPS32R2-NEXT:    addu $1, $2, $25

test/CodeGen/Mips/atomic64.ll

 ; MIPS64R6-NEXT:    beqzc $3, .LBB7_1
 ; MIPS64R6-NEXT:    nop
 ; MIPS64R6-NEXT:  .LBB7_3: # %entry
 ; MIPS64R6-NEXT:    jr $ra
 ; MIPS64R6-NEXT:    daddiu $sp, $sp, 16
 ;
 ; MIPS64R6O0-LABEL: AtomicCmpSwap64:
 ; MIPS64R6O0:       # %bb.0: # %entry
-; MIPS64R6O0-NEXT:    daddiu $sp, $sp, -32
+; MIPS64R6O0-NEXT:    daddiu $sp, $sp, -16
 ; MIPS64R6O0-NEXT:    lui $1, %hi(%neg(%gp_rel(AtomicCmpSwap64)))
 ; MIPS64R6O0-NEXT:    daddu $1, $1, $25
 ; MIPS64R6O0-NEXT:    daddiu $1, $1, %lo(%neg(%gp_rel(AtomicCmpSwap64)))
 ; MIPS64R6O0-NEXT:    move $2, $5
 ; MIPS64R6O0-NEXT:    move $3, $4
-; MIPS64R6O0-NEXT:    sd $5, 24($sp)
-; MIPS64R6O0-NEXT:    ld $5, 24($sp)
+; MIPS64R6O0-NEXT:    sd $5, 8($sp)
+; MIPS64R6O0-NEXT:    ld $5, 8($sp)
 ; MIPS64R6O0-NEXT:    ld $1, %got_disp(x)($1)
 ; MIPS64R6O0-NEXT:  .LBB7_1: # %entry
 ; MIPS64R6O0-NEXT:    # =>This Inner Loop Header: Depth=1
 ; MIPS64R6O0-NEXT:    lld $6, 0($1)
 ; MIPS64R6O0-NEXT:    bnec $6, $4, .LBB7_3
 ; MIPS64R6O0-NEXT:  # %bb.2: # %entry
 ; MIPS64R6O0-NEXT:    # in Loop: Header=BB7_1 Depth=1
 ; MIPS64R6O0-NEXT:    move $7, $5
 ; MIPS64R6O0-NEXT:    scd $7, 0($1)
 ; MIPS64R6O0-NEXT:    beqzc $7, .LBB7_1
 ; MIPS64R6O0-NEXT:  .LBB7_3: # %entry
-; MIPS64R6O0-NEXT:    sd $2, 16($sp) # 8-byte Folded Spill
+; MIPS64R6O0-NEXT:    sd $2, 0($sp) # 8-byte Folded Spill
 ; MIPS64R6O0-NEXT:    move $2, $6
-; MIPS64R6O0-NEXT:    sd $3, 8($sp) # 8-byte Folded Spill
-; MIPS64R6O0-NEXT:    daddiu $sp, $sp, 32
+; MIPS64R6O0-NEXT:    daddiu $sp, $sp, 16
 ; MIPS64R6O0-NEXT:    jrc $ra
 ;
 ; O1-LABEL: AtomicCmpSwap64:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    daddiu $sp, $sp, -16
 ; O1-NEXT:    lui $1, %hi(%neg(%gp_rel(AtomicCmpSwap64)))
 ; O1-NEXT:    daddu $1, $1, $25
 ; O1-NEXT:    daddiu $1, $1, %lo(%neg(%gp_rel(AtomicCmpSwap64)))

test/CodeGen/Mips/atomicCmpSwapPW.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -O0 -mtriple=mipsel-unknown-linux-gnu -mcpu=mips32r2 -target-abi=o32 < %s -filetype=asm -o - \
 ; RUN:   | FileCheck -check-prefixes=O32 %s
 ; RUN: llc -O0 -mtriple=mips64el-unknown-linux-gnu -mcpu=mips64r2 -target-abi=n32 < %s -filetype=asm -o - \
 ; RUN:   | FileCheck  -check-prefixes=N32,ALL %s
 ; RUN: llc -O0 -mtriple=mips64el-unknown-linux-gnu -mcpu=mips64r2 -target-abi=n64 < %s -filetype=asm -o - \
 ; RUN:   | FileCheck -check-prefixes=N64 %s
 
 @sym = external global i32 *
 
 define void @foo(i32 %new, i32 %old) {
 ; O32-LABEL: foo:
 ; O32:       # %bb.0: # %entry
-; O32-NEXT:    addiu $sp, $sp, -16
-; O32-NEXT:    .cfi_def_cfa_offset 16
 ; O32-NEXT:    move $1, $5
 ; O32-NEXT:    move $2, $4
 ; O32-NEXT:    lui $3, %hi(sym)
 ; O32-NEXT:    lw $3, %lo(sym)($3)
 ; O32-NEXT:    sync
 ; O32-NEXT:  $BB0_1: # %entry
 ; O32-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O32-NEXT:    ll $6, 0($3)
 ; O32-NEXT:    bne $6, $4, $BB0_3
 ; O32-NEXT:    nop
 ; O32-NEXT:  # %bb.2: # %entry
 ; O32-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; O32-NEXT:    move $7, $5
 ; O32-NEXT:    sc $7, 0($3)
 ; O32-NEXT:    beqz $7, $BB0_1
 ; O32-NEXT:    nop
 ; O32-NEXT:  $BB0_3: # %entry
 ; O32-NEXT:    sync
-; O32-NEXT:    sw $1, 12($sp) # 4-byte Folded Spill
-; O32-NEXT:    sw $2, 8($sp) # 4-byte Folded Spill
-; O32-NEXT:    sw $6, 4($sp) # 4-byte Folded Spill
-; O32-NEXT:    addiu $sp, $sp, 16
 ; O32-NEXT:    jr $ra
 ; O32-NEXT:    nop
 ;
 ; N32-LABEL: foo:
 ; N32:       # %bb.0: # %entry
-; N32-NEXT:    addiu $sp, $sp, -16
-; N32-NEXT:    .cfi_def_cfa_offset 16
 ; N32-NEXT:    move $1, $5
 ; N32-NEXT:    sll $1, $1, 0
 ; N32-NEXT:    move $2, $4
 ; N32-NEXT:    sll $2, $2, 0
 ; N32-NEXT:    lui $3, %hi(sym)
 ; N32-NEXT:    lw $3, %lo(sym)($3)
 ; N32-NEXT:    sync
 ; N32-NEXT:  .LBB0_1: # %entry
 ; N32-NEXT:    # =>This Inner Loop Header: Depth=1
 ; N32-NEXT:    ll $6, 0($3)
 ; N32-NEXT:    bne $6, $2, .LBB0_3
 ; N32-NEXT:    nop
 ; N32-NEXT:  # %bb.2: # %entry
 ; N32-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; N32-NEXT:    move $7, $1
 ; N32-NEXT:    sc $7, 0($3)
 ; N32-NEXT:    beqz $7, .LBB0_1
 ; N32-NEXT:    nop
 ; N32-NEXT:  .LBB0_3: # %entry
 ; N32-NEXT:    sync
-; N32-NEXT:    sw $6, 12($sp) # 4-byte Folded Spill
-; N32-NEXT:    addiu $sp, $sp, 16
 ; N32-NEXT:    jr $ra
 ; N32-NEXT:    nop
 ;
 ; N64-LABEL: foo:
 ; N64:       # %bb.0: # %entry
-; N64-NEXT:    daddiu $sp, $sp, -16
-; N64-NEXT:    .cfi_def_cfa_offset 16
 ; N64-NEXT:    move $1, $5
 ; N64-NEXT:    sll $1, $1, 0
 ; N64-NEXT:    move $2, $4
 ; N64-NEXT:    sll $2, $2, 0
 ; N64-NEXT:    lui $3, %highest(sym)
 ; N64-NEXT:    daddiu $3, $3, %higher(sym)
 ; N64-NEXT:    dsll $3, $3, 16
 ; N64-NEXT:    daddiu $3, $3, %hi(sym)
 ; N64-NEXT:    dsll $3, $3, 16
 ; N64-NEXT:    ld $3, %lo(sym)($3)
 ; N64-NEXT:    sync
 ; N64-NEXT:  .LBB0_1: # %entry
 ; N64-NEXT:    # =>This Inner Loop Header: Depth=1
 ; N64-NEXT:    ll $6, 0($3)
 ; N64-NEXT:    bne $6, $2, .LBB0_3
 ; N64-NEXT:    nop
 ; N64-NEXT:  # %bb.2: # %entry
 ; N64-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; N64-NEXT:    move $7, $1
 ; N64-NEXT:    sc $7, 0($3)
 ; N64-NEXT:    beqz $7, .LBB0_1
 ; N64-NEXT:    nop
 ; N64-NEXT:  .LBB0_3: # %entry
 ; N64-NEXT:    sync
-; N64-NEXT:    sw $6, 12($sp) # 4-byte Folded Spill
-; N64-NEXT:    daddiu $sp, $sp, 16
 ; N64-NEXT:    jr $ra
 ; N64-NEXT:    nop
 entry:
   %0 = load i32 *, i32 ** @sym
   cmpxchg i32 * %0, i32 %new, i32 %old seq_cst seq_cst
   ret void
 }
 

test/CodeGen/Mips/dsp-spill-reload.ll

 define <4 x i8>  @spill_reload(<4 x i8> %a, <4 x i8> %b, i32 %g) {
 entry:
   %c = tail call <4 x i8> @llvm.mips.addu.qb(<4 x i8> %a, <4 x i8> %b)
   %cond = icmp eq i32 %g, 0
   br i1 %cond, label %true, label %end
 
 ; ASM: SWDSP
 ; ASM: SWDSP
-; ASM: SWDSP
 
 ; MM-OBJ:   sw  ${{[0-9]+}}, {{[0-9]+}}($sp)
 ; MM-OBJ:   sw  ${{[0-9]+}}, {{[0-9]+}}($sp)
 ; MM-OBJ:   sw  ${{[0-9]+}}, {{[0-9]+}}($sp)
-; MM-OBJ:   sw  ${{[0-9]+}}, {{[0-9]+}}($sp)
 
 true:
   ret <4 x i8> %c
 
 ; ASM: LWDSP
 
 ; MM-OBJ: lw ${{[0-9]+}}, {{[0-9]+}}($sp)
 

test/CodeGen/PowerPC/addegluecrash.ll

 ; CHECK-NEXT:    add 4, 5, 4
 ; CHECK-NEXT:    cmpld 7, 4, 5
 ; CHECK-NEXT:    mfocrf 10, 1
 ; CHECK-NEXT:    rlwinm 10, 10, 29, 31, 31
 ; CHECK-NEXT:    # implicit-def: $x4
 ; CHECK-NEXT:    mr 4, 10
 ; CHECK-NEXT:    clrldi 4, 4, 32
 ; CHECK-NEXT:    std 4, 0(3)
-; CHECK-NEXT:    std 6, -8(1) # 8-byte Folded Spill
 ; CHECK-NEXT:    blr
   %1 = load i64, i64* %a, align 8
   %conv = zext i64 %1 to i128
   %2 = load i64, i64* %b, align 8
   %conv2 = zext i64 %2 to i128
   %mul = mul nuw i128 %conv2, %conv
   %shr = lshr i128 %mul, 64
   %agep = getelementptr inbounds i64, i64* %a, i64 1

test/CodeGen/PowerPC/vsx.ll

 ; CHECK-REG:       # %bb.0: # %entry
 ; CHECK-REG-NEXT:    xxlnor v2, v2, v3
 ; CHECK-REG-NEXT:    blr
 ;
 ; CHECK-FISL-LABEL: test14:
 ; CHECK-FISL:       # %bb.0: # %entry
 ; CHECK-FISL-NEXT:    xxlor vs0, v2, v3
 ; CHECK-FISL-NEXT:    xxlnor v2, v2, v3
-; CHECK-FISL-NEXT:    li r3, -16
-; CHECK-FISL-NEXT:    stxvd2x vs0, r1, r3 # 16-byte Folded Spill
 ; CHECK-FISL-NEXT:    blr
 ;
 ; CHECK-LE-LABEL: test14:
 ; CHECK-LE:       # %bb.0: # %entry
 ; CHECK-LE-NEXT:    xxlnor v2, v2, v3
 ; CHECK-LE-NEXT:    blr
 entry:
   %v = or <4 x i32> %a, %b

 ; CHECK-REG-NEXT:    blr
 ;
 ; CHECK-FISL-LABEL: test15:
 ; CHECK-FISL:       # %bb.0: # %entry
 ; CHECK-FISL-NEXT:    xxlor vs0, v2, v3
 ; CHECK-FISL-NEXT:    xxlor v4, vs0, vs0
 ; CHECK-FISL-NEXT:    xxlnor vs0, v2, v3
 ; CHECK-FISL-NEXT:    xxlor v2, vs0, vs0
-; CHECK-FISL-NEXT:    li r3, -16
-; CHECK-FISL-NEXT:    stxvd2x v4, r1, r3 # 16-byte Folded Spill
 ; CHECK-FISL-NEXT:    blr
 ;
 ; CHECK-LE-LABEL: test15:
 ; CHECK-LE:       # %bb.0: # %entry
 ; CHECK-LE-NEXT:    xxlnor v2, v2, v3
 ; CHECK-LE-NEXT:    blr
 entry:
   %v = or <8 x i16> %a, %b

 ; CHECK-REG-NEXT:    blr
 ;
 ; CHECK-FISL-LABEL: test16:
 ; CHECK-FISL:       # %bb.0: # %entry
 ; CHECK-FISL-NEXT:    xxlor vs0, v2, v3
 ; CHECK-FISL-NEXT:    xxlor v4, vs0, vs0
 ; CHECK-FISL-NEXT:    xxlnor vs0, v2, v3
 ; CHECK-FISL-NEXT:    xxlor v2, vs0, vs0
-; CHECK-FISL-NEXT:    li r3, -16
-; CHECK-FISL-NEXT:    stxvd2x v4, r1, r3 # 16-byte Folded Spill
 ; CHECK-FISL-NEXT:    blr
 ;
 ; CHECK-LE-LABEL: test16:
 ; CHECK-LE:       # %bb.0: # %entry
 ; CHECK-LE-NEXT:    xxlnor v2, v2, v3
 ; CHECK-LE-NEXT:    blr
 entry:
   %v = or <16 x i8> %a, %b

 ; CHECK-REG-NEXT:    blr
 ;
 ; CHECK-FISL-LABEL: test18:
 ; CHECK-FISL:       # %bb.0: # %entry
 ; CHECK-FISL-NEXT:    xxlnor vs0, v3, v3
 ; CHECK-FISL-NEXT:    xxlor v4, vs0, vs0
 ; CHECK-FISL-NEXT:    xxlandc vs0, v2, v3
 ; CHECK-FISL-NEXT:    xxlor v2, vs0, vs0
-; CHECK-FISL-NEXT:    li r3, -16
-; CHECK-FISL-NEXT:    stxvd2x v4, r1, r3 # 16-byte Folded Spill
 ; CHECK-FISL-NEXT:    blr
 ;
 ; CHECK-LE-LABEL: test18:
 ; CHECK-LE:       # %bb.0: # %entry
 ; CHECK-LE-NEXT:    xxlandc v2, v2, v3
 ; CHECK-LE-NEXT:    blr
 entry:
   %w = xor <8 x i16> %b, <i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1>

 ; CHECK-REG-NEXT:    blr
 ;
 ; CHECK-FISL-LABEL: test19:
 ; CHECK-FISL:       # %bb.0: # %entry
 ; CHECK-FISL-NEXT:    xxlnor vs0, v3, v3
 ; CHECK-FISL-NEXT:    xxlor v4, vs0, vs0
 ; CHECK-FISL-NEXT:    xxlandc vs0, v2, v3
 ; CHECK-FISL-NEXT:    xxlor v2, vs0, vs0
-; CHECK-FISL-NEXT:    li r3, -16
-; CHECK-FISL-NEXT:    stxvd2x v4, r1, r3 # 16-byte Folded Spill
 ; CHECK-FISL-NEXT:    blr
 ;
 ; CHECK-LE-LABEL: test19:
 ; CHECK-LE:       # %bb.0: # %entry
 ; CHECK-LE-NEXT:    xxlandc v2, v2, v3
 ; CHECK-LE-NEXT:    blr
 entry:
   %w = xor <16 x i8> %b, <i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1>

 ; CHECK-REG-LABEL: test51:
 ; CHECK-REG:       # %bb.0:
 ; CHECK-REG-NEXT:    xxspltd v2, v2, 0
 ; CHECK-REG-NEXT:    blr
 ;
 ; CHECK-FISL-LABEL: test51:
 ; CHECK-FISL:       # %bb.0:
 ; CHECK-FISL-NEXT:    xxspltd v2, v2, 0
-; CHECK-FISL-NEXT:    li r3, -16
-; CHECK-FISL-NEXT:    stxvd2x v3, r1, r3 # 16-byte Folded Spill
 ; CHECK-FISL-NEXT:    blr
 ;
 ; CHECK-LE-LABEL: test51:
 ; CHECK-LE:       # %bb.0:
 ; CHECK-LE-NEXT:    xxspltd v2, v2, 1
 ; CHECK-LE-NEXT:    blr
   %v = shufflevector <2 x double> %a, <2 x double> %b, <2 x i32> <i32 0, i32 0>
   ret <2 x double> %v

test/CodeGen/Thumb2/high-reg-spill.mir

bb.0.entry:
     tBX_RET 14, $noreg
 
 ...
 # CHECK: bb.0.entry:
 # CHECK-NEXT: renamable $r0 = tLDRspi %stack.0.i, 0, 14, $noreg :: (dereferenceable load 4 from %ir.i)
 # CHECK-NEXT: renamable $r12 = COPY killed renamable $r0
 # CHECK-NEXT: t2STRi12 killed $r12, %stack.1, 0, 14, $noreg :: (store 4 into %stack.1)
 # CHECK-NEXT: $r8 = t2LDRi12 %stack.1, 0, 14, $noreg :: (load 4 from %stack.1)
-# CHECK-NEXT: INLINEASM &"@ $0", 1, 589833, killed renamable $r8, 12, implicit-def early-clobber $r12
+# CHECK-NEXT: INLINEASM &"@ $0", 1, 589833, renamable $r8, 12, implicit-def early-clobber $r12
 # CHECK-NEXT: tBX_RET 14, $noreg

test/CodeGen/X86/atomic-unordered.ll

 }
 
 ; Legal if wider type is also atomic (TODO)
 define void @widen_zero_init(i32* %p0, i32 %v1, i32 %v2) {
 ; CHECK-O0-LABEL: widen_zero_init:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movl $0, (%rdi)
 ; CHECK-O0-NEXT:    movl $0, 4(%rdi)
-; CHECK-O0-NEXT:    movl %esi, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
-; CHECK-O0-NEXT:    movl %edx, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: widen_zero_init:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movl $0, (%rdi)
 ; CHECK-O3-NEXT:    movl $0, 4(%rdi)
 ; CHECK-O3-NEXT:    retq
   %p1 = getelementptr i32, i32* %p0, i64 1
   store atomic i32 0, i32* %p0 unordered, align 8
   store atomic i32 0, i32* %p1 unordered, align 4
   ret void
 }
 
 ; Not legal to widen due to alignment restriction
 define void @widen_zero_init_unaligned(i32* %p0, i32 %v1, i32 %v2) {
 ; CHECK-O0-LABEL: widen_zero_init_unaligned:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movl $0, (%rdi)
 ; CHECK-O0-NEXT:    movl $0, 4(%rdi)
-; CHECK-O0-NEXT:    movl %esi, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
-; CHECK-O0-NEXT:    movl %edx, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: widen_zero_init_unaligned:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movl $0, (%rdi)
 ; CHECK-O3-NEXT:    movl $0, 4(%rdi)
 ; CHECK-O3-NEXT:    retq
   %p1 = getelementptr i32, i32* %p0, i64 1

 
 ; Legal, as expected
 define void @rmw_fold_add1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_add1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    addq $15, %rax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_add1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    addq $15, (%rdi)
 ; CHECK-O3-NEXT:    retq
   %prev = load atomic i64, i64* %p unordered, align 8
   %val = add i64 %prev, 15

 
 ; Legal, as expected
 define void @rmw_fold_sub1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_sub1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    addq $-15, %rax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_sub1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    addq $-15, (%rdi)
 ; CHECK-O3-NEXT:    retq
   %prev = load atomic i64, i64* %p unordered, align 8
   %val = sub i64 %prev, 15

 ; Legal, as expected
 define void @rmw_fold_mul1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_mul1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    leaq (%rax,%rax,4), %rax
 ; CHECK-O0-NEXT:    leaq (%rax,%rax,2), %rax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_mul1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movq (%rdi), %rax
 ; CHECK-O3-NEXT:    leaq (%rax,%rax,4), %rax
 ; CHECK-O3-NEXT:    leaq (%rax,%rax,2), %rax
 ; CHECK-O3-NEXT:    movq %rax, (%rdi)

 ; CHECK-O0-NEXT:    imulq %rcx
 ; CHECK-O0-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rax # 8-byte Reload
 ; CHECK-O0-NEXT:    addq %rax, %rdx
 ; CHECK-O0-NEXT:    movq %rdx, %rcx
 ; CHECK-O0-NEXT:    shrq $63, %rcx
 ; CHECK-O0-NEXT:    sarq $3, %rdx
 ; CHECK-O0-NEXT:    addq %rcx, %rdx
 ; CHECK-O0-NEXT:    movq %rdx, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_sdiv1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movq (%rdi), %rcx
 ; CHECK-O3-NEXT:    movabsq $-8608480567731124087, %rdx # imm = 0x8888888888888889
 ; CHECK-O3-NEXT:    movq %rcx, %rax
 ; CHECK-O3-NEXT:    imulq %rdx

 define void @rmw_fold_udiv1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_udiv1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    movabsq $-8608480567731124087, %rcx # imm = 0x8888888888888889
 ; CHECK-O0-NEXT:    mulq %rcx
 ; CHECK-O0-NEXT:    shrq $3, %rdx
 ; CHECK-O0-NEXT:    movq %rdx, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_udiv1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movq (%rdi), %rax
 ; CHECK-O3-NEXT:    movabsq $-8608480567731124087, %rcx # imm = 0x8888888888888889
 ; CHECK-O3-NEXT:    mulq %rcx
 ; CHECK-O3-NEXT:    shrq $3, %rdx

 ; CHECK-O0-NEXT:    movq %rdx, %rcx
 ; CHECK-O0-NEXT:    shrq $63, %rcx
 ; CHECK-O0-NEXT:    sarq $3, %rdx
 ; CHECK-O0-NEXT:    addq %rcx, %rdx
 ; CHECK-O0-NEXT:    leaq (%rdx,%rdx,4), %rcx
 ; CHECK-O0-NEXT:    leaq (%rcx,%rcx,2), %rcx
 ; CHECK-O0-NEXT:    subq %rcx, %rax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_srem1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movq (%rdi), %rcx
 ; CHECK-O3-NEXT:    movabsq $-8608480567731124087, %rdx # imm = 0x8888888888888889
 ; CHECK-O3-NEXT:    movq %rcx, %rax
 ; CHECK-O3-NEXT:    imulq %rdx

 ; CHECK-O0-NEXT:    movq %rax, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    mulq %rcx
 ; CHECK-O0-NEXT:    shrq $3, %rdx
 ; CHECK-O0-NEXT:    leaq (%rdx,%rdx,4), %rax
 ; CHECK-O0-NEXT:    leaq (%rax,%rax,2), %rax
 ; CHECK-O0-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rcx # 8-byte Reload
 ; CHECK-O0-NEXT:    subq %rax, %rcx
 ; CHECK-O0-NEXT:    movq %rcx, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_urem1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movq (%rdi), %rcx
 ; CHECK-O3-NEXT:    movabsq $-8608480567731124087, %rdx # imm = 0x8888888888888889
 ; CHECK-O3-NEXT:    movq %rcx, %rax
 ; CHECK-O3-NEXT:    mulq %rdx

 
 ; Legal to fold (TODO)
 define void @rmw_fold_shl1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_shl1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    shlq $15, %rax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_shl1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movq (%rdi), %rax
 ; CHECK-O3-NEXT:    shlq $15, %rax
 ; CHECK-O3-NEXT:    movq %rax, (%rdi)
 ; CHECK-O3-NEXT:    retq

 
 ; Legal to fold (TODO)
 define void @rmw_fold_lshr1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_lshr1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    shrq $15, %rax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_lshr1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movq (%rdi), %rax
 ; CHECK-O3-NEXT:    shrq $15, %rax
 ; CHECK-O3-NEXT:    movq %rax, (%rdi)
 ; CHECK-O3-NEXT:    retq

 
 ; Legal to fold (TODO)
 define void @rmw_fold_ashr1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_ashr1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    sarq $15, %rax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_ashr1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    movq (%rdi), %rax
 ; CHECK-O3-NEXT:    sarq $15, %rax
 ; CHECK-O3-NEXT:    movq %rax, (%rdi)
 ; CHECK-O3-NEXT:    retq

 define void @rmw_fold_and1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_and1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    movl %eax, %ecx
 ; CHECK-O0-NEXT:    andl $15, %ecx
 ; CHECK-O0-NEXT:    movl %ecx, %eax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_and1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    andq $15, (%rdi)
 ; CHECK-O3-NEXT:    retq
   %prev = load atomic i64, i64* %p unordered, align 8
   %val = and i64 %prev, 15

 
 ; Legal, as expected
 define void @rmw_fold_or1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_or1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    orq $15, %rax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_or1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    orq $15, (%rdi)
 ; CHECK-O3-NEXT:    retq
   %prev = load atomic i64, i64* %p unordered, align 8
   %val = or i64 %prev, 15

 
 ; Legal, as expected
 define void @rmw_fold_xor1(i64* %p, i64 %v) {
 ; CHECK-O0-LABEL: rmw_fold_xor1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
 ; CHECK-O0-NEXT:    xorq $15, %rax
 ; CHECK-O0-NEXT:    movq %rax, (%rdi)
-; CHECK-O0-NEXT:    movq %rsi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-O0-NEXT:    retq
 ;
 ; CHECK-O3-LABEL: rmw_fold_xor1:
 ; CHECK-O3:       # %bb.0:
 ; CHECK-O3-NEXT:    xorq $15, (%rdi)
 ; CHECK-O3-NEXT:    retq
   %prev = load atomic i64, i64* %p unordered, align 8
   %val = xor i64 %prev, 15

test/CodeGen/X86/atomic32.ll

 ; X86-CMOV-NEXT:    addl $12, %esp
 ; X86-CMOV-NEXT:    popl %ebx
 ; X86-CMOV-NEXT:    retl
 ;
 ; X86-NOCMOV-LABEL: atomic_fetch_max32:
 ; X86-NOCMOV:       # %bb.0:
 ; X86-NOCMOV-NEXT:    pushl %ebx
 ; X86-NOCMOV-NEXT:    pushl %esi
-; X86-NOCMOV-NEXT:    subl $24, %esp
+; X86-NOCMOV-NEXT:    subl $20, %esp
 ; X86-NOCMOV-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NOCMOV-NEXT:    movl sc32, %ecx
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:  .LBB6_1: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # =>This Inner Loop Header: Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, %ecx
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    subl %edx, %ecx
 ; X86-NOCMOV-NEXT:    movl %eax, %esi
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NOCMOV-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    jge .LBB6_4
 ; X86-NOCMOV-NEXT:  # %bb.3: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # in Loop: Header=BB6_1 Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:  .LBB6_4: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # in Loop: Header=BB6_1 Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, (%esp) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %ecx, %eax
 ; X86-NOCMOV-NEXT:    movl (%esp), %edx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    lock cmpxchgl %edx, sc32
 ; X86-NOCMOV-NEXT:    sete %bl
 ; X86-NOCMOV-NEXT:    testb $1, %bl
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    jne .LBB6_2
 ; X86-NOCMOV-NEXT:    jmp .LBB6_1
 ; X86-NOCMOV-NEXT:  .LBB6_2: # %atomicrmw.end
-; X86-NOCMOV-NEXT:    addl $24, %esp
+; X86-NOCMOV-NEXT:    addl $20, %esp
 ; X86-NOCMOV-NEXT:    popl %esi
 ; X86-NOCMOV-NEXT:    popl %ebx
 ; X86-NOCMOV-NEXT:    retl
   %t1 = atomicrmw max  i32* @sc32, i32 %x acquire
   ret void
 }
 
 define void @atomic_fetch_min32(i32 %x) nounwind {

 ; X86-CMOV-NEXT:    addl $12, %esp
 ; X86-CMOV-NEXT:    popl %ebx
 ; X86-CMOV-NEXT:    retl
 ;
 ; X86-NOCMOV-LABEL: atomic_fetch_min32:
 ; X86-NOCMOV:       # %bb.0:
 ; X86-NOCMOV-NEXT:    pushl %ebx
 ; X86-NOCMOV-NEXT:    pushl %esi
-; X86-NOCMOV-NEXT:    subl $24, %esp
+; X86-NOCMOV-NEXT:    subl $20, %esp
 ; X86-NOCMOV-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NOCMOV-NEXT:    movl sc32, %ecx
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:  .LBB7_1: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # =>This Inner Loop Header: Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, %ecx
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    subl %edx, %ecx
 ; X86-NOCMOV-NEXT:    movl %eax, %esi
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NOCMOV-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    jle .LBB7_4
 ; X86-NOCMOV-NEXT:  # %bb.3: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # in Loop: Header=BB7_1 Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:  .LBB7_4: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # in Loop: Header=BB7_1 Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, (%esp) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %ecx, %eax
 ; X86-NOCMOV-NEXT:    movl (%esp), %edx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    lock cmpxchgl %edx, sc32
 ; X86-NOCMOV-NEXT:    sete %bl
 ; X86-NOCMOV-NEXT:    testb $1, %bl
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    jne .LBB7_2
 ; X86-NOCMOV-NEXT:    jmp .LBB7_1
 ; X86-NOCMOV-NEXT:  .LBB7_2: # %atomicrmw.end
-; X86-NOCMOV-NEXT:    addl $24, %esp
+; X86-NOCMOV-NEXT:    addl $20, %esp
 ; X86-NOCMOV-NEXT:    popl %esi
 ; X86-NOCMOV-NEXT:    popl %ebx
 ; X86-NOCMOV-NEXT:    retl
   %t1 = atomicrmw min  i32* @sc32, i32 %x acquire
   ret void
 }
 
 define void @atomic_fetch_umax32(i32 %x) nounwind {

 ; X86-CMOV-NEXT:    addl $12, %esp
 ; X86-CMOV-NEXT:    popl %ebx
 ; X86-CMOV-NEXT:    retl
 ;
 ; X86-NOCMOV-LABEL: atomic_fetch_umax32:
 ; X86-NOCMOV:       # %bb.0:
 ; X86-NOCMOV-NEXT:    pushl %ebx
 ; X86-NOCMOV-NEXT:    pushl %esi
-; X86-NOCMOV-NEXT:    subl $24, %esp
+; X86-NOCMOV-NEXT:    subl $20, %esp
 ; X86-NOCMOV-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NOCMOV-NEXT:    movl sc32, %ecx
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:  .LBB8_1: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # =>This Inner Loop Header: Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, %ecx
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    subl %edx, %ecx
 ; X86-NOCMOV-NEXT:    movl %eax, %esi
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NOCMOV-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    ja .LBB8_4
 ; X86-NOCMOV-NEXT:  # %bb.3: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # in Loop: Header=BB8_1 Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:  .LBB8_4: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # in Loop: Header=BB8_1 Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, (%esp) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %ecx, %eax
 ; X86-NOCMOV-NEXT:    movl (%esp), %edx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    lock cmpxchgl %edx, sc32
 ; X86-NOCMOV-NEXT:    sete %bl
 ; X86-NOCMOV-NEXT:    testb $1, %bl
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    jne .LBB8_2
 ; X86-NOCMOV-NEXT:    jmp .LBB8_1
 ; X86-NOCMOV-NEXT:  .LBB8_2: # %atomicrmw.end
-; X86-NOCMOV-NEXT:    addl $24, %esp
+; X86-NOCMOV-NEXT:    addl $20, %esp
 ; X86-NOCMOV-NEXT:    popl %esi
 ; X86-NOCMOV-NEXT:    popl %ebx
 ; X86-NOCMOV-NEXT:    retl
   %t1 = atomicrmw umax i32* @sc32, i32 %x acquire
   ret void
 }
 
 define void @atomic_fetch_umin32(i32 %x) nounwind {

 ; X86-CMOV-NEXT:    addl $12, %esp
 ; X86-CMOV-NEXT:    popl %ebx
 ; X86-CMOV-NEXT:    retl
 ;
 ; X86-NOCMOV-LABEL: atomic_fetch_umin32:
 ; X86-NOCMOV:       # %bb.0:
 ; X86-NOCMOV-NEXT:    pushl %ebx
 ; X86-NOCMOV-NEXT:    pushl %esi
-; X86-NOCMOV-NEXT:    subl $24, %esp
+; X86-NOCMOV-NEXT:    subl $20, %esp
 ; X86-NOCMOV-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NOCMOV-NEXT:    movl sc32, %ecx
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:  .LBB9_1: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # =>This Inner Loop Header: Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, %ecx
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    subl %edx, %ecx
 ; X86-NOCMOV-NEXT:    movl %eax, %esi
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NOCMOV-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    jbe .LBB9_4
 ; X86-NOCMOV-NEXT:  # %bb.3: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # in Loop: Header=BB9_1 Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:  .LBB9_4: # %atomicrmw.start
 ; X86-NOCMOV-NEXT:    # in Loop: Header=BB9_1 Depth=1
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    movl %eax, (%esp) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    movl %ecx, %eax
 ; X86-NOCMOV-NEXT:    movl (%esp), %edx # 4-byte Reload
 ; X86-NOCMOV-NEXT:    lock cmpxchgl %edx, sc32
 ; X86-NOCMOV-NEXT:    sete %bl
 ; X86-NOCMOV-NEXT:    testb $1, %bl
 ; X86-NOCMOV-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NOCMOV-NEXT:    jne .LBB9_2
 ; X86-NOCMOV-NEXT:    jmp .LBB9_1
 ; X86-NOCMOV-NEXT:  .LBB9_2: # %atomicrmw.end
-; X86-NOCMOV-NEXT:    addl $24, %esp
+; X86-NOCMOV-NEXT:    addl $20, %esp
 ; X86-NOCMOV-NEXT:    popl %esi
 ; X86-NOCMOV-NEXT:    popl %ebx
 ; X86-NOCMOV-NEXT:    retl
   %t1 = atomicrmw umin i32* @sc32, i32 %x acquire
   ret void
 }
 
 define void @atomic_fetch_cmpxchg32() nounwind {
 ; X64-LABEL: atomic_fetch_cmpxchg32:
 ; X64:       # %bb.0:
 ; X64-NEXT:    xorl %eax, %eax
 ; X64-NEXT:    movl $1, %ecx
 ; X64-NEXT:    lock cmpxchgl %ecx, {{.*}}(%rip)
-; X64-NEXT:    movl %eax, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: atomic_fetch_cmpxchg32:
 ; X86:       # %bb.0:
-; X86-NEXT:    pushl %eax
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    movl $1, %ecx
 ; X86-NEXT:    lock cmpxchgl %ecx, sc32
-; X86-NEXT:    movl %eax, (%esp) # 4-byte Spill
-; X86-NEXT:    popl %eax
 ; X86-NEXT:    retl
   %t1 = cmpxchg i32* @sc32, i32 0, i32 1 acquire acquire
   ret void
 }
 
 define void @atomic_fetch_store32(i32 %x) nounwind {
 ; X64-LABEL: atomic_fetch_store32:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, {{.*}}(%rip)
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: atomic_fetch_store32:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %eax, sc32
 ; X86-NEXT:    retl
   store atomic i32 %x, i32* @sc32 release, align 4
   ret void
 }
 
 define void @atomic_fetch_swap32(i32 %x) nounwind {
 ; X64-LABEL: atomic_fetch_swap32:
 ; X64:       # %bb.0:
 ; X64-NEXT:    xchgl %edi, {{.*}}(%rip)
-; X64-NEXT:    movl %edi, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: atomic_fetch_swap32:
 ; X86:       # %bb.0:
-; X86-NEXT:    pushl %eax
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    xchgl %eax, sc32
-; X86-NEXT:    movl %eax, (%esp) # 4-byte Spill
-; X86-NEXT:    popl %eax
 ; X86-NEXT:    retl
   %t1 = atomicrmw xchg i32* @sc32, i32 %x acquire
   ret void
 }
 
 define void @atomic_fetch_swapf32(float %x) nounwind {
 ; X64-LABEL: atomic_fetch_swapf32:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movd %xmm0, %eax
 ; X64-NEXT:    xchgl %eax, {{.*}}(%rip)
-; X64-NEXT:    movl %eax, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
 ; X64-NEXT:    retq
 ;
 ; X86-CMOV-LABEL: atomic_fetch_swapf32:
 ; X86-CMOV:       # %bb.0:
-; X86-CMOV-NEXT:    pushl %eax
 ; X86-CMOV-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
 ; X86-CMOV-NEXT:    movd %xmm0, %eax
 ; X86-CMOV-NEXT:    xchgl %eax, fsc32
-; X86-CMOV-NEXT:    movl %eax, (%esp) # 4-byte Spill
-; X86-CMOV-NEXT:    popl %eax
 ; X86-CMOV-NEXT:    retl
 ;
 ; X86-NOCMOV-LABEL: atomic_fetch_swapf32:
 ; X86-NOCMOV:       # %bb.0:
-; X86-NOCMOV-NEXT:    subl $8, %esp
+; X86-NOCMOV-NEXT:    pushl %eax
 ; X86-NOCMOV-NEXT:    flds {{[0-9]+}}(%esp)
-; X86-NOCMOV-NEXT:    fstps {{[0-9]+}}(%esp)
-; X86-NOCMOV-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NOCMOV-NEXT:    fstps (%esp)
+; X86-NOCMOV-NEXT:    movl (%esp), %eax
 ; X86-NOCMOV-NEXT:    xchgl %eax, fsc32
-; X86-NOCMOV-NEXT:    movl %eax, (%esp) # 4-byte Spill
-; X86-NOCMOV-NEXT:    addl $8, %esp
+; X86-NOCMOV-NEXT:    popl %eax
 ; X86-NOCMOV-NEXT:    retl
   %t1 = atomicrmw xchg float* @fsc32, float %x acquire
   ret void
 }

test/CodeGen/X86/atomic64.ll

 ; X64-NEXT:    movl $5, %eax
 ; X64-NEXT:    lock xaddq %rax, {{.*}}(%rip)
 ; X64-NEXT:    lock addq %rax, {{.*}}(%rip)
 ; X64-NEXT:    retq
 ;
 ; I486-LABEL: atomic_fetch_add64:
 ; I486:       # %bb.0: # %entry
 ; I486-NEXT:    pushl %esi
-; I486-NEXT:    subl $56, %esp
+; I486-NEXT:    subl $48, %esp
 ; I486-NEXT:    leal sc64, %eax
 ; I486-NEXT:    movl %esp, %ecx
 ; I486-NEXT:    movl $2, 12(%ecx)
 ; I486-NEXT:    movl $0, 8(%ecx)
 ; I486-NEXT:    movl $1, 4(%ecx)
 ; I486-NEXT:    movl $sc64, (%ecx)
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_add_8

 ; I486-NEXT:    leal sc64, %ecx
 ; I486-NEXT:    movl %esp, %esi
 ; I486-NEXT:    movl %edx, 8(%esi)
 ; I486-NEXT:    movl %eax, 4(%esi)
 ; I486-NEXT:    movl $2, 12(%esi)
 ; I486-NEXT:    movl $sc64, (%esi)
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_add_8
-; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    addl $56, %esp
+; I486-NEXT:    addl $48, %esp
 ; I486-NEXT:    popl %esi
 ; I486-NEXT:    retl
 entry:
   %t1 = atomicrmw add  i64* @sc64, i64 1 acquire
   %t2 = atomicrmw add  i64* @sc64, i64 3 acquire
   %t3 = atomicrmw add  i64* @sc64, i64 5 acquire
   %t4 = atomicrmw add  i64* @sc64, i64 %t3 acquire
   ret void
 }
 
 define void @atomic_fetch_sub64() nounwind {
 ; X64-LABEL: atomic_fetch_sub64:
 ; X64:       # %bb.0:
 ; X64-NEXT:    lock decq {{.*}}(%rip)
 ; X64-NEXT:    lock subq $3, {{.*}}(%rip)
 ; X64-NEXT:    movq $-5, %rax
 ; X64-NEXT:    lock xaddq %rax, {{.*}}(%rip)
 ; X64-NEXT:    lock subq %rax, {{.*}}(%rip)
 ; X64-NEXT:    retq
 ;
 ; I486-LABEL: atomic_fetch_sub64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %esi
-; I486-NEXT:    subl $56, %esp
+; I486-NEXT:    subl $48, %esp
 ; I486-NEXT:    leal sc64, %eax
 ; I486-NEXT:    movl %esp, %ecx
 ; I486-NEXT:    movl $2, 12(%ecx)
 ; I486-NEXT:    movl $0, 8(%ecx)
 ; I486-NEXT:    movl $1, 4(%ecx)
 ; I486-NEXT:    movl $sc64, (%ecx)
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_sub_8

 ; I486-NEXT:    leal sc64, %ecx
 ; I486-NEXT:    movl %esp, %esi
 ; I486-NEXT:    movl %edx, 8(%esi)
 ; I486-NEXT:    movl %eax, 4(%esi)
 ; I486-NEXT:    movl $2, 12(%esi)
 ; I486-NEXT:    movl $sc64, (%esi)
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_sub_8
-; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    addl $56, %esp
+; I486-NEXT:    addl $48, %esp
 ; I486-NEXT:    popl %esi
 ; I486-NEXT:    retl
   %t1 = atomicrmw sub  i64* @sc64, i64 1 acquire
   %t2 = atomicrmw sub  i64* @sc64, i64 3 acquire
   %t3 = atomicrmw sub  i64* @sc64, i64 5 acquire
   %t4 = atomicrmw sub  i64* @sc64, i64 %t3 acquire
   ret void
 }

 ; X64-NEXT:  .LBB2_2: # %atomicrmw.end
 ; X64-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rax # 8-byte Reload
 ; X64-NEXT:    lock andq %rax, {{.*}}(%rip)
 ; X64-NEXT:    retq
 ;
 ; I486-LABEL: atomic_fetch_and64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %esi
-; I486-NEXT:    subl $44, %esp
+; I486-NEXT:    subl $36, %esp
 ; I486-NEXT:    leal sc64, %eax
 ; I486-NEXT:    movl %esp, %ecx
 ; I486-NEXT:    movl $2, 12(%ecx)
 ; I486-NEXT:    movl $0, 8(%ecx)
 ; I486-NEXT:    movl $3, 4(%ecx)
 ; I486-NEXT:    movl $sc64, (%ecx)
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_and_8

 ; I486-NEXT:    leal sc64, %ecx
 ; I486-NEXT:    movl %esp, %esi
 ; I486-NEXT:    movl %edx, 8(%esi)
 ; I486-NEXT:    movl %eax, 4(%esi)
 ; I486-NEXT:    movl $2, 12(%esi)
 ; I486-NEXT:    movl $sc64, (%esi)
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_and_8
-; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    addl $44, %esp
+; I486-NEXT:    addl $36, %esp
 ; I486-NEXT:    popl %esi
 ; I486-NEXT:    retl
   %t1 = atomicrmw and  i64* @sc64, i64 3 acquire
   %t2 = atomicrmw and  i64* @sc64, i64 5 acquire
   %t3 = atomicrmw and  i64* @sc64, i64 %t2 acquire
   ret void
 }
 

 ; X64-NEXT:  .LBB3_2: # %atomicrmw.end
 ; X64-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rax # 8-byte Reload
 ; X64-NEXT:    lock orq %rax, {{.*}}(%rip)
 ; X64-NEXT:    retq
 ;
 ; I486-LABEL: atomic_fetch_or64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %esi
-; I486-NEXT:    subl $44, %esp
+; I486-NEXT:    subl $36, %esp
 ; I486-NEXT:    leal sc64, %eax
 ; I486-NEXT:    movl %esp, %ecx
 ; I486-NEXT:    movl $2, 12(%ecx)
 ; I486-NEXT:    movl $0, 8(%ecx)
 ; I486-NEXT:    movl $3, 4(%ecx)
 ; I486-NEXT:    movl $sc64, (%ecx)
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_or_8

 ; I486-NEXT:    leal sc64, %ecx
 ; I486-NEXT:    movl %esp, %esi
 ; I486-NEXT:    movl %edx, 8(%esi)
 ; I486-NEXT:    movl %eax, 4(%esi)
 ; I486-NEXT:    movl $2, 12(%esi)
 ; I486-NEXT:    movl $sc64, (%esi)
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_or_8
-; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    addl $44, %esp
+; I486-NEXT:    addl $36, %esp
 ; I486-NEXT:    popl %esi
 ; I486-NEXT:    retl
   %t1 = atomicrmw or   i64* @sc64, i64 3 acquire
   %t2 = atomicrmw or   i64* @sc64, i64 5 acquire
   %t3 = atomicrmw or   i64* @sc64, i64 %t2 acquire
   ret void
 }
 

 ; X64-NEXT:  .LBB4_2: # %atomicrmw.end
 ; X64-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rax # 8-byte Reload
 ; X64-NEXT:    lock xorq %rax, {{.*}}(%rip)
 ; X64-NEXT:    retq
 ;
 ; I486-LABEL: atomic_fetch_xor64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %esi
-; I486-NEXT:    subl $44, %esp
+; I486-NEXT:    subl $36, %esp
 ; I486-NEXT:    leal sc64, %eax
 ; I486-NEXT:    movl %esp, %ecx
 ; I486-NEXT:    movl $2, 12(%ecx)
 ; I486-NEXT:    movl $0, 8(%ecx)
 ; I486-NEXT:    movl $3, 4(%ecx)
 ; I486-NEXT:    movl $sc64, (%ecx)
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_xor_8

 ; I486-NEXT:    leal sc64, %ecx
 ; I486-NEXT:    movl %esp, %esi
 ; I486-NEXT:    movl %edx, 8(%esi)
 ; I486-NEXT:    movl %eax, 4(%esi)
 ; I486-NEXT:    movl $2, 12(%esi)
 ; I486-NEXT:    movl $sc64, (%esi)
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_xor_8
-; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    addl $44, %esp
+; I486-NEXT:    addl $36, %esp
 ; I486-NEXT:    popl %esi
 ; I486-NEXT:    retl
   %t1 = atomicrmw xor  i64* @sc64, i64 3 acquire
   %t2 = atomicrmw xor  i64* @sc64, i64 5 acquire
   %t3 = atomicrmw xor  i64* @sc64, i64 %t2 acquire
   ret void
 }
 

 ; X64-NEXT:    jne .LBB5_2
 ; X64-NEXT:    jmp .LBB5_1
 ; X64-NEXT:  .LBB5_2: # %atomicrmw.end
 ; X64-NEXT:    retq
 ;
 ; I486-LABEL: atomic_fetch_nand64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %esi
-; I486-NEXT:    subl $28, %esp
+; I486-NEXT:    subl $20, %esp
 ; I486-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; I486-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; I486-NEXT:    leal sc64, %edx
 ; I486-NEXT:    movl %esp, %esi
 ; I486-NEXT:    movl %eax, 8(%esi)
 ; I486-NEXT:    movl %ecx, 4(%esi)
 ; I486-NEXT:    movl $2, 12(%esi)
 ; I486-NEXT:    movl $sc64, (%esi)
 ; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_fetch_nand_8
-; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    addl $28, %esp
+; I486-NEXT:    addl $20, %esp
 ; I486-NEXT:    popl %esi
 ; I486-NEXT:    retl
   %t1 = atomicrmw nand i64* @sc64, i64 %x acquire
   ret void
 }
 
 define void @atomic_fetch_max64(i64 %x) nounwind {
 ; X64-LABEL: atomic_fetch_max64:

 ; I486-LABEL: atomic_fetch_max64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %ebp
 ; I486-NEXT:    movl %esp, %ebp
 ; I486-NEXT:    pushl %ebx
 ; I486-NEXT:    pushl %edi
 ; I486-NEXT:    pushl %esi
 ; I486-NEXT:    andl $-8, %esp
-; I486-NEXT:    subl $80, %esp
+; I486-NEXT:    subl $72, %esp
 ; I486-NEXT:    movl 12(%ebp), %eax
 ; I486-NEXT:    movl 8(%ebp), %ecx
 ; I486-NEXT:    movl sc64+4, %edx
 ; I486-NEXT:    movl sc64, %esi
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    jmp .LBB6_1
 ; I486-NEXT:  .LBB6_1: # %atomicrmw.start
 ; I486-NEXT:    # =>This Inner Loop Header: Depth=1
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; I486-NEXT:    movl %ecx, %edx
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
 ; I486-NEXT:    subl %esi, %edx
 ; I486-NEXT:    movl %eax, %edi
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; I486-NEXT:    sbbl %ebx, %edi
 ; I486-NEXT:    movl %ecx, %esi
 ; I486-NEXT:    movl %eax, %ebx
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    jge .LBB6_4
 ; I486-NEXT:  # %bb.3: # %atomicrmw.start
 ; I486-NEXT:    # in Loop: Header=BB6_1 Depth=1
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill

 ; I486-LABEL: atomic_fetch_min64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %ebp
 ; I486-NEXT:    movl %esp, %ebp
 ; I486-NEXT:    pushl %ebx
 ; I486-NEXT:    pushl %edi
 ; I486-NEXT:    pushl %esi
 ; I486-NEXT:    andl $-8, %esp
-; I486-NEXT:    subl $80, %esp
+; I486-NEXT:    subl $72, %esp
 ; I486-NEXT:    movl 12(%ebp), %eax
 ; I486-NEXT:    movl 8(%ebp), %ecx
 ; I486-NEXT:    movl sc64+4, %edx
 ; I486-NEXT:    movl sc64, %esi
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    jmp .LBB7_1
 ; I486-NEXT:  .LBB7_1: # %atomicrmw.start
 ; I486-NEXT:    # =>This Inner Loop Header: Depth=1
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; I486-NEXT:    subl %ecx, %edx
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
 ; I486-NEXT:    sbbl %eax, %esi
 ; I486-NEXT:    movl %ecx, %edi
 ; I486-NEXT:    movl %eax, %ebx
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    jge .LBB7_4
 ; I486-NEXT:  # %bb.3: # %atomicrmw.start
 ; I486-NEXT:    # in Loop: Header=BB7_1 Depth=1
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill

 ; I486-LABEL: atomic_fetch_umax64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %ebp
 ; I486-NEXT:    movl %esp, %ebp
 ; I486-NEXT:    pushl %ebx
 ; I486-NEXT:    pushl %edi
 ; I486-NEXT:    pushl %esi
 ; I486-NEXT:    andl $-8, %esp
-; I486-NEXT:    subl $80, %esp
+; I486-NEXT:    subl $72, %esp
 ; I486-NEXT:    movl 12(%ebp), %eax
 ; I486-NEXT:    movl 8(%ebp), %ecx
 ; I486-NEXT:    movl sc64+4, %edx
 ; I486-NEXT:    movl sc64, %esi
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    jmp .LBB8_1
 ; I486-NEXT:  .LBB8_1: # %atomicrmw.start
 ; I486-NEXT:    # =>This Inner Loop Header: Depth=1
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; I486-NEXT:    subl %ecx, %edx
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
 ; I486-NEXT:    sbbl %eax, %esi
 ; I486-NEXT:    movl %ecx, %edi
 ; I486-NEXT:    movl %eax, %ebx
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    jb .LBB8_4
 ; I486-NEXT:  # %bb.3: # %atomicrmw.start
 ; I486-NEXT:    # in Loop: Header=BB8_1 Depth=1
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill

 ; I486-LABEL: atomic_fetch_umin64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %ebp
 ; I486-NEXT:    movl %esp, %ebp
 ; I486-NEXT:    pushl %ebx
 ; I486-NEXT:    pushl %edi
 ; I486-NEXT:    pushl %esi
 ; I486-NEXT:    andl $-8, %esp
-; I486-NEXT:    subl $80, %esp
+; I486-NEXT:    subl $72, %esp
 ; I486-NEXT:    movl 12(%ebp), %eax
 ; I486-NEXT:    movl 8(%ebp), %ecx
 ; I486-NEXT:    movl sc64+4, %edx
 ; I486-NEXT:    movl sc64, %esi
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    jmp .LBB9_1
 ; I486-NEXT:  .LBB9_1: # %atomicrmw.start
 ; I486-NEXT:    # =>This Inner Loop Header: Depth=1
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; I486-NEXT:    subl %ecx, %edx
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
 ; I486-NEXT:    sbbl %eax, %esi
 ; I486-NEXT:    movl %ecx, %edi
 ; I486-NEXT:    movl %eax, %ebx
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    jae .LBB9_4
 ; I486-NEXT:  # %bb.3: # %atomicrmw.start
 ; I486-NEXT:    # in Loop: Header=BB9_1 Depth=1
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; I486-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill

 
 define void @atomic_fetch_cmpxchg64() nounwind {
 ; X64-LABEL: atomic_fetch_cmpxchg64:
 ; X64:       # %bb.0:
 ; X64-NEXT:    xorl %eax, %eax
 ; X64-NEXT:    # kill: def $rax killed $eax
 ; X64-NEXT:    movl $1, %ecx
 ; X64-NEXT:    lock cmpxchgq %rcx, {{.*}}(%rip)
-; X64-NEXT:    movq %rax, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; X64-NEXT:    retq
 ;
 ; I486-LABEL: atomic_fetch_cmpxchg64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %ebp
 ; I486-NEXT:    movl %esp, %ebp
 ; I486-NEXT:    andl $-8, %esp
 ; I486-NEXT:    subl $40, %esp
 ; I486-NEXT:    leal sc64, %eax
 ; I486-NEXT:    leal {{[0-9]+}}(%esp), %ecx
 ; I486-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; I486-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; I486-NEXT:    movl %esp, %edx
 ; I486-NEXT:    movl %ecx, 4(%edx)
 ; I486-NEXT:    movl $2, 20(%edx)
 ; I486-NEXT:    movl $2, 16(%edx)
 ; I486-NEXT:    movl $0, 12(%edx)
 ; I486-NEXT:    movl $1, 8(%edx)
 ; I486-NEXT:    movl $sc64, (%edx)
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_compare_exchange_8
-; I486-NEXT:    movb %al, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
 ; I486-NEXT:    movl %ebp, %esp
 ; I486-NEXT:    popl %ebp
 ; I486-NEXT:    retl
   %t1 = cmpxchg i64* @sc64, i64 0, i64 1 acquire acquire
   ret void
 }
 
 define void @atomic_fetch_store64(i64 %x) nounwind {
; I486-NEXT:    retl
   store atomic i64 %x, i64* @sc64 release, align 8
   ret void
 }
 
 define void @atomic_fetch_swap64(i64 %x) nounwind {
 ; X64-LABEL: atomic_fetch_swap64:
 ; X64:       # %bb.0:
 ; X64-NEXT:    xchgq %rdi, {{.*}}(%rip)
-; X64-NEXT:    movq %rdi, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; X64-NEXT:    retq
 ;
 ; I486-LABEL: atomic_fetch_swap64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %esi
-; I486-NEXT:    subl $28, %esp
+; I486-NEXT:    subl $20, %esp
 ; I486-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; I486-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; I486-NEXT:    leal sc64, %edx
 ; I486-NEXT:    movl %esp, %esi
 ; I486-NEXT:    movl %eax, 8(%esi)
 ; I486-NEXT:    movl %ecx, 4(%esi)
 ; I486-NEXT:    movl $2, 12(%esi)
 ; I486-NEXT:    movl $sc64, (%esi)
 ; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_exchange_8
-; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    addl $28, %esp
+; I486-NEXT:    addl $20, %esp
 ; I486-NEXT:    popl %esi
 ; I486-NEXT:    retl
   %t1 = atomicrmw xchg i64* @sc64, i64 %x acquire
   ret void
 }
 
 define void @atomic_fetch_swapf64(double %x) nounwind {
 ; X64-LABEL: atomic_fetch_swapf64:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %xmm0, %rax
 ; X64-NEXT:    xchgq %rax, {{.*}}(%rip)
-; X64-NEXT:    movq %rax, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; X64-NEXT:    retq
 ;
 ; I486-LABEL: atomic_fetch_swapf64:
 ; I486:       # %bb.0:
 ; I486-NEXT:    pushl %ebp
 ; I486-NEXT:    movl %esp, %ebp
 ; I486-NEXT:    pushl %esi
 ; I486-NEXT:    andl $-8, %esp
-; I486-NEXT:    subl $48, %esp
+; I486-NEXT:    subl $40, %esp
 ; I486-NEXT:    fldl 8(%ebp)
 ; I486-NEXT:    leal fsc64, %eax
 ; I486-NEXT:    fstpl {{[0-9]+}}(%esp)
 ; I486-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; I486-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; I486-NEXT:    movl %esp, %esi
 ; I486-NEXT:    movl %edx, 8(%esi)
 ; I486-NEXT:    movl %ecx, 4(%esi)
 ; I486-NEXT:    movl $2, 12(%esi)
 ; I486-NEXT:    movl $fsc64, (%esi)
 ; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    calll __atomic_exchange_8
-; I486-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; I486-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; I486-NEXT:    leal -4(%ebp), %esp
 ; I486-NEXT:    popl %esi
 ; I486-NEXT:    popl %ebp
 ; I486-NEXT:    retl
   %t1 = atomicrmw xchg double* @fsc64, double %x acquire
   ret void
 }

test/CodeGen/X86/atomic6432.ll

; X32-NEXT:    retl
   %t1 = atomicrmw umin i64* @sc64, i64 %x acquire
   ret void
 }
 
 define void @atomic_fetch_cmpxchg64() nounwind {
 ; X32-LABEL: atomic_fetch_cmpxchg64:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushl %ebx
-; X32-NEXT:    subl $12, %esp
+; X32-NEXT:    pushl %eax
 ; X32-NEXT:    xorl %eax, %eax
 ; X32-NEXT:    movl $1, %ebx
-; X32-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X32-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X32-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X32-NEXT:    movl %eax, (%esp) # 4-byte Spill
+; X32-NEXT:    movl (%esp), %edx # 4-byte Reload
+; X32-NEXT:    movl (%esp), %ecx # 4-byte Reload
 ; X32-NEXT:    lock cmpxchg8b sc64
-; X32-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X32-NEXT:    movl %edx, (%esp) # 4-byte Spill
-; X32-NEXT:    addl $12, %esp
+; X32-NEXT:    addl $4, %esp
 ; X32-NEXT:    popl %ebx
 ; X32-NEXT:    retl
   %t1 = cmpxchg i64* @sc64, i64 0, i64 1 acquire acquire
   ret void
 }
 
 define void @atomic_fetch_store64(i64 %x) nounwind {
 ; X32-LABEL: atomic_fetch_store64:

test/CodeGen/X86/avx512-mask-zext-bugfix.ll

 declare i64 @calc_expected_mask_val(i8* %valp, i32 %el_size, i32 %length)
 ; Function Attrs: nounwind uwtable
 declare i32 @check_mask16(i16 zeroext %res_mask, i16 zeroext %exp_mask, i8* %fname, i8* %input)
 
 ; Function Attrs: nounwind uwtable
 define void @test_xmm(i32 %shift, i32 %mulp, <2 x i64> %a,i8* %arraydecay,i8* %fname){
 ; CHECK-LABEL: test_xmm:
 ; CHECK:       ## %bb.0:
-; CHECK-NEXT:    subq $72, %rsp
-; CHECK-NEXT:    .cfi_def_cfa_offset 80
+; CHECK-NEXT:    subq $56, %rsp
+; CHECK-NEXT:    .cfi_def_cfa_offset 64
 ; CHECK-NEXT:    vpmovw2m %xmm0, %k0
 ; CHECK-NEXT:    movl $2, %esi
 ; CHECK-NEXT:    movl $8, %eax
 ; CHECK-NEXT:    movq %rdx, %rdi
 ; CHECK-NEXT:    movq %rdx, {{[-0-9]+}}(%r{{[sb]}}p) ## 8-byte Spill
 ; CHECK-NEXT:    movl %eax, %edx
 ; CHECK-NEXT:    movq %rcx, {{[-0-9]+}}(%r{{[sb]}}p) ## 8-byte Spill
 ; CHECK-NEXT:    vmovaps %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) ## 16-byte Spill

 ; CHECK-NEXT:    movzbl %r9b, %esi
 ; CHECK-NEXT:    movw %si, %r8w
 ; CHECK-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rdi ## 8-byte Reload
 ; CHECK-NEXT:    movl $4, %esi
 ; CHECK-NEXT:    movl %esi, {{[-0-9]+}}(%r{{[sb]}}p) ## 4-byte Spill
 ; CHECK-NEXT:    movl {{[-0-9]+}}(%r{{[sb]}}p), %edx ## 4-byte Reload
 ; CHECK-NEXT:    movl %eax, {{[-0-9]+}}(%r{{[sb]}}p) ## 4-byte Spill
 ; CHECK-NEXT:    kmovw %k1, {{[-0-9]+}}(%r{{[sb]}}p) ## 2-byte Spill
-; CHECK-NEXT:    movw %r8w, {{[-0-9]+}}(%r{{[sb]}}p) ## 2-byte Spill
+; CHECK-NEXT:    movw %r8w, (%rsp) ## 2-byte Spill
 ; CHECK-NEXT:    callq _calc_expected_mask_val
 ; CHECK-NEXT:    movw %ax, %r8w
-; CHECK-NEXT:    movw {{[-0-9]+}}(%r{{[sb]}}p), %r10w ## 2-byte Reload
+; CHECK-NEXT:    movw (%rsp), %r10w ## 2-byte Reload
 ; CHECK-NEXT:    movzwl %r10w, %edi
 ; CHECK-NEXT:    movzwl %r8w, %esi
 ; CHECK-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rdx ## 8-byte Reload
 ; CHECK-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rcx ## 8-byte Reload
 ; CHECK-NEXT:    callq _check_mask16
-; CHECK-NEXT:    movl %eax, {{[-0-9]+}}(%r{{[sb]}}p) ## 4-byte Spill
-; CHECK-NEXT:    addq $72, %rsp
+; CHECK-NEXT:    addq $56, %rsp
 ; CHECK-NEXT:    retq
   %d2 = bitcast <2 x i64> %a to <8 x i16>
   %m2 = call i8 @llvm.x86.avx512.cvtw2mask.128(<8 x i16> %d2)
   %conv7 = zext i8 %m2 to i16
   %call9 = call i64 @calc_expected_mask_val(i8* %arraydecay, i32 2, i32 8)
   %conv10 = trunc i64 %call9 to i16
   %call12 = call i32 @check_mask16(i16 zeroext %conv7, i16 zeroext %conv10, i8* %fname, i8* %arraydecay)
   %d3 = bitcast <2 x i64> %a to <4 x i32>

test/CodeGen/X86/pr11415.ll

 ; RUN: llc -mtriple=x86_64-pc-linux %s -o - -regalloc=fast -optimize-regalloc=0 | FileCheck %s
 
 ; We used to consider the early clobber in the second asm statement as
 ; defining %0 before it was read. This caused us to omit the
 ; movq  -8(%rsp), %rdx
 
 ; CHECK:  #APP
 ; CHECK-NEXT: #NO_APP
 ; CHECK-NEXT: movq  %rcx, %rax
 ; CHECK-NEXT: movq  %rax, -8(%rsp)
 ; CHECK-NEXT: movq  -8(%rsp), %rdx
 ; CHECK-NEXT: #APP
 ; CHECK-NEXT: #NO_APP
 ; CHECK-NEXT: movq  %rdx, %rax
-; CHECK-NEXT: movq  %rdx, -8(%rsp)
 ; CHECK-NEXT: ret
 
 define i64 @foo() {
 entry:
   %0 = tail call i64 asm "", "={cx}"() nounwind
   %1 = tail call i64 asm "", "=&r,0,r,~{rax}"(i64 %0, i64 %0) nounwind
   ret i64 %1
 }

test/CodeGen/X86/pr30430.ll

 ; CHECK-NEXT:    # implicit-def: $ymm3
 ; CHECK-NEXT:    vmovaps %xmm2, %xmm3
 ; CHECK-NEXT:    vinsertf128 $1, %xmm0, %ymm3, %ymm3
 ; CHECK-NEXT:    # implicit-def: $zmm24
 ; CHECK-NEXT:    vmovaps %zmm3, %zmm24
 ; CHECK-NEXT:    vinsertf64x4 $1, %ymm1, %zmm24, %zmm24
 ; CHECK-NEXT:    vmovaps %zmm24, {{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    vmovaps {{[0-9]+}}(%rsp), %zmm0
-; CHECK-NEXT:    vmovss %xmm8, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    vmovss %xmm9, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    vmovss %xmm10, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    vmovss %xmm11, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    vmovss %xmm12, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    vmovss %xmm13, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    vmovss %xmm14, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    vmovss %xmm15, (%rsp) # 4-byte Spill
 ; CHECK-NEXT:    movq %rbp, %rsp
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    .cfi_def_cfa %rsp, 8
 ; CHECK-NEXT:    retq
 entry:
   %__A.addr.i = alloca float, align 4
   %__B.addr.i = alloca float, align 4
   %__C.addr.i = alloca float, align 4

test/CodeGen/X86/pr32284.ll

 ; 686-O0-NEXT:    pushl %ebp
 ; 686-O0-NEXT:    .cfi_def_cfa_offset 8
 ; 686-O0-NEXT:    pushl %ebx
 ; 686-O0-NEXT:    .cfi_def_cfa_offset 12
 ; 686-O0-NEXT:    pushl %edi
 ; 686-O0-NEXT:    .cfi_def_cfa_offset 16
 ; 686-O0-NEXT:    pushl %esi
 ; 686-O0-NEXT:    .cfi_def_cfa_offset 20
-; 686-O0-NEXT:    subl $24, %esp
-; 686-O0-NEXT:    .cfi_def_cfa_offset 44
+; 686-O0-NEXT:    subl $1, %esp
+; 686-O0-NEXT:    .cfi_def_cfa_offset 21
 ; 686-O0-NEXT:    .cfi_offset %esi, -20
 ; 686-O0-NEXT:    .cfi_offset %edi, -16
 ; 686-O0-NEXT:    .cfi_offset %ebx, -12
 ; 686-O0-NEXT:    .cfi_offset %ebp, -8
 ; 686-O0-NEXT:    movl var_5, %eax
 ; 686-O0-NEXT:    movl %eax, %ecx
 ; 686-O0-NEXT:    sarl $31, %ecx
 ; 686-O0-NEXT:    xorl $208307499, %eax # imm = 0xC6A852B
 ; 686-O0-NEXT:    xorl $-2, %ecx
 ; 686-O0-NEXT:    orl %ecx, %eax
-; 686-O0-NEXT:    setne {{[0-9]+}}(%esp)
+; 686-O0-NEXT:    setne (%esp)
 ; 686-O0-NEXT:    movl var_5, %ecx
 ; 686-O0-NEXT:    movl %ecx, %edx
 ; 686-O0-NEXT:    sarl $31, %edx
 ; 686-O0-NEXT:    movl %ecx, %esi
 ; 686-O0-NEXT:    subl $-1, %esi
 ; 686-O0-NEXT:    sete %bl
 ; 686-O0-NEXT:    movzbl %bl, %edi
 ; 686-O0-NEXT:    addl $7093, %ecx # imm = 0x1BB5
 ; 686-O0-NEXT:    adcl $0, %edx
 ; 686-O0-NEXT:    subl %edi, %ecx
 ; 686-O0-NEXT:    sbbl $0, %edx
 ; 686-O0-NEXT:    setl %bl
 ; 686-O0-NEXT:    movzbl %bl, %edi
 ; 686-O0-NEXT:    movl %edi, var_57
 ; 686-O0-NEXT:    movl $0, var_57+4
 ; 686-O0-NEXT:    movl var_5, %edi
 ; 686-O0-NEXT:    subl $-1, %edi
 ; 686-O0-NEXT:    sete %bl
 ; 686-O0-NEXT:    movzbl %bl, %ebp
 ; 686-O0-NEXT:    movl %ebp, _ZN8struct_210member_2_0E
 ; 686-O0-NEXT:    movl $0, _ZN8struct_210member_2_0E+4
-; 686-O0-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; 686-O0-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; 686-O0-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; 686-O0-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; 686-O0-NEXT:    movl %edi, (%esp) # 4-byte Spill
-; 686-O0-NEXT:    addl $24, %esp
+; 686-O0-NEXT:    addl $1, %esp
 ; 686-O0-NEXT:    .cfi_def_cfa_offset 20
 ; 686-O0-NEXT:    popl %esi
 ; 686-O0-NEXT:    .cfi_def_cfa_offset 16
 ; 686-O0-NEXT:    popl %edi
 ; 686-O0-NEXT:    .cfi_def_cfa_offset 12
 ; 686-O0-NEXT:    popl %ebx
 ; 686-O0-NEXT:    .cfi_def_cfa_offset 8
 ; 686-O0-NEXT:    popl %ebp

test/CodeGen/X86/pr32484.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -fast-isel-sink-local-values -O0 -mtriple=x86_64-unknown-linux-gnu < %s | FileCheck %s
 
 define void @foo() {
 ; CHECK-LABEL: foo:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    # implicit-def: $rax
 ; CHECK-NEXT:    jmpq *%rax
 ; CHECK-NEXT:  .LBB0_1:
 ; CHECK-NEXT:    xorps %xmm0, %xmm0
 ; CHECK-NEXT:    pcmpeqd %xmm1, %xmm1
 ; CHECK-NEXT:    # implicit-def: $rax
 ; CHECK-NEXT:    movdqu %xmm1, (%rax)
-; CHECK-NEXT:    movaps %xmm0, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; CHECK-NEXT:  .LBB0_2:
 ; CHECK-NEXT:    retq
   indirectbr i8* undef, [label %9, label %1]
 
 ; <label>:1:                                      ; preds = %0
   %2 = shufflevector <16 x i8> zeroinitializer, <16 x i8> <i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef>, <16 x i32> <i32 8, i32 9, i32 10, i32 11, i32 12, i32 13, i32 14, i32 15, i32 16, i32 17, i32 18, i32 19, i32 20, i32 21, i32 22, i32 23>
   %3 = shufflevector <16 x i8> <i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0, i8 0>, <16 x i8> zeroinitializer, <16 x i32> <i32 8, i32 9, i32 10, i32 11, i32 12, i32 13, i32 14, i32 15, i32 16, i32 17, i32 18, i32 19, i32 20, i32 21, i32 22, i32 23>
   %4 = or <16 x i8> %3, %2

test/CodeGen/X86/pr34592.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=avx2 -O0 | FileCheck %s
 
 define <16 x i64> @pluto(<16 x i64> %arg, <16 x i64> %arg1, <16 x i64> %arg2, <16 x i64> %arg3, <16 x i64> %arg4) {
 ; CHECK-LABEL: pluto:
 ; CHECK:       # %bb.0: # %bb
 ; CHECK-NEXT:    pushq %rbp
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset %rbp, -16
 ; CHECK-NEXT:    movq %rsp, %rbp
 ; CHECK-NEXT:    .cfi_def_cfa_register %rbp
 ; CHECK-NEXT:    andq $-32, %rsp
-; CHECK-NEXT:    subq $320, %rsp # imm = 0x140
+; CHECK-NEXT:    subq $128, %rsp
 ; CHECK-NEXT:    vmovaps 240(%rbp), %ymm8
 ; CHECK-NEXT:    vmovaps 208(%rbp), %ymm9
 ; CHECK-NEXT:    vmovaps 176(%rbp), %ymm10
 ; CHECK-NEXT:    vmovaps 144(%rbp), %ymm11
 ; CHECK-NEXT:    vmovaps 112(%rbp), %ymm12
 ; CHECK-NEXT:    vmovaps 80(%rbp), %ymm13
 ; CHECK-NEXT:    vmovaps 48(%rbp), %ymm14
 ; CHECK-NEXT:    vmovaps 16(%rbp), %ymm15

 ; CHECK-NEXT:    vpermq {{.*#+}} ymm9 = ymm9[0,1,0,3]
 ; CHECK-NEXT:    vpblendd {{.*#+}} ymm9 = ymm11[0,1,2,3],ymm9[4,5,6,7]
 ; CHECK-NEXT:    vpblendd {{.*#+}} ymm7 = ymm7[0,1],ymm8[2,3],ymm7[4,5,6,7]
 ; CHECK-NEXT:    vpermq {{.*#+}} ymm7 = ymm7[2,1,1,3]
 ; CHECK-NEXT:    vpshufd {{.*#+}} ymm5 = ymm5[0,1,0,1,4,5,4,5]
 ; CHECK-NEXT:    vpblendd {{.*#+}} ymm5 = ymm7[0,1,2,3,4,5],ymm5[6,7]
 ; CHECK-NEXT:    vmovaps %ymm1, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
 ; CHECK-NEXT:    vmovaps %ymm5, %ymm1
-; CHECK-NEXT:    vmovaps %ymm3, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
+; CHECK-NEXT:    vmovaps %ymm3, (%rsp) # 32-byte Spill
 ; CHECK-NEXT:    vmovaps %ymm9, %ymm3
-; CHECK-NEXT:    vmovaps %ymm10, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
-; CHECK-NEXT:    vmovaps %ymm12, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
-; CHECK-NEXT:    vmovaps %ymm13, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
-; CHECK-NEXT:    vmovaps %ymm14, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
-; CHECK-NEXT:    vmovaps %ymm15, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
-; CHECK-NEXT:    vmovaps %ymm4, (%rsp) # 32-byte Spill
 ; CHECK-NEXT:    movq %rbp, %rsp
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    .cfi_def_cfa %rsp, 8
 ; CHECK-NEXT:    retq
 bb:
   %tmp = select <16 x i1> <i1 false, i1 false, i1 false, i1 false, i1 false, i1 false, i1 false, i1 false, i1 false, i1 false, i1 true, i1 true, i1 false, i1 false, i1 false, i1 false>, <16 x i64> %arg, <16 x i64> %arg1
   %tmp5 = select <16 x i1> <i1 true, i1 false, i1 false, i1 true, i1 true, i1 false, i1 false, i1 true, i1 false, i1 true, i1 false, i1 false, i1 false, i1 false, i1 false, i1 false>, <16 x i64> %arg2, <16 x i64> zeroinitializer
   %tmp6 = select <16 x i1> <i1 false, i1 true, i1 true, i1 true, i1 false, i1 false, i1 false, i1 false, i1 true, i1 true, i1 false, i1 false, i1 false, i1 true, i1 true, i1 true>, <16 x i64> %arg3, <16 x i64> %tmp5
   %tmp7 = shufflevector <16 x i64> %tmp, <16 x i64> %tmp6, <16 x i32> <i32 11, i32 18, i32 24, i32 9, i32 14, i32 29, i32 29, i32 6, i32 14, i32 28, i32 8, i32 9, i32 22, i32 12, i32 25, i32 6>
   ret <16 x i64> %tmp7
 }

test/CodeGen/X86/pr34653.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc %s -O0 -mtriple=x86_64-unknown-unknown -mattr=+avx512f -o - | FileCheck %s
 
 declare fastcc <38 x double> @test()
 
 define void @pr34653() {
 ; CHECK-LABEL: pr34653:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pushq %rbp
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset %rbp, -16
 ; CHECK-NEXT:    movq %rsp, %rbp
 ; CHECK-NEXT:    .cfi_def_cfa_register %rbp
 ; CHECK-NEXT:    andq $-512, %rsp # imm = 0xFE00
-; CHECK-NEXT:    subq $2048, %rsp # imm = 0x800
+; CHECK-NEXT:    subq $1536, %rsp # imm = 0x600
 ; CHECK-NEXT:    leaq {{[0-9]+}}(%rsp), %rdi
 ; CHECK-NEXT:    callq test
 ; CHECK-NEXT:    vmovupd {{[0-9]+}}(%rsp), %xmm0
 ; CHECK-NEXT:    vmovaps %xmm0, %xmm1
 ; CHECK-NEXT:    vmovupd {{[0-9]+}}(%rsp), %xmm2
 ; CHECK-NEXT:    vmovaps %xmm2, %xmm3
 ; CHECK-NEXT:    vmovupd {{[0-9]+}}(%rsp), %xmm4
 ; CHECK-NEXT:    vmovaps %xmm4, %xmm5

 ; CHECK-NEXT:    vmovsd {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 8-byte Reload
 ; CHECK-NEXT:    # xmm0 = mem[0],zero
 ; CHECK-NEXT:    vmovsd %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-NEXT:    vmovsd {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 8-byte Reload
 ; CHECK-NEXT:    # xmm0 = mem[0],zero
 ; CHECK-NEXT:    vmovsd %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-NEXT:    vmovsd {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 8-byte Reload
 ; CHECK-NEXT:    # xmm0 = mem[0],zero
-; CHECK-NEXT:    vmovsd %xmm1, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm3, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm5, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm7, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm9, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm11, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm13, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm15, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm16, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm17, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm18, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm19, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm20, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm21, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm22, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm23, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm24, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm25, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm26, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm27, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm2, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm4, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm6, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm8, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm10, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm12, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm14, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm28, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm29, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm30, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm31, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    vmovsd %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-NEXT:    movq %rbp, %rsp
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    .cfi_def_cfa %rsp, 8
 ; CHECK-NEXT:    vzeroupper
 ; CHECK-NEXT:    retq
 entry:
   %v = call fastcc <38 x double> @test()
   %v.0 = extractelement <38 x double> %v, i32 0
   ret void
 }
 

test/CodeGen/X86/swifterror.ll

 ; CHECK-APPLE:  popq %rax
 ; CHECK-APPLE:  retq
 
 ; CHECK-APPLE:  xorps %xmm0, %xmm0
 ; CHECK-APPLE:  popq %rax
 ; CHECK-APPLE:  retq
 
 ; CHECK-O0-LABEL: conditionally_forward_swifterror:
-; CHECK-O0:  subq $24, %rsp
-; CHECK-O0:  movq %r12, [[REG1:%[a-z0-9]+]]
+; CHECK-O0: pushq [[REG1:%[a-z0-9]+]]
+; CHECK-O0:  movq %r12, [[REG1]]
 ; CHECK-O0:  cmpl $0, %edi
-; CHECK-O0-DAG:  movq [[REG1]], [[STK:[0-9]+]](%rsp)
-; CHECK-O0-DAG:  movq %r12, [[STK2:[0-9]+]](%rsp)
+; CHECK-O0-DAG:  movq %r12, (%rsp)
 ; CHECK-O0:  je
 
-; CHECK-O0:  movq [[STK2]](%rsp), [[REG:%[a-z0-9]+]]
+; CHECK-O0:  movq (%rsp), [[REG:%[a-z0-9]+]]
 ; CHECK-O0:  movq [[REG]], %r12
 ; CHECK-O0:  callq _moo
-; CHECK-O0:  addq $24, %rsp
+; CHECK-O0:  popq [[REG1]]
 ; CHECK-O0:  retq
 
-; CHECK-O0:  movq [[STK2]](%rsp), [[REG:%[a-z0-9]+]]
+; CHECK-O0:  movq (%rsp), [[REG:%[a-z0-9]+]]
 ; CHECK-O0:  xorps %xmm0, %xmm0
 ; CHECK-O0:  movq [[REG]], %r12
-; CHECK-O0:  addq $24, %rsp
+; CHECK-O0:  popq [[REG1]]
 ; CHECK-O0:  retq
 entry:
   %cond = icmp ne i32 %cc, 0
   br i1 %cond, label %gen_error, label %normal
 
 gen_error:
   %call = call swiftcc float @moo(%swift_error** swifterror %error_ptr_ref)
   ret float %call

test/DebugInfo/X86/pieces-1.ll

 ;    }
 ;
 ;
 ; CHECK: .debug_loc contents:
 ;
 
 ; CHECK: [0x0000000000000000, 0x[[LTMP3:.*]]): DW_OP_reg5 RDI, DW_OP_piece 0x8, DW_OP_reg4 RSI, DW_OP_piece 0x4
 ; 0x0000000000000006 - 0x0000000000000008: rbp-8, piece 0x8, rax, piece 0x4 )
-; CHECK: [0x[[LTMP3]], {{.*}}): DW_OP_breg6 RBP-8, DW_OP_piece 0x8, DW_OP_reg4 RSI, DW_OP_piece 0x4
 
 target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-apple-macosx10.9.0"
 
 ; Function Attrs: nounwind ssp uwtable
 define i32 @foo(i64 %s.coerce0, i32 %s.coerce1) #0 !dbg !4 {
 entry:
   call void @llvm.dbg.value(metadata i64 %s.coerce0, metadata !20, metadata !24), !dbg !21