[LSR] Improved code generation for Zero Compare loops

[joanlluch]

Summary

Improves loop code generation. All targets are affected but most benefits are obtained for X86. Creates shorter code in a number of cases by allowing the Strength Reduce algorithm to consider both the direct and swapped forms of zero compare instructions, which enhances the opportunities to obtain an overall better LSR solution. Given equal LSR solution cost, the patch also honours the direction of the loop induction variable specified in the user source code, which in practice also tends to result in a better solution.

The patch broke a number of regression tests due to inherent test fragility, not because of intended test failures. I fixed the CodeGen tests for the ARM and X86 architectures.

An example of code improved by this patch:

int func(void);
void func2(void);

void LSRTest(int count)
{
  count += func();
  for ( ; count != 20; ++count ) {
    func2();
  }
}

Before:

	.section	__TEXT,__text,regular,pure_instructions
	.macosx_version_min 10, 12
	.globl	_LSRTest
_LSRTest:
	.cfi_startproc
	pushl	%ebp
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp
	.cfi_def_cfa_register %ebp
	pushl	%esi
	pushl	%eax
	.cfi_offset %esi, -12
	calll	_func
	addl	8(%ebp), %eax
	pushl	$20
	popl	%esi
	subl	%eax, %esi
	jmp	LBB0_1
LBB0_2:
	calll	_func2
	decl	%esi
LBB0_1:
	testl	%esi, %esi
	jne	LBB0_2
	addl	$4, %esp
	popl	%esi
	popl	%ebp
	retl
	.cfi_endproc

After:

	.section	__TEXT,__text,regular,pure_instructions
	.macosx_version_min 10, 12
	.globl	_LSRTest
_LSRTest:
	.cfi_startproc
	pushl	%ebp
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp
	.cfi_def_cfa_register %ebp
	pushl	%esi
	pushl	%eax
	.cfi_offset %esi, -12
	movl	8(%ebp), %esi
	calll	_func
	leal	-20(%eax,%esi), %esi
	jmp	LBB0_1
LBB0_2:
	calll	_func2
	incl	%esi
LBB0_1:
	testl	%esi, %esi
	jne	LBB0_2
	addl	$4, %esp
	popl	%esi
	popl	%ebp
	retl
	.cfi_endproc

[shchenz]

This patch causes a lot of cases fail except platform X86 and ARM.

And there is a compiling time crash:

: 'RUN: at line 6';   /home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc -switch-peel-threshold=101 < /home/czhengsz/llvm_new/llvm-project/llvm/test/CodeGen/SystemZ/loop-03.ll -mtriple=s390x-linux-gnu -mcpu=z13 | /home/czhengsz/llvm_new/llvm-project/llvm/build/bin/FileCheck /home/czhengsz/llvm_new/llvm-project/llvm/test/CodeGen/SystemZ/loop-03.ll
--
Exit Code: 2

Command Output (stderr):
--
llc: /home/czhengsz/llvm_new/llvm-project/llvm/lib/Transforms/Scalar/LoopStrengthReduce.cpp:3350: void {anonymous}::LSRInstance::InsertSupplementalFormula(const llvm::SCEV*, {anonymous}::LSRUse&, size_t): Assertion `Inserted && "Supplemental formula already exists!"' failed.
Stack dump:
0.      Program arguments: /home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc -switch-peel-threshold=101 -mtriple=s390x-linux-gnu -mcpu=z13 
1.      Running pass 'Function Pass Manager' on module '<stdin>'.
2.      Running pass 'Loop Pass Manager' on function '@fun0'
3.      Running pass 'Loop Strength Reduction' on basic block '%23'
 #0 0x00000000127b8768 llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x127b8768)
 #1 0x00000000127b8890 PrintStackTraceSignalHandler(void*) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x127b8890)
 #2 0x00000000127b6324 llvm::sys::RunSignalHandlers() (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x127b6324)
 #3 0x00000000127b6504 SignalHandler(int) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x127b6504)
 #4 0x00003fffb6470478 (linux-vdso64.so.1+0x478)
 #5 0x00003fffb5d0e100 raise (/opt/at12.0/lib64/power8/libc.so.6+0x4e100)
 #6 0x00003fffb5ce4598 abort (/opt/at12.0/lib64/power8/libc.so.6+0x24598)
 #7 0x00003fffb5cfb624 (/opt/at12.0/lib64/power8/libc.so.6+0x3b624)
 #8 0x00003fffb5cfb6c4 __assert_fail (/opt/at12.0/lib64/power8/libc.so.6+0x3b6c4)
 #9 0x000000001220151c (anonymous namespace)::LSRInstance::InsertSupplementalFormula(llvm::SCEV const*, (anonymous namespace)::LSRUse&, unsigned long) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x1220151c)
#10 0x0000000012209050 (anonymous namespace)::LSRInstance::CollectFixupsAndInitialFormulae() (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x12209050)
#11 0x0000000012212908 ReduceLoopStrength(llvm::Loop*, llvm::IVUsers&, llvm::ScalarEvolution&, llvm::DominatorTree&, llvm::LoopInfo&, llvm::TargetTransformInfo const&) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x12212908)
#12 0x00000000117e5938 llvm::LPPassManager::runOnFunction(llvm::Function&) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x117e5938)
#13 0x00000000120240b0 llvm::FPPassManager::runOnFunction(llvm::Function&) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x120240b0)
#14 0x00000000120242f8 llvm::FPPassManager::runOnModule(llvm::Module&) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x120242f8)
#15 0x00000000120230e0 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x120230e0)
#16 0x000000001202350c llvm::legacy::PassManager::run(llvm::Module&) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x1202350c)
#17 0x00000000103b585c compileModule(char**, llvm::LLVMContext&) (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x103b585c)
#18 0x0000000010318678 main (/home/czhengsz/llvm_new/llvm-project/llvm/build/bin/llc+0x10318678)
#19 0x00003fffb5ce4bf8 (/opt/at12.0/lib64/power8/libc.so.6+0x24bf8)
#20 0x00003fffb5ce4e04 __libc_start_main (/opt/at12.0/lib64/power8/libc.so.6+0x24e04)
FileCheck error: '-' is empty.
FileCheck command line:  /home/czhengsz/llvm_new/llvm-project/llvm/build/bin/FileCheck /home/czhengsz/llvm_new/llvm-project/llvm/test/CodeGen/SystemZ/loop-03.ll

I think these failures should be fixed firstly.

One opinion about this patch's example, seems you found the improvement for X86, because incl loop(negative loop count) on X86 can use the instruction movl+ leal, but decl loop(positive loop count) uses addl + pushl + popl + subl.

As I know, PowerPC has hardware loop, it needs the loop count must be a positive value. This patch changes all loop to a incl loop with negative loop count which is surely a deg for PowerPC. So maybe you need to treat this improvement for specific platform by using some target hook function?

[joanlluch]

Fixes reported assertion crash on SystemZ

[joanlluch]

Hi shchenz,

Thanks for taking the time for reviewing this. I solved your reported assertion crash on SystemZ, and it is now fixed.

I stated on the description comment that I fixed codegen tests for X86 and ARM, but I'm waiting feedback for other platforms because I'm not that versed on them. It would just be very easy for me to just run "update_llc_test_checks.py" and forget, but I do not think that this is what the community would want to see. On the other hand, I understand that you can help with the PowerPC platform, so maybe you can look at that.

This patch improves code generation not because it choses a particular induction variable direction (increment or decrement) but because it choses the less costly or most natural one. Following there are a couple of examples that show improvements for both positive and negative increments:

void LSRTestA( int a, unsigned ammount )
{
  ammount += 8;
  while ( ammount-- )
    bar(a);
}

void LSRTestB( int a, unsigned ammount )
{
  ammount += 8;
  for ( ; ammount != 0 ; ammount++ )
    bar(a);
}

Before:

	.section	__TEXT,__text,regular,pure_instructions
	.macosx_version_min 10, 12
	.globl	_LSRTestA
_LSRTestA:
	.cfi_startproc
	pushl	%ebp
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp
	.cfi_def_cfa_register %ebp
	pushl	%edi
	pushl	%esi
	subl	$16, %esp
	.cfi_offset %esi, -16
	.cfi_offset %edi, -12
	movl	8(%ebp), %esi
	pushl	$-8
	popl	%edi
	subl	12(%ebp), %edi
	jmp	LBB0_1
LBB0_2:
	movl	%esi, (%esp)
	calll	_bar
	incl	%edi
LBB0_1:
	testl	%edi, %edi
	jne	LBB0_2
	addl	$16, %esp
	popl	%esi
	popl	%edi
	popl	%ebp
	retl
	.cfi_endproc

	.globl	_LSRTestB
_LSRTestB:
	.cfi_startproc
	pushl	%ebp
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp
	.cfi_def_cfa_register %ebp
	pushl	%edi
	pushl	%esi
	subl	$16, %esp
	.cfi_offset %esi, -16
	.cfi_offset %edi, -12
	movl	8(%ebp), %esi
	pushl	$-8
	popl	%edi
	subl	12(%ebp), %edi
	jmp	LBB1_1
LBB1_2:
	movl	%esi, (%esp)
	calll	_bar
	decl	%edi
LBB1_1:
	testl	%edi, %edi
	jne	LBB1_2
	addl	$16, %esp
	popl	%esi
	popl	%edi
	popl	%ebp
	retl
	.cfi_endproc

After:

	.section	__TEXT,__text,regular,pure_instructions
	.macosx_version_min 10, 12
	.globl	_LSRTestA
_LSRTestA:
	.cfi_startproc
	pushl	%ebp
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp
	.cfi_def_cfa_register %ebp
	pushl	%edi
	pushl	%esi
	subl	$16, %esp
	.cfi_offset %esi, -16
	.cfi_offset %edi, -12
	movl	8(%ebp), %esi
	movl	12(%ebp), %edi
	addl	$8, %edi
	jmp	LBB0_1
LBB0_2:
	decl	%edi
	movl	%esi, (%esp)
	calll	_bar
LBB0_1:
	testl	%edi, %edi
	jne	LBB0_2
	addl	$16, %esp
	popl	%esi
	popl	%edi
	popl	%ebp
	retl
	.cfi_endproc

	.globl	_LSRTestB
_LSRTestB:
	.cfi_startproc
	pushl	%ebp
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp
	.cfi_def_cfa_register %ebp
	pushl	%edi
	pushl	%esi
	subl	$16, %esp
	.cfi_offset %esi, -16
	.cfi_offset %edi, -12
	movl	8(%ebp), %esi
	movl	12(%ebp), %edi
	addl	$8, %edi
	jmp	LBB1_1
LBB1_2:
	movl	%esi, (%esp)
	calll	_bar
	incl	%edi
LBB1_1:
	testl	%edi, %edi
	jne	LBB1_2
	addl	$16, %esp
	popl	%esi
	popl	%edi
	popl	%ebp
	retl
	.cfi_endproc

On these examples, a pair of potentially expensive push, pop instructions are replaced by a mov.

About PowerPC, I'm not that versed on it. If it needs the loop count to be always positive to take advantage of hardware, then I suggest you to propose a patch for that. Maybe we can work together if you agree. My current proposal does not particularly favour any count direction, but given equal cost it tends to honour the one specified on the source code. So at the end of the day, I think that even for PowerPC it should still result in overall improvements because most loops are positive counting anyway. Please correct me if I am wrong.

[craig.topper]

I'm not sure why your examples have push instructions to load an immediate into registers unless you're compiling with -Oz? And if that's the case then changing push/pop to mov is an increase in code size.

[joanlluch]

Hi Craig, thanks for commenting.
Yes, I was actually compiling for -Oz, but the differences when using -Os are even bigger. Let me try to explain every case.

• For Oz, the compiler indeed generates expensive push/pop instructions as an attempt to reduce code size. However, after the patch is applied these instructions are removed without any code size increase. I have checked that in several scenarios and the result is either the same size or less size. This is because the patch reduces the overall number of instructions. On the examples above, the resulting code size is identical, this is because the sequence:

	pushl	$-8                     ## encoding: [0x6a,0xf8]
	popl	%edi                    ## encoding: [0x5f]
	subl	12(%ebp), %edi          ## encoding: [0x2b,0x7d,0x0c]

is replaced by:

	movl	12(%ebp), %edi          ## encoding: [0x8b,0x7d,0x0c]
	addl	$8, %edi                ## encoding: [0x83,0xc7,0x08]

It's a total of 6 machine code bytes in both cases, but one less instruction after the patch.
This is just an example, there are many cases where both the code size and number of instructions are reduced.

• Now, if we compile the same with the -Os flag we get even further improvements. this is the resulting code before and after for the same source code:

Before:

	.section	__TEXT,__text,regular,pure_instructions
	.macosx_version_min 10, 12
	.globl	_LSRTestA               ## -- Begin function LSRTestA
_LSRTestA:                              ## @LSRTestA
	.cfi_startproc
## %bb.0:                               ## %entry
	pushl	%ebp                    ## encoding: [0x55]
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp              ## encoding: [0x89,0xe5]
	.cfi_def_cfa_register %ebp
	pushl	%edi                    ## encoding: [0x57]
	pushl	%esi                    ## encoding: [0x56]
	subl	$16, %esp               ## encoding: [0x83,0xec,0x10]
	.cfi_offset %esi, -16
	.cfi_offset %edi, -12
	movl	12(%ebp), %eax          ## encoding: [0x8b,0x45,0x0c]
	cmpl	$-8, %eax               ## encoding: [0x83,0xf8,0xf8]
	je	LBB0_3                  ## encoding: [0x74,A]
                                        ##   fixup A - offset: 1, value: LBB0_3-1, kind: FK_PCRel_1
## %bb.1:                               ## %while.body.preheader
	movl	8(%ebp), %esi           ## encoding: [0x8b,0x75,0x08]
	movl	$-8, %edi               ## encoding: [0xbf,0xf8,0xff,0xff,0xff]
	subl	%eax, %edi              ## encoding: [0x29,0xc7]
LBB0_2:                                 ## %while.body
                                        ## =>This Inner Loop Header: Depth=1
	movl	%esi, (%esp)            ## encoding: [0x89,0x34,0x24]
	calll	_bar                    ## encoding: [0xe8,A,A,A,A]
                                        ##   fixup A - offset: 1, value: _bar-4, kind: FK_PCRel_4
	incl	%edi                    ## encoding: [0x47]
	jne	LBB0_2                  ## encoding: [0x75,A]
                                        ##   fixup A - offset: 1, value: LBB0_2-1, kind: FK_PCRel_1
LBB0_3:                                 ## %while.end
	addl	$16, %esp               ## encoding: [0x83,0xc4,0x10]
	popl	%esi                    ## encoding: [0x5e]
	popl	%edi                    ## encoding: [0x5f]
	popl	%ebp                    ## encoding: [0x5d]
	retl                            ## encoding: [0xc3]
	.cfi_endproc
                                        ## -- End function
	.globl	_LSRTestB               ## -- Begin function LSRTestB
_LSRTestB:                              ## @LSRTestB
	.cfi_startproc
## %bb.0:                               ## %entry
	pushl	%ebp                    ## encoding: [0x55]
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp              ## encoding: [0x89,0xe5]
	.cfi_def_cfa_register %ebp
	pushl	%edi                    ## encoding: [0x57]
	pushl	%esi                    ## encoding: [0x56]
	subl	$16, %esp               ## encoding: [0x83,0xec,0x10]
	.cfi_offset %esi, -16
	.cfi_offset %edi, -12
	movl	12(%ebp), %eax          ## encoding: [0x8b,0x45,0x0c]
	cmpl	$-8, %eax               ## encoding: [0x83,0xf8,0xf8]
	je	LBB1_3                  ## encoding: [0x74,A]
                                        ##   fixup A - offset: 1, value: LBB1_3-1, kind: FK_PCRel_1
## %bb.1:                               ## %for.body.preheader
	movl	8(%ebp), %esi           ## encoding: [0x8b,0x75,0x08]
	movl	$-8, %edi               ## encoding: [0xbf,0xf8,0xff,0xff,0xff]
	subl	%eax, %edi              ## encoding: [0x29,0xc7]
LBB1_2:                                 ## %for.body
                                        ## =>This Inner Loop Header: Depth=1
	movl	%esi, (%esp)            ## encoding: [0x89,0x34,0x24]
	calll	_bar                    ## encoding: [0xe8,A,A,A,A]
                                        ##   fixup A - offset: 1, value: _bar-4, kind: FK_PCRel_4
	decl	%edi                    ## encoding: [0x4f]
	jne	LBB1_2                  ## encoding: [0x75,A]
                                        ##   fixup A - offset: 1, value: LBB1_2-1, kind: FK_PCRel_1
LBB1_3:                                 ## %for.end
	addl	$16, %esp               ## encoding: [0x83,0xc4,0x10]
	popl	%esi                    ## encoding: [0x5e]
	popl	%edi                    ## encoding: [0x5f]
	popl	%ebp                    ## encoding: [0x5d]
	retl                            ## encoding: [0xc3]
	.cfi_endproc

After:

	.section	__TEXT,__text,regular,pure_instructions
	.macosx_version_min 10, 12
	.globl	_LSRTestA               ## -- Begin function LSRTestA
_LSRTestA:                              ## @LSRTestA
	.cfi_startproc
## %bb.0:                               ## %entry
	pushl	%ebp                    ## encoding: [0x55]
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp              ## encoding: [0x89,0xe5]
	.cfi_def_cfa_register %ebp
	pushl	%edi                    ## encoding: [0x57]
	pushl	%esi                    ## encoding: [0x56]
	subl	$16, %esp               ## encoding: [0x83,0xec,0x10]
	.cfi_offset %esi, -16
	.cfi_offset %edi, -12
	movl	12(%ebp), %esi          ## encoding: [0x8b,0x75,0x0c]
	addl	$8, %esi                ## encoding: [0x83,0xc6,0x08]
	je	LBB0_3                  ## encoding: [0x74,A]
                                        ##   fixup A - offset: 1, value: LBB0_3-1, kind: FK_PCRel_1
## %bb.1:                               ## %while.body.preheader
	movl	8(%ebp), %edi           ## encoding: [0x8b,0x7d,0x08]
LBB0_2:                                 ## %while.body
                                        ## =>This Inner Loop Header: Depth=1
	movl	%edi, (%esp)            ## encoding: [0x89,0x3c,0x24]
	calll	_bar                    ## encoding: [0xe8,A,A,A,A]
                                        ##   fixup A - offset: 1, value: _bar-4, kind: FK_PCRel_4
	decl	%esi                    ## encoding: [0x4e]
	jne	LBB0_2                  ## encoding: [0x75,A]
                                        ##   fixup A - offset: 1, value: LBB0_2-1, kind: FK_PCRel_1
LBB0_3:                                 ## %while.end
	addl	$16, %esp               ## encoding: [0x83,0xc4,0x10]
	popl	%esi                    ## encoding: [0x5e]
	popl	%edi                    ## encoding: [0x5f]
	popl	%ebp                    ## encoding: [0x5d]
	retl                            ## encoding: [0xc3]
	.cfi_endproc
                                        ## -- End function
	.globl	_LSRTestB               ## -- Begin function LSRTestB
_LSRTestB:                              ## @LSRTestB
	.cfi_startproc
## %bb.0:                               ## %entry
	pushl	%ebp                    ## encoding: [0x55]
	.cfi_def_cfa_offset 8
	.cfi_offset %ebp, -8
	movl	%esp, %ebp              ## encoding: [0x89,0xe5]
	.cfi_def_cfa_register %ebp
	pushl	%edi                    ## encoding: [0x57]
	pushl	%esi                    ## encoding: [0x56]
	subl	$16, %esp               ## encoding: [0x83,0xec,0x10]
	.cfi_offset %esi, -16
	.cfi_offset %edi, -12
	movl	12(%ebp), %esi          ## encoding: [0x8b,0x75,0x0c]
	addl	$8, %esi                ## encoding: [0x83,0xc6,0x08]
	je	LBB1_3                  ## encoding: [0x74,A]
                                        ##   fixup A - offset: 1, value: LBB1_3-1, kind: FK_PCRel_1
## %bb.1:                               ## %for.body.preheader
	movl	8(%ebp), %edi           ## encoding: [0x8b,0x7d,0x08]
LBB1_2:                                 ## %for.body
                                        ## =>This Inner Loop Header: Depth=1
	movl	%edi, (%esp)            ## encoding: [0x89,0x3c,0x24]
	calll	_bar                    ## encoding: [0xe8,A,A,A,A]
                                        ##   fixup A - offset: 1, value: _bar-4, kind: FK_PCRel_4
	incl	%esi                    ## encoding: [0x46]
	jne	LBB1_2                  ## encoding: [0x75,A]
                                        ##   fixup A - offset: 1, value: LBB1_2-1, kind: FK_PCRel_1
LBB1_3:                                 ## %for.end
	addl	$16, %esp               ## encoding: [0x83,0xc4,0x10]
	popl	%esi                    ## encoding: [0x5e]
	popl	%edi                    ## encoding: [0x5f]
	popl	%ebp                    ## encoding: [0x5d]
	retl                            ## encoding: [0xc3]
	.cfi_endproc

So in this case we get an improvement of 2 less instructions per function and 7 less bytes per function.

• The same sort of improvements are obtained with the -O3 setting and the remaining ones.

Please let me know if you need me to perform some specific test.

Thanks,
[Edited for grammar typos]

[joanlluch]

@craig.topper , can this be considered ok for the X86?. Thanks.

lib/Transforms/Scalar/LoopStrengthReduce.cpp

if (isAddressUse(TTI, UserInst, U.getOperandValToReplace())) {
       Kind = LSRUse::Address;
       AccessTy = getAccessType(TTI, UserInst, U.getOperandValToReplace());
     }
 
     const SCEV *S = IU.getExpr(U);
     PostIncLoopSet TmpPostIncLoops = U.getPostIncLoops();
 
     // Equality (== and !=) ICmps are special. We can rewrite (i == N) as
-    // (N - i == 0), and this allows (N - i) to be the expression that we work
-    // with rather than just N or i, so we can consider the register
-    // requirements for both N and i at the same time. Limiting this code to
+    // (i - N == 0), and this allows (i - N) to be the expression that we work
+    // with rather than just i or N, so we can consider the register
+    // requirements for both i and N at the same time. Limiting this code to
     // equality icmps is not a problem because all interesting loops use
     // equality icmps, thanks to IndVarSimplify.
     if (ICmpInst *CI = dyn_cast<ICmpInst>(UserInst))
       if (CI->isEquality()) {
         // Swap the operands if needed to put the OperandValToReplace on the
         // left, for consistency.
         Value *NV = CI->getOperand(1);
         if (NV == U.getOperandValToReplace()) {
           CI->setOperand(1, CI->getOperand(0));
           CI->setOperand(0, NV);
           NV = CI->getOperand(1);
           Changed = true;
         }
 
         // x == y  -->  x - y == 0
         const SCEV *N = SE.getSCEV(NV);
         if (SE.isLoopInvariant(N, L) && isSafeToExpand(N, SE)) {
           // S is normalized, so normalize N before folding it into S
           // to keep the result normalized.
           N = normalizeForPostIncUse(N, TmpPostIncLoops, SE);
           Kind = LSRUse::ICmpZero;
-          S = SE.getMinusSCEV(N, S);
+          S = SE.getMinusSCEV(S, N);
         }
 
         // -1 and the negations of all interesting strides (except the negation
         // of -1) are now also interesting.
         for (size_t i = 0, e = Factors.size(); i != e; ++i)
           if (Factors[i] != -1)
             Factors.insert(-(uint64_t)Factors[i]);
         Factors.insert(-1);
if (!LU.WidestFixupType ||
         SE.getTypeSizeInBits(LU.WidestFixupType) <
         SE.getTypeSizeInBits(LF.OperandValToReplace->getType()))
       LU.WidestFixupType = LF.OperandValToReplace->getType();
 
     // If this is the first use of this LSRUse, give it a formula.
     if (LU.Formulae.empty()) {
       InsertInitialFormula(S, LU, LUIdx);
       CountRegisters(LU.Formulae.back(), LUIdx);
+      if ( LU.Kind == LSRUse::ICmpZero && isSafeToExpand(S, SE)) {
+        // Account for the negated ICmpZero SCEV
+        const SCEV *Z = SE.getMinusSCEV(SE.getZero(S->getType()), S);
+        InsertSupplementalFormula(Z, LU, LUIdx);
+        CountRegisters(LU.Formulae.back(), LUIdx);
+      }
     }
   }
 
   LLVM_DEBUG(print_fixups(dbgs()));
 }
 
 /// Insert a formula for the given expression into the given use, separating out
 /// loop-variant portions from loop-invariant and loop-computable portions.

test/CodeGen/ARM/arm-shrink-wrapping.ll

 ; ARM-ENABLE-LABEL: freqSaveAndRestoreOutsideLoop:
 ; ARM-ENABLE:       @ %bb.0: @ %entry
 ; ARM-ENABLE-NEXT:    cmp r0, #0
 ; ARM-ENABLE-NEXT:    beq LBB1_4
 ; ARM-ENABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; ARM-ENABLE-NEXT:    push {r4, r7, lr}
 ; ARM-ENABLE-NEXT:    add r7, sp, #4
 ; ARM-ENABLE-NEXT:    mov r0, #0
-; ARM-ENABLE-NEXT:    mov r1, #10
+; ARM-ENABLE-NEXT:    mvn r1, #9
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    nop
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:  LBB1_2: @ %for.body
 ; ARM-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    mov r2, #1
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:    add r0, r2, r0
-; ARM-ENABLE-NEXT:    subs r1, r1, #1
-; ARM-ENABLE-NEXT:    bne LBB1_2
+; ARM-ENABLE-NEXT:    adds r1, r1, #1
+; ARM-ENABLE-NEXT:    blo LBB1_2
 ; ARM-ENABLE-NEXT:  @ %bb.3: @ %for.end
 ; ARM-ENABLE-NEXT:    lsl r0, r0, #3
 ; ARM-ENABLE-NEXT:    pop {r4, r7, pc}
 ; ARM-ENABLE-NEXT:  LBB1_4: @ %if.else
 ; ARM-ENABLE-NEXT:    lsl r0, r1, #1
 ; ARM-ENABLE-NEXT:    bx lr
 ;
 ; ARM-DISABLE-LABEL: freqSaveAndRestoreOutsideLoop:
 ; ARM-DISABLE:       @ %bb.0: @ %entry
 ; ARM-DISABLE-NEXT:    push {r4, r7, lr}
 ; ARM-DISABLE-NEXT:    add r7, sp, #4
 ; ARM-DISABLE-NEXT:    cmp r0, #0
 ; ARM-DISABLE-NEXT:    beq LBB1_4
 ; ARM-DISABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; ARM-DISABLE-NEXT:    mov r0, #0
-; ARM-DISABLE-NEXT:    mov r1, #10
+; ARM-DISABLE-NEXT:    mvn r1, #9
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    nop
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:  LBB1_2: @ %for.body
 ; ARM-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    mov r2, #1
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:    add r0, r2, r0
-; ARM-DISABLE-NEXT:    subs r1, r1, #1
-; ARM-DISABLE-NEXT:    bne LBB1_2
+; ARM-DISABLE-NEXT:    adds r1, r1, #1
+; ARM-DISABLE-NEXT:    blo LBB1_2
 ; ARM-DISABLE-NEXT:  @ %bb.3: @ %for.end
 ; ARM-DISABLE-NEXT:    lsl r0, r0, #3
 ; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
 ; ARM-DISABLE-NEXT:  LBB1_4: @ %if.else
 ; ARM-DISABLE-NEXT:    lsl r0, r1, #1
 ; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
 ;
 ; THUMB-ENABLE-LABEL: freqSaveAndRestoreOutsideLoop:
 ; THUMB-ENABLE:       @ %bb.0: @ %entry
 ; THUMB-ENABLE-NEXT:    cbz r0, LBB1_4
 ; THUMB-ENABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; THUMB-ENABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-ENABLE-NEXT:    add r7, sp, #4
 ; THUMB-ENABLE-NEXT:    movs r0, #0
-; THUMB-ENABLE-NEXT:    movs r1, #10
+; THUMB-ENABLE-NEXT:    mvn r1, #9
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    nop
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:  LBB1_2: @ %for.body
 ; THUMB-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    mov.w r2, #1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:    add r0, r2
-; THUMB-ENABLE-NEXT:    subs r1, #1
-; THUMB-ENABLE-NEXT:    bne LBB1_2
+; THUMB-ENABLE-NEXT:    adds r1, #1
+; THUMB-ENABLE-NEXT:    blo LBB1_2
 ; THUMB-ENABLE-NEXT:  @ %bb.3: @ %for.end
 ; THUMB-ENABLE-NEXT:    lsls r0, r0, #3
 ; THUMB-ENABLE-NEXT:    pop {r4, r7, pc}
 ; THUMB-ENABLE-NEXT:  LBB1_4: @ %if.else
 ; THUMB-ENABLE-NEXT:    lsls r0, r1, #1
 ; THUMB-ENABLE-NEXT:    bx lr
 ;
 ; THUMB-DISABLE-LABEL: freqSaveAndRestoreOutsideLoop:
 ; THUMB-DISABLE:       @ %bb.0: @ %entry
 ; THUMB-DISABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-DISABLE-NEXT:    add r7, sp, #4
 ; THUMB-DISABLE-NEXT:    cbz r0, LBB1_4
 ; THUMB-DISABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; THUMB-DISABLE-NEXT:    movs r0, #0
-; THUMB-DISABLE-NEXT:    movs r1, #10
+; THUMB-DISABLE-NEXT:    mvn r1, #9
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    nop
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:  LBB1_2: @ %for.body
 ; THUMB-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    mov.w r2, #1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:    add r0, r2
-; THUMB-DISABLE-NEXT:    subs r1, #1
-; THUMB-DISABLE-NEXT:    bne LBB1_2
+; THUMB-DISABLE-NEXT:    adds r1, #1
+; THUMB-DISABLE-NEXT:    blo LBB1_2
 ; THUMB-DISABLE-NEXT:  @ %bb.3: @ %for.end
 ; THUMB-DISABLE-NEXT:    lsls r0, r0, #3
 ; THUMB-DISABLE-NEXT:    pop {r4, r7, pc}
 ; THUMB-DISABLE-NEXT:  LBB1_4: @ %if.else
 ; THUMB-DISABLE-NEXT:    lsls r0, r1, #1
 ; THUMB-DISABLE-NEXT:    pop {r4, r7, pc}
 entry:
   %tobool = icmp eq i32 %cond, 0

 ; subs [[IV]], [[IV]], #1
 ; subs [[IV]], #1
 ; bne [[LOOP_LABEL]]
 ; Next BB.
 ; @ %for.exit
 ; nop
 ; pop {r4
 define i32 @freqSaveAndRestoreOutsideLoop2(i32 %cond) "no-frame-pointer-elim"="true" {
-; ARM-LABEL: freqSaveAndRestoreOutsideLoop2:
-; ARM:       @ %bb.0: @ %entry
-; ARM-NEXT:    push {r4, r7, lr}
-; ARM-NEXT:    add r7, sp, #4
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    mov r1, #10
-; ARM-NEXT:    @ InlineAsm Start
-; ARM-NEXT:    nop
-; ARM-NEXT:    @ InlineAsm End
-; ARM-NEXT:  LBB2_1: @ %for.body
-; ARM-NEXT:    @ =>This Inner Loop Header: Depth=1
-; ARM-NEXT:    @ InlineAsm Start
-; ARM-NEXT:    mov r2, #1
-; ARM-NEXT:    @ InlineAsm End
-; ARM-NEXT:    add r0, r2, r0
-; ARM-NEXT:    subs r1, r1, #1
-; ARM-NEXT:    bne LBB2_1
-; ARM-NEXT:  @ %bb.2: @ %for.exit
-; ARM-NEXT:    @ InlineAsm Start
-; ARM-NEXT:    nop
-; ARM-NEXT:    @ InlineAsm End
-; ARM-NEXT:    pop {r4, r7, pc}
-;
-; THUMB-LABEL: freqSaveAndRestoreOutsideLoop2:
-; THUMB:       @ %bb.0: @ %entry
-; THUMB-NEXT:    push {r4, r7, lr}
-; THUMB-NEXT:    add r7, sp, #4
-; THUMB-NEXT:    movs r0, #0
-; THUMB-NEXT:    movs r1, #10
-; THUMB-NEXT:    @ InlineAsm Start
-; THUMB-NEXT:    nop
-; THUMB-NEXT:    @ InlineAsm End
-; THUMB-NEXT:  LBB2_1: @ %for.body
-; THUMB-NEXT:    @ =>This Inner Loop Header: Depth=1
-; THUMB-NEXT:    @ InlineAsm Start
-; THUMB-NEXT:    mov.w r2, #1
-; THUMB-NEXT:    @ InlineAsm End
-; THUMB-NEXT:    add r0, r2
-; THUMB-NEXT:    subs r1, #1
-; THUMB-NEXT:    bne LBB2_1
-; THUMB-NEXT:  @ %bb.2: @ %for.exit
-; THUMB-NEXT:    @ InlineAsm Start
-; THUMB-NEXT:    nop
-; THUMB-NEXT:    @ InlineAsm End
-; THUMB-NEXT:    pop {r4, r7, pc}
 ; ARM-ENABLE-LABEL: freqSaveAndRestoreOutsideLoop2:
 ; ARM-ENABLE:       @ %bb.0: @ %entry
 ; ARM-ENABLE-NEXT:    push {r4, r7, lr}
 ; ARM-ENABLE-NEXT:    add r7, sp, #4
 ; ARM-ENABLE-NEXT:    mov r0, #0
-; ARM-ENABLE-NEXT:    mov r1, #10
+; ARM-ENABLE-NEXT:    mvn r1, #9
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    nop
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:  LBB2_1: @ %for.body
 ; ARM-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    mov r2, #1
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:    add r0, r2, r0
-; ARM-ENABLE-NEXT:    subs r1, r1, #1
-; ARM-ENABLE-NEXT:    bne LBB2_1
+; ARM-ENABLE-NEXT:    adds r1, r1, #1
+; ARM-ENABLE-NEXT:    blo LBB2_1
 ; ARM-ENABLE-NEXT:  @ %bb.2: @ %for.exit
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    nop
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:    pop {r4, r7, pc}
 ;
 ; ARM-DISABLE-LABEL: freqSaveAndRestoreOutsideLoop2:
 ; ARM-DISABLE:       @ %bb.0: @ %entry
 ; ARM-DISABLE-NEXT:    push {r4, r7, lr}
 ; ARM-DISABLE-NEXT:    add r7, sp, #4
 ; ARM-DISABLE-NEXT:    mov r0, #0
-; ARM-DISABLE-NEXT:    mov r1, #10
+; ARM-DISABLE-NEXT:    mvn r1, #9
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    nop
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:  LBB2_1: @ %for.body
 ; ARM-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    mov r2, #1
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:    add r0, r2, r0
-; ARM-DISABLE-NEXT:    subs r1, r1, #1
-; ARM-DISABLE-NEXT:    bne LBB2_1
+; ARM-DISABLE-NEXT:    adds r1, r1, #1
+; ARM-DISABLE-NEXT:    blo LBB2_1
 ; ARM-DISABLE-NEXT:  @ %bb.2: @ %for.exit
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    nop
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
 ;
 ; THUMB-ENABLE-LABEL: freqSaveAndRestoreOutsideLoop2:
 ; THUMB-ENABLE:       @ %bb.0: @ %entry
 ; THUMB-ENABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-ENABLE-NEXT:    add r7, sp, #4
 ; THUMB-ENABLE-NEXT:    movs r0, #0
-; THUMB-ENABLE-NEXT:    movs r1, #10
+; THUMB-ENABLE-NEXT:    mvn r1, #9
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    nop
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:  LBB2_1: @ %for.body
 ; THUMB-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    mov.w r2, #1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:    add r0, r2
-; THUMB-ENABLE-NEXT:    subs r1, #1
-; THUMB-ENABLE-NEXT:    bne LBB2_1
+; THUMB-ENABLE-NEXT:    adds r1, #1
+; THUMB-ENABLE-NEXT:    blo LBB2_1
 ; THUMB-ENABLE-NEXT:  @ %bb.2: @ %for.exit
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    nop
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:    pop {r4, r7, pc}
 ;
 ; THUMB-DISABLE-LABEL: freqSaveAndRestoreOutsideLoop2:
 ; THUMB-DISABLE:       @ %bb.0: @ %entry
 ; THUMB-DISABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-DISABLE-NEXT:    add r7, sp, #4
 ; THUMB-DISABLE-NEXT:    movs r0, #0
-; THUMB-DISABLE-NEXT:    movs r1, #10
+; THUMB-DISABLE-NEXT:    mvn r1, #9
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    nop
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:  LBB2_1: @ %for.body
 ; THUMB-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    mov.w r2, #1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:    add r0, r2
-; THUMB-DISABLE-NEXT:    subs r1, #1
-; THUMB-DISABLE-NEXT:    bne LBB2_1
+; THUMB-DISABLE-NEXT:    adds r1, #1
+; THUMB-DISABLE-NEXT:    blo LBB2_1
 ; THUMB-DISABLE-NEXT:  @ %bb.2: @ %for.exit
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    nop
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:    pop {r4, r7, pc}
 entry:
   br label %for.preheader
 

 ; ARM-ENABLE-LABEL: loopInfoSaveOutsideLoop:
 ; ARM-ENABLE:       @ %bb.0: @ %entry
 ; ARM-ENABLE-NEXT:    cmp r0, #0
 ; ARM-ENABLE-NEXT:    beq LBB3_4
 ; ARM-ENABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; ARM-ENABLE-NEXT:    push {r4, r7, lr}
 ; ARM-ENABLE-NEXT:    add r7, sp, #4
 ; ARM-ENABLE-NEXT:    mov r0, #0
-; ARM-ENABLE-NEXT:    mov r1, #10
+; ARM-ENABLE-NEXT:    mvn r1, #9
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    nop
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:  LBB3_2: @ %for.body
 ; ARM-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    mov r2, #1
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:    add r0, r2, r0
-; ARM-ENABLE-NEXT:    subs r1, r1, #1
-; ARM-ENABLE-NEXT:    bne LBB3_2
+; ARM-ENABLE-NEXT:    adds r1, r1, #1
+; ARM-ENABLE-NEXT:    blo LBB3_2
 ; ARM-ENABLE-NEXT:  @ %bb.3: @ %for.end
 ; ARM-ENABLE-NEXT:    lsl r0, r0, #3
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    nop
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:    pop {r4, r7, pc}
 ; ARM-ENABLE-NEXT:  LBB3_4: @ %if.else
 ; ARM-ENABLE-NEXT:    lsl r0, r1, #1
 ; ARM-ENABLE-NEXT:    bx lr
 ;
 ; ARM-DISABLE-LABEL: loopInfoSaveOutsideLoop:
 ; ARM-DISABLE:       @ %bb.0: @ %entry
 ; ARM-DISABLE-NEXT:    push {r4, r7, lr}
 ; ARM-DISABLE-NEXT:    add r7, sp, #4
 ; ARM-DISABLE-NEXT:    cmp r0, #0
 ; ARM-DISABLE-NEXT:    beq LBB3_4
 ; ARM-DISABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; ARM-DISABLE-NEXT:    mov r0, #0
-; ARM-DISABLE-NEXT:    mov r1, #10
+; ARM-DISABLE-NEXT:    mvn r1, #9
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    nop
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:  LBB3_2: @ %for.body
 ; ARM-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    mov r2, #1
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:    add r0, r2, r0
-; ARM-DISABLE-NEXT:    subs r1, r1, #1
-; ARM-DISABLE-NEXT:    bne LBB3_2
+; ARM-DISABLE-NEXT:    adds r1, r1, #1
+; ARM-DISABLE-NEXT:    blo LBB3_2
 ; ARM-DISABLE-NEXT:  @ %bb.3: @ %for.end
 ; ARM-DISABLE-NEXT:    lsl r0, r0, #3
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    nop
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
 ; ARM-DISABLE-NEXT:  LBB3_4: @ %if.else
 ; ARM-DISABLE-NEXT:    lsl r0, r1, #1
 ; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
 ;
 ; THUMB-ENABLE-LABEL: loopInfoSaveOutsideLoop:
 ; THUMB-ENABLE:       @ %bb.0: @ %entry
 ; THUMB-ENABLE-NEXT:    cbz r0, LBB3_4
 ; THUMB-ENABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; THUMB-ENABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-ENABLE-NEXT:    add r7, sp, #4
 ; THUMB-ENABLE-NEXT:    movs r0, #0
-; THUMB-ENABLE-NEXT:    movs r1, #10
+; THUMB-ENABLE-NEXT:    mvn r1, #9
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    nop
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:  LBB3_2: @ %for.body
 ; THUMB-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    mov.w r2, #1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:    add r0, r2
-; THUMB-ENABLE-NEXT:    subs r1, #1
-; THUMB-ENABLE-NEXT:    bne LBB3_2
+; THUMB-ENABLE-NEXT:    adds r1, #1
+; THUMB-ENABLE-NEXT:    blo LBB3_2
 ; THUMB-ENABLE-NEXT:  @ %bb.3: @ %for.end
 ; THUMB-ENABLE-NEXT:    lsls r0, r0, #3
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    nop
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:    pop {r4, r7, pc}
 ; THUMB-ENABLE-NEXT:  LBB3_4: @ %if.else
 ; THUMB-ENABLE-NEXT:    lsls r0, r1, #1
 ; THUMB-ENABLE-NEXT:    bx lr
 ;
 ; THUMB-DISABLE-LABEL: loopInfoSaveOutsideLoop:
 ; THUMB-DISABLE:       @ %bb.0: @ %entry
 ; THUMB-DISABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-DISABLE-NEXT:    add r7, sp, #4
 ; THUMB-DISABLE-NEXT:    cbz r0, LBB3_4
 ; THUMB-DISABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; THUMB-DISABLE-NEXT:    movs r0, #0
-; THUMB-DISABLE-NEXT:    movs r1, #10
+; THUMB-DISABLE-NEXT:    mvn r1, #9
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    nop
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:  LBB3_2: @ %for.body
 ; THUMB-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    mov.w r2, #1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:    add r0, r2
-; THUMB-DISABLE-NEXT:    subs r1, #1
-; THUMB-DISABLE-NEXT:    bne LBB3_2
+; THUMB-DISABLE-NEXT:    adds r1, #1
+; THUMB-DISABLE-NEXT:    blo LBB3_2
 ; THUMB-DISABLE-NEXT:  @ %bb.3: @ %for.end
 ; THUMB-DISABLE-NEXT:    lsls r0, r0, #3
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    nop
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:    pop {r4, r7, pc}
 ; THUMB-DISABLE-NEXT:  LBB3_4: @ %if.else
 ; THUMB-DISABLE-NEXT:    lsls r0, r1, #1

 ; ARM-ENABLE-LABEL: loopInfoRestoreOutsideLoop:
 ; ARM-ENABLE:       @ %bb.0: @ %entry
 ; ARM-ENABLE-NEXT:    cmp r0, #0
 ; ARM-ENABLE-NEXT:    beq LBB4_4
 ; ARM-ENABLE-NEXT:  @ %bb.1: @ %if.then
 ; ARM-ENABLE-NEXT:    push {r4, r7, lr}
 ; ARM-ENABLE-NEXT:    add r7, sp, #4
 ; ARM-ENABLE-NEXT:    mov r0, #0
-; ARM-ENABLE-NEXT:    mov r1, #10
+; ARM-ENABLE-NEXT:    mvn r1, #9
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    nop
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:  LBB4_2: @ %for.body
 ; ARM-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    mov r2, #1
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:    add r0, r2, r0
-; ARM-ENABLE-NEXT:    subs r1, r1, #1
-; ARM-ENABLE-NEXT:    bne LBB4_2
+; ARM-ENABLE-NEXT:    adds r1, r1, #1
+; ARM-ENABLE-NEXT:    blo LBB4_2
 ; ARM-ENABLE-NEXT:  @ %bb.3: @ %for.end
 ; ARM-ENABLE-NEXT:    lsl r0, r0, #3
 ; ARM-ENABLE-NEXT:    pop {r4, r7, pc}
 ; ARM-ENABLE-NEXT:  LBB4_4: @ %if.else
 ; ARM-ENABLE-NEXT:    lsl r0, r1, #1
 ; ARM-ENABLE-NEXT:    bx lr
 ;
 ; ARM-DISABLE-LABEL: loopInfoRestoreOutsideLoop:
 ; ARM-DISABLE:       @ %bb.0: @ %entry
 ; ARM-DISABLE-NEXT:    push {r4, r7, lr}
 ; ARM-DISABLE-NEXT:    add r7, sp, #4
 ; ARM-DISABLE-NEXT:    cmp r0, #0
 ; ARM-DISABLE-NEXT:    beq LBB4_4
 ; ARM-DISABLE-NEXT:  @ %bb.1: @ %if.then
 ; ARM-DISABLE-NEXT:    mov r0, #0
-; ARM-DISABLE-NEXT:    mov r1, #10
+; ARM-DISABLE-NEXT:    mvn r1, #9
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    nop
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:  LBB4_2: @ %for.body
 ; ARM-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    mov r2, #1
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:    add r0, r2, r0
-; ARM-DISABLE-NEXT:    subs r1, r1, #1
-; ARM-DISABLE-NEXT:    bne LBB4_2
+; ARM-DISABLE-NEXT:    adds r1, r1, #1
+; ARM-DISABLE-NEXT:    blo LBB4_2
 ; ARM-DISABLE-NEXT:  @ %bb.3: @ %for.end
 ; ARM-DISABLE-NEXT:    lsl r0, r0, #3
 ; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
 ; ARM-DISABLE-NEXT:  LBB4_4: @ %if.else
 ; ARM-DISABLE-NEXT:    lsl r0, r1, #1
 ; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
 ;
 ; THUMB-ENABLE-LABEL: loopInfoRestoreOutsideLoop:
 ; THUMB-ENABLE:       @ %bb.0: @ %entry
 ; THUMB-ENABLE-NEXT:    cbz r0, LBB4_4
 ; THUMB-ENABLE-NEXT:  @ %bb.1: @ %if.then
 ; THUMB-ENABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-ENABLE-NEXT:    add r7, sp, #4
 ; THUMB-ENABLE-NEXT:    movs r0, #0
-; THUMB-ENABLE-NEXT:    movs r1, #10
+; THUMB-ENABLE-NEXT:    mvn r1, #9
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    nop
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:  LBB4_2: @ %for.body
 ; THUMB-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    mov.w r2, #1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:    add r0, r2
-; THUMB-ENABLE-NEXT:    subs r1, #1
-; THUMB-ENABLE-NEXT:    bne LBB4_2
+; THUMB-ENABLE-NEXT:    adds r1, #1
+; THUMB-ENABLE-NEXT:    blo LBB4_2
 ; THUMB-ENABLE-NEXT:  @ %bb.3: @ %for.end
 ; THUMB-ENABLE-NEXT:    lsls r0, r0, #3
 ; THUMB-ENABLE-NEXT:    pop {r4, r7, pc}
 ; THUMB-ENABLE-NEXT:  LBB4_4: @ %if.else
 ; THUMB-ENABLE-NEXT:    lsls r0, r1, #1
 ; THUMB-ENABLE-NEXT:    bx lr
 ;
 ; THUMB-DISABLE-LABEL: loopInfoRestoreOutsideLoop:
 ; THUMB-DISABLE:       @ %bb.0: @ %entry
 ; THUMB-DISABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-DISABLE-NEXT:    add r7, sp, #4
 ; THUMB-DISABLE-NEXT:    cbz r0, LBB4_4
 ; THUMB-DISABLE-NEXT:  @ %bb.1: @ %if.then
 ; THUMB-DISABLE-NEXT:    movs r0, #0
-; THUMB-DISABLE-NEXT:    movs r1, #10
+; THUMB-DISABLE-NEXT:    mvn r1, #9
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    nop
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:  LBB4_2: @ %for.body
 ; THUMB-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    mov.w r2, #1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:    add r0, r2
-; THUMB-DISABLE-NEXT:    subs r1, #1
-; THUMB-DISABLE-NEXT:    bne LBB4_2
+; THUMB-DISABLE-NEXT:    adds r1, #1
+; THUMB-DISABLE-NEXT:    blo LBB4_2
 ; THUMB-DISABLE-NEXT:  @ %bb.3: @ %for.end
 ; THUMB-DISABLE-NEXT:    lsls r0, r0, #3
 ; THUMB-DISABLE-NEXT:    pop {r4, r7, pc}
 ; THUMB-DISABLE-NEXT:  LBB4_4: @ %if.else
 ; THUMB-DISABLE-NEXT:    lsls r0, r1, #1
 ; THUMB-DISABLE-NEXT:    pop {r4, r7, pc}
 entry:
   %tobool = icmp eq i32 %cond, 0

 }
 
 ; Check that we handle function with no frame information correctly.
 ; emptyFrame:
 ; @ %entry
 ; mov{{s?}} r0, #0
 ; bx lr
 define i32 @emptyFrame() {
-; ARM-LABEL: emptyFrame:
-; ARM:       @ %bb.0: @ %entry
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    bx lr
-;
-; THUMB-LABEL: emptyFrame:
-; THUMB:       @ %bb.0: @ %entry
-; THUMB-NEXT:    movs r0, #0
-; THUMB-NEXT:    bx lr
 ; ARM-ENABLE-LABEL: emptyFrame:
 ; ARM-ENABLE:       @ %bb.0: @ %entry
 ; ARM-ENABLE-NEXT:    mov r0, #0
 ; ARM-ENABLE-NEXT:    bx lr
 ;
 ; ARM-DISABLE-LABEL: emptyFrame:
 ; ARM-DISABLE:       @ %bb.0: @ %entry
 ; ARM-DISABLE-NEXT:    mov r0, #0

 define i32 @inlineAsm(i32 %cond, i32 %N) "no-frame-pointer-elim"="true" {
 ; ARM-ENABLE-LABEL: inlineAsm:
 ; ARM-ENABLE:       @ %bb.0: @ %entry
 ; ARM-ENABLE-NEXT:    cmp r0, #0
 ; ARM-ENABLE-NEXT:    beq LBB6_4
 ; ARM-ENABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; ARM-ENABLE-NEXT:    push {r4, r7, lr}
 ; ARM-ENABLE-NEXT:    add r7, sp, #4
-; ARM-ENABLE-NEXT:    mov r0, #10
+; ARM-ENABLE-NEXT:    mvn r0, #9
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    nop
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:  LBB6_2: @ %for.body
 ; ARM-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
-; ARM-ENABLE-NEXT:    subs r0, r0, #1
+; ARM-ENABLE-NEXT:    adds r0, r0, #1
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    add r4, r4, #1
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
-; ARM-ENABLE-NEXT:    bne LBB6_2
+; ARM-ENABLE-NEXT:    blo LBB6_2
 ; ARM-ENABLE-NEXT:  @ %bb.3: @ %for.exit
 ; ARM-ENABLE-NEXT:    mov r0, #0
 ; ARM-ENABLE-NEXT:    @ InlineAsm Start
 ; ARM-ENABLE-NEXT:    nop
 ; ARM-ENABLE-NEXT:    @ InlineAsm End
 ; ARM-ENABLE-NEXT:    pop {r4, r7, pc}
 ; ARM-ENABLE-NEXT:  LBB6_4: @ %if.else
 ; ARM-ENABLE-NEXT:    lsl r0, r1, #1
 ; ARM-ENABLE-NEXT:    bx lr
 ;
 ; ARM-DISABLE-LABEL: inlineAsm:
 ; ARM-DISABLE:       @ %bb.0: @ %entry
 ; ARM-DISABLE-NEXT:    push {r4, r7, lr}
 ; ARM-DISABLE-NEXT:    add r7, sp, #4
 ; ARM-DISABLE-NEXT:    cmp r0, #0
 ; ARM-DISABLE-NEXT:    beq LBB6_4
 ; ARM-DISABLE-NEXT:  @ %bb.1: @ %for.preheader
-; ARM-DISABLE-NEXT:    mov r0, #10
+; ARM-DISABLE-NEXT:    mvn r0, #9
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    nop
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:  LBB6_2: @ %for.body
 ; ARM-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
-; ARM-DISABLE-NEXT:    subs r0, r0, #1
+; ARM-DISABLE-NEXT:    adds r0, r0, #1
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    add r4, r4, #1
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
-; ARM-DISABLE-NEXT:    bne LBB6_2
+; ARM-DISABLE-NEXT:    blo LBB6_2
 ; ARM-DISABLE-NEXT:  @ %bb.3: @ %for.exit
 ; ARM-DISABLE-NEXT:    mov r0, #0
 ; ARM-DISABLE-NEXT:    @ InlineAsm Start
 ; ARM-DISABLE-NEXT:    nop
 ; ARM-DISABLE-NEXT:    @ InlineAsm End
 ; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
 ; ARM-DISABLE-NEXT:  LBB6_4: @ %if.else
 ; ARM-DISABLE-NEXT:    lsl r0, r1, #1
 ; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
 ;
 ; THUMB-ENABLE-LABEL: inlineAsm:
 ; THUMB-ENABLE:       @ %bb.0: @ %entry
 ; THUMB-ENABLE-NEXT:    cbz r0, LBB6_4
 ; THUMB-ENABLE-NEXT:  @ %bb.1: @ %for.preheader
 ; THUMB-ENABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-ENABLE-NEXT:    add r7, sp, #4
-; THUMB-ENABLE-NEXT:    movs r0, #10
+; THUMB-ENABLE-NEXT:    mvn r0, #9
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    nop
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:  LBB6_2: @ %for.body
 ; THUMB-ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
-; THUMB-ENABLE-NEXT:    subs r0, #1
+; THUMB-ENABLE-NEXT:    adds r0, #1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    add.w r4, r4, #1
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
-; THUMB-ENABLE-NEXT:    bne LBB6_2
+; THUMB-ENABLE-NEXT:    blo LBB6_2
 ; THUMB-ENABLE-NEXT:  @ %bb.3: @ %for.exit
 ; THUMB-ENABLE-NEXT:    movs r0, #0
 ; THUMB-ENABLE-NEXT:    @ InlineAsm Start
 ; THUMB-ENABLE-NEXT:    nop
 ; THUMB-ENABLE-NEXT:    @ InlineAsm End
 ; THUMB-ENABLE-NEXT:    pop {r4, r7, pc}
 ; THUMB-ENABLE-NEXT:  LBB6_4: @ %if.else
 ; THUMB-ENABLE-NEXT:    lsls r0, r1, #1
 ; THUMB-ENABLE-NEXT:    bx lr
 ;
 ; THUMB-DISABLE-LABEL: inlineAsm:
 ; THUMB-DISABLE:       @ %bb.0: @ %entry
 ; THUMB-DISABLE-NEXT:    push {r4, r7, lr}
 ; THUMB-DISABLE-NEXT:    add r7, sp, #4
 ; THUMB-DISABLE-NEXT:    cbz r0, LBB6_4
 ; THUMB-DISABLE-NEXT:  @ %bb.1: @ %for.preheader
-; THUMB-DISABLE-NEXT:    movs r0, #10
+; THUMB-DISABLE-NEXT:    mvn r0, #9
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    nop
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:  LBB6_2: @ %for.body
 ; THUMB-DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
-; THUMB-DISABLE-NEXT:    subs r0, #1
+; THUMB-DISABLE-NEXT:    adds r0, #1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    add.w r4, r4, #1
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
-; THUMB-DISABLE-NEXT:    bne LBB6_2
+; THUMB-DISABLE-NEXT:    blo LBB6_2
 ; THUMB-DISABLE-NEXT:  @ %bb.3: @ %for.exit
 ; THUMB-DISABLE-NEXT:    movs r0, #0
 ; THUMB-DISABLE-NEXT:    @ InlineAsm Start
 ; THUMB-DISABLE-NEXT:    nop
 ; THUMB-DISABLE-NEXT:    @ InlineAsm End
 ; THUMB-DISABLE-NEXT:    pop {r4, r7, pc}
 ; THUMB-DISABLE-NEXT:  LBB6_4: @ %if.else
 ; THUMB-DISABLE-NEXT:    lsls r0, r1, #1

 ; immediate (post) dominator for the (restore) save blocks. When either the Save
 ; or Restore block is located in an infinite loop the only immediate (post)
 ; dominator is itself. In this case, we cannot perform shrink wrapping, but we
 ; should return gracefully and continue compilation.
 ; The only condition for this test is the compilation finishes correctly.
 ; infiniteloop
 ; pop
 define void @infiniteloop() "no-frame-pointer-elim"="true" {
-; ARM-LABEL: infiniteloop:
-; ARM:       @ %bb.0: @ %entry
-; ARM-NEXT:    push {r4, r5, r7, lr}
-; ARM-NEXT:    add r7, sp, #8
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    cmp r0, #0
-; ARM-NEXT:    bne LBB9_3
-; ARM-NEXT:  @ %bb.1: @ %if.then
-; ARM-NEXT:    sub r1, sp, #16
-; ARM-NEXT:    mov sp, r1
-; ARM-NEXT:  LBB9_2: @ %for.body
-; ARM-NEXT:    @ =>This Inner Loop Header: Depth=1
-; ARM-NEXT:    @ InlineAsm Start
-; ARM-NEXT:    mov r2, #1
-; ARM-NEXT:    @ InlineAsm End
-; ARM-NEXT:    add r0, r2, r0
-; ARM-NEXT:    str r0, [r1]
-; ARM-NEXT:    b LBB9_2
-; ARM-NEXT:  LBB9_3: @ %if.end
-; ARM-NEXT:    sub sp, r7, #8
-; ARM-NEXT:    pop {r4, r5, r7, pc}
-;
-; THUMB-LABEL: infiniteloop:
-; THUMB:       @ %bb.0: @ %entry
-; THUMB-NEXT:    push {r4, r5, r7, lr}
-; THUMB-NEXT:    add r7, sp, #8
-; THUMB-NEXT:    movs r0, #0
-; THUMB-NEXT:    cbnz r0, LBB9_3
-; THUMB-NEXT:  @ %bb.1: @ %if.then
-; THUMB-NEXT:    sub.w r0, sp, #16
-; THUMB-NEXT:    mov sp, r0
-; THUMB-NEXT:    movs r1, #0
-; THUMB-NEXT:  LBB9_2: @ %for.body
-; THUMB-NEXT:    @ =>This Inner Loop Header: Depth=1
-; THUMB-NEXT:    @ InlineAsm Start
-; THUMB-NEXT:    mov.w r2, #1
-; THUMB-NEXT:    @ InlineAsm End
-; THUMB-NEXT:    add r1, r2
-; THUMB-NEXT:    str r1, [r0]
-; THUMB-NEXT:    b LBB9_2
-; THUMB-NEXT:  LBB9_3: @ %if.end
-; THUMB-NEXT:    sub.w r4, r7, #8
-; THUMB-NEXT:    mov sp, r4
-; THUMB-NEXT:    pop {r4, r5, r7, pc}
 ; ARM-ENABLE-LABEL: infiniteloop:
 ; ARM-ENABLE:       @ %bb.0: @ %entry
 ; ARM-ENABLE-NEXT:    push {r4, r5, r7, lr}
 ; ARM-ENABLE-NEXT:    add r7, sp, #8
 ; ARM-ENABLE-NEXT:    mov r0, #0
 ; ARM-ENABLE-NEXT:    cmp r0, #0
 ; ARM-ENABLE-NEXT:    bne LBB9_3
 ; ARM-ENABLE-NEXT:  @ %bb.1: @ %if.then

 if.end:
   ret void
 }
 
 ; Another infinite loop test this time with two nested infinite loop.
 ; infiniteloop3
 ; bx lr
 define void @infiniteloop3() "no-frame-pointer-elim"="true" {
-; ARM-LABEL: infiniteloop3:
-; ARM:       @ %bb.0: @ %entry
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    cmp r0, #0
-; ARM-NEXT:    bne LBB11_5
-; ARM-NEXT:  @ %bb.1: @ %loop2a.preheader
-; ARM-NEXT:    mov r1, #0
-; ARM-NEXT:    mov r2, r0
-; ARM-NEXT:    b LBB11_3
-; ARM-NEXT:  LBB11_2: @ %loop2b
-; ARM-NEXT:    @ in Loop: Header=BB11_3 Depth=1
-; ARM-NEXT:    str r1, [r2]
-; ARM-NEXT:    mov r2, r1
-; ARM-NEXT:    mov r1, r3
-; ARM-NEXT:  LBB11_3: @ %loop1
-; ARM-NEXT:    @ =>This Inner Loop Header: Depth=1
-; ARM-NEXT:    ldr r3, [r0]
-; ARM-NEXT:    cmp r0, #0
-; ARM-NEXT:    bne LBB11_2
-; ARM-NEXT:  @ %bb.4: @ in Loop: Header=BB11_3 Depth=1
-; ARM-NEXT:    mov r0, r1
-; ARM-NEXT:    mov r1, r3
-; ARM-NEXT:    mov r2, r0
-; ARM-NEXT:    b LBB11_3
-; ARM-NEXT:  LBB11_5: @ %end
-; ARM-NEXT:    bx lr
-;
-; THUMB-LABEL: infiniteloop3:
-; THUMB:       @ %bb.0: @ %entry
-; THUMB-NEXT:    movs r0, #0
-; THUMB-NEXT:    cbnz r0, LBB11_5
-; THUMB-NEXT:  @ %bb.1: @ %loop2a.preheader
-; THUMB-NEXT:    movs r0, #0
-; THUMB-NEXT:    movs r1, #0
-; THUMB-NEXT:    mov r2, r0
-; THUMB-NEXT:    b LBB11_3
-; THUMB-NEXT:  LBB11_2: @ %loop2b
-; THUMB-NEXT:    @ in Loop: Header=BB11_3 Depth=1
-; THUMB-NEXT:    str r1, [r2]
-; THUMB-NEXT:    mov r2, r1
-; THUMB-NEXT:    mov r1, r3
-; THUMB-NEXT:  LBB11_3: @ %loop1
-; THUMB-NEXT:    @ =>This Inner Loop Header: Depth=1
-; THUMB-NEXT:    ldr r3, [r0]
-; THUMB-NEXT:    cmp r0, #0
-; THUMB-NEXT:    bne LBB11_2
-; THUMB-NEXT:  @ %bb.4: @ in Loop: Header=BB11_3 Depth=1
-; THUMB-NEXT:    mov r0, r1
-; THUMB-NEXT:    mov r1, r3
-; THUMB-NEXT:    mov r2, r0
-; THUMB-NEXT:    b LBB11_3
-; THUMB-NEXT:  LBB11_5: @ %end
-; THUMB-NEXT:    bx lr
 ; ARM-ENABLE-LABEL: infiniteloop3:
 ; ARM-ENABLE:       @ %bb.0: @ %entry
 ; ARM-ENABLE-NEXT:    mov r0, #0
 ; ARM-ENABLE-NEXT:    cmp r0, #0
 ; ARM-ENABLE-NEXT:    bne LBB11_5
 ; ARM-ENABLE-NEXT:  @ %bb.1: @ %loop2a.preheader
 ; ARM-ENABLE-NEXT:    mov r1, #0
 ; ARM-ENABLE-NEXT:    mov r2, r0

 ;
 ; pop
 ;
 ; FIXME: This is flakey passing by finding 'bl' somewhere amongst the debug
 ; info (like labels named 'line_table) not because it's found a bl instruction.
 ;
 ; bl
 define float @debug_info(float %gamma, float %slopeLimit, i1 %or.cond, double %tmp) "no-frame-pointer-elim"="true" {
-; ARM-LABEL: debug_info:
-; ARM:       @ %bb.0: @ %bb
-; ARM-NEXT:    push {r4, r7, lr}
-; ARM-NEXT:    add r7, sp, #4
-; ARM-NEXT:    sub r4, sp, #16
-; ARM-NEXT:    bfc r4, #0, #4
-; ARM-NEXT:    mov sp, r4
-; ARM-NEXT:    tst r2, #1
-; ARM-NEXT:    vst1.64 {d8, d9}, [r4:128]
-; ARM-NEXT:    beq LBB12_2
-; ARM-NEXT:  @ %bb.1: @ %bb3
-; ARM-NEXT:    ldr r1, [r7, #8]
-; ARM-NEXT:    vmov s16, r0
-; ARM-NEXT:    mov r0, r3
-; ARM-NEXT:    mov r2, r3
-; ARM-NEXT:    vmov d9, r3, r1
-; ARM-NEXT:    mov r3, r1
-; ARM-NEXT:    bl _pow
-; ARM-NEXT:    vmov.f32 s0, #1.000000e+00
-; ARM-NEXT:    vmov.f64 d16, #1.000000e+00
-; ARM-NEXT:    vadd.f64 d16, d9, d16
-; ARM-NEXT:    vcmpe.f32 s16, s0
-; ARM-NEXT:    vmrs APSR_nzcv, fpscr
-; ARM-NEXT:    vmov d17, r0, r1
-; ARM-NEXT:    vmov.f64 d18, d9
-; ARM-NEXT:    vadd.f64 d17, d17, d17
-; ARM-NEXT:    vmovgt.f64 d18, d16
-; ARM-NEXT:    vcmp.f64 d18, d9
-; ARM-NEXT:    vmrs APSR_nzcv, fpscr
-; ARM-NEXT:    vmovne.f64 d9, d17
-; ARM-NEXT:    vcvt.f32.f64 s0, d9
-; ARM-NEXT:    b LBB12_3
-; ARM-NEXT:  LBB12_2:
-; ARM-NEXT:    vldr s0, LCPI12_0
-; ARM-NEXT:  LBB12_3: @ %bb13
-; ARM-NEXT:    mov r4, sp
-; ARM-NEXT:    vld1.64 {d8, d9}, [r4:128]
-; ARM-NEXT:    vmov r0, s0
-; ARM-NEXT:    sub sp, r7, #4
-; ARM-NEXT:    pop {r4, r7, pc}
-; ARM-NEXT:    .p2align 2
-; ARM-NEXT:  @ %bb.4:
-; ARM-NEXT:    .data_region
-; ARM-NEXT:  LCPI12_0:
-; ARM-NEXT:    .long 0 @ float 0
-; ARM-NEXT:    .end_data_region
-;
-; THUMB-LABEL: debug_info:
-; THUMB:       @ %bb.0: @ %bb
-; THUMB-NEXT:    push {r4, r7, lr}
-; THUMB-NEXT:    add r7, sp, #4
-; THUMB-NEXT:    sub.w r4, sp, #16
-; THUMB-NEXT:    bfc r4, #0, #4
-; THUMB-NEXT:    mov sp, r4
-; THUMB-NEXT:    lsls r1, r2, #31
-; THUMB-NEXT:    vst1.64 {d8, d9}, [r4:128]
-; THUMB-NEXT:    beq LBB12_2
-; THUMB-NEXT:  @ %bb.1: @ %bb3
-; THUMB-NEXT:    ldr r1, [r7, #8]
-; THUMB-NEXT:    vmov s16, r0
-; THUMB-NEXT:    mov r0, r3
-; THUMB-NEXT:    mov r2, r3
-; THUMB-NEXT:    vmov d9, r3, r1
-; THUMB-NEXT:    mov r3, r1
-; THUMB-NEXT:    bl _pow
-; THUMB-NEXT:    vmov.f32 s0, #1.000000e+00
-; THUMB-NEXT:    vmov.f64 d16, #1.000000e+00
-; THUMB-NEXT:    vmov.f64 d18, d9
-; THUMB-NEXT:    vcmpe.f32 s16, s0
-; THUMB-NEXT:    vadd.f64 d16, d9, d16
-; THUMB-NEXT:    vmrs APSR_nzcv, fpscr
-; THUMB-NEXT:    it gt
-; THUMB-NEXT:    vmovgt.f64 d18, d16
-; THUMB-NEXT:    vcmp.f64 d18, d9
-; THUMB-NEXT:    vmov d17, r0, r1
-; THUMB-NEXT:    vmrs APSR_nzcv, fpscr
-; THUMB-NEXT:    vadd.f64 d17, d17, d17
-; THUMB-NEXT:    it ne
-; THUMB-NEXT:    vmovne.f64 d9, d17
-; THUMB-NEXT:    vcvt.f32.f64 s0, d9
-; THUMB-NEXT:    b LBB12_3
-; THUMB-NEXT:  LBB12_2:
-; THUMB-NEXT:    vldr s0, LCPI12_0
-; THUMB-NEXT:  LBB12_3: @ %bb13
-; THUMB-NEXT:    mov r4, sp
-; THUMB-NEXT:    vld1.64 {d8, d9}, [r4:128]
-; THUMB-NEXT:    subs r4, r7, #4
-; THUMB-NEXT:    vmov r0, s0
-; THUMB-NEXT:    mov sp, r4
-; THUMB-NEXT:    pop {r4, r7, pc}
-; THUMB-NEXT:    .p2align 2
-; THUMB-NEXT:  @ %bb.4:
-; THUMB-NEXT:    .data_region
-; THUMB-NEXT:  LCPI12_0:
-; THUMB-NEXT:    .long 0 @ float 0
-; THUMB-NEXT:    .end_data_region
+; ARM-ENABLE-LABEL: debug_info:
+; ARM-ENABLE:       @ %bb.0: @ %bb
+; ARM-ENABLE-NEXT:    push {r4, r7, lr}
+; ARM-ENABLE-NEXT:    add r7, sp, #4
+; ARM-ENABLE-NEXT:    sub r4, sp, #16
+; ARM-ENABLE-NEXT:    bfc r4, #0, #4
+; ARM-ENABLE-NEXT:    mov sp, r4
+; ARM-ENABLE-NEXT:    tst r2, #1
+; ARM-ENABLE-NEXT:    vst1.64 {d8, d9}, [r4:128]
+; ARM-ENABLE-NEXT:    beq LBB12_2
+; ARM-ENABLE-NEXT:  @ %bb.1: @ %bb3
+; ARM-ENABLE-NEXT:    ldr r1, [r7, #8]
+; ARM-ENABLE-NEXT:    vmov s16, r0
+; ARM-ENABLE-NEXT:    mov r0, r3
+; ARM-ENABLE-NEXT:    mov r2, r3
+; ARM-ENABLE-NEXT:    vmov d9, r3, r1
+; ARM-ENABLE-NEXT:    mov r3, r1
+; ARM-ENABLE-NEXT:    bl _pow
+; ARM-ENABLE-NEXT:    vmov.f32 s0, #1.000000e+00
+; ARM-ENABLE-NEXT:    vmov.f64 d16, #1.000000e+00
+; ARM-ENABLE-NEXT:    vadd.f64 d16, d9, d16
+; ARM-ENABLE-NEXT:    vcmpe.f32 s16, s0
+; ARM-ENABLE-NEXT:    vmrs APSR_nzcv, fpscr
+; ARM-ENABLE-NEXT:    vmov d17, r0, r1
+; ARM-ENABLE-NEXT:    vmov.f64 d18, d9
+; ARM-ENABLE-NEXT:    vadd.f64 d17, d17, d17
+; ARM-ENABLE-NEXT:    vmovgt.f64 d18, d16
+; ARM-ENABLE-NEXT:    vcmp.f64 d18, d9
+; ARM-ENABLE-NEXT:    vmrs APSR_nzcv, fpscr
+; ARM-ENABLE-NEXT:    vmovne.f64 d9, d17
+; ARM-ENABLE-NEXT:    vcvt.f32.f64 s0, d9
+; ARM-ENABLE-NEXT:    b LBB12_3
+; ARM-ENABLE-NEXT:  LBB12_2:
+; ARM-ENABLE-NEXT:    vldr s0, LCPI12_0
+; ARM-ENABLE-NEXT:  LBB12_3: @ %bb13
+; ARM-ENABLE-NEXT:    mov r4, sp
+; ARM-ENABLE-NEXT:    vld1.64 {d8, d9}, [r4:128]
+; ARM-ENABLE-NEXT:    vmov r0, s0
+; ARM-ENABLE-NEXT:    sub sp, r7, #4
+; ARM-ENABLE-NEXT:    pop {r4, r7, pc}
+; ARM-ENABLE-NEXT:    .p2align 2
+; ARM-ENABLE-NEXT:  @ %bb.4:
+; ARM-ENABLE-NEXT:    .data_region
+; ARM-ENABLE-NEXT:  LCPI12_0:
+; ARM-ENABLE-NEXT:    .long 0 @ float 0
+; ARM-ENABLE-NEXT:    .end_data_region
+;
+; ARM-DISABLE-LABEL: debug_info:
+; ARM-DISABLE:       @ %bb.0: @ %bb
+; ARM-DISABLE-NEXT:    push {r4, r7, lr}
+; ARM-DISABLE-NEXT:    add r7, sp, #4
+; ARM-DISABLE-NEXT:    sub r4, sp, #16
+; ARM-DISABLE-NEXT:    bfc r4, #0, #4
+; ARM-DISABLE-NEXT:    mov sp, r4
+; ARM-DISABLE-NEXT:    tst r2, #1
+; ARM-DISABLE-NEXT:    vst1.64 {d8, d9}, [r4:128]
+; ARM-DISABLE-NEXT:    beq LBB12_2
+; ARM-DISABLE-NEXT:  @ %bb.1: @ %bb3
+; ARM-DISABLE-NEXT:    ldr r1, [r7, #8]
+; ARM-DISABLE-NEXT:    vmov s16, r0
+; ARM-DISABLE-NEXT:    mov r0, r3
+; ARM-DISABLE-NEXT:    mov r2, r3
+; ARM-DISABLE-NEXT:    vmov d9, r3, r1
+; ARM-DISABLE-NEXT:    mov r3, r1
+; ARM-DISABLE-NEXT:    bl _pow
+; ARM-DISABLE-NEXT:    vmov.f32 s0, #1.000000e+00
+; ARM-DISABLE-NEXT:    vmov.f64 d16, #1.000000e+00
+; ARM-DISABLE-NEXT:    vadd.f64 d16, d9, d16
+; ARM-DISABLE-NEXT:    vcmpe.f32 s16, s0
+; ARM-DISABLE-NEXT:    vmrs APSR_nzcv, fpscr
+; ARM-DISABLE-NEXT:    vmov d17, r0, r1
+; ARM-DISABLE-NEXT:    vmov.f64 d18, d9
+; ARM-DISABLE-NEXT:    vadd.f64 d17, d17, d17
+; ARM-DISABLE-NEXT:    vmovgt.f64 d18, d16
+; ARM-DISABLE-NEXT:    vcmp.f64 d18, d9
+; ARM-DISABLE-NEXT:    vmrs APSR_nzcv, fpscr
+; ARM-DISABLE-NEXT:    vmovne.f64 d9, d17
+; ARM-DISABLE-NEXT:    vcvt.f32.f64 s0, d9
+; ARM-DISABLE-NEXT:    b LBB12_3
+; ARM-DISABLE-NEXT:  LBB12_2:
+; ARM-DISABLE-NEXT:    vldr s0, LCPI12_0
+; ARM-DISABLE-NEXT:  LBB12_3: @ %bb13
+; ARM-DISABLE-NEXT:    mov r4, sp
+; ARM-DISABLE-NEXT:    vld1.64 {d8, d9}, [r4:128]
+; ARM-DISABLE-NEXT:    vmov r0, s0
+; ARM-DISABLE-NEXT:    sub sp, r7, #4
+; ARM-DISABLE-NEXT:    pop {r4, r7, pc}
+; ARM-DISABLE-NEXT:    .p2align 2
+; ARM-DISABLE-NEXT:  @ %bb.4:
+; ARM-DISABLE-NEXT:    .data_region
+; ARM-DISABLE-NEXT:  LCPI12_0:
+; ARM-DISABLE-NEXT:    .long 0 @ float 0
+; ARM-DISABLE-NEXT:    .end_data_region
+;
+; THUMB-ENABLE-LABEL: debug_info:
+; THUMB-ENABLE:       @ %bb.0: @ %bb
+; THUMB-ENABLE-NEXT:    push {r4, r7, lr}
+; THUMB-ENABLE-NEXT:    add r7, sp, #4
+; THUMB-ENABLE-NEXT:    sub.w r4, sp, #16
+; THUMB-ENABLE-NEXT:    bfc r4, #0, #4
+; THUMB-ENABLE-NEXT:    mov sp, r4
+; THUMB-ENABLE-NEXT:    lsls r1, r2, #31
+; THUMB-ENABLE-NEXT:    vst1.64 {d8, d9}, [r4:128]
+; THUMB-ENABLE-NEXT:    beq LBB12_2
+; THUMB-ENABLE-NEXT:  @ %bb.1: @ %bb3
+; THUMB-ENABLE-NEXT:    ldr r1, [r7, #8]
+; THUMB-ENABLE-NEXT:    vmov s16, r0
+; THUMB-ENABLE-NEXT:    mov r0, r3
+; THUMB-ENABLE-NEXT:    mov r2, r3
+; THUMB-ENABLE-NEXT:    vmov d9, r3, r1
+; THUMB-ENABLE-NEXT:    mov r3, r1
+; THUMB-ENABLE-NEXT:    bl _pow
+; THUMB-ENABLE-NEXT:    vmov.f32 s0, #1.000000e+00
+; THUMB-ENABLE-NEXT:    vmov.f64 d16, #1.000000e+00
+; THUMB-ENABLE-NEXT:    vmov.f64 d18, d9
+; THUMB-ENABLE-NEXT:    vcmpe.f32 s16, s0
+; THUMB-ENABLE-NEXT:    vadd.f64 d16, d9, d16
+; THUMB-ENABLE-NEXT:    vmrs APSR_nzcv, fpscr
+; THUMB-ENABLE-NEXT:    it gt
+; THUMB-ENABLE-NEXT:    vmovgt.f64 d18, d16
+; THUMB-ENABLE-NEXT:    vcmp.f64 d18, d9
+; THUMB-ENABLE-NEXT:    vmov d17, r0, r1
+; THUMB-ENABLE-NEXT:    vmrs APSR_nzcv, fpscr
+; THUMB-ENABLE-NEXT:    vadd.f64 d17, d17, d17
+; THUMB-ENABLE-NEXT:    it ne
+; THUMB-ENABLE-NEXT:    vmovne.f64 d9, d17
+; THUMB-ENABLE-NEXT:    vcvt.f32.f64 s0, d9
+; THUMB-ENABLE-NEXT:    b LBB12_3
+; THUMB-ENABLE-NEXT:  LBB12_2:
+; THUMB-ENABLE-NEXT:    vldr s0, LCPI12_0
+; THUMB-ENABLE-NEXT:  LBB12_3: @ %bb13
+; THUMB-ENABLE-NEXT:    mov r4, sp
+; THUMB-ENABLE-NEXT:    vld1.64 {d8, d9}, [r4:128]
+; THUMB-ENABLE-NEXT:    subs r4, r7, #4
+; THUMB-ENABLE-NEXT:    vmov r0, s0
+; THUMB-ENABLE-NEXT:    mov sp, r4
+; THUMB-ENABLE-NEXT:    pop {r4, r7, pc}
+; THUMB-ENABLE-NEXT:    .p2align 2
+; THUMB-ENABLE-NEXT:  @ %bb.4:
+; THUMB-ENABLE-NEXT:    .data_region
+; THUMB-ENABLE-NEXT:  LCPI12_0:
+; THUMB-ENABLE-NEXT:    .long 0 @ float 0
+; THUMB-ENABLE-NEXT:    .end_data_region
+;
+; THUMB-DISABLE-LABEL: debug_info:
+; THUMB-DISABLE:       @ %bb.0: @ %bb
+; THUMB-DISABLE-NEXT:    push {r4, r7, lr}
+; THUMB-DISABLE-NEXT:    add r7, sp, #4
+; THUMB-DISABLE-NEXT:    sub.w r4, sp, #16
+; THUMB-DISABLE-NEXT:    bfc r4, #0, #4
+; THUMB-DISABLE-NEXT:    mov sp, r4
+; THUMB-DISABLE-NEXT:    lsls r1, r2, #31
+; THUMB-DISABLE-NEXT:    vst1.64 {d8, d9}, [r4:128]
+; THUMB-DISABLE-NEXT:    beq LBB12_2
+; THUMB-DISABLE-NEXT:  @ %bb.1: @ %bb3
+; THUMB-DISABLE-NEXT:    ldr r1, [r7, #8]
+; THUMB-DISABLE-NEXT:    vmov s16, r0
+; THUMB-DISABLE-NEXT:    mov r0, r3
+; THUMB-DISABLE-NEXT:    mov r2, r3
+; THUMB-DISABLE-NEXT:    vmov d9, r3, r1
+; THUMB-DISABLE-NEXT:    mov r3, r1
+; THUMB-DISABLE-NEXT:    bl _pow
+; THUMB-DISABLE-NEXT:    vmov.f32 s0, #1.000000e+00
+; THUMB-DISABLE-NEXT:    vmov.f64 d16, #1.000000e+00
+; THUMB-DISABLE-NEXT:    vmov.f64 d18, d9
+; THUMB-DISABLE-NEXT:    vcmpe.f32 s16, s0
+; THUMB-DISABLE-NEXT:    vadd.f64 d16, d9, d16
+; THUMB-DISABLE-NEXT:    vmrs APSR_nzcv, fpscr
+; THUMB-DISABLE-NEXT:    it gt
+; THUMB-DISABLE-NEXT:    vmovgt.f64 d18, d16
+; THUMB-DISABLE-NEXT:    vcmp.f64 d18, d9
+; THUMB-DISABLE-NEXT:    vmov d17, r0, r1
+; THUMB-DISABLE-NEXT:    vmrs APSR_nzcv, fpscr
+; THUMB-DISABLE-NEXT:    vadd.f64 d17, d17, d17
+; THUMB-DISABLE-NEXT:    it ne
+; THUMB-DISABLE-NEXT:    vmovne.f64 d9, d17
+; THUMB-DISABLE-NEXT:    vcvt.f32.f64 s0, d9
+; THUMB-DISABLE-NEXT:    b LBB12_3
+; THUMB-DISABLE-NEXT:  LBB12_2:
+; THUMB-DISABLE-NEXT:    vldr s0, LCPI12_0
+; THUMB-DISABLE-NEXT:  LBB12_3: @ %bb13
+; THUMB-DISABLE-NEXT:    mov r4, sp
+; THUMB-DISABLE-NEXT:    vld1.64 {d8, d9}, [r4:128]
+; THUMB-DISABLE-NEXT:    subs r4, r7, #4
+; THUMB-DISABLE-NEXT:    vmov r0, s0
+; THUMB-DISABLE-NEXT:    mov sp, r4
+; THUMB-DISABLE-NEXT:    pop {r4, r7, pc}
+; THUMB-DISABLE-NEXT:    .p2align 2
+; THUMB-DISABLE-NEXT:  @ %bb.4:
+; THUMB-DISABLE-NEXT:    .data_region
+; THUMB-DISABLE-NEXT:  LCPI12_0:
+; THUMB-DISABLE-NEXT:    .long 0 @ float 0
+; THUMB-DISABLE-NEXT:    .end_data_region
 bb:
   br i1 %or.cond, label %bb3, label %bb13
 
 bb3:                                              ; preds = %bb
   %tmp4 = fcmp ogt float %gamma, 1.000000e+00
   %tmp5 = fadd double 1.000000e+00, %tmp
   %tmp6 = select i1 %tmp4, double %tmp5, double %tmp
   %tmp10 = tail call double @llvm.pow.f64(double %tmp, double %tmp)

test/CodeGen/X86/avx-vzeroupper.ll

 ; VZ-NEXT:    vmovaps %xmm0, (%rsp) # 16-byte Spill
 ; VZ-NEXT:    .p2align 4, 0x90
 ; VZ-NEXT:  .LBB3_1: # %while.cond
 ; VZ-NEXT:    # =>This Inner Loop Header: Depth=1
 ; VZ-NEXT:    callq foo
 ; VZ-NEXT:    testl %eax, %eax
 ; VZ-NEXT:    jne .LBB3_1
 ; VZ-NEXT:  # %bb.2: # %for.body.preheader
-; VZ-NEXT:    movl $4, %ebx
+; VZ-NEXT:    movl $-4, %ebx
 ; VZ-NEXT:    vmovaps (%rsp), %xmm0 # 16-byte Reload
 ; VZ-NEXT:    .p2align 4, 0x90
 ; VZ-NEXT:  .LBB3_3: # %for.body
 ; VZ-NEXT:    # =>This Inner Loop Header: Depth=1
 ; VZ-NEXT:    callq do_sse
 ; VZ-NEXT:    callq do_sse
 ; VZ-NEXT:    vmovaps g+{{.*}}(%rip), %xmm0
 ; VZ-NEXT:    callq do_sse
-; VZ-NEXT:    decl %ebx
+; VZ-NEXT:    incl %ebx
 ; VZ-NEXT:    jne .LBB3_3
 ; VZ-NEXT:  # %bb.4: # %for.end
 ; VZ-NEXT:    addq $16, %rsp
 ; VZ-NEXT:    popq %rbx
 ; VZ-NEXT:    retq
 ;
 ; FAST-ymm-zmm-LABEL: test03:
 ; FAST-ymm-zmm:       # %bb.0: # %entry
 ; FAST-ymm-zmm-NEXT:    pushq %rbx
 ; FAST-ymm-zmm-NEXT:    subq $16, %rsp
 ; FAST-ymm-zmm-NEXT:    vaddps %xmm1, %xmm0, %xmm0
 ; FAST-ymm-zmm-NEXT:    vmovaps %xmm0, (%rsp) # 16-byte Spill
 ; FAST-ymm-zmm-NEXT:    .p2align 4, 0x90
 ; FAST-ymm-zmm-NEXT:  .LBB3_1: # %while.cond
 ; FAST-ymm-zmm-NEXT:    # =>This Inner Loop Header: Depth=1
 ; FAST-ymm-zmm-NEXT:    callq foo
 ; FAST-ymm-zmm-NEXT:    testl %eax, %eax
 ; FAST-ymm-zmm-NEXT:    jne .LBB3_1
 ; FAST-ymm-zmm-NEXT:  # %bb.2: # %for.body.preheader
-; FAST-ymm-zmm-NEXT:    movl $4, %ebx
+; FAST-ymm-zmm-NEXT:    movl $-4, %ebx
 ; FAST-ymm-zmm-NEXT:    vmovaps (%rsp), %xmm0 # 16-byte Reload
 ; FAST-ymm-zmm-NEXT:    .p2align 4, 0x90
 ; FAST-ymm-zmm-NEXT:  .LBB3_3: # %for.body
 ; FAST-ymm-zmm-NEXT:    # =>This Inner Loop Header: Depth=1
 ; FAST-ymm-zmm-NEXT:    callq do_sse
 ; FAST-ymm-zmm-NEXT:    callq do_sse
 ; FAST-ymm-zmm-NEXT:    vmovaps g+{{.*}}(%rip), %xmm0
 ; FAST-ymm-zmm-NEXT:    callq do_sse
-; FAST-ymm-zmm-NEXT:    decl %ebx
+; FAST-ymm-zmm-NEXT:    incl %ebx
 ; FAST-ymm-zmm-NEXT:    jne .LBB3_3
 ; FAST-ymm-zmm-NEXT:  # %bb.4: # %for.end
 ; FAST-ymm-zmm-NEXT:    addq $16, %rsp
 ; FAST-ymm-zmm-NEXT:    popq %rbx
 ; FAST-ymm-zmm-NEXT:    retq
 ;
 ; BDVER2-LABEL: test03:
 ; BDVER2:       # %bb.0: # %entry
 ; BDVER2-NEXT:    pushq %rbx
 ; BDVER2-NEXT:    subq $16, %rsp
 ; BDVER2-NEXT:    vaddps %xmm1, %xmm0, %xmm0
 ; BDVER2-NEXT:    vmovaps %xmm0, (%rsp) # 16-byte Spill
 ; BDVER2-NEXT:    .p2align 4, 0x90
 ; BDVER2-NEXT:  .LBB3_1: # %while.cond
 ; BDVER2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; BDVER2-NEXT:    callq foo
 ; BDVER2-NEXT:    testl %eax, %eax
 ; BDVER2-NEXT:    jne .LBB3_1
 ; BDVER2-NEXT:  # %bb.2: # %for.body.preheader
 ; BDVER2-NEXT:    vmovaps (%rsp), %xmm0 # 16-byte Reload
-; BDVER2-NEXT:    movl $4, %ebx
+; BDVER2-NEXT:    movl $-4, %ebx
 ; BDVER2-NEXT:    .p2align 4, 0x90
 ; BDVER2-NEXT:  .LBB3_3: # %for.body
 ; BDVER2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; BDVER2-NEXT:    callq do_sse
 ; BDVER2-NEXT:    callq do_sse
 ; BDVER2-NEXT:    vmovaps g+{{.*}}(%rip), %xmm0
 ; BDVER2-NEXT:    callq do_sse
-; BDVER2-NEXT:    decl %ebx
+; BDVER2-NEXT:    incl %ebx
 ; BDVER2-NEXT:    jne .LBB3_3
 ; BDVER2-NEXT:  # %bb.4: # %for.end
 ; BDVER2-NEXT:    addq $16, %rsp
 ; BDVER2-NEXT:    popq %rbx
 ; BDVER2-NEXT:    retq
 ;
 ; BTVER2-LABEL: test03:
 ; BTVER2:       # %bb.0: # %entry
 ; BTVER2-NEXT:    pushq %rbx
 ; BTVER2-NEXT:    subq $16, %rsp
 ; BTVER2-NEXT:    vaddps %xmm1, %xmm0, %xmm0
 ; BTVER2-NEXT:    vmovaps %xmm0, (%rsp) # 16-byte Spill
 ; BTVER2-NEXT:    .p2align 4, 0x90
 ; BTVER2-NEXT:  .LBB3_1: # %while.cond
 ; BTVER2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; BTVER2-NEXT:    callq foo
 ; BTVER2-NEXT:    testl %eax, %eax
 ; BTVER2-NEXT:    jne .LBB3_1
 ; BTVER2-NEXT:  # %bb.2: # %for.body.preheader
 ; BTVER2-NEXT:    vmovaps (%rsp), %xmm0 # 16-byte Reload
-; BTVER2-NEXT:    movl $4, %ebx
+; BTVER2-NEXT:    movl $-4, %ebx
 ; BTVER2-NEXT:    .p2align 4, 0x90
 ; BTVER2-NEXT:  .LBB3_3: # %for.body
 ; BTVER2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; BTVER2-NEXT:    callq do_sse
 ; BTVER2-NEXT:    callq do_sse
 ; BTVER2-NEXT:    vmovaps g+{{.*}}(%rip), %xmm0
 ; BTVER2-NEXT:    callq do_sse
-; BTVER2-NEXT:    decl %ebx
+; BTVER2-NEXT:    incl %ebx
 ; BTVER2-NEXT:    jne .LBB3_3
 ; BTVER2-NEXT:  # %bb.4: # %for.end
 ; BTVER2-NEXT:    addq $16, %rsp
 ; BTVER2-NEXT:    popq %rbx
 ; BTVER2-NEXT:    retq
 entry:
   %add.i = fadd <4 x float> %a, %b
   br label %while.cond

test/CodeGen/X86/lsr-wrap.ll

 ; RUN: llc -mtriple=x86_64-- < %s | FileCheck %s
 
 ; LSR would like to use a single IV for both of these, however it's
 ; not safe due to wraparound.
 
 ; CHECK: addb  $-4, %
-; CHECK: decw  %
+; CHECK: incw  %
 
 @g_19 = common global i32 0                       ; <i32*> [#uses=2]
 
 declare i32 @func_8(i8 zeroext) nounwind
 
 declare i32 @func_3(i8 signext) nounwind
 
 define void @func_1() nounwind {

test/CodeGen/X86/masked-iv-safe.ll

 ; RUN: llc < %s -mcpu=generic -mtriple=x86_64-- | FileCheck %s
 
 ; Optimize away zext-inreg and sext-inreg on the loop induction
 ; variable using trip-count information.
 
 ; CHECK-LABEL: count_up
 ; CHECK-NOT: {{and|movz|sar|shl}}
-; CHECK: addq $8
+; CHECK: incq
 ; CHECK-NOT: {{and|movz|sar|shl}}
 ; CHECK: jne
 define void @count_up(double* %d, i64 %n) nounwind {
 entry:
   br label %loop
 
 loop:
   %indvar = phi i64 [ 0, %entry ], [ %indvar.next, %loop ]
loop:
   br i1 %exitcond, label %return, label %loop
 
 return:
   ret void
 }
 
 ; CHECK-LABEL: count_down
 ; CHECK-NOT: {{and|movz|sar|shl}}
-; CHECK: addq $-8
+; CHECK: decq
 ; CHECK-NOT: {{and|movz|sar|shl}}
 ; CHECK: jne
 define void @count_down(double* %d, i64 %n) nounwind {
 entry:
   br label %loop
 
 loop:
   %indvar = phi i64 [ 10, %entry ], [ %indvar.next, %loop ]
loop:
   br i1 %exitcond, label %return, label %loop
 
 return:
   ret void
 }
 
 ; CHECK-LABEL: count_up_signed
 ; CHECK-NOT: {{and|movz|sar|shl}}
-; CHECK: addq $8
+; CHECK: incq
 ; CHECK-NOT: {{and|movz|sar|shl}}
 ; CHECK: jne
 define void @count_up_signed(double* %d, i64 %n) nounwind {
 entry:
   br label %loop
 
 loop:
   %indvar = phi i64 [ 0, %entry ], [ %indvar.next, %loop ]
loop:
   br i1 %exitcond, label %return, label %loop
 
 return:
   ret void
 }
 
 ; CHECK-LABEL: count_down_signed
 ; CHECK-NOT: {{and|movz|sar|shl}}
-; CHECK: addq $-8
+; CHECK: decq
 ; CHECK-NOT: {{and|movz|sar|shl}}
 ; CHECK: jne
 define void @count_down_signed(double* %d, i64 %n) nounwind {
 entry:
   br label %loop
 
 loop:
   %indvar = phi i64 [ 10, %entry ], [ %indvar.next, %loop ]
loop:
   br i1 %exitcond, label %return, label %loop
 
 return:
   ret void
 }
 
 ; CHECK-LABEL: another_count_up
 ; CHECK-NOT: {{and|movz|sar|shl}}
-; CHECK: addq $8
+; CHECK: incq
 ; CHECK-NOT: {{and|movz|sar|shl}}
 ; CHECK: jne
 define void @another_count_up(double* %d, i64 %n) nounwind {
 entry:
   br label %loop
 
 loop:
   %indvar = phi i64 [ 18446744073709551615, %entry ], [ %indvar.next, %loop ]
loop:
   br i1 %exitcond, label %return, label %loop
 
 return:
   ret void
 }
 
 ; CHECK-LABEL: another_count_up_signed
 ; CHECK-NOT: {{and|movz|sar|shl}}
-; CHECK: addq $8
+; CHECK: incq
 ; CHECK-NOT: {{and|movz|sar|shl}}
 ; CHECK: jne
 define void @another_count_up_signed(double* %d, i64 %n) nounwind {
 entry:
   br label %loop
 
 loop:
   %indvar = phi i64 [ 18446744073709551615, %entry ], [ %indvar.next, %loop ]
loop:
   br i1 %exitcond, label %return, label %loop
 
 return:
   ret void
 }
 
 ; CHECK-LABEL: another_count_down_signed
 ; CHECK-NOT: {{and|movz|sar|shl}}
-; CHECK: addq $-8
+; CHECK: decq
 ; CHECK-NOT: {{and|movz|sar|shl}}
 ; CHECK: jne
 define void @another_count_down_signed(double* %d, i64 %n) nounwind {
 entry:
   br label %loop
 
 loop:
   %indvar = phi i64 [ 0, %entry ], [ %indvar.next, %loop ]

test/CodeGen/X86/reverse_branches.ll

 ; CHECK-NEXT:    cmpl $999, %eax ## imm = 0x3E7
 ; CHECK-NEXT:    jg LBB0_16
 ; CHECK-NEXT:  LBB0_9: ## %for.cond18.preheader
 ; CHECK-NEXT:    ## =>This Loop Header: Depth=1
 ; CHECK-NEXT:    ## Child Loop BB0_11 Depth 2
 ; CHECK-NEXT:    ## Child Loop BB0_12 Depth 3
 ; CHECK-NEXT:    movq %rcx, %rdx
 ; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    xorl %edi, %edi
-; CHECK-NEXT:    cmpl $999, %edi ## imm = 0x3E7
+; CHECK-NEXT:    cmpl $999, %esi ## imm = 0x3E7
 ; CHECK-NEXT:    jle LBB0_11
 ; CHECK-NEXT:    jmp LBB0_15
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_14: ## %exit
 ; CHECK-NEXT:    ## in Loop: Header=BB0_11 Depth=2
-; CHECK-NEXT:    addq %rsi, %rbp
-; CHECK-NEXT:    incq %rdi
-; CHECK-NEXT:    decq %rsi
+; CHECK-NEXT:    addq $1000, %rdi ## imm = 0x3E8
 ; CHECK-NEXT:    addq $1001, %rdx ## imm = 0x3E9
-; CHECK-NEXT:    cmpq $-1000, %rbp ## imm = 0xFC18
+; CHECK-NEXT:    cmpq %rsi, %rdi
+; CHECK-NEXT:    leaq 1(%rsi), %rsi
 ; CHECK-NEXT:    jne LBB0_5
 ; CHECK-NEXT:  ## %bb.10: ## %for.cond18
 ; CHECK-NEXT:    ## in Loop: Header=BB0_11 Depth=2
-; CHECK-NEXT:    cmpl $999, %edi ## imm = 0x3E7
+; CHECK-NEXT:    cmpl $999, %esi ## imm = 0x3E7
 ; CHECK-NEXT:    jg LBB0_15
 ; CHECK-NEXT:  LBB0_11: ## %for.body20
 ; CHECK-NEXT:    ## Parent Loop BB0_9 Depth=1
 ; CHECK-NEXT:    ## => This Loop Header: Depth=2
 ; CHECK-NEXT:    ## Child Loop BB0_12 Depth 3
-; CHECK-NEXT:    movq $-1000, %rbp ## imm = 0xFC18
+; CHECK-NEXT:    movq $-1000, %rdi ## imm = 0xFC18
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_12: ## %do.body.i
 ; CHECK-NEXT:    ## Parent Loop BB0_9 Depth=1
 ; CHECK-NEXT:    ## Parent Loop BB0_11 Depth=2
 ; CHECK-NEXT:    ## => This Inner Loop Header: Depth=3
-; CHECK-NEXT:    cmpb $120, 1000(%rdx,%rbp)
+; CHECK-NEXT:    cmpb $120, 1000(%rdx,%rdi)
 ; CHECK-NEXT:    je LBB0_14
 ; CHECK-NEXT:  ## %bb.13: ## %do.cond.i
 ; CHECK-NEXT:    ## in Loop: Header=BB0_12 Depth=3
-; CHECK-NEXT:    incq %rbp
+; CHECK-NEXT:    incq %rdi
 ; CHECK-NEXT:    jne LBB0_12
 ; CHECK-NEXT:  LBB0_5: ## %if.then
 ; CHECK-NEXT:    leaq {{.*}}(%rip), %rdi
 ; CHECK-NEXT:    callq _puts
 ; CHECK-NEXT:    movl $1, %edi
 ; CHECK-NEXT:    callq _exit
 ; CHECK-NEXT:  LBB0_16: ## %for.end40
 ; CHECK-NEXT:    leaq {{.*}}(%rip), %rdi

test/CodeGen/X86/x86-shrink-wrapping.ll

 ; ENABLE-NEXT:  ## %bb.1: ## %for.preheader
 ; ENABLE-NEXT:    pushq %rbx
 ; ENABLE-NEXT:    .cfi_def_cfa_offset 16
 ; ENABLE-NEXT:    .cfi_offset %rbx, -16
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    nop
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    xorl %eax, %eax
-; ENABLE-NEXT:    movl $10, %ecx
+; ENABLE-NEXT:    movl $-10, %ecx
 ; ENABLE-NEXT:    .p2align 4, 0x90
 ; ENABLE-NEXT:  LBB1_2: ## %for.body
 ; ENABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    movl $1, %edx
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    addl %edx, %eax
-; ENABLE-NEXT:    decl %ecx
+; ENABLE-NEXT:    incl %ecx
 ; ENABLE-NEXT:    jne LBB1_2
 ; ENABLE-NEXT:  ## %bb.3: ## %for.end
 ; ENABLE-NEXT:    shll $3, %eax
 ; ENABLE-NEXT:    popq %rbx
 ; ENABLE-NEXT:    retq
 ; ENABLE-NEXT:  LBB1_4: ## %if.else
 ; ENABLE-NEXT:    movl %esi, %eax
 ; ENABLE-NEXT:    addl %esi, %eax
 ; ENABLE-NEXT:    retq
 ;
 ; DISABLE-LABEL: freqSaveAndRestoreOutsideLoop:
 ; DISABLE:       ## %bb.0: ## %entry
 ; DISABLE-NEXT:    pushq %rbx
 ; DISABLE-NEXT:    .cfi_def_cfa_offset 16
 ; DISABLE-NEXT:    .cfi_offset %rbx, -16
 ; DISABLE-NEXT:    testl %edi, %edi
 ; DISABLE-NEXT:    je LBB1_4
 ; DISABLE-NEXT:  ## %bb.1: ## %for.preheader
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    nop
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    xorl %eax, %eax
-; DISABLE-NEXT:    movl $10, %ecx
+; DISABLE-NEXT:    movl $-10, %ecx
 ; DISABLE-NEXT:    .p2align 4, 0x90
 ; DISABLE-NEXT:  LBB1_2: ## %for.body
 ; DISABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    movl $1, %edx
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    addl %edx, %eax
-; DISABLE-NEXT:    decl %ecx
+; DISABLE-NEXT:    incl %ecx
 ; DISABLE-NEXT:    jne LBB1_2
 ; DISABLE-NEXT:  ## %bb.3: ## %for.end
 ; DISABLE-NEXT:    shll $3, %eax
 ; DISABLE-NEXT:    popq %rbx
 ; DISABLE-NEXT:    retq
 ; DISABLE-NEXT:  LBB1_4: ## %if.else
 ; DISABLE-NEXT:    movl %esi, %eax
 ; DISABLE-NEXT:    addl %esi, %eax

 ; ENABLE:       ## %bb.0: ## %entry
 ; ENABLE-NEXT:    pushq %rbx
 ; ENABLE-NEXT:    .cfi_def_cfa_offset 16
 ; ENABLE-NEXT:    .cfi_offset %rbx, -16
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    nop
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    xorl %eax, %eax
-; ENABLE-NEXT:    movl $10, %ecx
+; ENABLE-NEXT:    movl $-10, %ecx
 ; ENABLE-NEXT:    .p2align 4, 0x90
 ; ENABLE-NEXT:  LBB2_1: ## %for.body
 ; ENABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    movl $1, %edx
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    addl %edx, %eax
-; ENABLE-NEXT:    decl %ecx
+; ENABLE-NEXT:    incl %ecx
 ; ENABLE-NEXT:    jne LBB2_1
 ; ENABLE-NEXT:  ## %bb.2: ## %for.exit
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    nop
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    popq %rbx
 ; ENABLE-NEXT:    retq
 ;
 ; DISABLE-LABEL: freqSaveAndRestoreOutsideLoop2:
 ; DISABLE:       ## %bb.0: ## %entry
 ; DISABLE-NEXT:    pushq %rbx
 ; DISABLE-NEXT:    .cfi_def_cfa_offset 16
 ; DISABLE-NEXT:    .cfi_offset %rbx, -16
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    nop
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    xorl %eax, %eax
-; DISABLE-NEXT:    movl $10, %ecx
+; DISABLE-NEXT:    movl $-10, %ecx
 ; DISABLE-NEXT:    .p2align 4, 0x90
 ; DISABLE-NEXT:  LBB2_1: ## %for.body
 ; DISABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    movl $1, %edx
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    addl %edx, %eax
-; DISABLE-NEXT:    decl %ecx
+; DISABLE-NEXT:    incl %ecx
 ; DISABLE-NEXT:    jne LBB2_1
 ; DISABLE-NEXT:  ## %bb.2: ## %for.exit
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    nop
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    popq %rbx
 ; DISABLE-NEXT:    retq
 entry:

 ; ENABLE-NEXT:  ## %bb.1: ## %for.preheader
 ; ENABLE-NEXT:    pushq %rbx
 ; ENABLE-NEXT:    .cfi_def_cfa_offset 16
 ; ENABLE-NEXT:    .cfi_offset %rbx, -16
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    nop
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    xorl %eax, %eax
-; ENABLE-NEXT:    movl $10, %ecx
+; ENABLE-NEXT:    movl $-10, %ecx
 ; ENABLE-NEXT:    .p2align 4, 0x90
 ; ENABLE-NEXT:  LBB3_2: ## %for.body
 ; ENABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    movl $1, %edx
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    addl %edx, %eax
-; ENABLE-NEXT:    decl %ecx
+; ENABLE-NEXT:    incl %ecx
 ; ENABLE-NEXT:    jne LBB3_2
 ; ENABLE-NEXT:  ## %bb.3: ## %for.end
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    nop
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    shll $3, %eax
 ; ENABLE-NEXT:    popq %rbx
 ; ENABLE-NEXT:    retq

 ; DISABLE-NEXT:    .cfi_offset %rbx, -16
 ; DISABLE-NEXT:    testl %edi, %edi
 ; DISABLE-NEXT:    je LBB3_4
 ; DISABLE-NEXT:  ## %bb.1: ## %for.preheader
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    nop
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    xorl %eax, %eax
-; DISABLE-NEXT:    movl $10, %ecx
+; DISABLE-NEXT:    movl $-10, %ecx
 ; DISABLE-NEXT:    .p2align 4, 0x90
 ; DISABLE-NEXT:  LBB3_2: ## %for.body
 ; DISABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    movl $1, %edx
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    addl %edx, %eax
-; DISABLE-NEXT:    decl %ecx
+; DISABLE-NEXT:    incl %ecx
 ; DISABLE-NEXT:    jne LBB3_2
 ; DISABLE-NEXT:  ## %bb.3: ## %for.end
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    nop
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    shll $3, %eax
 ; DISABLE-NEXT:    popq %rbx
 ; DISABLE-NEXT:    retq

 ; ENABLE-NEXT:    testl %edi, %edi
 ; ENABLE-NEXT:    je LBB4_4
 ; ENABLE-NEXT:  ## %bb.1: ## %if.then
 ; ENABLE-NEXT:    pushq %rbx
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    nop
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    xorl %eax, %eax
-; ENABLE-NEXT:    movl $10, %ecx
+; ENABLE-NEXT:    movl $-10, %ecx
 ; ENABLE-NEXT:    .p2align 4, 0x90
 ; ENABLE-NEXT:  LBB4_2: ## %for.body
 ; ENABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    movl $1, %edx
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    addl %edx, %eax
-; ENABLE-NEXT:    decl %ecx
+; ENABLE-NEXT:    incl %ecx
 ; ENABLE-NEXT:    jne LBB4_2
 ; ENABLE-NEXT:  ## %bb.3: ## %for.end
 ; ENABLE-NEXT:    shll $3, %eax
 ; ENABLE-NEXT:    popq %rbx
 ; ENABLE-NEXT:    retq
 ; ENABLE-NEXT:  LBB4_4: ## %if.else
 ; ENABLE-NEXT:    movl %esi, %eax
 ; ENABLE-NEXT:    addl %esi, %eax
 ; ENABLE-NEXT:    retq
 ;
 ; DISABLE-LABEL: loopInfoRestoreOutsideLoop:
 ; DISABLE:       ## %bb.0: ## %entry
 ; DISABLE-NEXT:    pushq %rbx
 ; DISABLE-NEXT:    testl %edi, %edi
 ; DISABLE-NEXT:    je LBB4_4
 ; DISABLE-NEXT:  ## %bb.1: ## %if.then
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    nop
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    xorl %eax, %eax
-; DISABLE-NEXT:    movl $10, %ecx
+; DISABLE-NEXT:    movl $-10, %ecx
 ; DISABLE-NEXT:    .p2align 4, 0x90
 ; DISABLE-NEXT:  LBB4_2: ## %for.body
 ; DISABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    movl $1, %edx
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    addl %edx, %eax
-; DISABLE-NEXT:    decl %ecx
+; DISABLE-NEXT:    incl %ecx
 ; DISABLE-NEXT:    jne LBB4_2
 ; DISABLE-NEXT:  ## %bb.3: ## %for.end
 ; DISABLE-NEXT:    shll $3, %eax
 ; DISABLE-NEXT:    popq %rbx
 ; DISABLE-NEXT:    retq
 ; DISABLE-NEXT:  LBB4_4: ## %if.else
 ; DISABLE-NEXT:    movl %esi, %eax
 ; DISABLE-NEXT:    addl %esi, %eax

 ; ENABLE-NEXT:    je LBB6_4
 ; ENABLE-NEXT:  ## %bb.1: ## %for.preheader
 ; ENABLE-NEXT:    pushq %rbx
 ; ENABLE-NEXT:    .cfi_def_cfa_offset 16
 ; ENABLE-NEXT:    .cfi_offset %rbx, -16
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    nop
 ; ENABLE-NEXT:    ## InlineAsm End
-; ENABLE-NEXT:    movl $10, %eax
+; ENABLE-NEXT:    movl $-10, %eax
 ; ENABLE-NEXT:    .p2align 4, 0x90
 ; ENABLE-NEXT:  LBB6_2: ## %for.body
 ; ENABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    addl $1, %ebx
 ; ENABLE-NEXT:    ## InlineAsm End
-; ENABLE-NEXT:    decl %eax
+; ENABLE-NEXT:    incl %eax
 ; ENABLE-NEXT:    jne LBB6_2
 ; ENABLE-NEXT:  ## %bb.3: ## %for.exit
 ; ENABLE-NEXT:    ## InlineAsm Start
 ; ENABLE-NEXT:    nop
 ; ENABLE-NEXT:    ## InlineAsm End
 ; ENABLE-NEXT:    xorl %eax, %eax
 ; ENABLE-NEXT:    popq %rbx
 ; ENABLE-NEXT:    retq
 ; ENABLE-NEXT:  LBB6_4: ## %if.else
 ; ENABLE-NEXT:    movl %esi, %eax
 ; ENABLE-NEXT:    addl %esi, %eax
 ; ENABLE-NEXT:    retq
 ;
 ; DISABLE-LABEL: inlineAsm:
 ; DISABLE:       ## %bb.0: ## %entry
 ; DISABLE-NEXT:    pushq %rbx
 ; DISABLE-NEXT:    .cfi_def_cfa_offset 16
 ; DISABLE-NEXT:    .cfi_offset %rbx, -16
 ; DISABLE-NEXT:    testl %edi, %edi
 ; DISABLE-NEXT:    je LBB6_4
 ; DISABLE-NEXT:  ## %bb.1: ## %for.preheader
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    nop
 ; DISABLE-NEXT:    ## InlineAsm End
-; DISABLE-NEXT:    movl $10, %eax
+; DISABLE-NEXT:    movl $-10, %eax
 ; DISABLE-NEXT:    .p2align 4, 0x90
 ; DISABLE-NEXT:  LBB6_2: ## %for.body
 ; DISABLE-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    addl $1, %ebx
 ; DISABLE-NEXT:    ## InlineAsm End
-; DISABLE-NEXT:    decl %eax
+; DISABLE-NEXT:    incl %eax
 ; DISABLE-NEXT:    jne LBB6_2
 ; DISABLE-NEXT:  ## %bb.3: ## %for.exit
 ; DISABLE-NEXT:    ## InlineAsm Start
 ; DISABLE-NEXT:    nop
 ; DISABLE-NEXT:    ## InlineAsm End
 ; DISABLE-NEXT:    xorl %eax, %eax
 ; DISABLE-NEXT:    popq %rbx
 ; DISABLE-NEXT:    retq

test/CodeGen/X86/x86-win64-shrink-wrapping.ll

 ; Make sure we save the CSR used in the inline asm: rbx.
 ; CHECK: pushq %rbx
 ;
 ; DISABLE: testl %ecx, %ecx
 ; DISABLE-NEXT: je [[ELSE_LABEL:.LBB[0-9_]+]]
 ;
 ; CHECK: nop
 ; CHECK: xorl [[SUM:%eax]], [[SUM]]
-; CHECK-NEXT: movl $10, [[IV:%e[a-z]+]]
+; CHECK-NEXT: movl $-10, [[IV:%e[a-z]+]]
 ;
 ; CHECK: [[LOOP_LABEL:.LBB[0-9_]+]]: # %for.body
 ; CHECK: movl $1, [[TMP:%e[a-z]+]]
 ; CHECK: addl [[TMP]], [[SUM]]
-; CHECK-NEXT: decl [[IV]]
+; CHECK-NEXT: incl [[IV]]
 ; CHECK-NEXT: jne [[LOOP_LABEL]]
 ; Next BB.
 ; CHECK: nop
 ; CHECK: shll $3, [[SUM]]
 ;
 ; DISABLE: jmp [[EPILOG_BB:.LBB[0-9_]+]]
 ;
 ; ENABLE-NEXT: popq %rbx
 ; ENABLE-NEXT: retq
 ;
 ; CHECK: [[ELSE_LABEL]]: # %if.else
 ; Shift second argument by one and store into returned register.
 ; CHECK: addl %edx, %edx
 ; CHECK: movl %edx, %eax
 ;
-; DISABLE: [[EPILOG_BB]]: # %if.end
+; DISABLE: [[EPILOG_BB]]: # %for.end
 ; DISABLE-NEXT: popq %rbx
 ;
 ; CHECK: retq
 ;
 define i32 @loopInfoSaveOutsideLoop2(i32 %cond, i32 %N) #0 {
 entry:
   %tobool = icmp eq i32 %cond, 0
   br i1 %tobool, label %if.else, label %for.preheader