[SelectionDAG] Add icmp UNDEF handling to SelectionDAG::FoldSetCC

[RKSimon]

Summary

First half of PR40800, this patch adds DAG undef handling to icmp instructions to match the behaviour in SimplifyICmpInst, this permits constant folding of vector comparisons where some elements had been reduced to UNDEF (by SimplifyDemandedVectorElts etc.).

This involves a lot of tweaking to reduced tests as bugpoint loves to reduce icmp arguments to undef........

I've gone through these tests best I can, and AFAICT I haven't changed their purposes in any critical way - many are fragile and difficult to reduce from scratch as they are often 5+ years old.... I'd appreciate it if the respective target specialists could check my changes. I'd like to pre-commit some of these changes where its just a case of 'unreducing' lost arguments etc.

[efriedma]

test/CodeGen/ARM/ifcvt-branch-weight-bug.ll

1

LGTM to commit separately.

test/CodeGen/ARM/struct-byval-frame-index.ll

1

Also LGTM to commit separately. (It's sort of hard to tell what this is aiming for, but I think you're fine as long as you preserve the CFG.)

[tlively]

test/CodeGen/WebAssembly/cfg-stackify.ll

370

WebAssembly changes LGTM. Thanks for checking!

[RKSimon]

rebase - thanks @efriedma and @tlively

[RKSimon]

rebase after regenerating sparc test in rL356253

[kparzysz]

The Hexagon changes look good.

[RKSimon]

rebase - thanks @kparzysz

[jonpa]

I think the SystemZ test changes look ok (by replacing the undef operands with an argument the tests are more or less unaffected by this patch), but as usual I will let Uli do the formal approval.

[RKSimon]

@craig.topper Are you happy with the x86 changes?

[uweigand]

In D59363#1431933, @jonpa wrote:

I think the SystemZ test changes look ok (by replacing the undef operands with an argument the tests are more or less unaffected by this patch), but as usual I will let Uli do the formal approval.

The SystemZ test changes LGTM. It's probably better anyway to have real arguments instead of undefs ...

[RKSimon]

In D59363#1432683, @uweigand wrote:

In D59363#1431933, @jonpa wrote:

I think the SystemZ test changes look ok (by replacing the undef operands with an argument the tests are more or less unaffected by this patch), but as usual I will let Uli do the formal approval.

The SystemZ test changes LGTM. It's probably better anyway to have real arguments instead of undefs ...

Thanks @uweigand - once this patch goes in bugpoint should be less likely to reduce icmp arguments.

[RKSimon]

rebase - thanks @uweigand

[lebedev.ri]

test/CodeGen/X86/urem-seteq-vec-nonsplat.ll

686

Hmm, but -instsimplify says it's 0 https://godbolt.org/z/1dpyQG
Either answer is correct?

[RKSimon]

test/CodeGen/X86/urem-seteq-vec-nonsplat.ll

686

Interesting (please can you raise a bug?) - the patch isn't actually changing the result in DAG, its just managing to constant fold.

[spatel]

test/CodeGen/X86/urem-seteq-vec-nonsplat.ll

686

Probably not necessary? The urem invokes UB:
https://llvm.org/docs/LangRef.html#urem-instruction

So it's not that either answer is correct...*any* answer is correct. :)

[lebedev.ri]

test/CodeGen/X86/urem-seteq-vec-nonsplat.ll

686

https://bugs.llvm.org/show_bug.cgi?id=41125

[spatel]

test/CodeGen/X86/urem-seteq-vec-nonsplat.ll

686

So there are 2 questions of UB/undef/poison:

1. The urem causes immediate UB (same as div-by-0), so anything goes in this example.
2. Independent of that, what is the output of an icmp with undef operand? PR41125 shows that we have potentially 2 different answers in IR, so we need to decide which behavior is correct/optimal to mimic here in the DAG.

[RKSimon]

test/CodeGen/X86/urem-seteq-vec-nonsplat.ll

686

What I think I'm going to do is add the llvm::ConstantFoldCompareInstruction logic to both SimplifyICmpInst (in a new patch) and to FoldSetCC (in this patch):

If the predicate is eq/ne and any operand is undef then return undef - else return a bool constant using isTrueWhenEqual. That way we are consistent for all cases where the other operand is constant/non-constant,

Does that make sense?

[RKSimon]

Updated undef behaviour to match llvm::ConstantFoldCompareInstruction (see D59541 as well).

[RKSimon]

test/CodeGen/SPARC/missinglabel.ll

7–23

@dcederman Are you OK with this change please?

[RKSimon]

@spatel @craig.topper @efriedma any more comments?

[spatel]

The x86 regression test changes to preserve behavior look good. Pre-commit those, so we're just left actual diffs here?

[RKSimon]

rebase

[spatel]

LGTM

lib/CodeGen/SelectionDAG/SelectionDAG.cpp

SDValue SelectionDAG::FoldSetCC(EVT VT, SDValue N1, SDValue N2,
   case ISD::SETO:
   case ISD::SETUO:
   case ISD::SETUEQ:
   case ISD::SETUNE:
     assert(!OpVT.isInteger() && "Illegal setcc for integer!");
     break;
   }
 
-  // We can always fold X == X for integer setcc's.
-  if (N1 == N2 && OpVT.isInteger())
+  // icmp X, X -> true/false
+  // icmp X, undef -> true/false because undef could be X.
+  if (OpVT.isInteger())
+    if (N1 == N2 || N1.isUndef() || N2.isUndef())
       return getBoolConstant(ISD::isTrueWhenEqual(Cond), dl, VT, OpVT);
 
   if (ConstantSDNode *N2C = dyn_cast<ConstantSDNode>(N2)) {
     const APInt &C2 = N2C->getAPIntValue();
     if (ConstantSDNode *N1C = dyn_cast<ConstantSDNode>(N1)) {
       const APInt &C1 = N1C->getAPIntValue();
 
       switch (Cond) {
       default: llvm_unreachable("Unknown integer setcc!");

test/CodeGen/AArch64/urem-seteq-vec-nonsplat.ll

; CHECK-NEXT:    ret
   %cmp = icmp eq <4 x i32> %urem, <i32 1, i32 0, i32 1, i32 0>
   %ret = zext <4 x i1> %cmp to <4 x i32>
   ret <4 x i32> %ret
 }
 
 define <4 x i32> @test_urem_div_undef(<4 x i32> %X) nounwind readnone {
 ; CHECK-LABEL: test_urem_div_undef:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    cmeq v0.4s, v0.4s, #0
-; CHECK-NEXT:    movi v1.4s, #1
-; CHECK-NEXT:    and v0.16b, v0.16b, v1.16b
+; CHECK-NEXT:    movi v0.4s, #1
 ; CHECK-NEXT:    ret
   %urem = urem <4 x i32> %X, <i32 5, i32 5, i32 undef, i32 5>
   %cmp = icmp eq <4 x i32> %urem, <i32 0, i32 0, i32 0, i32 0>
   %ret = zext <4 x i1> %cmp to <4 x i32>
   ret <4 x i32> %ret
 }
 
 define <4 x i32> @test_urem_comp_undef(<4 x i32> %X) nounwind readnone {
; CHECK-NEXT:    ret
   %cmp = icmp eq <4 x i32> %urem, <i32 0, i32 undef, i32 0, i32 0>
   %ret = zext <4 x i1> %cmp to <4 x i32>
   ret <4 x i32> %ret
 }
 
 define <4 x i32> @test_urem_both_undef(<4 x i32> %X) nounwind readnone {
 ; CHECK-LABEL: test_urem_both_undef:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    cmeq v0.4s, v0.4s, #0
-; CHECK-NEXT:    movi v1.4s, #1
-; CHECK-NEXT:    and v0.16b, v0.16b, v1.16b
+; CHECK-NEXT:    movi v0.4s, #1
 ; CHECK-NEXT:    ret
   %urem = urem <4 x i32> %X, <i32 5, i32 5, i32 undef, i32 5>
   %cmp = icmp eq <4 x i32> %urem, <i32 0, i32 undef, i32 0, i32 0>
   %ret = zext <4 x i1> %cmp to <4 x i32>
   ret <4 x i32> %ret
 }
 
 define <4 x i32> @test_urem_div_even_odd(<4 x i32> %X) nounwind readnone {

test/CodeGen/ARM/ifcvt-branch-weight-bug.ll

 ; RUN: llc < %s -mtriple=armv4t--linux-androideabi -print-machineinstrs=if-converter -o /dev/null 2>&1 | FileCheck %s
 ; Fix a bug triggered in IfConverterTriangle when CvtBB has multiple
 ; predecessors.
 ; PR18752
 
 %classK = type { i8, %classF }
 %classF = type { i8 }
 %classL = type { %classG, i32, i32 }
 %classG = type { %classL* }
 %classM2 = type { %classL }
 
-define zeroext i1 @test(%classK* %this, %classM2* nocapture readnone %p1, %classM2* nocapture readnone %p2) align 2 {
+define zeroext i1 @test(%classK* %this, %classL* nocapture readnone %p0, %classM2* nocapture readnone %p1, %classM2* nocapture readnone %p2, i32 %a0) align 2 {
 entry:
   br i1 undef, label %for.end, label %for.body
 
 ; Before if conversion, we have
 ; for.body -> lor.lhs.false.i (50%)
 ;          -> for.cond.backedge (50%)
 ; lor.lhs.false.i -> for.cond.backedge (100%)
 ;                 -> cond.false.i (0%)
 ; Afer if conversion, we have
 ; for.body -> for.cond.backedge (100%)
 ;          -> cond.false.i (0%)
 ; CHECK: bb.1.for.body:
 ; CHECK: successors: %bb.2(0x80000000), %bb.4(0x00000001)
 for.body:
   br i1 undef, label %for.cond.backedge, label %lor.lhs.false.i, !prof !1
 
 for.cond.backedge:
-  %tobool = icmp eq %classL* undef, null
+  %tobool = icmp eq %classL* %p0, null
   br i1 %tobool, label %for.end, label %for.body
 
 lor.lhs.false.i:
-  %tobool.i.i7 = icmp eq i32 undef, 0
+  %tobool.i.i7 = icmp eq i32 %a0, 0
   br i1 %tobool.i.i7, label %for.cond.backedge, label %cond.false.i
 
 cond.false.i:
   call void @_Z3fn1v()
   unreachable
 
 for.end:
-  br i1 undef, label %if.else.i.i, label %if.then.i.i
+  %tobool.i.i9 = icmp eq i32 %a0, 32
+  br i1 %tobool.i.i9, label %if.else.i.i, label %if.then.i.i
 
 if.then.i.i:
   store %classL* null, %classL** undef, align 4
   br label %_ZN1M6spliceEv.exit
 
 if.else.i.i:
   store %classL* null, %classL** null, align 4
   br label %_ZN1M6spliceEv.exit

test/CodeGen/ARM/struct-byval-frame-index.ll

 ; RUN: llc < %s -mcpu=cortex-a15 -verify-machineinstrs -arm-atomic-cfg-tidy=0 | FileCheck %s
 
 ; Check a spill right after a function call with large struct byval is correctly
 ; generated.
 ; PR16393
 
 ; We expect 4-byte spill and reload to be generated.
 
 ; CHECK: set_stored_macroblock_parameters

 
 ; Function Attrs: nounwind
 declare void @llvm.memset.p0i8.i32(i8* nocapture, i8, i32, i1) #0
 
 ; Function Attrs: nounwind
 declare void @SetMotionVectorsMB(%structK* nocapture, i32) #1
 
 ; Function Attrs: nounwind
-define void @set_stored_macroblock_parameters() #1 {
+define void @set_stored_macroblock_parameters(i16 %a0, i32 %a1) #1 {
 entry:
   %0 = load %structB*, %structB** @img, align 4
   %1 = load i32, i32* undef, align 4
   %mb_data = getelementptr inbounds %structB, %structB* %0, i32 0, i32 61
   %2 = load %structK*, %structK** %mb_data, align 4
   br label %for.body
 
 for.body:                                         ; preds = %for.body, %entry

 for.cond210.preheader:                            ; preds = %if.then169
   unreachable
 
 if.end230:                                        ; preds = %if.end164
   tail call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 undef, i8* align 4 bitcast ([4 x i32]* @b8mode to i8*), i32 16, i1 false)
   %b8pdir = getelementptr inbounds %structK, %structK* %2, i32 %1, i32 15
   %3 = bitcast [4 x i32]* %b8pdir to i8*
   tail call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 %3, i8* align 4 bitcast ([4 x i32]* @b8pdir to i8*), i32 16, i1 false)
-  br i1 undef, label %if.end236, label %if.then233
+  %tobool.if.end230 = icmp ne i8* %3, null
+  br i1 %tobool.if.end230, label %if.end236, label %if.then233
 
 if.then233:                                       ; preds = %if.end230
   unreachable
 
 if.end236:                                        ; preds = %if.end230
-  %cmp242 = icmp ne i16 undef, 8
+  %cmp242 = icmp ne i16 %a0, 8
   %4 = load i32, i32* @luma_transform_size_8x8_flag, align 4
   %tobool245 = icmp ne i32 %4, 0
   %or.cond812 = or i1 %cmp242, %tobool245
   br i1 %or.cond812, label %if.end249, label %land.lhs.true246
 
 land.lhs.true246:                                 ; preds = %if.end236
-  br i1 undef, label %if.end249, label %if.then248
+  %tobool246 = icmp sgt i32 %4, 1
+  br i1 %tobool246, label %if.end249, label %if.then248
 
 if.then248:                                       ; preds = %land.lhs.true246
   tail call void asm sideeffect "", "~{r1},~{r2},~{r3},~{r4},~{r5},~{r6},~{r7},~{r8},~{r9},~{r10},~{r11}"() nounwind
   tail call void @RestoreMVBlock8x8(i32 1, i32 0, %structN* byval @tr8x8, i32 0) #0
   tail call void @RestoreMVBlock8x8(i32 1, i32 2, %structN* byval @tr8x8, i32 0) #0
   tail call void @RestoreMVBlock8x8(i32 1, i32 3, %structN* byval @tr8x8, i32 0) #0
   br label %if.end249
 
 if.end249:                                        ; preds = %if.then248, %land.lhs.true246, %if.end236
   %5 = load i32, i32* @luma_transform_size_8x8_flag, align 4
   %6 = load %structA*, %structA** @rdopt, align 4
   %luma_transform_size_8x8_flag264 = getelementptr inbounds %structA, %structA* %6, i32 0, i32 21
   store i32 %5, i32* %luma_transform_size_8x8_flag264, align 4
   %7 = load i32, i32* undef, align 4
-  %add281 = add nsw i32 %7, 0
+  %add281 = add nsw i32 %7, %4
   br label %for.body285
 
 for.body285:                                      ; preds = %for.inc503, %if.end249
   %8 = phi %structB* [ undef, %if.end249 ], [ %.pre1155, %for.inc503 ]
   %i.21103 = phi i32 [ 0, %if.end249 ], [ %inc504, %for.inc503 ]
   %block_x286 = getelementptr inbounds %structB, %structB* %8, i32 0, i32 37
   %9 = load i32, i32* %block_x286, align 4
   %add287 = add nsw i32 %9, %i.21103
   %shr289 = ashr i32 %i.21103, 1
   %add290 = add nsw i32 %shr289, 0
   %arrayidx292 = getelementptr inbounds %structK, %structK* %2, i32 %1, i32 15, i32 %add290
   %10 = load %structM*, %structM** @enc_picture, align 4
   %ref_idx = getelementptr inbounds %structM, %structM* %10, i32 0, i32 35
   %11 = load i8***, i8**** %ref_idx, align 4
   %12 = load i8**, i8*** %11, align 4
   %arrayidx313 = getelementptr inbounds i8*, i8** %12, i32 %add281
   %13 = load i8*, i8** %arrayidx313, align 4
   %arrayidx314 = getelementptr inbounds i8, i8* %13, i32 %add287
   store i8 -1, i8* %arrayidx314, align 1
   %14 = load %structB*, %structB** @img, align 4
   %MbaffFrameFlag327 = getelementptr inbounds %structB, %structB* %14, i32 0, i32 100
   %15 = load i32, i32* %MbaffFrameFlag327, align 4
-  %tobool328 = icmp eq i32 %15, 0
+  %tobool328 = icmp eq i32 %15, %a1
   br i1 %tobool328, label %if.end454, label %if.then329
 
 if.then329:                                       ; preds = %for.body285
   %16 = load %structA*, %structA** @rdopt, align 4
   br label %if.end454
 
 if.end454:                                        ; preds = %if.then329, %for.body285
   %17 = load i32, i32* %arrayidx292, align 4

test/CodeGen/Hexagon/autohvx/isel-setcc-pair.ll

 ; RUN: llc -march=hexagon -hexagon-initial-cfg-cleanup=0 < %s | FileCheck %s
 
 ; Check that a setcc of a vector pair is handled (without crashing).
 ; CHECK: vcmp
 
 target datalayout = "e-m:e-p:32:32:32-a:0-n16:32-i64:64:64-i32:32:32-i16:16:16-i1:8:8-f32:32:32-f64:64:64-v32:32:32-v64:64:64-v512:512:512-v1024:1024:1024-v2048:2048:2048"
 target triple = "hexagon"
 
 ; Function Attrs: nounwind
 define hidden fastcc void @fred(i32 %a0) #0 {
 b1:
   %v2 = insertelement <32 x i32> undef, i32 %a0, i32 0
   %v3 = shufflevector <32 x i32> %v2, <32 x i32> undef, <32 x i32> zeroinitializer
-  %v4 = icmp eq <32 x i32> %v3, undef
+  %v4 = icmp eq <32 x i32> %v2, %v3
   %v5 = and <32 x i1> undef, %v4
   br label %b6
 
 b6:                                               ; preds = %b1
   %v7 = extractelement <32 x i1> %v5, i32 22
   br i1 %v7, label %b8, label %b9
 
 b8:                                               ; preds = %b6
   unreachable
 
 b9:                                               ; preds = %b6
   unreachable
 }
 
 attributes #0 = { nounwind "target-cpu"="hexagonv60" "target-features"="+hvx,+hvx-length64b" }

test/CodeGen/Hexagon/expand-condsets-dead-pred.ll

 ; RUN: llc -march=hexagon -verify-machineinstrs -hexagon-initial-cfg-cleanup=0 < %s | FileCheck %s
 ; REQUIRES: asserts
 
 ; Check for some output (as opposed to a crash).
-; CHECK: loop0
+; CHECK: fred
 
 target triple = "hexagon"
 
 @x = external local_unnamed_addr global [80 x i32], align 8
 
 ; Function Attrs: nounwind
 define void @fred() local_unnamed_addr #0 {
 b0:

test/CodeGen/Hexagon/packetize-impdef-1.ll

 b23:                                              ; preds = %b21
   br label %b24
 
 b24:                                              ; preds = %b23, %b19
   %v27 = phi i64 [ 0, %b19 ], [ %v17, %b23 ]
   br label %b25
 
 b25:                                              ; preds = %b24
-  %v28 = icmp sgt i64 undef, %v27
+  %v28 = icmp sgt i64 0, %v27
   br i1 %v28, label %b28, label %b26
 
 b26:                                              ; preds = %b25
   unreachable
 
 b27:                                              ; preds = %b17, %b5
   call void @f2(i8* %v4)
   call void @f2(i8* %v0)

test/CodeGen/Hexagon/redundant-branching2.ll

 ; RUN: llc -march=hexagon -O2 < %s | FileCheck %s
 ; This test checks if redundant conditional branches are removed.
 
 ; CHECK: memub
 ; CHECK: memub
 ; CHECK: memub
 ; CHECK-NOT: if{{.*}}jump .LBB
 ; CHECK: cmp.eq
 
 target triple = "hexagon-unknown--elf"
 
 ; Function Attrs: nounwind
 declare void @f0() #0
 
 ; Function Attrs: nounwind
-define void @f1(i8* %a0, i32 %a1, i8* %a2, i32* %a3) #0 {
+define void @f1(i8* %a0, i32 %a1, i8* %a2, i32* %a3, i32 %a4) #0 {
 b0:
   br i1 undef, label %b8, label %b1
 
 b1:                                               ; preds = %b0
   tail call void @f0() #0
   br i1 false, label %b8, label %b2
 
 b2:                                               ; preds = %b1
   %v0 = getelementptr inbounds i8, i8* %a0, i32 undef
   %v1 = sub i32 0, %a1
-  %v2 = icmp eq i32 undef, undef
+  %v2 = icmp eq i32 %a1, %a4
   br label %b3
 
 b3:                                               ; preds = %b6, %b2
   %v3 = phi i8* [ %a2, %b2 ], [ %v17, %b6 ]
   %v4 = phi i8* [ %v0, %b2 ], [ null, %b6 ]
   %v5 = phi i32 [ 1, %b2 ], [ 0, %b6 ]
   br i1 %v2, label %b4, label %b5
 
b6:                                               ; preds = %b5, %b4
   %v19 = mul nsw i32 %v18, %v18
   %v20 = add nuw i32 %v16, 0
   %v21 = add i32 %v20, 0
   %v22 = add i32 %v21, 0
   %v23 = lshr i32 %v22, 1
   %v24 = add nuw nsw i32 %v23, %v19
   %v25 = add nsw i32 %v24, 0
   store i32 %v25, i32* %a3, align 4
-  %v26 = icmp eq i32 %v5, undef
+  %v26 = icmp eq i32 %v5, %a4
   br i1 %v26, label %b7, label %b3
 
 b7:                                               ; preds = %b6
   ret void
 
 b8:                                               ; preds = %b1, %b0
   ret void
 }
 
 attributes #0 = { nounwind "target-cpu"="hexagonv55" }

test/CodeGen/Hexagon/swp-change-deps.ll

 declare i32 @llvm.hexagon.A2.asrh(i32) #1
 
 ; Function Attrs: nounwind readnone
 declare i32 @llvm.hexagon.A2.addsat(i32, i32) #1
 
 ; Function Attrs: nounwind readnone
 declare i32 @llvm.hexagon.M2.mpy.sat.ll.s1(i32, i32) #1
 
-define void @f0() #0 align 2 {
+define void @f0(i32 %a0) #0 align 2 {
 b0:
   br label %b1
 
 b1:                                               ; preds = %b0
   br label %b2
 
 b2:                                               ; preds = %b2, %b1
   %v0 = phi i16* [ undef, %b1 ], [ %v14, %b2 ]
   %v1 = phi i32 [ 0, %b1 ], [ %v12, %b2 ]
   %v2 = load i16, i16* %v0, align 2
   %v3 = sext i16 %v2 to i32
   %v4 = call i32 @llvm.hexagon.M2.mpy.sat.ll.s1(i32 undef, i32 %v3)
   %v5 = call i32 @llvm.hexagon.S2.asr.r.r.sat(i32 %v4, i32 undef)
   %v6 = call i32 @llvm.hexagon.A2.addsat(i32 %v5, i32 32768)
   %v7 = call i32 @llvm.hexagon.A2.asrh(i32 %v6)
   %v8 = call i32 @llvm.hexagon.S2.asr.r.r.sat(i32 %v7, i32 undef)
   %v9 = call i32 @llvm.hexagon.A2.sath(i32 %v8)
   %v10 = trunc i32 %v9 to i16
   store i16 %v10, i16* null, align 2
   %v11 = trunc i32 %v7 to i16
   store i16 %v11, i16* %v0, align 2
   %v12 = add nsw i32 %v1, 1
-  %v13 = icmp slt i32 %v12, undef
+  %v13 = icmp slt i32 %v12, %a0
   %v14 = getelementptr i16, i16* %v0, i32 1
   br i1 %v13, label %b2, label %b3
 
 b3:                                               ; preds = %b2
   unreachable
 
 b4:                                               ; No predecessors!
   unreachable
 }
 
 attributes #0 = { nounwind "target-cpu"="hexagonv55" }
 attributes #1 = { nounwind readnone }

test/CodeGen/Hexagon/swp-epilog-phi2.ll

 ; RUN: llc -march=hexagon -enable-pipeliner -pipeliner-max-stages=3 < %s | FileCheck %s
 
 %s.0 = type { i16, i8, i8, i16, i8, i8, i16, i8, i8, i8, i8, i8, i8, i8, i8, i8, i8, i8, i8, i8, i8, i32, i16, i8, i8, %s.1, [2 x [16 x %s.2]], i32 (i8*, i8*, i8*, i8*, i8*)*, %s.3*, %s.3*, [120 x i8], i8, i8, %s.4*, [2 x [120 x [8 x i8]]], [56 x i8], [2 x [121 x %s.5]], [2 x %s.5], %s.5*, %s.5*, i32, i32, i16, i8, i8, %s.7, %s.9, %s.11, %s.8*, %s.8* }
 %s.1 = type { i8, i8, i8, i8, i8, i8, i8, i8, i32, i8, [16 x i8], i8, [4 x i8], [32 x i16], [32 x i16], [2 x i8], [4 x i8], [2 x [4 x i8]], [2 x [4 x i8]], i32, i32, i16, i8 }
 %s.2 = type { [2 x i16] }
 %s.3 = type { i16*, i16*, i32, i32 }
 %s.4 = type { i8*, i8*, i8*, i32, i32, i32, i32 }
 %s.5 = type { %s.6, [2 x [4 x %s.2]], [2 x [2 x i8]], [2 x i8] }
 %s.6 = type { i8, i8, i8, i8, i8, i8, i8, i8, i32 }
 %s.7 = type { [12 x %s.8], [4 x %s.8], [2 x %s.8], [4 x %s.8], [6 x %s.8], [2 x [7 x %s.8]], [4 x %s.8], [3 x [4 x %s.8]], [3 x %s.8], [3 x %s.8] }
 %s.8 = type { i8, i8 }
 %s.9 = type { [371 x %s.8], [6 x %s.10] }
 %s.10 = type { %s.8*, i8, i8, i8, i8, i8, i8, i8, i8, i8, i8, i8 }
 %s.11 = type { i32, i32, i8* }
 
 ; Function Attrs: nounwind
-define void @f0(%s.0* %a0, i8* %a1, i16* %a2, i16** %a3, i16** %a4) #0 {
+define void @f0(%s.0* %a0, i8* %a1, i16* %a2, i16** %a3, i16** %a4, i32 %a5) #0 {
 b0:
   %v0 = load i8, i8* %a1, align 1, !tbaa !0
   %v1 = icmp eq i8 %v0, 1
   br i1 %v1, label %b1, label %b2
 
 ; CHECK: loop0(.LBB0_[[LOOP:.]],
 ; CHECK: .LBB0_[[LOOP]]:
 ; CHECK: memh([[REG0:(r[0-9]+)]]+#0) = #0
b1:                                               ; preds = %b1, %b0
   %v15 = or i16 %v14, 0
   %v16 = getelementptr inbounds i16, i16* %v2, i32 1
   store i16* %v16, i16** %a3, align 4, !tbaa !3
   store i16 %v15, i16* %v2, align 2, !tbaa !5
   %v17 = getelementptr inbounds i16, i16* %v2, i32 2
   store i16* %v17, i16** %a4, align 4, !tbaa !3
   store i16 0, i16* %v16, align 2, !tbaa !5
   %v18 = add nsw i32 %v3, 8
-  %v19 = icmp slt i32 %v18, undef
+  %v19 = icmp slt i32 %v18, %a5
   br i1 %v19, label %b1, label %b2
 
 b2:                                               ; preds = %b1, %b0
   ret void
 }
 
 attributes #0 = { nounwind "target-cpu"="hexagonv55" }
 
 !0 = !{!1, !1, i64 0}
 !1 = !{!"omnipotent char", !2}
 !2 = !{!"Simple C/C++ TBAA"}
 !3 = !{!4, !4, i64 0}
 !4 = !{!"any pointer", !1}
 !5 = !{!6, !6, i64 0}
 !6 = !{!"short", !1}

test/CodeGen/Hexagon/swp-epilog-phi9.ll

 ; RUN: llc -march=hexagon -hexagon-initial-cfg-cleanup=0 < %s | FileCheck %s
 
 ; Test that we generate the correct Phi name in the last couple of epilog
 ; blocks, when there are 3 epilog blocks. The Phi was scheduled in stage
 ; 2, so the computation for the number of Phis needs to be adjusted when
 ; the incoming prolog block is from prolog 0 or prolog 1.
 ; Note: the pipeliner no longer generates a 3 stage pipeline for this test.
 
 ; CHECK: loop0
 ; CHECK: [[REG0:r([0-9]+)]] = add(r{{[0-8]+}},#8)
 ; CHECK: endloop0
 ; CHECK: [[REG0]] = add(r{{[0-9]+}},#8)
 
 ; Function Attrs: nounwind
-define void @f0(i16* nocapture readonly %a0) #0 {
+define void @f0(i16* nocapture readonly %a0, i32 %a1) #0 {
 b0:
   %v0 = alloca [129 x i32], align 8
   br i1 undef, label %b1, label %b3
 
 b1:                                               ; preds = %b0
   br label %b2
 
 b2:                                               ; preds = %b2, %b1
   %v1 = phi i16* [ %a0, %b1 ], [ %v2, %b2 ]
   %v2 = phi i16* [ undef, %b1 ], [ %v15, %b2 ]
   %v3 = phi i32* [ null, %b1 ], [ %v4, %b2 ]
   %v4 = phi i32* [ null, %b1 ], [ %v14, %b2 ]
   %v5 = phi i32 [ 0, %b1 ], [ %v13, %b2 ]
   %v6 = phi i16* [ undef, %b1 ], [ %v12, %b2 ]
   %v7 = load i16, i16* %v2, align 2
   %v8 = sext i16 %v7 to i32
   %v9 = call i32 @llvm.hexagon.M2.mpy.ll.s0(i32 %v8, i32 %v8) #2
   %v10 = load i16, i16* %v6, align 2
   %v11 = call i32 @llvm.hexagon.M2.mpy.acc.sat.ll.s0(i32 %v9, i32 undef, i32 undef) #2
   store i32 %v11, i32* %v4, align 4
   %v12 = getelementptr inbounds i16, i16* %v6, i32 -1
   %v13 = add i32 %v5, 1
   %v14 = getelementptr inbounds i32, i32* %v3, i32 2
   %v15 = getelementptr inbounds i16, i16* %v1, i32 2
-  %v16 = icmp slt i32 %v13, undef
+  %v16 = icmp slt i32 %v13, %a1
   br i1 %v16, label %b2, label %b3
 
 b3:                                               ; preds = %b2, %b0
   unreachable
 }
 
 ; Function Attrs: nounwind readnone
 declare i32 @llvm.hexagon.M2.mpy.ll.s0(i32, i32) #1
 
 ; Function Attrs: nounwind readnone
 declare i32 @llvm.hexagon.M2.mpy.acc.sat.ll.s0(i32, i32, i32) #1
 
 attributes #0 = { nounwind "target-cpu"="hexagonv60" }
 attributes #1 = { nounwind readnone }
 attributes #2 = { nounwind }

test/CodeGen/Hexagon/swp-prolog-phi.ll

 ; RUN: llc -march=hexagon -rdf-opt=0 < %s | FileCheck %s
 
 ; Test that we generate the correct name for a value in a prolog block. The
 ; pipeliner was using an incorrect value for an instruction in the 2nd prolog
 ; block for a value defined by a Phi. The result was that an instruction in
 ; the 1st and 2nd prolog blocks contain the same operands.
 
 ; CHECK: vcmp.gt([[VREG:(v[0-9]+)]].uh,v{{[0-9]+}}.uh)
 ; CHECK-NOT: vcmp.gt([[VREG]].uh,v{{[0-9]+}}.uh)
 ; CHECK: loop0
 
-define void @f0(<64 x i32> %a0, <32 x i32> %a1) #0 {
+define void @f0(<64 x i32> %a0, <32 x i32> %a1, i32 %a2) #0 {
 b0:
   br i1 undef, label %b1, label %b5
 
 b1:                                               ; preds = %b0
   %v0 = tail call <32 x i32> @llvm.hexagon.V6.hi.128B(<64 x i32> %a0)
   br label %b2
 
 b2:                                               ; preds = %b4, %b1
   %v1 = phi <32 x i32> [ %a1, %b1 ], [ %v7, %b4 ]
   br label %b3
 
 b3:                                               ; preds = %b3, %b2
   %v2 = phi i32 [ 0, %b2 ], [ %v8, %b3 ]
   %v3 = phi <32 x i32> [ zeroinitializer, %b2 ], [ %v0, %b3 ]
   %v4 = phi <32 x i32> [ %v1, %b2 ], [ %v7, %b3 ]
   %v5 = tail call <1024 x i1> @llvm.hexagon.V6.vgtuh.128B(<32 x i32> %v3, <32 x i32> undef)
   %v6 = tail call <1024 x i1> @llvm.hexagon.V6.veqh.and.128B(<1024 x i1> %v5, <32 x i32> undef, <32 x i32> undef)
   %v7 = tail call <32 x i32> @llvm.hexagon.V6.vaddhq.128B(<1024 x i1> %v6, <32 x i32> %v4, <32 x i32> undef)
   %v8 = add nsw i32 %v2, 1
-  %v9 = icmp slt i32 %v8, undef
+  %v9 = icmp slt i32 %v8, %a2
   br i1 %v9, label %b3, label %b4
 
 b4:                                               ; preds = %b3
   br i1 undef, label %b5, label %b2
 
 b5:                                               ; preds = %b4, %b0
   ret void
 }

test/CodeGen/Hexagon/swp-rename.ll

 ; RUN: llc -march=hexagon -enable-pipeliner < %s | FileCheck %s
 
 ; A test that the Phi rewrite logic is correct.
 
 ; CHECK: [[REG0:(r[0-9]+)]] = #0
 ; CHECK: loop0(.LBB0_[[LOOP:.]],
 ; CHECK: .LBB0_[[LOOP]]:
 ; CHECK: memh([[REG0]]+#0) = #0
 
-define void @f0() #0 {
+define void @f0(i32 %a0) #0 {
 b0:
-  %v0 = add i32 undef, -4
+  %v0 = add i32 %a0, -4
   br label %b1
 
 b1:                                               ; preds = %b1, %b0
   %v1 = phi i16* [ %v4, %b1 ], [ null, %b0 ]
   %v2 = phi i32 [ %v5, %b1 ], [ 0, %b0 ]
   %v3 = getelementptr inbounds i16, i16* %v1, i32 1
   store i16 0, i16* %v1, align 2
   %v4 = getelementptr inbounds i16, i16* %v1, i32 2

test/CodeGen/Hexagon/tail-dup-subreg-map.ll

 ; RUN: llc -march=hexagon -hexagon-initial-cfg-cleanup=0 < %s | FileCheck %s
 ; REQUIRES: asserts
 
 ; When tail-duplicating a block with PHI nodes that use subregisters, the
 ; subregisters were dropped by the tail duplicator, resulting in invalid
 ; COPY instructions being generated.
 
 ; CHECK: = asl(r{{[0-9]+}}:{{[0-9]+}},#15)
 
 target triple = "hexagon"
 
 %struct.0 = type { i64, i16 }
 %struct.1 = type { i64, i64 }
 
 declare hidden fastcc void @foo(%struct.0* noalias nocapture, i8 signext, i8 zeroext, i32, i64, i64) unnamed_addr #0
 
-define void @fred(%struct.0* noalias nocapture sret %agg.result, %struct.1* byval nocapture readonly align 8 %a) #1 {
+define void @fred(%struct.0* noalias nocapture sret %agg.result, %struct.1* byval nocapture readonly align 8 %a, i32 %a0) #1 {
 entry:
   %0 = load i64, i64* undef, align 8
-  switch i32 undef, label %if.else [
+  switch i32 %a0, label %if.else [
     i32 32767, label %if.then
     i32 0, label %if.then7
   ]
 
 if.then:                                          ; preds = %entry
   ret void
 
 if.then7:                                         ; preds = %entry

test/CodeGen/SPARC/missinglabel.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -verify-machineinstrs | FileCheck %s
+
 target datalayout = "E-m:e-i64:64-n32:64-S128"
 target triple = "sparc64-unknown-linux-gnu"
 
-define void @f() align 2 {
+define void @f(i64 %a0) align 2 {
+; CHECK-LABEL: f:
+; CHECK:         .cfi_startproc
+; CHECK-NEXT:  ! %bb.0: ! %entry
+; CHECK-NEXT:    cmp %o0, 0
+; CHECK-NEXT:    be %xcc, .LBB0_2
+; CHECK-NEXT:    nop
+; CHECK-NEXT:    ba .LBB0_1
+; CHECK-NEXT:    nop
+; CHECK-NEXT:  .LBB0_2: ! %targetblock
+; CHECK-NEXT:    mov %g0, %o0
+; CHECK-NEXT:    cmp %o0, 0
+; CHECK-NEXT:    bne .LBB0_4
+; CHECK-NEXT:    nop
+; CHECK-NEXT:  ! %bb.3: ! %cond.false.i83
+; CHECK-NEXT:  .LBB0_1: ! %cond.false
+; CHECK-NEXT:  .LBB0_4: ! %exit.i85
 entry:
-; CHECK: %xcc, .LBB0_2
-  %cmp = icmp eq i64 undef, 0
+  %cmp = icmp eq i64 %a0, 0
   br i1 %cmp, label %targetblock, label %cond.false
 
 cond.false:
   unreachable
 
-; CHECK: .LBB0_2: ! %targetblock
 targetblock:
   br i1 undef, label %cond.false.i83, label %exit.i85
 
 cond.false.i83:
   unreachable
 
 exit.i85:
   unreachable
 }

test/CodeGen/SystemZ/buildvector-00.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 | FileCheck %s
 
 ; Test that the dag combiner can understand that some vector operands are
 ; all-zeros and then optimize the logical operations.
 
-define void @f1() {
+define void @f1(<2 x i64> %a0) {
 ; CHECK-LABEL: f1:
 ; CHECK:       # %bb.0: # %bb
 ; CHECK-NEXT:    vlrepg %v0, 0(%r1)
 ; CHECK-NEXT:    vgbm %v1, 0
-; CHECK-NEXT:    vceqg %v2, %v0, %v1
+; CHECK-NEXT:    vceqg %v2, %v24, %v1
 ; CHECK-NEXT:    vn %v0, %v0, %v0
 ; CHECK-NEXT:    vno %v2, %v2, %v2
 ; CHECK-NEXT:    vceqg %v0, %v0, %v1
 ; CHECK-NEXT:    vx %v0, %v2, %v0
 ; CHECK-NEXT:    vnc %v0, %v2, %v0
 ; CHECK-NEXT:    vlgvf %r0, %v0, 1
 ; CHECK-NEXT:    tmll %r0, 1
 ; CHECK-NEXT:  # %bb.1: # %bb15
 
 bb:
   %tmp = shufflevector <2 x i64> undef, <2 x i64> undef, <2 x i32> zeroinitializer
   br label %bb1
 
 bb1:                                              ; preds = %bb
   %tmp2 = load i64, i64* undef, align 8
   %tmp3 = insertelement <2 x i64> undef, i64 %tmp2, i32 1
-  %tmp4 = icmp ne <2 x i64> undef, zeroinitializer
+  %tmp4 = icmp ne <2 x i64> %a0, zeroinitializer
   %tmp5 = xor <2 x i1> %tmp4, zeroinitializer
   %tmp6 = xor <2 x i1> zeroinitializer, %tmp5
   %tmp7 = and <2 x i64> %tmp3, %tmp
   %tmp8 = icmp ne <2 x i64> %tmp7, zeroinitializer
   %tmp9 = xor <2 x i1> zeroinitializer, %tmp8
-  %tmp10 = icmp ne <2 x i64> undef, zeroinitializer
+  %tmp10 = icmp ne <2 x i64> %a0, zeroinitializer
   %tmp11 = xor <2 x i1> %tmp10, %tmp9
   %tmp12 = and <2 x i1> %tmp6, %tmp11
   %tmp13 = extractelement <2 x i1> %tmp12, i32 0
   br i1 %tmp13, label %bb14, label %bb15
 
 bb14:                                             ; preds = %bb1
   store i64 undef, i64* undef, align 8
   br label %bb15
 
 bb15:                                             ; preds = %bb14, %bb1
   unreachable
 }

test/CodeGen/SystemZ/dag-combine-03.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=s390x-linux-gnu -mcpu=z13 < %s  | FileCheck %s
 
 ; Test that DAGCombiner gets helped by getKnownBitsForTargetNode() when
 ; BITCAST nodes are involved on a big-endian target.
 ;
 ; The EXTRACT_VECTOR_ELT is done first into an i32, and then AND:ed with
 ; 1. The AND is not actually necessary since the element contains a CC (i1)
 ; value. Test that the BITCAST nodes in the DAG when computing KnownBits is
 ; handled so that the AND is removed. If this succeeds, this results in a CHI
 ; instead of TMLL.
 
-define void @fun() {
+define void @fun(i64 %a0) {
 ; CHECK-LABEL: fun:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lghi %r1, 0
 ; CHECK-NEXT:  .LBB0_1: # %lab0
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    la %r0, 2(%r1)
 ; CHECK-NEXT:    la %r1, 1(%r1)
-; CHECK-NEXT:    cgr %r1, %r0
-; CHECK-NEXT:    lhi %r2, 0
-; CHECK-NEXT:    lochie %r2, 1
-; CHECK-NEXT:    cgr %r0, %r0
+; CHECK-NEXT:    cgr %r1, %r2
+; CHECK-NEXT:    lhi %r3, 0
+; CHECK-NEXT:    lochie %r3, 1
+; CHECK-NEXT:    cgr %r0, %r2
 ; CHECK-NEXT:    lhi %r0, 0
 ; CHECK-NEXT:    lochie %r0, 1
-; CHECK-NEXT:    vlvgp %v0, %r2, %r2
+; CHECK-NEXT:    vlvgp %v0, %r3, %r3
 ; CHECK-NEXT:    vlvgp %v1, %r0, %r0
 ; CHECK-NEXT:    vx %v0, %v0, %v1
 ; CHECK-NEXT:    vlgvf %r0, %v0, 1
 ; CHECK-NEXT:    chi %r0, 0
 ; CHECK-NEXT:    locghie %r1, 0
 ; CHECK-NEXT:    j .LBB0_1
 entry:
   br label %lab0
 
 lab0:
   %phi = phi i64 [ %sel, %lab0 ], [ 0, %entry ]
   %add = add nuw nsw i64 %phi, 1
   %add2 = add nuw nsw i64 %phi, 2
-  %cmp = icmp eq i64 %add, undef
-  %cmp2 = icmp eq i64 %add2, undef
+  %cmp = icmp eq i64 %add, %a0
+  %cmp2 = icmp eq i64 %add2, %a0
   %ins = insertelement <2 x i1> undef, i1 %cmp, i32 0
   %ins2 = insertelement <2 x i1> undef, i1 %cmp2, i32 0
   %xor = xor <2 x i1> %ins, %ins2
   %extr = extractelement <2 x i1> %xor, i32 0
 
   %sel = select i1 %extr, i64 %add, i64 0
   br label %lab0
 }

test/CodeGen/WebAssembly/cfg-stackify.ll

 ; Test that nested loops are handled.
 
 ; CHECK-LABEL: test3:
 ; CHECK: loop
 ; CHECK-NEXT: br_if
 ; CHECK-NEXT: .LBB{{[0-9]+}}_{{[0-9]+}}:
 ; CHECK-NEXT: loop
 declare void @bar()
-define void @test3(i32 %w)  {
+define void @test3(i32 %w, i32 %x)  {
 entry:
   br i1 undef, label %outer.ph, label %exit
 
 outer.ph:
   br label %outer
 
 outer:
-  %tobool = icmp eq i32 undef, 0
+  %tobool = icmp eq i32 %x, 0
   br i1 %tobool, label %inner, label %unreachable
 
 unreachable:
   unreachable
 
 inner:
-  %c = icmp eq i32 undef, %w
+  %c = icmp eq i32 %x, %w
   br i1 %c, label %if.end, label %inner
 
 exit:
   ret void
 
 if.end:
   call void @bar()
   br label %outer

test/CodeGen/X86/2006-11-17-IllegalMove.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s
 
 define void @handle_vector_size_attribute() nounwind {
 ; CHECK-LABEL: handle_vector_size_attribute:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl 0, %eax
 ; CHECK-NEXT:    decl %eax
 ; CHECK-NEXT:    cmpl $1, %eax
 ; CHECK-NEXT:    ja .LBB0_2
 ; CHECK-NEXT:  # %bb.1: # %bb77
 ; CHECK-NEXT:    movb 0, %al
-; CHECK-NEXT:    movzbl %al, %eax
-; CHECK-NEXT:    # kill: def $eax killed $eax def $ax
-; CHECK-NEXT:    divb 0
-; CHECK-NEXT:    movzbl %al, %eax
-; CHECK-NEXT:    cmpq %rax, %rax
+; CHECK-NEXT:    movb 0, %al
+; CHECK-NEXT:    movb $1, %al
+; CHECK-NEXT:    testb %al, %al
 ; CHECK-NEXT:  .LBB0_2: # %bb84
 ; CHECK-NEXT:    retq
 entry:
   %tmp69 = load i32, i32* null    ; <i32> [#uses=1]
   switch i32 %tmp69, label %bb84 [
      i32 2, label %bb77
      i32 1, label %bb77
   ]

test/CodeGen/X86/hoist-spill.ll

 ; RUN: llc < %s | FileCheck %s
 
 ; Check no spills to the same stack slot after hoisting.
 ; CHECK: mov{{.}} %{{.*}}, [[SPOFFSET1:-?[0-9]*]](%rsp)
 ; CHECK: mov{{.}} %{{.*}}, [[SPOFFSET2:-?[0-9]*]](%rsp)
 ; CHECK-NOT: mov{{.}} %{{.*}}, [[SPOFFSET1]](%rsp)
 ; CHECK-NOT: mov{{.}} %{{.*}}, [[SPOFFSET2]](%rsp)
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 @a = external global i32*, align 8
 @b = external global i32, align 4
 @d = external global i32*, align 8
 
 ; Function Attrs: norecurse noreturn nounwind uwtable
-define void @fn1(i32 %p1) {
+define void @fn1(i32 %p1, i32 %p2, i64 %p3) {
 entry:
   %tmp = load i32*, i32** @d, align 8
   %tmp1 = load i32*, i32** @a, align 8
   %tmp2 = sext i32 %p1 to i64
   br label %for.cond
 
 for.cond:                                         ; preds = %for.inc14, %entry
   %indvar = phi i32 [ %indvar.next, %for.inc14 ], [ 0, %entry ]
   %indvars.iv30.in = phi i32 [ %indvars.iv30, %for.inc14 ], [ %p1, %entry ]
   %c.0 = phi i32 [ %inc15, %for.inc14 ], [ 1, %entry ]
   %k.0 = phi i32 [ %k.1.lcssa, %for.inc14 ], [ undef, %entry ]
-  %tmp3 = icmp sgt i32 undef, 0
+  %tmp3 = icmp sgt i32 %p2, 0
   %smax52 = select i1 %tmp3, i32 %c.0, i32 0
   %tmp4 = zext i32 %smax52 to i64
-  %tmp5 = icmp sgt i64 undef, %tmp4
+  %tmp5 = icmp sgt i64 %p3, %tmp4
   %smax53 = select i1 %tmp5, i64 %tmp2, i64 %tmp4
   %tmp6 = add nsw i64 %smax53, 1
   %tmp7 = sub nsw i64 %tmp6, %tmp4
   %tmp8 = add nsw i64 %tmp7, -8
   %tmp9 = sub i32 undef, %indvar
   %tmp10 = icmp sgt i64 %tmp2, 0
   %smax40 = select i1 %tmp10, i64 %tmp2, i64 0
   %scevgep41 = getelementptr i32, i32* %tmp1, i64 %smax40

test/CodeGen/X86/select.ll

; MCU-NEXT:    retl
   %sel1 = select i1 %cmp, i32 32767, i32 %src
   %cmp1 = icmp slt i32 %sel1, -32768
   %sel2 = select i1 %cmp1, i32 -32768, i32 %sel1
   %conv = trunc i32 %sel2 to i16
   store i16 %conv, i16* %dst, align 2
   ret void
 }
 
-define void @test19() {
-; This is a massive reduction of an llvm-stress test case that generates
-; interesting chains feeding setcc and eventually a f32 select operation. This
-; is intended to exercise the SELECT formation in the DAG combine simplifying
-; a simplified select_cc node. If it it regresses and is no longer triggering
-; that code path, it can be deleted.
-;
-; CHECK-LABEL: test19:
-; CHECK:       ## %bb.0: ## %BB
-; CHECK-NEXT:    movl $-1, %eax
-; CHECK-NEXT:    movb $1, %cl
-; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  LBB23_1: ## %CF
-; CHECK-NEXT:    ## =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    testb %cl, %cl
-; CHECK-NEXT:    jne LBB23_1
-; CHECK-NEXT:  ## %bb.2: ## %CF250
-; CHECK-NEXT:    ## in Loop: Header=BB23_1 Depth=1
-; CHECK-NEXT:    jne LBB23_1
-; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  LBB23_3: ## %CF242
-; CHECK-NEXT:    ## =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    cmpl %eax, %eax
-; CHECK-NEXT:    ucomiss %xmm0, %xmm0
-; CHECK-NEXT:    jp LBB23_3
-; CHECK-NEXT:  ## %bb.4: ## %CF244
-; CHECK-NEXT:    retq
-;
-; ATHLON-LABEL: test19:
-; ATHLON:       ## %bb.0: ## %BB
-; ATHLON-NEXT:    movb $1, %al
-; ATHLON-NEXT:    .p2align 4, 0x90
-; ATHLON-NEXT:  LBB23_1: ## %CF
-; ATHLON-NEXT:    ## =>This Inner Loop Header: Depth=1
-; ATHLON-NEXT:    testb %al, %al
-; ATHLON-NEXT:    jne LBB23_1
-; ATHLON-NEXT:  ## %bb.2: ## %CF250
-; ATHLON-NEXT:    ## in Loop: Header=BB23_1 Depth=1
-; ATHLON-NEXT:    jne LBB23_1
-; ATHLON-NEXT:  ## %bb.3: ## %CF242.preheader
-; ATHLON-NEXT:    fldz
-; ATHLON-NEXT:    .p2align 4, 0x90
-; ATHLON-NEXT:  LBB23_4: ## %CF242
-; ATHLON-NEXT:    ## =>This Inner Loop Header: Depth=1
-; ATHLON-NEXT:    fucomi %st(0), %st
-; ATHLON-NEXT:    jp LBB23_4
-; ATHLON-NEXT:  ## %bb.5: ## %CF244
-; ATHLON-NEXT:    fstp %st(0)
-; ATHLON-NEXT:    retl
-;
-; MCU-LABEL: test19:
-; MCU:       # %bb.0: # %BB
-; MCU-NEXT:    movl $-1, %ecx
-; MCU-NEXT:    movb $1, %al
-; MCU-NEXT:    .p2align 4, 0x90
-; MCU-NEXT:  .LBB23_1: # %CF
-; MCU-NEXT:    # =>This Inner Loop Header: Depth=1
-; MCU-NEXT:    testb %al, %al
-; MCU-NEXT:    jne .LBB23_1
-; MCU-NEXT:  # %bb.2: # %CF250
-; MCU-NEXT:    # in Loop: Header=BB23_1 Depth=1
-; MCU-NEXT:    jne .LBB23_1
-; MCU-NEXT:  # %bb.3: # %CF242.preheader
-; MCU-NEXT:    fldz
-; MCU-NEXT:    .p2align 4, 0x90
-; MCU-NEXT:  .LBB23_4: # %CF242
-; MCU-NEXT:    # =>This Inner Loop Header: Depth=1
-; MCU-NEXT:    cmpl %eax, %ecx
-; MCU-NEXT:    fucom %st(0)
-; MCU-NEXT:    fnstsw %ax
-; MCU-NEXT:    # kill: def $ah killed $ah killed $ax
-; MCU-NEXT:    sahf
-; MCU-NEXT:    jp .LBB23_4
-; MCU-NEXT:  # %bb.5: # %CF244
-; MCU-NEXT:    fstp %st(0)
-; MCU-NEXT:    retl
-BB:
-  br label %CF
-
-CF:
-  %Cmp10 = icmp ule i8 undef, undef
-  br i1 %Cmp10, label %CF, label %CF250
-
-CF250:
-  %E12 = extractelement <4 x i32> <i32 -1, i32 -1, i32 -1, i32 -1>, i32 2
-  %Cmp32 = icmp ugt i1 %Cmp10, false
-  br i1 %Cmp32, label %CF, label %CF242
-
-CF242:
-  %Cmp38 = icmp uge i32 %E12, undef
-  %FC = uitofp i1 %Cmp38 to float
-  %Sl59 = select i1 %Cmp32, float %FC, float undef
-  %Cmp60 = fcmp ugt float undef, undef
-  br i1 %Cmp60, label %CF242, label %CF244
-
-CF244:
-  %B122 = fadd float %Sl59, undef
-  ret void
-}
-
 define i16 @select_xor_1(i16 %A, i8 %cond) {
 ; CHECK-LABEL: select_xor_1:
 ; CHECK:       ## %bb.0: ## %entry
 ; CHECK-NEXT:    movl %edi, %eax
 ; CHECK-NEXT:    xorl $43, %eax
 ; CHECK-NEXT:    testb $1, %sil
 ; CHECK-NEXT:    cmovel %edi, %eax
 ; CHECK-NEXT:    ## kill: def $ax killed $ax killed $eax

 ; ATHLON-NEXT:    testb $1, {{[0-9]+}}(%esp)
 ; ATHLON-NEXT:    cmovel %ecx, %eax
 ; ATHLON-NEXT:    ## kill: def $ax killed $ax killed $eax
 ; ATHLON-NEXT:    retl
 ;
 ; MCU-LABEL: select_xor_1b:
 ; MCU:       # %bb.0: # %entry
 ; MCU-NEXT:    testb $1, %dl
-; MCU-NEXT:    je .LBB25_2
+; MCU-NEXT:    je .LBB24_2
 ; MCU-NEXT:  # %bb.1:
 ; MCU-NEXT:    xorl $43, %eax
-; MCU-NEXT:  .LBB25_2: # %entry
+; MCU-NEXT:  .LBB24_2: # %entry
 ; MCU-NEXT:    # kill: def $ax killed $ax killed $eax
 ; MCU-NEXT:    retl
 entry:
  %and = and i8 %cond, 1
  %cmp10 = icmp ne i8 %and, 1
  %0 = xor i16 %A, 43
  %1 = select i1 %cmp10, i16 %A, i16 %0
  ret i16 %1

 ; ATHLON-NEXT:    xorl %ecx, %eax
 ; ATHLON-NEXT:    testb $1, {{[0-9]+}}(%esp)
 ; ATHLON-NEXT:    cmovel %ecx, %eax
 ; ATHLON-NEXT:    retl
 ;
 ; MCU-LABEL: select_xor_2b:
 ; MCU:       # %bb.0: # %entry
 ; MCU-NEXT:    testb $1, %cl
-; MCU-NEXT:    je .LBB27_2
+; MCU-NEXT:    je .LBB26_2
 ; MCU-NEXT:  # %bb.1:
 ; MCU-NEXT:    xorl %edx, %eax
-; MCU-NEXT:  .LBB27_2: # %entry
+; MCU-NEXT:  .LBB26_2: # %entry
 ; MCU-NEXT:    retl
 entry:
  %and = and i8 %cond, 1
  %cmp10 = icmp ne i8 %and, 1
  %0 = xor i32 %B, %A
  %1 = select i1 %cmp10, i32 %A, i32 %0
  ret i32 %1
 }

 ; ATHLON-NEXT:    orl %ecx, %eax
 ; ATHLON-NEXT:    testb $1, {{[0-9]+}}(%esp)
 ; ATHLON-NEXT:    cmovel %ecx, %eax
 ; ATHLON-NEXT:    retl
 ;
 ; MCU-LABEL: select_or_b:
 ; MCU:       # %bb.0: # %entry
 ; MCU-NEXT:    testb $1, %cl
-; MCU-NEXT:    je .LBB29_2
+; MCU-NEXT:    je .LBB28_2
 ; MCU-NEXT:  # %bb.1:
 ; MCU-NEXT:    orl %edx, %eax
-; MCU-NEXT:  .LBB29_2: # %entry
+; MCU-NEXT:  .LBB28_2: # %entry
 ; MCU-NEXT:    retl
 entry:
  %and = and i8 %cond, 1
  %cmp10 = icmp ne i8 %and, 1
  %0 = or i32 %B, %A
  %1 = select i1 %cmp10, i32 %A, i32 %0
  ret i32 %1
 }

 ; ATHLON-NEXT:    orl %ecx, %eax
 ; ATHLON-NEXT:    testb $1, {{[0-9]+}}(%esp)
 ; ATHLON-NEXT:    cmovel %ecx, %eax
 ; ATHLON-NEXT:    retl
 ;
 ; MCU-LABEL: select_or_1b:
 ; MCU:       # %bb.0: # %entry
 ; MCU-NEXT:    testb $1, %cl
-; MCU-NEXT:    je .LBB31_2
+; MCU-NEXT:    je .LBB30_2
 ; MCU-NEXT:  # %bb.1:
 ; MCU-NEXT:    orl %edx, %eax
-; MCU-NEXT:  .LBB31_2: # %entry
+; MCU-NEXT:  .LBB30_2: # %entry
 ; MCU-NEXT:    retl
 entry:
  %and = and i32 %cond, 1
  %cmp10 = icmp ne i32 %and, 1
  %0 = or i32 %B, %A
  %1 = select i1 %cmp10, i32 %A, i32 %0
  ret i32 %1
 }

test/CodeGen/X86/tail-dup-merge-loop-headers.ll

 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pushq %rbp
 ; CHECK-NEXT:    pushq %r15
 ; CHECK-NEXT:    pushq %r14
 ; CHECK-NEXT:    pushq %r13
 ; CHECK-NEXT:    pushq %r12
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    pushq %rax
-; CHECK-NEXT:    movl $1, %ebx
+; CHECK-NEXT:    movl $1, %r14d
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    testb %al, %al
-; CHECK-NEXT:    jne .LBB1_26
+; CHECK-NEXT:    jne .LBB1_15
 ; CHECK-NEXT:  # %bb.1: # %if.end19
 ; CHECK-NEXT:    movl %esi, %ebp
 ; CHECK-NEXT:    movq %rdi, %r12
-; CHECK-NEXT:    movl (%rax), %r14d
-; CHECK-NEXT:    leal (,%r14,4), %r13d
+; CHECK-NEXT:    movl (%rax), %ebx
+; CHECK-NEXT:    leal (,%rbx,4), %r13d
 ; CHECK-NEXT:    movl %r13d, %r15d
 ; CHECK-NEXT:    movl $1, %esi
 ; CHECK-NEXT:    movq %r15, %rdi
 ; CHECK-NEXT:    callq cli_calloc
 ; CHECK-NEXT:    testl %ebp, %ebp
-; CHECK-NEXT:    je .LBB1_25
+; CHECK-NEXT:    je .LBB1_15
 ; CHECK-NEXT:  # %bb.2: # %if.end19
-; CHECK-NEXT:    testl %r14d, %r14d
-; CHECK-NEXT:    je .LBB1_25
+; CHECK-NEXT:    testl %ebx, %ebx
+; CHECK-NEXT:    je .LBB1_15
 ; CHECK-NEXT:  # %bb.3: # %if.end19
-; CHECK-NEXT:    movq %rax, %rbx
-; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    testb %al, %al
-; CHECK-NEXT:    jne .LBB1_25
+; CHECK-NEXT:    xorl %ecx, %ecx
+; CHECK-NEXT:    testb %cl, %cl
+; CHECK-NEXT:    jne .LBB1_15
 ; CHECK-NEXT:  # %bb.4: # %if.end19
 ; CHECK-NEXT:    cmpq %r12, %rax
-; CHECK-NEXT:    jb .LBB1_25
+; CHECK-NEXT:    jb .LBB1_15
 ; CHECK-NEXT:  # %bb.5: # %if.end50
-; CHECK-NEXT:    movq %rbx, %rdi
+; CHECK-NEXT:    movq %rax, %rdi
 ; CHECK-NEXT:    movq %r15, %rdx
 ; CHECK-NEXT:    callq memcpy
 ; CHECK-NEXT:    cmpl $4, %r13d
-; CHECK-NEXT:    jb .LBB1_28
+; CHECK-NEXT:    jb .LBB1_23
 ; CHECK-NEXT:  # %bb.6: # %shared_preheader
-; CHECK-NEXT:    movb $32, %dl
+; CHECK-NEXT:    movb $32, %sil
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    # implicit-def: $rcx
-; CHECK-NEXT:    testl %eax, %eax
-; CHECK-NEXT:    je .LBB1_18
+; CHECK-NEXT:    movb $1, %cl
+; CHECK-NEXT:    # implicit-def: $rdx
+; CHECK-NEXT:    testl %ebx, %ebx
+; CHECK-NEXT:    jne .LBB1_11
+; CHECK-NEXT:    jmp .LBB1_8
+; CHECK-NEXT:  .LBB1_21: # %outer_loop_latch
+; CHECK-NEXT:    # in Loop: Header=BB1_11 Depth=1
+; CHECK-NEXT:    movzwl %di, %edi
+; CHECK-NEXT:    decl %edi
+; CHECK-NEXT:    movzwl %di, %edi
+; CHECK-NEXT:    leaq 1(%rdx,%rdi), %rdx
+; CHECK-NEXT:    testl %ebx, %ebx
+; CHECK-NEXT:    je .LBB1_8
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB1_8: # %shared_loop_header
+; CHECK-NEXT:  .LBB1_11: # %shared_loop_header
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    cmpq %rbx, %rax
-; CHECK-NEXT:    jb .LBB1_27
-; CHECK-NEXT:  # %bb.9: # %inner_loop_body
-; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
-; CHECK-NEXT:    testl %eax, %eax
-; CHECK-NEXT:    jns .LBB1_8
-; CHECK-NEXT:  # %bb.10: # %if.end96.i
-; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
-; CHECK-NEXT:    cmpl $3, %eax
-; CHECK-NEXT:    jae .LBB1_22
-; CHECK-NEXT:  # %bb.11: # %if.end287.i
-; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    cmpl $1, %eax
-; CHECK-NEXT:    setne %dl
 ; CHECK-NEXT:    testb %al, %al
-; CHECK-NEXT:    jne .LBB1_15
-; CHECK-NEXT:  # %bb.12: # %if.end308.i
-; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
+; CHECK-NEXT:    jne .LBB1_22
+; CHECK-NEXT:  # %bb.12: # %inner_loop_body
+; CHECK-NEXT:    # in Loop: Header=BB1_11 Depth=1
 ; CHECK-NEXT:    testb %al, %al
-; CHECK-NEXT:    je .LBB1_17
-; CHECK-NEXT:  # %bb.13: # %if.end335.i
-; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
-; CHECK-NEXT:    xorl %edx, %edx
-; CHECK-NEXT:    testb %dl, %dl
-; CHECK-NEXT:    movl $0, %esi
-; CHECK-NEXT:    jne .LBB1_7
-; CHECK-NEXT:  # %bb.14: # %merge_other
-; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    jmp .LBB1_16
-; CHECK-NEXT:  .LBB1_15: # in Loop: Header=BB1_8 Depth=1
-; CHECK-NEXT:    movb %dl, %sil
-; CHECK-NEXT:    addl $3, %esi
-; CHECK-NEXT:  .LBB1_16: # %outer_loop_latch
-; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
-; CHECK-NEXT:    # implicit-def: $dl
-; CHECK-NEXT:    jmp .LBB1_7
-; CHECK-NEXT:  .LBB1_17: # %merge_predecessor_split
-; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
-; CHECK-NEXT:    movb $32, %dl
+; CHECK-NEXT:    je .LBB1_11
+; CHECK-NEXT:  # %bb.13: # %if.end96.i
+; CHECK-NEXT:    # in Loop: Header=BB1_11 Depth=1
+; CHECK-NEXT:    testb %cl, %cl
+; CHECK-NEXT:    je .LBB1_14
+; CHECK-NEXT:  # %bb.16: # %if.end287.i
+; CHECK-NEXT:    # in Loop: Header=BB1_11 Depth=1
+; CHECK-NEXT:    movw $3, %di
+; CHECK-NEXT:    testb %al, %al
+; CHECK-NEXT:    # implicit-def: $sil
+; CHECK-NEXT:    jne .LBB1_21
+; CHECK-NEXT:  # %bb.17: # %if.end308.i
+; CHECK-NEXT:    # in Loop: Header=BB1_11 Depth=1
+; CHECK-NEXT:    testb %al, %al
+; CHECK-NEXT:    je .LBB1_18
+; CHECK-NEXT:  # %bb.19: # %if.end335.i
+; CHECK-NEXT:    # in Loop: Header=BB1_11 Depth=1
 ; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:  .LBB1_7: # %outer_loop_latch
-; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
-; CHECK-NEXT:    movzwl %si, %esi
-; CHECK-NEXT:    decl %esi
-; CHECK-NEXT:    movzwl %si, %esi
-; CHECK-NEXT:    leaq 1(%rcx,%rsi), %rcx
-; CHECK-NEXT:    testl %eax, %eax
-; CHECK-NEXT:    jne .LBB1_8
-; CHECK-NEXT:  .LBB1_18: # %while.cond.us1412.i
+; CHECK-NEXT:    testb %sil, %sil
+; CHECK-NEXT:    movl $0, %edi
+; CHECK-NEXT:    jne .LBB1_21
+; CHECK-NEXT:  # %bb.20: # %merge_other
+; CHECK-NEXT:    # in Loop: Header=BB1_11 Depth=1
+; CHECK-NEXT:    xorl %edi, %edi
+; CHECK-NEXT:    # implicit-def: $sil
+; CHECK-NEXT:    jmp .LBB1_21
+; CHECK-NEXT:  .LBB1_18: # %merge_predecessor_split
+; CHECK-NEXT:    # in Loop: Header=BB1_11 Depth=1
+; CHECK-NEXT:    movb $32, %sil
+; CHECK-NEXT:    xorl %edi, %edi
+; CHECK-NEXT:    jmp .LBB1_21
+; CHECK-NEXT:  .LBB1_8: # %while.cond.us1412.i
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    testb %al, %al
-; CHECK-NEXT:    movl $1, %ebx
-; CHECK-NEXT:    jne .LBB1_20
-; CHECK-NEXT:  # %bb.19: # %while.cond.us1412.i
-; CHECK-NEXT:    decb %dl
-; CHECK-NEXT:    jne .LBB1_26
-; CHECK-NEXT:  .LBB1_20: # %if.end41.us1436.i
-; CHECK-NEXT:  .LBB1_25:
-; CHECK-NEXT:    movl $1, %ebx
-; CHECK-NEXT:    jmp .LBB1_26
-; CHECK-NEXT:  .LBB1_22: # %if.then99.i
-; CHECK-NEXT:    xorl %ebx, %ebx
+; CHECK-NEXT:    jne .LBB1_10
+; CHECK-NEXT:  # %bb.9: # %while.cond.us1412.i
+; CHECK-NEXT:    decb %sil
+; CHECK-NEXT:    jne .LBB1_15
+; CHECK-NEXT:  .LBB1_10: # %if.end41.us1436.i
+; CHECK-NEXT:  .LBB1_14: # %if.then99.i
+; CHECK-NEXT:    xorl %r14d, %r14d
 ; CHECK-NEXT:    movl $.str.6, %edi
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    callq cli_dbgmsg
-; CHECK-NEXT:  .LBB1_26: # %cleanup
-; CHECK-NEXT:    movl %ebx, %eax
+; CHECK-NEXT:  .LBB1_15: # %cleanup
+; CHECK-NEXT:    movl %r14d, %eax
 ; CHECK-NEXT:    addq $8, %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %r12
 ; CHECK-NEXT:    popq %r13
 ; CHECK-NEXT:    popq %r14
 ; CHECK-NEXT:    popq %r15
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    retq
-; CHECK-NEXT:  .LBB1_27: # %wunpsect.exit.thread.loopexit389
-; CHECK-NEXT:  .LBB1_28: # %wunpsect.exit.thread.loopexit391
+; CHECK-NEXT:  .LBB1_22: # %wunpsect.exit.thread.loopexit389
+; CHECK-NEXT:  .LBB1_23: # %wunpsect.exit.thread.loopexit391
 entry:
   %0 = load i32, i32* undef, align 4
   %mul = shl nsw i32 %0, 2
   br i1 undef, label %if.end19, label %cleanup
 
 if.end19:                                         ; preds = %entry
   %conv = zext i32 %mul to i64
   %call = tail call i8* @cli_calloc(i64 %conv, i64 1)
   %1 = icmp eq i32 %exesz, 0
   %notrhs = icmp eq i32 %0, 0
   %or.cond117.not = or i1 %1, %notrhs
   %or.cond202 = or i1 %or.cond117.not, undef
-  %cmp35 = icmp ult i8* undef, %exe
+  %cmp35 = icmp ult i8* %call, %exe
   %or.cond203 = or i1 %or.cond202, %cmp35
   br i1 %or.cond203, label %cleanup, label %if.end50
 
 if.end50:                                         ; preds = %if.end19
   tail call void @llvm.memcpy.p0i8.p0i8.i64(i8* nonnull %call, i8* undef, i64 %conv, i1 false)
   %cmp1.i.i = icmp ugt i32 %mul, 3
   br i1 %cmp1.i.i, label %shared_preheader, label %wunpsect.exit.thread.loopexit391
 
 shared_preheader:                                 ; preds = %if.end50
   br label %outer_loop_header
 
 outer_loop_header:                                ; preds = %outer_loop_latch, %shared_preheader
   %bits.1.i = phi i8 [ 32, %shared_preheader ], [ %bits.43.i, %outer_loop_latch ]
   %dst.0.ph.i = phi i8* [ undef, %shared_preheader ], [ %scevgep679.i, %outer_loop_latch ]
-  %2 = icmp eq i32 undef, 0
+  %2 = icmp eq i32 %0, 0
   br i1 %2, label %while.cond.us1412.i, label %shared_loop_header
 
 while.cond.us1412.i:                              ; preds = %outer_loop_header
   %.pre.i = add i8 %bits.1.i, -1
   %tobool2.us1420.i = icmp eq i8 %.pre.i, 0
   %or.cond.us1421.i = or i1 undef, %tobool2.us1420.i
   br i1 %or.cond.us1421.i, label %if.end41.us1436.i, label %cleanup
 

test/CodeGen/X86/tail-dup-repeat.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s
 
 ; Function Attrs: uwtable
 ; When tail-duplicating during placement, we work backward from blocks with
 ; multiple successors. In this case, the block dup1 gets duplicated into dup2
 ; and if.then64, and then the block dup2 gets duplicated into land.lhs.true
 ; and if.end70
 
-define void @repeated_tail_dup(i1 %a1, i1 %a2, i32* %a4, i32* %a5, i8* %a6) #0 align 2 {
+define void @repeated_tail_dup(i1 %a1, i1 %a2, i32* %a4, i32* %a5, i8* %a6, i32 %a7) #0 align 2 {
 ; CHECK-LABEL: repeated_tail_dup:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_1: # %for.cond
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    testb $1, %dil
 ; CHECK-NEXT:    je .LBB0_3
 ; CHECK-NEXT:  # %bb.2: # %land.lhs.true
 ; CHECK-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; CHECK-NEXT:    movl $10, (%rdx)
 ; CHECK-NEXT:    jmp .LBB0_6
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_3: # %if.end56
 ; CHECK-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; CHECK-NEXT:    testb $1, %sil
 ; CHECK-NEXT:    je .LBB0_5
 ; CHECK-NEXT:  # %bb.4: # %if.then64
 ; CHECK-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; CHECK-NEXT:    movb $1, (%r8)
-; CHECK-NEXT:    testl %eax, %eax
+; CHECK-NEXT:    testl %r9d, %r9d
 ; CHECK-NEXT:    je .LBB0_1
 ; CHECK-NEXT:    jmp .LBB0_8
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_5: # %if.end70
 ; CHECK-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; CHECK-NEXT:    movl $12, (%rdx)
 ; CHECK-NEXT:  .LBB0_6: # %dup2
 ; CHECK-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; CHECK-NEXT:    movl $2, (%rcx)
-; CHECK-NEXT:    testl %eax, %eax
+; CHECK-NEXT:    testl %r9d, %r9d
 ; CHECK-NEXT:    je .LBB0_1
 ; CHECK-NEXT:  .LBB0_8: # %for.end
 ; CHECK-NEXT:    retq
 entry:
   br label %for.cond
 
 for.cond:                                         ; preds = %dup1, %entry
   br i1 %a1, label %land.lhs.true, label %if.end56
if.end70:                                         ; preds = %if.end56
   br label %dup2
 
 dup2:                                             ; preds = %if.end70, %land.lhs.true
   store i32 2, i32* %a5, align 4
   br label %dup1
 
 dup1:                                             ; preds = %dup2, %if.then64
   %val = load i32, i32* %a4, align 8
-  %switch = icmp ult i32 undef, 1
+  %switch = icmp ult i32 %a7, 1
   br i1 %switch, label %for.cond, label %for.end
 
 for.end:                                          ; preds = %dup1
   ret void
 }
 
 attributes #0 = { uwtable }

test/CodeGen/X86/undef-ops.ll

 
 ; This would crash because the shift amount is an i8 operand,
 ; but the result of the shift is i32. We can't just propagate
 ; the existing undef as the result.
 
 define i1 @undef_operand_size_not_same_as_result() {
 ; CHECK-LABEL: undef_operand_size_not_same_as_result:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    testl %eax, %eax
-; CHECK-NEXT:    sete %al
+; CHECK-NEXT:    movb $1, %al
 ; CHECK-NEXT:    retq
   %sh = shl i32 7, undef
   %cmp = icmp eq i32 0, %sh
   ret i1 %cmp
 }
 

test/CodeGen/X86/urem-seteq-vec-nonsplat.ll

; CHECK-AVX512VL-NEXT:    retq
   %cmp = icmp eq <4 x i32> %urem, <i32 1, i32 0, i32 1, i32 0>
   %ret = zext <4 x i1> %cmp to <4 x i32>
   ret <4 x i32> %ret
 }
 
 define <4 x i32> @test_urem_div_undef(<4 x i32> %X) nounwind readnone {
 ; CHECK-SSE-LABEL: test_urem_div_undef:
 ; CHECK-SSE:       # %bb.0:
-; CHECK-SSE-NEXT:    pxor %xmm0, %xmm0
-; CHECK-SSE-NEXT:    pcmpeqd %xmm0, %xmm0
-; CHECK-SSE-NEXT:    psrld $31, %xmm0
+; CHECK-SSE-NEXT:    movaps {{.*#+}} xmm0 = [1,1,1,1]
 ; CHECK-SSE-NEXT:    retq
 ;
-; CHECK-AVX-LABEL: test_urem_div_undef:
-; CHECK-AVX:       # %bb.0:
-; CHECK-AVX-NEXT:    vpxor %xmm0, %xmm0, %xmm0
-; CHECK-AVX-NEXT:    vpcmpeqd %xmm0, %xmm0, %xmm0
-; CHECK-AVX-NEXT:    vpsrld $31, %xmm0, %xmm0
-; CHECK-AVX-NEXT:    retq
+; CHECK-AVX1-LABEL: test_urem_div_undef:
+; CHECK-AVX1:       # %bb.0:
+; CHECK-AVX1-NEXT:    vmovaps {{.*#+}} xmm0 = [1,1,1,1]
+; CHECK-AVX1-NEXT:    retq
+;
+; CHECK-AVX2-LABEL: test_urem_div_undef:
+; CHECK-AVX2:       # %bb.0:
+; CHECK-AVX2-NEXT:    vbroadcastss {{.*#+}} xmm0 = [1,1,1,1]
+; CHECK-AVX2-NEXT:    retq
+;
+; CHECK-AVX512VL-LABEL: test_urem_div_undef:
+; CHECK-AVX512VL:       # %bb.0:
+; CHECK-AVX512VL-NEXT:    vbroadcastss {{.*#+}} xmm0 = [1,1,1,1]
+; CHECK-AVX512VL-NEXT:    retq
   %urem = urem <4 x i32> %X, <i32 5, i32 5, i32 undef, i32 5>
   %cmp = icmp eq <4 x i32> %urem, <i32 0, i32 0, i32 0, i32 0>
   %ret = zext <4 x i1> %cmp to <4 x i32>
   ret <4 x i32> %ret
 }
 
 define <4 x i32> @test_urem_comp_undef(<4 x i32> %X) nounwind readnone {
 ; CHECK-SSE2-LABEL: test_urem_comp_undef:
; CHECK-AVX512VL-NEXT:    retq
   %cmp = icmp eq <4 x i32> %urem, <i32 0, i32 undef, i32 0, i32 0>
   %ret = zext <4 x i1> %cmp to <4 x i32>
   ret <4 x i32> %ret
 }
 
 define <4 x i32> @test_urem_both_undef(<4 x i32> %X) nounwind readnone {
 ; CHECK-SSE-LABEL: test_urem_both_undef:
 ; CHECK-SSE:       # %bb.0:
-; CHECK-SSE-NEXT:    pxor %xmm0, %xmm0
-; CHECK-SSE-NEXT:    pcmpeqd %xmm0, %xmm0
-; CHECK-SSE-NEXT:    psrld $31, %xmm0
+; CHECK-SSE-NEXT:    movaps {{.*#+}} xmm0 = [1,1,1,1]
 ; CHECK-SSE-NEXT:    retq
 ;
-; CHECK-AVX-LABEL: test_urem_both_undef:
-; CHECK-AVX:       # %bb.0:
-; CHECK-AVX-NEXT:    vpxor %xmm0, %xmm0, %xmm0
-; CHECK-AVX-NEXT:    vpcmpeqd %xmm0, %xmm0, %xmm0
-; CHECK-AVX-NEXT:    vpsrld $31, %xmm0, %xmm0
-; CHECK-AVX-NEXT:    retq
+; CHECK-AVX1-LABEL: test_urem_both_undef:
+; CHECK-AVX1:       # %bb.0:
+; CHECK-AVX1-NEXT:    vmovaps {{.*#+}} xmm0 = [1,1,1,1]
+; CHECK-AVX1-NEXT:    retq
+;
+; CHECK-AVX2-LABEL: test_urem_both_undef:
+; CHECK-AVX2:       # %bb.0:
+; CHECK-AVX2-NEXT:    vbroadcastss {{.*#+}} xmm0 = [1,1,1,1]
+; CHECK-AVX2-NEXT:    retq
+;
+; CHECK-AVX512VL-LABEL: test_urem_both_undef:
+; CHECK-AVX512VL:       # %bb.0:
+; CHECK-AVX512VL-NEXT:    vbroadcastss {{.*#+}} xmm0 = [1,1,1,1]
+; CHECK-AVX512VL-NEXT:    retq
   %urem = urem <4 x i32> %X, <i32 5, i32 5, i32 undef, i32 5>
   %cmp = icmp eq <4 x i32> %urem, <i32 0, i32 undef, i32 0, i32 0>
   %ret = zext <4 x i1> %cmp to <4 x i32>
   ret <4 x i32> %ret
 }
 
 define <4 x i32> @test_urem_div_even_odd(<4 x i32> %X) nounwind readnone {
 ; CHECK-SSE2-LABEL: test_urem_div_even_odd:

test/CodeGen/X86/vec_int_to_fp-widen.ll

 ; VEX-NEXT:    movq %rax, %rcx
 ; VEX-NEXT:    shrq %rcx
 ; VEX-NEXT:    andl $1, %eax
 ; VEX-NEXT:    orq %rcx, %rax
 ; VEX-NEXT:    vcvtsi2ssq %rax, %xmm2, %xmm0
 ; VEX-NEXT:    vaddss %xmm0, %xmm0, %xmm0
 ; VEX-NEXT:  .LBB39_6:
 ; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[2,3]
-; VEX-NEXT:    testq %rax, %rax
-; VEX-NEXT:    vxorps %xmm1, %xmm1, %xmm1
-; VEX-NEXT:    js .LBB39_8
-; VEX-NEXT:  # %bb.7:
 ; VEX-NEXT:    vcvtsi2ssq %rax, %xmm2, %xmm1
-; VEX-NEXT:  .LBB39_8:
 ; VEX-NEXT:    vshufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0,0]
 ; VEX-NEXT:    retq
 ;
 ; AVX512F-LABEL: uitofp_2i64_to_4f32:
 ; AVX512F:       # %bb.0:
 ; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
 ; AVX512F-NEXT:    vcvtusi2ssq %rax, %xmm1, %xmm1
 ; AVX512F-NEXT:    vmovq %xmm0, %rax

 ; SSE2-NEXT:    shrq %rcx
 ; SSE2-NEXT:    andl $1, %eax
 ; SSE2-NEXT:    orq %rcx, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2ssq %rax, %xmm1
 ; SSE2-NEXT:    addss %xmm1, %xmm1
 ; SSE2-NEXT:  .LBB41_6:
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
-; SSE2-NEXT:    testq %rax, %rax
-; SSE2-NEXT:    xorps %xmm1, %xmm1
-; SSE2-NEXT:    js .LBB41_8
-; SSE2-NEXT:  # %bb.7:
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2ssq %rax, %xmm1
-; SSE2-NEXT:  .LBB41_8:
 ; SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0,0]
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: uitofp_4i64_to_4f32_undef:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    pextrq $1, %xmm0, %rax
 ; SSE41-NEXT:    testq %rax, %rax
 ; SSE41-NEXT:    js .LBB41_1

 ; SSE41-NEXT:    shrq %rcx
 ; SSE41-NEXT:    andl $1, %eax
 ; SSE41-NEXT:    orq %rcx, %rax
 ; SSE41-NEXT:    xorps %xmm0, %xmm0
 ; SSE41-NEXT:    cvtsi2ssq %rax, %xmm0
 ; SSE41-NEXT:    addss %xmm0, %xmm0
 ; SSE41-NEXT:  .LBB41_6:
 ; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[2,3]
-; SSE41-NEXT:    testq %rax, %rax
-; SSE41-NEXT:    xorps %xmm1, %xmm1
-; SSE41-NEXT:    js .LBB41_8
-; SSE41-NEXT:  # %bb.7:
 ; SSE41-NEXT:    xorps %xmm1, %xmm1
 ; SSE41-NEXT:    cvtsi2ssq %rax, %xmm1
-; SSE41-NEXT:  .LBB41_8:
 ; SSE41-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0,0]
 ; SSE41-NEXT:    retq
 ;
 ; VEX-LABEL: uitofp_4i64_to_4f32_undef:
 ; VEX:       # %bb.0:
 ; VEX-NEXT:    vpextrq $1, %xmm0, %rax
 ; VEX-NEXT:    testq %rax, %rax
 ; VEX-NEXT:    js .LBB41_1

 ; VEX-NEXT:    movq %rax, %rcx
 ; VEX-NEXT:    shrq %rcx
 ; VEX-NEXT:    andl $1, %eax
 ; VEX-NEXT:    orq %rcx, %rax
 ; VEX-NEXT:    vcvtsi2ssq %rax, %xmm2, %xmm0
 ; VEX-NEXT:    vaddss %xmm0, %xmm0, %xmm0
 ; VEX-NEXT:  .LBB41_6:
 ; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[2,3]
-; VEX-NEXT:    testq %rax, %rax
-; VEX-NEXT:    vxorps %xmm1, %xmm1, %xmm1
-; VEX-NEXT:    js .LBB41_8
-; VEX-NEXT:  # %bb.7:
 ; VEX-NEXT:    vcvtsi2ssq %rax, %xmm2, %xmm1
-; VEX-NEXT:  .LBB41_8:
 ; VEX-NEXT:    vshufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0,0]
 ; VEX-NEXT:    retq
 ;
 ; AVX512F-LABEL: uitofp_4i64_to_4f32_undef:
 ; AVX512F:       # %bb.0:
 ; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
 ; AVX512F-NEXT:    vcvtusi2ssq %rax, %xmm1, %xmm1
 ; AVX512F-NEXT:    vmovq %xmm0, %rax

test/CodeGen/X86/vec_int_to_fp.ll

 ; VEX-NEXT:    movq %rax, %rcx
 ; VEX-NEXT:    shrq %rcx
 ; VEX-NEXT:    andl $1, %eax
 ; VEX-NEXT:    orq %rcx, %rax
 ; VEX-NEXT:    vcvtsi2ssq %rax, %xmm2, %xmm0
 ; VEX-NEXT:    vaddss %xmm0, %xmm0, %xmm0
 ; VEX-NEXT:  .LBB39_6:
 ; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[2,3]
-; VEX-NEXT:    testq %rax, %rax
-; VEX-NEXT:    vxorps %xmm1, %xmm1, %xmm1
-; VEX-NEXT:    js .LBB39_8
-; VEX-NEXT:  # %bb.7:
 ; VEX-NEXT:    vcvtsi2ssq %rax, %xmm2, %xmm1
-; VEX-NEXT:  .LBB39_8:
 ; VEX-NEXT:    vshufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0,0]
 ; VEX-NEXT:    retq
 ;
 ; AVX512F-LABEL: uitofp_2i64_to_4f32:
 ; AVX512F:       # %bb.0:
 ; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
 ; AVX512F-NEXT:    vcvtusi2ssq %rax, %xmm1, %xmm1
 ; AVX512F-NEXT:    vmovq %xmm0, %rax

 ; SSE2-NEXT:    shrq %rcx
 ; SSE2-NEXT:    andl $1, %eax
 ; SSE2-NEXT:    orq %rcx, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2ssq %rax, %xmm1
 ; SSE2-NEXT:    addss %xmm1, %xmm1
 ; SSE2-NEXT:  .LBB41_6:
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
-; SSE2-NEXT:    testq %rax, %rax
-; SSE2-NEXT:    xorps %xmm1, %xmm1
-; SSE2-NEXT:    js .LBB41_8
-; SSE2-NEXT:  # %bb.7:
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2ssq %rax, %xmm1
-; SSE2-NEXT:  .LBB41_8:
 ; SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0,0]
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: uitofp_4i64_to_4f32_undef:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    pextrq $1, %xmm0, %rax
 ; SSE41-NEXT:    testq %rax, %rax
 ; SSE41-NEXT:    js .LBB41_1

 ; SSE41-NEXT:    shrq %rcx
 ; SSE41-NEXT:    andl $1, %eax
 ; SSE41-NEXT:    orq %rcx, %rax
 ; SSE41-NEXT:    xorps %xmm0, %xmm0
 ; SSE41-NEXT:    cvtsi2ssq %rax, %xmm0
 ; SSE41-NEXT:    addss %xmm0, %xmm0
 ; SSE41-NEXT:  .LBB41_6:
 ; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[2,3]
-; SSE41-NEXT:    testq %rax, %rax
-; SSE41-NEXT:    xorps %xmm1, %xmm1
-; SSE41-NEXT:    js .LBB41_8
-; SSE41-NEXT:  # %bb.7:
 ; SSE41-NEXT:    xorps %xmm1, %xmm1
 ; SSE41-NEXT:    cvtsi2ssq %rax, %xmm1
-; SSE41-NEXT:  .LBB41_8:
 ; SSE41-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0,0]
 ; SSE41-NEXT:    retq
 ;
 ; VEX-LABEL: uitofp_4i64_to_4f32_undef:
 ; VEX:       # %bb.0:
 ; VEX-NEXT:    vpextrq $1, %xmm0, %rax
 ; VEX-NEXT:    testq %rax, %rax
 ; VEX-NEXT:    js .LBB41_1

 ; VEX-NEXT:    movq %rax, %rcx
 ; VEX-NEXT:    shrq %rcx
 ; VEX-NEXT:    andl $1, %eax
 ; VEX-NEXT:    orq %rcx, %rax
 ; VEX-NEXT:    vcvtsi2ssq %rax, %xmm2, %xmm0
 ; VEX-NEXT:    vaddss %xmm0, %xmm0, %xmm0
 ; VEX-NEXT:  .LBB41_6:
 ; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[2,3]
-; VEX-NEXT:    testq %rax, %rax
-; VEX-NEXT:    vxorps %xmm1, %xmm1, %xmm1
-; VEX-NEXT:    js .LBB41_8
-; VEX-NEXT:  # %bb.7:
 ; VEX-NEXT:    vcvtsi2ssq %rax, %xmm2, %xmm1
-; VEX-NEXT:  .LBB41_8:
 ; VEX-NEXT:    vshufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0,0]
 ; VEX-NEXT:    retq
 ;
 ; AVX512F-LABEL: uitofp_4i64_to_4f32_undef:
 ; AVX512F:       # %bb.0:
 ; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
 ; AVX512F-NEXT:    vcvtusi2ssq %rax, %xmm1, %xmm1
 ; AVX512F-NEXT:    vmovq %xmm0, %rax

test/CodeGen/X86/vector-shift-ashr-sub128-widen.ll

 ; SSE2-NEXT:    andps %xmm2, %xmm0
 ; SSE2-NEXT:    psraw $1, %xmm1
 ; SSE2-NEXT:    andnps %xmm1, %xmm2
 ; SSE2-NEXT:    orps %xmm2, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: constant_shift_v4i16:
 ; SSE41:       # %bb.0:
-; SSE41-NEXT:    movdqa %xmm0, %xmm1
-; SSE41-NEXT:    pxor %xmm0, %xmm0
-; SSE41-NEXT:    movdqa {{.*#+}} xmm2 = <0,1,2,3,u,u,u,u>
-; SSE41-NEXT:    pcmpeqw %xmm2, %xmm0
-; SSE41-NEXT:    movdqa {{.*#+}} xmm3 = <u,32768,16384,8192,u,u,u,u>
-; SSE41-NEXT:    pmulhw %xmm1, %xmm3
-; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm3
-; SSE41-NEXT:    pcmpeqw {{.*}}(%rip), %xmm2
-; SSE41-NEXT:    psraw $1, %xmm1
-; SSE41-NEXT:    movdqa %xmm2, %xmm0
-; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm3
-; SSE41-NEXT:    movdqa %xmm3, %xmm0
+; SSE41-NEXT:    movdqa {{.*#+}} xmm1 = <u,32768,16384,8192,u,u,u,u>
+; SSE41-NEXT:    pmulhw %xmm0, %xmm1
+; SSE41-NEXT:    pblendw {{.*#+}} xmm1 = xmm0[0],xmm1[1,2,3],xmm0[4,5,6,7]
+; SSE41-NEXT:    psraw $1, %xmm0
+; SSE41-NEXT:    pblendw {{.*#+}} xmm0 = xmm1[0],xmm0[1],xmm1[2,3],xmm0[4,5,6,7]
 ; SSE41-NEXT:    retq
 ;
-; AVX1-LABEL: constant_shift_v4i16:
-; AVX1:       # %bb.0:
-; AVX1-NEXT:    vpxor %xmm1, %xmm1, %xmm1
-; AVX1-NEXT:    vmovddup {{.*#+}} xmm2 = [4.1720559249406128E-309,4.1720559249406128E-309]
-; AVX1-NEXT:    # xmm2 = mem[0,0]
-; AVX1-NEXT:    vpcmpeqw %xmm1, %xmm2, %xmm1
-; AVX1-NEXT:    vpmulhw {{.*}}(%rip), %xmm0, %xmm3
-; AVX1-NEXT:    vpblendvb %xmm1, %xmm0, %xmm3, %xmm1
-; AVX1-NEXT:    vpcmpeqw {{.*}}(%rip), %xmm2, %xmm2
-; AVX1-NEXT:    vpsraw $1, %xmm0, %xmm0
-; AVX1-NEXT:    vpblendvb %xmm2, %xmm0, %xmm1, %xmm0
-; AVX1-NEXT:    retq
-;
-; AVX2-LABEL: constant_shift_v4i16:
-; AVX2:       # %bb.0:
-; AVX2-NEXT:    vpxor %xmm1, %xmm1, %xmm1
-; AVX2-NEXT:    vpbroadcastq {{.*#+}} xmm2 = [844433520132096,844433520132096]
-; AVX2-NEXT:    vpcmpeqw %xmm1, %xmm2, %xmm1
-; AVX2-NEXT:    vpmulhw {{.*}}(%rip), %xmm0, %xmm3
-; AVX2-NEXT:    vpblendvb %xmm1, %xmm0, %xmm3, %xmm1
-; AVX2-NEXT:    vpcmpeqw {{.*}}(%rip), %xmm2, %xmm2
-; AVX2-NEXT:    vpsraw $1, %xmm0, %xmm0
-; AVX2-NEXT:    vpblendvb %xmm2, %xmm0, %xmm1, %xmm0
-; AVX2-NEXT:    retq
+; AVX-LABEL: constant_shift_v4i16:
+; AVX:       # %bb.0:
+; AVX-NEXT:    vpmulhw {{.*}}(%rip), %xmm0, %xmm1
+; AVX-NEXT:    vpblendw {{.*#+}} xmm1 = xmm0[0],xmm1[1,2,3],xmm0[4,5,6,7]
+; AVX-NEXT:    vpsraw $1, %xmm0, %xmm0
+; AVX-NEXT:    vpblendw {{.*#+}} xmm0 = xmm1[0],xmm0[1],xmm1[2,3],xmm0[4,5,6,7]
+; AVX-NEXT:    retq
 ;
 ; XOP-LABEL: constant_shift_v4i16:
 ; XOP:       # %bb.0:
 ; XOP-NEXT:    vpshaw {{.*}}(%rip), %xmm0, %xmm0
 ; XOP-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: constant_shift_v4i16:
 ; AVX512DQ:       # %bb.0:

test/CodeGen/X86/vector-shift-lshr-sub128-widen.ll

 ; X32-SSE-NEXT:    retl
   %shift = lshr <2 x i32> %a, <i32 4, i32 5>
   ret <2 x i32> %shift
 }
 
 define <4 x i16> @constant_shift_v4i16(<4 x i16> %a) nounwind {
 ; SSE2-LABEL: constant_shift_v4i16:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    movdqa {{.*#+}} xmm1 = <u,32768,16384,8192,u,u,u,u>
-; SSE2-NEXT:    pmulhuw %xmm0, %xmm1
-; SSE2-NEXT:    pxor %xmm2, %xmm2
-; SSE2-NEXT:    pcmpeqw {{.*}}(%rip), %xmm2
-; SSE2-NEXT:    pand %xmm2, %xmm0
-; SSE2-NEXT:    pandn %xmm1, %xmm2
-; SSE2-NEXT:    por %xmm2, %xmm0
+; SSE2-NEXT:    movdqa {{.*#+}} xmm2 = [65535,0,0,0,65535,65535,65535,65535]
+; SSE2-NEXT:    movdqa %xmm0, %xmm1
+; SSE2-NEXT:    pand %xmm2, %xmm1
+; SSE2-NEXT:    pmulhuw {{.*}}(%rip), %xmm0
+; SSE2-NEXT:    pandn %xmm0, %xmm2
+; SSE2-NEXT:    por %xmm2, %xmm1
+; SSE2-NEXT:    movdqa %xmm1, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: constant_shift_v4i16:
 ; SSE41:       # %bb.0:
-; SSE41-NEXT:    movdqa %xmm0, %xmm1
-; SSE41-NEXT:    pxor %xmm0, %xmm0
-; SSE41-NEXT:    pcmpeqw {{.*}}(%rip), %xmm0
-; SSE41-NEXT:    movdqa {{.*#+}} xmm2 = <u,32768,16384,8192,u,u,u,u>
-; SSE41-NEXT:    pmulhuw %xmm1, %xmm2
-; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm2
-; SSE41-NEXT:    movdqa %xmm2, %xmm0
+; SSE41-NEXT:    movdqa {{.*#+}} xmm1 = <u,32768,16384,8192,u,u,u,u>
+; SSE41-NEXT:    pmulhuw %xmm0, %xmm1
+; SSE41-NEXT:    pblendw {{.*#+}} xmm0 = xmm0[0],xmm1[1,2,3],xmm0[4,5,6,7]
 ; SSE41-NEXT:    retq
 ;
 ; AVX-LABEL: constant_shift_v4i16:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vpxor %xmm1, %xmm1, %xmm1
-; AVX-NEXT:    vpcmpeqw {{.*}}(%rip), %xmm1, %xmm1
-; AVX-NEXT:    vpmulhuw {{.*}}(%rip), %xmm0, %xmm2
-; AVX-NEXT:    vpblendvb %xmm1, %xmm0, %xmm2, %xmm0
+; AVX-NEXT:    vpmulhuw {{.*}}(%rip), %xmm0, %xmm1
+; AVX-NEXT:    vpblendw {{.*#+}} xmm0 = xmm0[0],xmm1[1,2,3],xmm0[4,5,6,7]
 ; AVX-NEXT:    retq
 ;
 ; XOP-LABEL: constant_shift_v4i16:
 ; XOP:       # %bb.0:
 ; XOP-NEXT:    vpshlw {{.*}}(%rip), %xmm0, %xmm0
 ; XOP-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: constant_shift_v4i16:
 ; AVX512DQ:       # %bb.0:
-; AVX512DQ-NEXT:    vpxor %xmm1, %xmm1, %xmm1
-; AVX512DQ-NEXT:    vpcmpeqw {{.*}}(%rip), %xmm1, %xmm1
-; AVX512DQ-NEXT:    vpmulhuw {{.*}}(%rip), %xmm0, %xmm2
-; AVX512DQ-NEXT:    vpblendvb %xmm1, %xmm0, %xmm2, %xmm0
+; AVX512DQ-NEXT:    vpmulhuw {{.*}}(%rip), %xmm0, %xmm1
+; AVX512DQ-NEXT:    vpblendw {{.*#+}} xmm0 = xmm0[0],xmm1[1,2,3],xmm0[4,5,6,7]
 ; AVX512DQ-NEXT:    retq
 ;
 ; AVX512BW-LABEL: constant_shift_v4i16:
 ; AVX512BW:       # %bb.0:
 ; AVX512BW-NEXT:    # kill: def $xmm0 killed $xmm0 def $zmm0
 ; AVX512BW-NEXT:    vmovdqa {{.*#+}} xmm1 = <0,1,2,3,u,u,u,u>
 ; AVX512BW-NEXT:    vpsrlvw %zmm1, %zmm0, %zmm0
 ; AVX512BW-NEXT:    # kill: def $xmm0 killed $xmm0 killed $zmm0
 ; AVX512BW-NEXT:    vzeroupper
 ; AVX512BW-NEXT:    retq
 ;
 ; AVX512DQVL-LABEL: constant_shift_v4i16:
 ; AVX512DQVL:       # %bb.0:
-; AVX512DQVL-NEXT:    vpxor %xmm1, %xmm1, %xmm1
-; AVX512DQVL-NEXT:    vpcmpeqw {{.*}}(%rip), %xmm1, %xmm1
-; AVX512DQVL-NEXT:    vpmulhuw {{.*}}(%rip), %xmm0, %xmm2
-; AVX512DQVL-NEXT:    vpblendvb %xmm1, %xmm0, %xmm2, %xmm0
+; AVX512DQVL-NEXT:    vpmulhuw {{.*}}(%rip), %xmm0, %xmm1
+; AVX512DQVL-NEXT:    vpblendw {{.*#+}} xmm0 = xmm0[0],xmm1[1,2,3],xmm0[4,5,6,7]
 ; AVX512DQVL-NEXT:    retq
 ;
 ; AVX512BWVL-LABEL: constant_shift_v4i16:
 ; AVX512BWVL:       # %bb.0:
 ; AVX512BWVL-NEXT:    vpsrlvw {{.*}}(%rip), %xmm0, %xmm0
 ; AVX512BWVL-NEXT:    retq
 ;
 ; X32-SSE-LABEL: constant_shift_v4i16:
 ; X32-SSE:       # %bb.0:
-; X32-SSE-NEXT:    movdqa {{.*#+}} xmm1 = <u,32768,16384,8192,u,u,u,u>
-; X32-SSE-NEXT:    pmulhuw %xmm0, %xmm1
-; X32-SSE-NEXT:    pxor %xmm2, %xmm2
-; X32-SSE-NEXT:    pcmpeqw {{\.LCPI.*}}, %xmm2
-; X32-SSE-NEXT:    pand %xmm2, %xmm0
-; X32-SSE-NEXT:    pandn %xmm1, %xmm2
-; X32-SSE-NEXT:    por %xmm2, %xmm0
+; X32-SSE-NEXT:    movdqa {{.*#+}} xmm2 = [65535,0,0,0,65535,65535,65535,65535]
+; X32-SSE-NEXT:    movdqa %xmm0, %xmm1
+; X32-SSE-NEXT:    pand %xmm2, %xmm1
+; X32-SSE-NEXT:    pmulhuw {{\.LCPI.*}}, %xmm0
+; X32-SSE-NEXT:    pandn %xmm0, %xmm2
+; X32-SSE-NEXT:    por %xmm2, %xmm1
+; X32-SSE-NEXT:    movdqa %xmm1, %xmm0
 ; X32-SSE-NEXT:    retl
   %shift = lshr <4 x i16> %a, <i16 0, i16 1, i16 2, i16 3>
   ret <4 x i16> %shift
 }
 
 define <2 x i16> @constant_shift_v2i16(<2 x i16> %a) nounwind {
 ; SSE2-LABEL: constant_shift_v2i16:
 ; SSE2:       # %bb.0: