[VectorCombine] scalarize compares with insertelement operand(s)

[spatel]

Summary

Generalize scalarization (recently enhanced with D80885) to allow compares as well as binops.
Similar to binops, we are avoiding scalarization of a loaded value because that could avoid a register transfer in codegen.
This requires 1 extra predicate that I am aware of: we do not want to scalarize the condition value of a vector select. That might also invert a transform that we do in instcombine that prefers a vector condition operand for a vector select.

I think this is the final step in solving PR37463:
https://bugs.llvm.org/show_bug.cgi?id=37463

[lebedev.ri]

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

326–327

This needs more words i'd say.

https://godbolt.org/z/4tqgwP
I believe the problem we are preventing is the differences in bit patterns for true/false between scalar and vector comparisons.

• https://www.felixcloutier.com/x86/setcc Sets the destination operand to 0 or 1 depending on the settings of the status flags
• https://www.felixcloutier.com/x86/pcmpeqb:pcmpeqw:pcmpeqd If a pair of data elements is equal, the corresponding data element in the destination operand is set to all 1s
• https://www.felixcloutier.com/x86/blendvps depending on mask bits defined in the mask register operand. The mask bits are the most significant bit in each dword element of the mask register.

And there are other variants on other targets. (there's enum, can't find it right now)

Can we just model that new extra cost?

[spatel]

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

326–327

Yes, the bool format (enum BooleanContent in SDAG) is one problem. The other is the potential transfer between register files as the x86 example demonstrates.

IMO, this is similar to the existing bailout for loaded operands below here. We can try to improve the cost model to deal with larger patterns, but the ability to handle this situation doesn't exist currently AFAIK. We don't have a way to specify the combined cost of the not-yet-created scalar op + insertelt with existing users of the vector op.

[spatel]

Patch updated:
Added comments to better explain the select-user bailout and TODO for possible cost model improvement.

[lebedev.ri]

lg

[vdmitrie]

Vector combine dies on assertion with this test case.

; opt -vector-combine t.ll -S

target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

%struct.s = type { i8* }

; Function Attrs: nounwind uwtable
define <4 x i1> @test(%struct.s** nocapture readonly %t1) #0 {

%t5 = insertelement <4 x %struct.s**> undef, %struct.s** %t1, i32 0
%t6 = icmp ne <4 x %struct.s**> %t5, zeroinitializer
ret <4 x i1> %t6

}

attributes #0 = { nounwind uwtable }

[spatel]

In D81661#2105403, @vdmitrie wrote:

Vector combine dies on assertion with this test case.

; opt -vector-combine t.ll -S

target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

%struct.s = type { i8* }

; Function Attrs: nounwind uwtable
define <4 x i1> @test(%struct.s** nocapture readonly %t1) #0 {

%t5 = insertelement <4 x %struct.s**> undef, %struct.s** %t1, i32 0
%t6 = icmp ne <4 x %struct.s**> %t5, zeroinitializer
ret <4 x i1> %t6

}

attributes #0 = { nounwind uwtable }

Thanks - seems like the assert just became too strict for compares, so I changed it here:
rG741e20f3d61

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

 using namespace llvm;
 using namespace llvm::PatternMatch;
 
 #define DEBUG_TYPE "vector-combine"
 STATISTIC(NumVecCmp, "Number of vector compares formed");
 STATISTIC(NumVecBO, "Number of vector binops formed");
 STATISTIC(NumShufOfBitcast, "Number of shuffles moved after bitcast");
 STATISTIC(NumScalarBO, "Number of scalar binops formed");
+STATISTIC(NumScalarCmp, "Number of scalar compares formed");
 
 static cl::opt<bool> DisableVectorCombine(
     "disable-vector-combine", cl::init(false), cl::Hidden,
     cl::desc("Disable all vector combine transforms"));
 
 static cl::opt<bool> DisableBinopExtractShuffle(
     "disable-binop-extract-shuffle", cl::init(false), cl::Hidden,
     cl::desc("Disable binop extract to shuffle transforms"));
static bool foldBitcastShuf(Instruction &I, const TargetTransformInfo &TTI) {
   IRBuilder<> Builder(&I);
   Value *CastV = Builder.CreateBitCast(V, DestTy);
   Value *Shuf =
       Builder.CreateShuffleVector(CastV, UndefValue::get(DestTy), NewMask);
   I.replaceAllUsesWith(Shuf);
   return true;
 }
 
-/// Match a vector binop instruction with inserted scalar operands and convert
-/// to scalar binop followed by insertelement.
-static bool scalarizeBinop(Instruction &I, const TargetTransformInfo &TTI) {
+/// Match a vector binop or compare instruction with at least one inserted
+/// scalar operand and convert to scalar binop/cmp followed by insertelement.
+static bool scalarizeBinopOrCmp(Instruction &I,
+                                const TargetTransformInfo &TTI) {
+  CmpInst::Predicate Pred = CmpInst::BAD_ICMP_PREDICATE;
   Value *Ins0, *Ins1;
-  if (!match(&I, m_BinOp(m_Value(Ins0), m_Value(Ins1))))
+  if (!match(&I, m_BinOp(m_Value(Ins0), m_Value(Ins1))) &&
+      !match(&I, m_Cmp(Pred, m_Value(Ins0), m_Value(Ins1))))
+    return false;
+
+  // Do not convert the vector condition of a vector select into a scalar
+  // condition. That may cause problems for codegen because of differences in
+  // boolean formats and register-file transfers.
+  // TODO: Can we account for that in the cost model?
+  bool IsCmp = Pred != CmpInst::Predicate::BAD_ICMP_PREDICATE;
+  if (IsCmp)
+    for (User *U : I.users())
+      if (match(U, m_Select(m_Specific(&I), m_Value(), m_Value())))
         return false;
 
   // Match against one or both scalar values being inserted into constant
   // vectors:
-  // vec_bo VecC0, (inselt VecC1, V1, Index)
-  // vec_bo (inselt VecC0, V0, Index), VecC1
-  // vec_bo (inselt VecC0, V0, Index), (inselt VecC1, V1, Index)
+  // vec_op VecC0, (inselt VecC1, V1, Index)
+  // vec_op (inselt VecC0, V0, Index), VecC1
+  // vec_op (inselt VecC0, V0, Index), (inselt VecC1, V1, Index)
   // TODO: Deal with mismatched index constants and variable indexes?
   Constant *VecC0 = nullptr, *VecC1 = nullptr;
   Value *V0 = nullptr, *V1 = nullptr;
   uint64_t Index0 = 0, Index1 = 0;
   if (!match(Ins0, m_InsertElt(m_Constant(VecC0), m_Value(V0),
                                m_ConstantInt(Index0))) &&
       !match(Ins0, m_Constant(VecC0)))
     return false;
static bool scalarizeBinopOrCmp(Instruction &I,
   uint64_t Index = IsConst0 ? Index1 : Index0;
   Type *ScalarTy = IsConst0 ? V1->getType() : V0->getType();
   Type *VecTy = I.getType();
   assert(VecTy->isVectorTy() &&
          (IsConst0 || IsConst1 || V0->getType() == V1->getType()) &&
          (ScalarTy->isIntegerTy() || ScalarTy->isFloatingPointTy()) &&
          "Unexpected types for insert into binop");
 
-  Instruction::BinaryOps Opcode = cast<BinaryOperator>(&I)->getOpcode();
-  int ScalarOpCost = TTI.getArithmeticInstrCost(Opcode, ScalarTy);
-  int VectorOpCost = TTI.getArithmeticInstrCost(Opcode, VecTy);
+  unsigned Opcode = I.getOpcode();
+  int ScalarOpCost, VectorOpCost;
+  if (IsCmp) {
+    ScalarOpCost = TTI.getCmpSelInstrCost(Opcode, ScalarTy);
+    VectorOpCost = TTI.getCmpSelInstrCost(Opcode, VecTy);
+  } else {
+    ScalarOpCost = TTI.getArithmeticInstrCost(Opcode, ScalarTy);
+    VectorOpCost = TTI.getArithmeticInstrCost(Opcode, VecTy);
+  }
 
   // Get cost estimate for the insert element. This cost will factor into
   // both sequences.
   int InsertCost =
       TTI.getVectorInstrCost(Instruction::InsertElement, VecTy, Index);
   int OldCost = (IsConst0 ? 0 : InsertCost) + (IsConst1 ? 0 : InsertCost) +
                 VectorOpCost;
   int NewCost = ScalarOpCost + InsertCost +
                 (IsConst0 ? 0 : !Ins0->hasOneUse() * InsertCost) +
                 (IsConst1 ? 0 : !Ins1->hasOneUse() * InsertCost);
 
   // We want to scalarize unless the vector variant actually has lower cost.
   if (OldCost < NewCost)
     return false;
 
-  // vec_bo (inselt VecC0, V0, Index), (inselt VecC1, V1, Index) -->
-  // inselt NewVecC, (scalar_bo V0, V1), Index
+  // vec_op (inselt VecC0, V0, Index), (inselt VecC1, V1, Index) -->
+  // inselt NewVecC, (scalar_op V0, V1), Index
+  if (IsCmp)
+    ++NumScalarCmp;
+  else
     ++NumScalarBO;
-  IRBuilder<> Builder(&I);
 
   // For constant cases, extract the scalar element, this should constant fold.
+  IRBuilder<> Builder(&I);
   if (IsConst0)
     V0 = ConstantExpr::getExtractElement(VecC0, Builder.getInt64(Index));
   if (IsConst1)
     V1 = ConstantExpr::getExtractElement(VecC1, Builder.getInt64(Index));
 
-  Value *Scalar = Builder.CreateBinOp(Opcode, V0, V1, I.getName() + ".scalar");
+  Value *Scalar =
+      IsCmp ? Opcode == Instruction::FCmp ? Builder.CreateFCmp(Pred, V0, V1)
+                                          : Builder.CreateICmp(Pred, V0, V1)
+            : Builder.CreateBinOp((Instruction::BinaryOps)Opcode, V0, V1);
+
+  Scalar->setName(I.getName() + ".scalar");
 
   // All IR flags are safe to back-propagate. There is no potential for extra
   // poison to be created by the scalar instruction.
   if (auto *ScalarInst = dyn_cast<Instruction>(Scalar))
     ScalarInst->copyIRFlags(&I);
 
   // Fold the vector constants in the original vectors into a new base vector.
-  Constant *NewVecC = ConstantExpr::get(Opcode, VecC0, VecC1);
+  Constant *NewVecC = IsCmp ? ConstantExpr::getCompare(Pred, VecC0, VecC1)
+                            : ConstantExpr::get(Opcode, VecC0, VecC1);
   Value *Insert = Builder.CreateInsertElement(NewVecC, Scalar, Index);
   I.replaceAllUsesWith(Insert);
   Insert->takeName(&I);
   return true;
 }
 
 /// This is the entry point for all transforms. Pass manager differences are
 /// handled in the callers of this function.
for (BasicBlock &BB : F) {
     // Walk the block forwards to enable simple iterative chains of transforms.
     // TODO: It could be more efficient to remove dead instructions
     //       iteratively in this loop rather than waiting until the end.
     for (Instruction &I : BB) {
       if (isa<DbgInfoIntrinsic>(I))
         continue;
       MadeChange |= foldExtractExtract(I, TTI);
       MadeChange |= foldBitcastShuf(I, TTI);
-      MadeChange |= scalarizeBinop(I, TTI);
+      MadeChange |= scalarizeBinopOrCmp(I, TTI);
     }
   }
 
   // We're done with transforms, so remove dead instructions.
   if (MadeChange)
     for (BasicBlock &BB : F)
       SimplifyInstructionsInBlock(&BB);
 

llvm/test/Transforms/VectorCombine/X86/scalarize-cmp.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt < %s -vector-combine -S -mtriple=x86_64-- -mattr=SSE2 | FileCheck %s --check-prefixes=CHECK,SSE
 ; RUN: opt < %s -vector-combine -S -mtriple=x86_64-- -mattr=AVX2 | FileCheck %s --check-prefixes=CHECK,AVX
 
 declare void @use(<4 x i32>)
 declare void @usef(<4 x float>)
 
 ; Eliminating an insert is profitable.
 
 define <16 x i1> @ins0_ins0_i8(i8 %x, i8 %y) {
 ; CHECK-LABEL: @ins0_ins0_i8(
-; CHECK-NEXT:    [[I0:%.*]] = insertelement <16 x i8> undef, i8 [[X:%.*]], i32 0
-; CHECK-NEXT:    [[I1:%.*]] = insertelement <16 x i8> undef, i8 [[Y:%.*]], i32 0
-; CHECK-NEXT:    [[R:%.*]] = icmp eq <16 x i8> [[I0]], [[I1]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = icmp eq i8 [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <16 x i1> undef, i1 [[R_SCALAR]], i64 0
 ; CHECK-NEXT:    ret <16 x i1> [[R]]
 ;
   %i0 = insertelement <16 x i8> undef, i8 %x, i32 0
   %i1 = insertelement <16 x i8> undef, i8 %y, i32 0
   %r = icmp eq <16 x i8> %i0, %i1
   ret <16 x i1> %r
 }
 
 ; Eliminating an insert is still profitable. Mismatch types on index is ok.
 
 define <8 x i1> @ins5_ins5_i16(i16 %x, i16 %y) {
 ; CHECK-LABEL: @ins5_ins5_i16(
-; CHECK-NEXT:    [[I0:%.*]] = insertelement <8 x i16> undef, i16 [[X:%.*]], i8 5
-; CHECK-NEXT:    [[I1:%.*]] = insertelement <8 x i16> undef, i16 [[Y:%.*]], i32 5
-; CHECK-NEXT:    [[R:%.*]] = icmp sgt <8 x i16> [[I0]], [[I1]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = icmp sgt i16 [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <8 x i1> undef, i1 [[R_SCALAR]], i64 5
 ; CHECK-NEXT:    ret <8 x i1> [[R]]
 ;
   %i0 = insertelement <8 x i16> undef, i16 %x, i8 5
   %i1 = insertelement <8 x i16> undef, i16 %y, i32 5
   %r = icmp sgt <8 x i16> %i0, %i1
   ret <8 x i1> %r
 }
 
 ; The new vector constant is calculated by constant folding.
 
 define <2 x i1> @ins1_ins1_i64(i64 %x, i64 %y) {
 ; CHECK-LABEL: @ins1_ins1_i64(
-; CHECK-NEXT:    [[I0:%.*]] = insertelement <2 x i64> zeroinitializer, i64 [[X:%.*]], i64 1
-; CHECK-NEXT:    [[I1:%.*]] = insertelement <2 x i64> <i64 1, i64 -1>, i64 [[Y:%.*]], i32 1
-; CHECK-NEXT:    [[R:%.*]] = icmp sle <2 x i64> [[I0]], [[I1]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = icmp sle i64 [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <2 x i1> <i1 true, i1 false>, i1 [[R_SCALAR]], i64 1
 ; CHECK-NEXT:    ret <2 x i1> [[R]]
 ;
   %i0 = insertelement <2 x i64> zeroinitializer, i64 %x, i64 1
   %i1 = insertelement <2 x i64> <i64 1, i64 -1>, i64 %y, i32 1
   %r = icmp sle <2 x i64> %i0, %i1
   ret <2 x i1> %r
 }
 
 ; The inserts are free, but it's still better to scalarize.
 
 define <2 x i1> @ins0_ins0_f64(double %x, double %y) {
 ; CHECK-LABEL: @ins0_ins0_f64(
-; CHECK-NEXT:    [[I0:%.*]] = insertelement <2 x double> undef, double [[X:%.*]], i32 0
-; CHECK-NEXT:    [[I1:%.*]] = insertelement <2 x double> undef, double [[Y:%.*]], i32 0
-; CHECK-NEXT:    [[R:%.*]] = fcmp nnan ninf uge <2 x double> [[I0]], [[I1]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = fcmp nnan ninf uge double [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <2 x i1> <i1 true, i1 true>, i1 [[R_SCALAR]], i64 0
 ; CHECK-NEXT:    ret <2 x i1> [[R]]
 ;
   %i0 = insertelement <2 x double> undef, double %x, i32 0
   %i1 = insertelement <2 x double> undef, double %y, i32 0
   %r = fcmp nnan ninf uge <2 x double> %i0, %i1
   ret <2 x i1> %r
 }
 
;
   %r = icmp sle <16 x i8> %i0, %i1
   ret <16 x i1> %r
 }
 
 ; Base vector does not have to be undef.
 
 define <4 x i1> @ins0_ins0_i32(i32 %x, i32 %y) {
 ; CHECK-LABEL: @ins0_ins0_i32(
-; CHECK-NEXT:    [[I0:%.*]] = insertelement <4 x i32> zeroinitializer, i32 [[X:%.*]], i32 0
-; CHECK-NEXT:    [[I1:%.*]] = insertelement <4 x i32> undef, i32 [[Y:%.*]], i32 0
-; CHECK-NEXT:    [[R:%.*]] = icmp ne <4 x i32> [[I0]], [[I1]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = icmp ne i32 [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <4 x i1> undef, i1 [[R_SCALAR]], i64 0
 ; CHECK-NEXT:    ret <4 x i1> [[R]]
 ;
   %i0 = insertelement <4 x i32> zeroinitializer, i32 %x, i32 0
   %i1 = insertelement <4 x i32> undef, i32 %y, i32 0
   %r = icmp ne <4 x i32> %i0, %i1
   ret <4 x i1> %r
 }
 
 ; Extra use is accounted for in cost calculation.
 
 define <4 x i1> @ins0_ins0_i32_use(i32 %x, i32 %y) {
 ; CHECK-LABEL: @ins0_ins0_i32_use(
 ; CHECK-NEXT:    [[I0:%.*]] = insertelement <4 x i32> undef, i32 [[X:%.*]], i32 0
 ; CHECK-NEXT:    call void @use(<4 x i32> [[I0]])
-; CHECK-NEXT:    [[I1:%.*]] = insertelement <4 x i32> undef, i32 [[Y:%.*]], i32 0
-; CHECK-NEXT:    [[R:%.*]] = icmp ugt <4 x i32> [[I0]], [[I1]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = icmp ugt i32 [[X]], [[Y:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <4 x i1> undef, i1 [[R_SCALAR]], i64 0
 ; CHECK-NEXT:    ret <4 x i1> [[R]]
 ;
   %i0 = insertelement <4 x i32> undef, i32 %x, i32 0
   call void @use(<4 x i32> %i0)
   %i1 = insertelement <4 x i32> undef, i32 %y, i32 0
   %r = icmp ugt <4 x i32> %i0, %i1
   ret <4 x i1> %r
 }
 
 ; Extra use is accounted for in cost calculation.
 
 define <4 x i1> @ins1_ins1_f32_use(float %x, float %y) {
 ; CHECK-LABEL: @ins1_ins1_f32_use(
-; CHECK-NEXT:    [[I0:%.*]] = insertelement <4 x float> undef, float [[X:%.*]], i32 1
 ; CHECK-NEXT:    [[I1:%.*]] = insertelement <4 x float> undef, float [[Y:%.*]], i32 1
 ; CHECK-NEXT:    call void @usef(<4 x float> [[I1]])
-; CHECK-NEXT:    [[R:%.*]] = fcmp ogt <4 x float> [[I0]], [[I1]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = fcmp ogt float [[X:%.*]], [[Y]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <4 x i1> zeroinitializer, i1 [[R_SCALAR]], i64 1
 ; CHECK-NEXT:    ret <4 x i1> [[R]]
 ;
   %i0 = insertelement <4 x float> undef, float %x, i32 1
   %i1 = insertelement <4 x float> undef, float %y, i32 1
   call void @usef(<4 x float> %i1)
   %r = fcmp ogt <4 x float> %i0, %i1
   ret <4 x i1> %r
 }
;
   %i1 = insertelement <4 x float> undef, float %y, i32 2
   call void @usef(<4 x float> %i1)
   %r = fcmp oeq <4 x float> %i0, %i1
   ret <4 x i1> %r
 }
 
 define <2 x i1> @constant_op1_i64(i64 %x) {
 ; CHECK-LABEL: @constant_op1_i64(
-; CHECK-NEXT:    [[INS:%.*]] = insertelement <2 x i64> undef, i64 [[X:%.*]], i32 0
-; CHECK-NEXT:    [[R:%.*]] = icmp ne <2 x i64> [[INS]], <i64 42, i64 undef>
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = icmp ne i64 [[X:%.*]], 42
+; CHECK-NEXT:    [[R:%.*]] = insertelement <2 x i1> undef, i1 [[R_SCALAR]], i64 0
 ; CHECK-NEXT:    ret <2 x i1> [[R]]
 ;
   %ins = insertelement <2 x i64> undef, i64 %x, i32 0
   %r = icmp ne <2 x i64> %ins, <i64 42, i64 undef>
   ret <2 x i1> %r
 }
 
 define <2 x i1> @constant_op1_i64_not_undef_lane(i64 %x) {
 ; CHECK-LABEL: @constant_op1_i64_not_undef_lane(
-; CHECK-NEXT:    [[INS:%.*]] = insertelement <2 x i64> undef, i64 [[X:%.*]], i32 0
-; CHECK-NEXT:    [[R:%.*]] = icmp sge <2 x i64> [[INS]], <i64 42, i64 -42>
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = icmp sge i64 [[X:%.*]], 42
+; CHECK-NEXT:    [[R:%.*]] = insertelement <2 x i1> <i1 true, i1 true>, i1 [[R_SCALAR]], i64 0
 ; CHECK-NEXT:    ret <2 x i1> [[R]]
 ;
   %ins = insertelement <2 x i64> undef, i64 %x, i32 0
   %r = icmp sge <2 x i64> %ins, <i64 42, i64 -42>
   ret <2 x i1> %r
 }
 
+; negative test - load prevents the transform
+
 define <2 x i1> @constant_op1_i64_load(i64* %p) {
 ; CHECK-LABEL: @constant_op1_i64_load(
 ; CHECK-NEXT:    [[LD:%.*]] = load i64, i64* [[P:%.*]], align 4
 ; CHECK-NEXT:    [[INS:%.*]] = insertelement <2 x i64> undef, i64 [[LD]], i32 0
 ; CHECK-NEXT:    [[R:%.*]] = icmp eq <2 x i64> [[INS]], <i64 42, i64 -42>
 ; CHECK-NEXT:    ret <2 x i1> [[R]]
 ;
   %ld = load i64, i64* %p
   %ins = insertelement <2 x i64> undef, i64 %ld, i32 0
   %r = icmp eq <2 x i64> %ins, <i64 42, i64 -42>
   ret <2 x i1> %r
 }
 
 define <4 x i1> @constant_op0_i32(i32 %x) {
 ; CHECK-LABEL: @constant_op0_i32(
-; CHECK-NEXT:    [[INS:%.*]] = insertelement <4 x i32> undef, i32 [[X:%.*]], i32 1
-; CHECK-NEXT:    [[R:%.*]] = icmp ult <4 x i32> <i32 undef, i32 -42, i32 undef, i32 undef>, [[INS]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = icmp ult i32 -42, [[X:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <4 x i1> zeroinitializer, i1 [[R_SCALAR]], i64 1
 ; CHECK-NEXT:    ret <4 x i1> [[R]]
 ;
   %ins = insertelement <4 x i32> undef, i32 %x, i32 1
   %r = icmp ult <4 x i32> <i32 undef, i32 -42, i32 undef, i32 undef>, %ins
   ret <4 x i1> %r
 }
 
 define <4 x i1> @constant_op0_i32_not_undef_lane(i32 %x) {
 ; CHECK-LABEL: @constant_op0_i32_not_undef_lane(
-; CHECK-NEXT:    [[INS:%.*]] = insertelement <4 x i32> undef, i32 [[X:%.*]], i32 1
-; CHECK-NEXT:    [[R:%.*]] = icmp ule <4 x i32> <i32 1, i32 42, i32 42, i32 -42>, [[INS]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = icmp ule i32 42, [[X:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <4 x i1> <i1 true, i1 true, i1 true, i1 true>, i1 [[R_SCALAR]], i64 1
 ; CHECK-NEXT:    ret <4 x i1> [[R]]
 ;
   %ins = insertelement <4 x i32> undef, i32 %x, i32 1
   %r = icmp ule <4 x i32> <i32 1, i32 42, i32 42, i32 -42>, %ins
   ret <4 x i1> %r
 }
 
 define <2 x i1> @constant_op0_f64(double %x) {
 ; CHECK-LABEL: @constant_op0_f64(
-; CHECK-NEXT:    [[INS:%.*]] = insertelement <2 x double> undef, double [[X:%.*]], i32 0
-; CHECK-NEXT:    [[R:%.*]] = fcmp fast olt <2 x double> <double 4.200000e+01, double undef>, [[INS]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = fcmp fast olt double 4.200000e+01, [[X:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <2 x i1> zeroinitializer, i1 [[R_SCALAR]], i64 0
 ; CHECK-NEXT:    ret <2 x i1> [[R]]
 ;
   %ins = insertelement <2 x double> undef, double %x, i32 0
   %r = fcmp fast olt <2 x double> <double 42.0, double undef>, %ins
   ret <2 x i1> %r
 }
 
 define <2 x i1> @constant_op0_f64_not_undef_lane(double %x) {
 ; CHECK-LABEL: @constant_op0_f64_not_undef_lane(
-; CHECK-NEXT:    [[INS:%.*]] = insertelement <2 x double> undef, double [[X:%.*]], i32 1
-; CHECK-NEXT:    [[R:%.*]] = fcmp nnan ueq <2 x double> <double 4.200000e+01, double -4.200000e+01>, [[INS]]
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = fcmp nnan ueq double -4.200000e+01, [[X:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = insertelement <2 x i1> <i1 true, i1 true>, i1 [[R_SCALAR]], i64 1
 ; CHECK-NEXT:    ret <2 x i1> [[R]]
 ;
   %ins = insertelement <2 x double> undef, double %x, i32 1
   %r = fcmp nnan ueq <2 x double> <double 42.0, double -42.0>, %ins
   ret <2 x i1> %r
 }
 
 define <2 x i1> @constant_op1_f64(double %x) {
 ; CHECK-LABEL: @constant_op1_f64(
-; CHECK-NEXT:    [[INS:%.*]] = insertelement <2 x double> undef, double [[X:%.*]], i32 1
-; CHECK-NEXT:    [[R:%.*]] = fcmp one <2 x double> [[INS]], <double undef, double 4.200000e+01>
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = fcmp one double [[X:%.*]], 4.200000e+01
+; CHECK-NEXT:    [[R:%.*]] = insertelement <2 x i1> zeroinitializer, i1 [[R_SCALAR]], i64 1
 ; CHECK-NEXT:    ret <2 x i1> [[R]]
 ;
   %ins = insertelement <2 x double> undef, double %x, i32 1
   %r = fcmp one <2 x double> %ins, <double undef, double 42.0>
   ret <2 x i1> %r
 }
 
 define <4 x i1> @constant_op1_f32_not_undef_lane(float %x) {
 ; CHECK-LABEL: @constant_op1_f32_not_undef_lane(
-; CHECK-NEXT:    [[INS:%.*]] = insertelement <4 x float> undef, float [[X:%.*]], i32 0
-; CHECK-NEXT:    [[R:%.*]] = fcmp uge <4 x float> [[INS]], <float 4.200000e+01, float -4.200000e+01, float 0.000000e+00, float 1.000000e+00>
+; CHECK-NEXT:    [[R_SCALAR:%.*]] = fcmp uge float [[X:%.*]], 4.200000e+01
+; CHECK-NEXT:    [[R:%.*]] = insertelement <4 x i1> <i1 true, i1 true, i1 true, i1 true>, i1 [[R_SCALAR]], i64 0
 ; CHECK-NEXT:    ret <4 x i1> [[R]]
 ;
   %ins = insertelement <4 x float> undef, float %x, i32 0
   %r = fcmp uge <4 x float> %ins, <float 42.0, float -42.0, float 0.0, float 1.0>
   ret <4 x i1> %r
 }
 
+; negative test - select prevents the transform
+
 define <4 x float> @vec_select_use1(<4 x float> %x, <4 x float> %y, i32 %a, i32 %b) {
 ; CHECK-LABEL: @vec_select_use1(
 ; CHECK-NEXT:    [[VECA:%.*]] = insertelement <4 x i32> undef, i32 [[A:%.*]], i8 0
 ; CHECK-NEXT:    [[VECB:%.*]] = insertelement <4 x i32> undef, i32 [[B:%.*]], i8 0
 ; CHECK-NEXT:    [[COND:%.*]] = icmp eq <4 x i32> [[VECA]], [[VECB]]
 ; CHECK-NEXT:    [[R:%.*]] = select <4 x i1> [[COND]], <4 x float> [[X:%.*]], <4 x float> [[Y:%.*]]
 ; CHECK-NEXT:    ret <4 x float> [[R]]
 ;
   %veca = insertelement <4 x i32> undef, i32 %a, i8 0
   %vecb = insertelement <4 x i32> undef, i32 %b, i8 0
   %cond = icmp eq <4 x i32> %veca, %vecb
   %r = select <4 x i1> %cond, <4 x float> %x, <4 x float> %y
   ret <4 x float> %r
 }
 
+; negative test - select prevents the transform
+
 define <4 x float> @vec_select_use2(<4 x float> %x, <4 x float> %y, float %a) {
 ; CHECK-LABEL: @vec_select_use2(
 ; CHECK-NEXT:    [[VECA:%.*]] = insertelement <4 x float> undef, float [[A:%.*]], i8 0
 ; CHECK-NEXT:    [[COND:%.*]] = fcmp oeq <4 x float> [[VECA]], zeroinitializer
 ; CHECK-NEXT:    [[R:%.*]] = select <4 x i1> [[COND]], <4 x float> [[X:%.*]], <4 x float> [[Y:%.*]]
 ; CHECK-NEXT:    ret <4 x float> [[R]]
 ;
   %veca = insertelement <4 x float> undef, float %a, i8 0
   %cond = fcmp oeq <4 x float> %veca, zeroinitializer
   %r = select <4 x i1> %cond, <4 x float> %x, <4 x float> %y
   ret <4 x float> %r
 }