[llvm][LV] Drop poison-generating flags from FP instructions

[dcaballe]

Summary

See D111846 for more context. This patch extends Instruction::dropPoisonGeneratingFlags to also
drop poison-generating flags from FP instructions (i.e., nnan and ninf0). Two new tests for nnan and
ninf were pre-committed to LoopVectorize/X86/drop-poison-generating-flags.ll in D112335.

[dcaballe]

• Extended Instruction::dropPoisonGeneratedFlags to support FP
• Updated impacted tests

[spatel]

I didn't go through all of the tests, but the first reduction expansion diff seems unintended to me (see inline comment).
We could enable dropping FMF only with an optional bool parameter? That would allow incremental updates.
Or we handle that as a one-off in the motivating LV caller?

llvm/test/CodeGen/Generic/expand-experimental-reductions.ll

100

This isn't the result that we want. This might be due to the vague definitions and interactions of FMF, but I think 'fast' should propagate through this expansion.

For example, if the intrinsic has 'nnan reassoc', then we should be able say that the result is also 'nnan reassoc'. That is, we disallowed NAN inputs and allowed reassociation, so we can assume that a NAN was not accidentally/unknowingly created through this transform where it would not have been in the original code.

The use of dropPoison for integer ops in the ExpandReductions pass is correct though...

[dcaballe]

In D112117#3090902, @spatel wrote:

I didn't go through all of the tests, but the first reduction expansion diff seems unintended to me (see inline comment).
We could enable dropping FMF only with an optional bool parameter? That would allow incremental updates.
Or we handle that as a one-off in the motivating LV caller?

Good catch! I like the optional bool parameter. Honestly, I don't feel very comfortable with dropping all the FMF flags by default since I don't understand in detail all the users of this utility. Let's go with that.

[spatel]

In D112117#3092834, @dcaballe wrote:

In D112117#3090902, @spatel wrote:

I didn't go through all of the tests, but the first reduction expansion diff seems unintended to me (see inline comment).
We could enable dropping FMF only with an optional bool parameter? That would allow incremental updates.
Or we handle that as a one-off in the motivating LV caller?

Good catch! I like the optional bool parameter. Honestly, I don't feel very comfortable with dropping all the FMF flags by default since I don't understand in detail all the users of this utility. Let's go with that.

Sounds good...I was probably too optimistic when I wrote that it would just be minor regressions (perf) if we got this wrong. :)
We need to audit the callers to decide if/when FMF can be propagated.

[dcaballe]

Dropping FMF doesn't seem to be needed after all. See D111846

[reames]

JFYI, I've been taking a look at trying to get FMF flags into the generic dropPoisonGeneratingFlags. So far, the test diff (which looks a lot like this one for the same reasons) has exposed a bunch of cases where the existing code had problems. I'm working through fixing them individually, with the goal of getting a reasonable diff.

The most relevant one which got some discussion on this review is the reduction expansion. That appears to be entirely an artifact of code which a) does nothing without the FMF handling in the generic code, and b) serves no purpose even with it. 0d13f94c and b24db85 simply remove most of it, and fix up some comments.

[dcaballe]

In D112117#3183418, @reames wrote:

JFYI, I've been taking a look at trying to get FMF flags into the generic dropPoisonGeneratingFlags. So far, the test diff (which looks a lot like this one for the same reasons) has exposed a bunch of cases where the existing code had problems. I'm working through fixing them individually, with the goal of getting a reasonable diff.

The most relevant one which got some discussion on this review is the reduction expansion. That appears to be entirely an artifact of code which a) does nothing without the FMF handling in the generic code, and b) serves no purpose even with it. 0d13f94c and b24db85 simply remove most of it, and fix up some comments.

Thanks for the notice, Philip. Based on the feedback that @nlopes provided in D111846, we wouldn't need to drop FMF as part of the vectorization process. That is why I abandoned this review. Maybe you could do something similar to what was suggested above:

We could enable dropping FMF only with an optional bool parameter?

and make sure that the flag is disabled in the vectorizer to preserve what I understand is the expected behavior?

[reames]

The change I'm working on is D115460. To my knowledge, there's no interaction with the vectorizer here, it's simply a code cleanup. e.g. Do the same thing for all types of poison generating flags. We may be too conservative in some places, but consistency has a strong value of it's own. At least per the tests being changed, the vectorizer is not impacted.

[dcaballe]

In D112117#3185648, @reames wrote:

The change I'm working on is D115460. To my knowledge, there's no interaction with the vectorizer here, it's simply a code cleanup. e.g. Do the same thing for all types of poison generating flags. We may be too conservative in some places, but consistency has a strong value of it's own. At least per the tests being changed, the vectorizer is not impacted.

Yes, sorry, I introduced new calls to dropPoisonGeneratingFlags as part of this change that didn't land so the vectorizer shouldn't be impacted by your change. There are some existing calls but they are very specific and shouldn't have a significant impact on FMF.

llvm/lib/IR/Instruction.cpp

bool Instruction::hasNoUnsignedWrap() const {
   return cast<OverflowingBinaryOperator>(this)->hasNoUnsignedWrap();
 }
 
 bool Instruction::hasNoSignedWrap() const {
   return cast<OverflowingBinaryOperator>(this)->hasNoSignedWrap();
 }
 
 void Instruction::dropPoisonGeneratingFlags() {
-  switch (getOpcode()) {
-  case Instruction::Add:
-  case Instruction::Sub:
-  case Instruction::Mul:
-  case Instruction::Shl:
-    cast<OverflowingBinaryOperator>(this)->setHasNoUnsignedWrap(false);
-    cast<OverflowingBinaryOperator>(this)->setHasNoSignedWrap(false);
-    break;
-
-  case Instruction::UDiv:
-  case Instruction::SDiv:
-  case Instruction::AShr:
-  case Instruction::LShr:
-    cast<PossiblyExactOperator>(this)->setIsExact(false);
-    break;
-
-  case Instruction::GetElementPtr:
-    cast<GetElementPtrInst>(this)->setIsInBounds(false);
-    break;
+  if (auto *OverflowingOp = dyn_cast<OverflowingBinaryOperator>(this)) {
+    OverflowingOp->setHasNoUnsignedWrap(false);
+    OverflowingOp->setHasNoSignedWrap(false);
+    return;
+  }
+  if (auto *ExactOp = dyn_cast<PossiblyExactOperator>(this)) {
+    ExactOp->setIsExact(false);
+    return;
+  }
+  if (auto *Gep = dyn_cast<GetElementPtrInst>(this)) {
+    Gep->setIsInBounds(false);
+    return;
+  }
+  if (auto *FPMOp = dyn_cast<FPMathOperator>(this)) {
+    FPMOp->setHasNoNaNs(false);
+    FPMOp->setHasNoInfs(false);
   }
-  // TODO: FastMathFlags!
 }
 
 void Instruction::dropUndefImplyingAttrsAndUnknownMetadata(
     ArrayRef<unsigned> KnownIDs) {
   dropUnknownNonDebugMetadata(KnownIDs);
   auto *CB = dyn_cast<CallBase>(this);
   if (!CB)
     return;

llvm/test/CodeGen/Generic/expand-experimental-reductions.ll

entry:
   %r = call i64 @llvm.vector.reduce.xor.v2i64(<2 x i64> %vec)
   ret i64 %r
 }
 
 define float @fadd_f32(<4 x float> %vec) {
 ; CHECK-LABEL: @fadd_f32(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RDX_SHUF:%.*]] = shufflevector <4 x float> [[VEC:%.*]], <4 x float> poison, <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX:%.*]] = fadd fast <4 x float> [[VEC]], [[RDX_SHUF]]
+; CHECK-NEXT:    [[BIN_RDX:%.*]] = fadd reassoc nsz arcp contract afn <4 x float> [[VEC]], [[RDX_SHUF]]
 ; CHECK-NEXT:    [[RDX_SHUF1:%.*]] = shufflevector <4 x float> [[BIN_RDX]], <4 x float> poison, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX2:%.*]] = fadd fast <4 x float> [[BIN_RDX]], [[RDX_SHUF1]]
+; CHECK-NEXT:    [[BIN_RDX2:%.*]] = fadd reassoc nsz arcp contract afn <4 x float> [[BIN_RDX]], [[RDX_SHUF1]]
 ; CHECK-NEXT:    [[TMP0:%.*]] = extractelement <4 x float> [[BIN_RDX2]], i32 0
 ; CHECK-NEXT:    [[BIN_RDX3:%.*]] = fadd fast float 0.000000e+00, [[TMP0]]
 ; CHECK-NEXT:    ret float [[BIN_RDX3]]
 ;
 entry:
   %r = call fast float @llvm.vector.reduce.fadd.f32.v4f32(float 0.0, <4 x float> %vec)
   ret float %r
 }
 
 define float @fadd_f32_accum(float %accum, <4 x float> %vec) {
 ; CHECK-LABEL: @fadd_f32_accum(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RDX_SHUF:%.*]] = shufflevector <4 x float> [[VEC:%.*]], <4 x float> poison, <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX:%.*]] = fadd fast <4 x float> [[VEC]], [[RDX_SHUF]]
+; CHECK-NEXT:    [[BIN_RDX:%.*]] = fadd reassoc nsz arcp contract afn <4 x float> [[VEC]], [[RDX_SHUF]]
 ; CHECK-NEXT:    [[RDX_SHUF1:%.*]] = shufflevector <4 x float> [[BIN_RDX]], <4 x float> poison, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX2:%.*]] = fadd fast <4 x float> [[BIN_RDX]], [[RDX_SHUF1]]
+; CHECK-NEXT:    [[BIN_RDX2:%.*]] = fadd reassoc nsz arcp contract afn <4 x float> [[BIN_RDX]], [[RDX_SHUF1]]
 ; CHECK-NEXT:    [[TMP0:%.*]] = extractelement <4 x float> [[BIN_RDX2]], i32 0
 ; CHECK-NEXT:    [[BIN_RDX3:%.*]] = fadd fast float [[ACCUM:%.*]], [[TMP0]]
 ; CHECK-NEXT:    ret float [[BIN_RDX3]]
 ;
 entry:
   %r = call fast float @llvm.vector.reduce.fadd.f32.v4f32(float %accum, <4 x float> %vec)
   ret float %r
 }
entry:
   %r = call float @llvm.vector.reduce.fadd.f32.v4f32(float %accum, <4 x float> %vec)
   ret float %r
 }
 
 define float @fmul_f32(<4 x float> %vec) {
 ; CHECK-LABEL: @fmul_f32(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RDX_SHUF:%.*]] = shufflevector <4 x float> [[VEC:%.*]], <4 x float> poison, <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX:%.*]] = fmul fast <4 x float> [[VEC]], [[RDX_SHUF]]
+; CHECK-NEXT:    [[BIN_RDX:%.*]] = fmul reassoc nsz arcp contract afn <4 x float> [[VEC]], [[RDX_SHUF]]
 ; CHECK-NEXT:    [[RDX_SHUF1:%.*]] = shufflevector <4 x float> [[BIN_RDX]], <4 x float> poison, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX2:%.*]] = fmul fast <4 x float> [[BIN_RDX]], [[RDX_SHUF1]]
+; CHECK-NEXT:    [[BIN_RDX2:%.*]] = fmul reassoc nsz arcp contract afn <4 x float> [[BIN_RDX]], [[RDX_SHUF1]]
 ; CHECK-NEXT:    [[TMP0:%.*]] = extractelement <4 x float> [[BIN_RDX2]], i32 0
 ; CHECK-NEXT:    [[BIN_RDX3:%.*]] = fmul fast float 1.000000e+00, [[TMP0]]
 ; CHECK-NEXT:    ret float [[BIN_RDX3]]
 ;
 entry:
   %r = call fast float @llvm.vector.reduce.fmul.f32.v4f32(float 1.0, <4 x float> %vec)
   ret float %r
 }
 
 define float @fmul_f32_accum(float %accum, <4 x float> %vec) {
 ; CHECK-LABEL: @fmul_f32_accum(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RDX_SHUF:%.*]] = shufflevector <4 x float> [[VEC:%.*]], <4 x float> poison, <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX:%.*]] = fmul fast <4 x float> [[VEC]], [[RDX_SHUF]]
+; CHECK-NEXT:    [[BIN_RDX:%.*]] = fmul reassoc nsz arcp contract afn <4 x float> [[VEC]], [[RDX_SHUF]]
 ; CHECK-NEXT:    [[RDX_SHUF1:%.*]] = shufflevector <4 x float> [[BIN_RDX]], <4 x float> poison, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX2:%.*]] = fmul fast <4 x float> [[BIN_RDX]], [[RDX_SHUF1]]
+; CHECK-NEXT:    [[BIN_RDX2:%.*]] = fmul reassoc nsz arcp contract afn <4 x float> [[BIN_RDX]], [[RDX_SHUF1]]
 ; CHECK-NEXT:    [[TMP0:%.*]] = extractelement <4 x float> [[BIN_RDX2]], i32 0
 ; CHECK-NEXT:    [[BIN_RDX3:%.*]] = fmul fast float [[ACCUM:%.*]], [[TMP0]]
 ; CHECK-NEXT:    ret float [[BIN_RDX3]]
 ;
 entry:
   %r = call fast float @llvm.vector.reduce.fmul.f32.v4f32(float %accum, <4 x float> %vec)
   ret float %r
 }

llvm/test/Transforms/InstCombine/scalarization-inseltpoison.ll

;
   %add = fadd <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
   %r = extractelement <4 x float> %add, i32 %y
   ret float %r
 }
 
 define float @extelt_binop_insertelt(<4 x float> %A, <4 x float> %B, float %f) {
 ; CHECK-LABEL: @extelt_binop_insertelt(
 ; CHECK-NEXT:    [[TMP1:%.*]] = extractelement <4 x float> [[B:%.*]], i32 0
-; CHECK-NEXT:    [[E:%.*]] = fmul nnan float [[TMP1]], [[F:%.*]]
+; CHECK-NEXT:    [[E:%.*]] = fmul float [[TMP1]], [[F:%.*]]
 ; CHECK-NEXT:    ret float [[E]]
 ;
   %C = insertelement <4 x float> %A, float %f, i32 0
   %D = fmul nnan <4 x float> %C, %B
   %E = extractelement <4 x float> %D, i32 0
   ret float %E
 }
 

llvm/test/Transforms/InstCombine/scalarization.ll

;
   %add = fadd <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
   %r = extractelement <4 x float> %add, i32 %y
   ret float %r
 }
 
 define float @extelt_binop_insertelt(<4 x float> %A, <4 x float> %B, float %f) {
 ; CHECK-LABEL: @extelt_binop_insertelt(
 ; CHECK-NEXT:    [[TMP1:%.*]] = extractelement <4 x float> [[B:%.*]], i32 0
-; CHECK-NEXT:    [[E:%.*]] = fmul nnan float [[TMP1]], [[F:%.*]]
+; CHECK-NEXT:    [[E:%.*]] = fmul float [[TMP1]], [[F:%.*]]
 ; CHECK-NEXT:    ret float [[E]]
 ;
   %C = insertelement <4 x float> %A, float %f, i32 0
   %D = fmul nnan <4 x float> %C, %B
   %E = extractelement <4 x float> %D, i32 0
   ret float %E
 }
 

llvm/test/Transforms/InstCombine/shuffle_select-inseltpoison.ll

 ;
   %b = fmul nnan ninf <4 x float> %v, <float 41.0, float 42.0, float 43.0, float 44.0>
   %s = shufflevector <4 x float> %b, <4 x float> %v, <4 x i32> <i32 0, i32 5, i32 6, i32 7>
   ret <4 x float> %s
 }
 
 define <4 x double> @fdiv_constant_op0(<4 x double> %v) {
 ; CHECK-LABEL: @fdiv_constant_op0(
-; CHECK-NEXT:    [[B:%.*]] = fdiv fast <4 x double> <double poison, double poison, double 4.300000e+01, double 4.400000e+01>, [[V:%.*]]
+; CHECK-NEXT:    [[B:%.*]] = fdiv reassoc nsz arcp contract afn <4 x double> <double poison, double poison, double 4.300000e+01, double 4.400000e+01>, [[V:%.*]]
 ; CHECK-NEXT:    [[S:%.*]] = shufflevector <4 x double> [[V]], <4 x double> [[B]], <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
 ; CHECK-NEXT:    ret <4 x double> [[S]]
 ;
   %b = fdiv fast <4 x double> <double 41.0, double 42.0, double 43.0, double 44.0>, %v
   %s = shufflevector <4 x double> %v, <4 x double> %b, <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
   ret <4 x double> %s
 }
 
;
   %t1 = fmul nnan ninf <4 x float> %v0, <float 1.0, float 2.0, float 3.0, float 4.0>
   %t2 = fmul nnan ninf <4 x float> %v0, <float 5.0, float 6.0, float 7.0, float 8.0>
   %t3 = shufflevector <4 x float> %t1, <4 x float> %t2, <4 x i32> <i32 0, i32 5, i32 6, i32 7>
   ret <4 x float> %t3
 }
 
 define <4 x double> @fdiv_fdiv(<4 x double> %v0) {
 ; CHECK-LABEL: @fdiv_fdiv(
-; CHECK-NEXT:    [[T3:%.*]] = fdiv nnan arcp <4 x double> <double undef, double 2.000000e+00, double 7.000000e+00, double 8.000000e+00>, [[V0:%.*]]
+; CHECK-NEXT:    [[T3:%.*]] = fdiv arcp <4 x double> <double undef, double 2.000000e+00, double 7.000000e+00, double 8.000000e+00>, [[V0:%.*]]
 ; CHECK-NEXT:    ret <4 x double> [[T3]]
 ;
   %t1 = fdiv fast <4 x double> <double 1.0, double 2.0, double 3.0, double 4.0>, %v0
   %t2 = fdiv nnan arcp <4 x double> <double 5.0, double 6.0, double 7.0, double 8.0>, %v0
   %t3 = shufflevector <4 x double> %t1, <4 x double> %t2, <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
   ret <4 x double> %t3
 }
 
;
   %t2 = fmul reassoc nsz <4 x float> %v1, <float 5.0, float 6.0, float 7.0, float 8.0>
   %t3 = shufflevector <4 x float> %t1, <4 x float> %t2, <4 x i32> <i32 0, i32 5, i32 6, i32 7>
   ret <4 x float> %t3
 }
 
 define <4 x double> @frem_2_vars(<4 x double> %v0, <4 x double> %v1) {
 ; CHECK-LABEL: @frem_2_vars(
 ; CHECK-NEXT:    [[TMP1:%.*]] = shufflevector <4 x double> [[V0:%.*]], <4 x double> [[V1:%.*]], <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
-; CHECK-NEXT:    [[T3:%.*]] = frem nnan <4 x double> <double undef, double 2.000000e+00, double 7.000000e+00, double 8.000000e+00>, [[TMP1]]
+; CHECK-NEXT:    [[T3:%.*]] = frem <4 x double> <double undef, double 2.000000e+00, double 7.000000e+00, double 8.000000e+00>, [[TMP1]]
 ; CHECK-NEXT:    ret <4 x double> [[T3]]
 ;
   %t1 = frem nnan ninf <4 x double> <double 1.0, double 2.0, double 3.0, double 4.0>, %v0
   %t2 = frem nnan arcp <4 x double> <double 5.0, double 6.0, double 7.0, double 8.0>, %v1
   %t3 = shufflevector <4 x double> %t1, <4 x double> %t2, <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
   ret <4 x double> %t3
 }
 

llvm/test/Transforms/InstCombine/shuffle_select.ll

 ;
   %b = fmul nnan ninf <4 x float> %v, <float 41.0, float 42.0, float 43.0, float 44.0>
   %s = shufflevector <4 x float> %b, <4 x float> %v, <4 x i32> <i32 0, i32 5, i32 6, i32 7>
   ret <4 x float> %s
 }
 
 define <4 x double> @fdiv_constant_op0(<4 x double> %v) {
 ; CHECK-LABEL: @fdiv_constant_op0(
-; CHECK-NEXT:    [[B:%.*]] = fdiv fast <4 x double> <double poison, double poison, double 4.300000e+01, double 4.400000e+01>, [[V:%.*]]
+; CHECK-NEXT:    [[B:%.*]] = fdiv reassoc nsz arcp contract afn <4 x double> <double poison, double poison, double 4.300000e+01, double 4.400000e+01>, [[V:%.*]]
 ; CHECK-NEXT:    [[S:%.*]] = shufflevector <4 x double> [[V]], <4 x double> [[B]], <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
 ; CHECK-NEXT:    ret <4 x double> [[S]]
 ;
   %b = fdiv fast <4 x double> <double 41.0, double 42.0, double 43.0, double 44.0>, %v
   %s = shufflevector <4 x double> %v, <4 x double> %b, <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
   ret <4 x double> %s
 }
 
;
   %t1 = fmul nnan ninf <4 x float> %v0, <float 1.0, float 2.0, float 3.0, float 4.0>
   %t2 = fmul nnan ninf <4 x float> %v0, <float 5.0, float 6.0, float 7.0, float 8.0>
   %t3 = shufflevector <4 x float> %t1, <4 x float> %t2, <4 x i32> <i32 0, i32 5, i32 6, i32 7>
   ret <4 x float> %t3
 }
 
 define <4 x double> @fdiv_fdiv(<4 x double> %v0) {
 ; CHECK-LABEL: @fdiv_fdiv(
-; CHECK-NEXT:    [[T3:%.*]] = fdiv nnan arcp <4 x double> <double undef, double 2.000000e+00, double 7.000000e+00, double 8.000000e+00>, [[V0:%.*]]
+; CHECK-NEXT:    [[T3:%.*]] = fdiv arcp <4 x double> <double undef, double 2.000000e+00, double 7.000000e+00, double 8.000000e+00>, [[V0:%.*]]
 ; CHECK-NEXT:    ret <4 x double> [[T3]]
 ;
   %t1 = fdiv fast <4 x double> <double 1.0, double 2.0, double 3.0, double 4.0>, %v0
   %t2 = fdiv nnan arcp <4 x double> <double 5.0, double 6.0, double 7.0, double 8.0>, %v0
   %t3 = shufflevector <4 x double> %t1, <4 x double> %t2, <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
   ret <4 x double> %t3
 }
 
;
   %t2 = fmul reassoc nsz <4 x float> %v1, <float 5.0, float 6.0, float 7.0, float 8.0>
   %t3 = shufflevector <4 x float> %t1, <4 x float> %t2, <4 x i32> <i32 0, i32 5, i32 6, i32 7>
   ret <4 x float> %t3
 }
 
 define <4 x double> @frem_2_vars(<4 x double> %v0, <4 x double> %v1) {
 ; CHECK-LABEL: @frem_2_vars(
 ; CHECK-NEXT:    [[TMP1:%.*]] = shufflevector <4 x double> [[V0:%.*]], <4 x double> [[V1:%.*]], <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
-; CHECK-NEXT:    [[T3:%.*]] = frem nnan <4 x double> <double undef, double 2.000000e+00, double 7.000000e+00, double 8.000000e+00>, [[TMP1]]
+; CHECK-NEXT:    [[T3:%.*]] = frem <4 x double> <double undef, double 2.000000e+00, double 7.000000e+00, double 8.000000e+00>, [[TMP1]]
 ; CHECK-NEXT:    ret <4 x double> [[T3]]
 ;
   %t1 = frem nnan ninf <4 x double> <double 1.0, double 2.0, double 3.0, double 4.0>, %v0
   %t2 = frem nnan arcp <4 x double> <double 5.0, double 6.0, double 7.0, double 8.0>, %v1
   %t3 = shufflevector <4 x double> %t1, <4 x double> %t2, <4 x i32> <i32 undef, i32 1, i32 6, i32 7>
   ret <4 x double> %t3
 }
 

llvm/test/Transforms/InstCombine/vec_demanded_elts-inseltpoison.ll

 ;
   %bo = fadd <3 x float> %x, <float 1.0, float 2.0, float 3.0>
   %r = shufflevector <3 x float> %bo, <3 x float> poison, <3 x i32> <i32 undef, i32 1, i32 0>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_fsub(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_fsub(
-; CHECK-NEXT:    [[BO:%.*]] = fsub fast <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[BO:%.*]] = fsub reassoc nsz arcp contract afn <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> poison, <3 x i32> <i32 undef, i32 0, i32 2>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = fsub fast <3 x float> <float 1.0, float 2.0, float 3.0>, %x
   %r = shufflevector <3 x float> %bo, <3 x float> poison, <3 x i32> <i32 undef, i32 0, i32 2>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_fmul(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_fmul(
 ; CHECK-NEXT:    [[BO:%.*]] = fmul reassoc <3 x float> [[X:%.*]], <float 1.000000e+00, float 2.000000e+00, float poison>
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> poison, <3 x i32> <i32 undef, i32 1, i32 0>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = fmul reassoc <3 x float> %x, <float 1.0, float 2.0, float 3.0>
   %r = shufflevector <3 x float> %bo, <3 x float> poison, <3 x i32> <i32 undef, i32 1, i32 0>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_fdiv_const_op0(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_fdiv_const_op0(
-; CHECK-NEXT:    [[BO:%.*]] = fdiv reassoc ninf <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[BO:%.*]] = fdiv reassoc <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> poison, <3 x i32> <i32 undef, i32 0, i32 2>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = fdiv ninf reassoc <3 x float> <float 1.0, float 2.0, float 3.0>, %x
   %r = shufflevector <3 x float> %bo, <3 x float> poison, <3 x i32> <i32 undef, i32 0, i32 2>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_fdiv_const_op1(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_fdiv_const_op1(
-; CHECK-NEXT:    [[BO:%.*]] = fdiv nnan ninf <3 x float> [[X:%.*]], <float 1.000000e+00, float 2.000000e+00, float poison>
+; CHECK-NEXT:    [[BO:%.*]] = fdiv <3 x float> [[X:%.*]], <float 1.000000e+00, float 2.000000e+00, float poison>
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> poison, <3 x i32> <i32 undef, i32 1, i32 0>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = fdiv ninf nnan <3 x float> %x, <float 1.0, float 2.0, float 3.0>
   %r = shufflevector <3 x float> %bo, <3 x float> poison, <3 x i32> <i32 undef, i32 1, i32 0>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_frem_const_op0(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_frem_const_op0(
-; CHECK-NEXT:    [[BO:%.*]] = frem nnan <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[BO:%.*]] = frem <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> poison, <3 x i32> <i32 undef, i32 2, i32 0>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = frem nnan <3 x float> <float 1.0, float 2.0, float 3.0>, %x
   %r = shufflevector <3 x float> %bo, <3 x float> poison, <3 x i32> <i32 undef, i32 2, i32 0>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_frem_const_op1(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_frem_const_op1(
-; CHECK-NEXT:    [[BO:%.*]] = frem reassoc ninf <3 x float> [[X:%.*]], <float poison, float 2.000000e+00, float 3.000000e+00>
+; CHECK-NEXT:    [[BO:%.*]] = frem reassoc <3 x float> [[X:%.*]], <float poison, float 2.000000e+00, float 3.000000e+00>
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> poison, <3 x i32> <i32 1, i32 undef, i32 2>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = frem ninf reassoc <3 x float> %x, <float 1.0, float 2.0, float 3.0>
   %r = shufflevector <3 x float> %bo, <3 x float> poison, <3 x i32> <i32 1, i32 undef, i32 2>
   ret <3 x float> %r
 }
 

llvm/test/Transforms/InstCombine/vec_demanded_elts.ll

 ;
   %bo = fadd <3 x float> %x, <float 1.0, float 2.0, float 3.0>
   %r = shufflevector <3 x float> %bo, <3 x float> undef, <3 x i32> <i32 undef, i32 1, i32 0>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_fsub(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_fsub(
-; CHECK-NEXT:    [[BO:%.*]] = fsub fast <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[BO:%.*]] = fsub reassoc nsz arcp contract afn <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> undef, <3 x i32> <i32 undef, i32 0, i32 2>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = fsub fast <3 x float> <float 1.0, float 2.0, float 3.0>, %x
   %r = shufflevector <3 x float> %bo, <3 x float> undef, <3 x i32> <i32 undef, i32 0, i32 2>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_fmul(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_fmul(
 ; CHECK-NEXT:    [[BO:%.*]] = fmul reassoc <3 x float> [[X:%.*]], <float 1.000000e+00, float 2.000000e+00, float poison>
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> undef, <3 x i32> <i32 undef, i32 1, i32 0>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = fmul reassoc <3 x float> %x, <float 1.0, float 2.0, float 3.0>
   %r = shufflevector <3 x float> %bo, <3 x float> undef, <3 x i32> <i32 undef, i32 1, i32 0>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_fdiv_const_op0(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_fdiv_const_op0(
-; CHECK-NEXT:    [[BO:%.*]] = fdiv reassoc ninf <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[BO:%.*]] = fdiv reassoc <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> undef, <3 x i32> <i32 undef, i32 0, i32 2>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = fdiv ninf reassoc <3 x float> <float 1.0, float 2.0, float 3.0>, %x
   %r = shufflevector <3 x float> %bo, <3 x float> undef, <3 x i32> <i32 undef, i32 0, i32 2>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_fdiv_const_op1(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_fdiv_const_op1(
-; CHECK-NEXT:    [[BO:%.*]] = fdiv nnan ninf <3 x float> [[X:%.*]], <float 1.000000e+00, float 2.000000e+00, float poison>
+; CHECK-NEXT:    [[BO:%.*]] = fdiv <3 x float> [[X:%.*]], <float 1.000000e+00, float 2.000000e+00, float poison>
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> undef, <3 x i32> <i32 undef, i32 1, i32 0>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = fdiv ninf nnan <3 x float> %x, <float 1.0, float 2.0, float 3.0>
   %r = shufflevector <3 x float> %bo, <3 x float> undef, <3 x i32> <i32 undef, i32 1, i32 0>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_frem_const_op0(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_frem_const_op0(
-; CHECK-NEXT:    [[BO:%.*]] = frem nnan <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[BO:%.*]] = frem <3 x float> <float 1.000000e+00, float poison, float 3.000000e+00>, [[X:%.*]]
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> undef, <3 x i32> <i32 undef, i32 2, i32 0>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = frem nnan <3 x float> <float 1.0, float 2.0, float 3.0>, %x
   %r = shufflevector <3 x float> %bo, <3 x float> undef, <3 x i32> <i32 undef, i32 2, i32 0>
   ret <3 x float> %r
 }
 
 define <3 x float> @shuf_frem_const_op1(<3 x float> %x) {
 ; CHECK-LABEL: @shuf_frem_const_op1(
-; CHECK-NEXT:    [[BO:%.*]] = frem reassoc ninf <3 x float> [[X:%.*]], <float poison, float 2.000000e+00, float 3.000000e+00>
+; CHECK-NEXT:    [[BO:%.*]] = frem reassoc <3 x float> [[X:%.*]], <float poison, float 2.000000e+00, float 3.000000e+00>
 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> undef, <3 x i32> <i32 1, i32 undef, i32 2>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = frem ninf reassoc <3 x float> %x, <float 1.0, float 2.0, float 3.0>
   %r = shufflevector <3 x float> %bo, <3 x float> undef, <3 x i32> <i32 1, i32 undef, i32 2>
   ret <3 x float> %r
 }
 

llvm/test/Transforms/LoopVectorize/X86/drop-poison-generating-flags.ll

 ; CHECK-NEXT:    [[TMP2:%.*]] = getelementptr inbounds float, float* [[TMP1]], i32 0
 ; CHECK-NEXT:    [[TMP3:%.*]] = bitcast float* [[TMP2]] to <4 x float>*
 ; CHECK-NEXT:    [[WIDE_LOAD:%.*]] = load <4 x float>, <4 x float>* [[TMP3]], align 4
 ; CHECK-NEXT:    [[TMP4:%.*]] = fcmp uno <4 x float> [[WIDE_LOAD]], [[WIDE_LOAD]]
 ; CHECK-NEXT:    [[TMP5:%.*]] = getelementptr float, float* [[OUTPUT:%.*]], i64 [[TMP0]]
 ; CHECK-NEXT:    [[TMP6:%.*]] = getelementptr float, float* [[TMP5]], i32 0
 ; CHECK-NEXT:    [[TMP7:%.*]] = bitcast float* [[TMP6]] to <4 x float>*
 ; CHECK-NEXT:    [[WIDE_MASKED_LOAD:%.*]] = call <4 x float> @llvm.masked.load.v4f32.p0v4f32(<4 x float>* [[TMP7]], i32 4, <4 x i1> [[TMP4]], <4 x float> poison)
-; CHECK-NEXT:    [[TMP8:%.*]] = fadd nnan <4 x float> [[WIDE_LOAD]], [[WIDE_MASKED_LOAD]]
+; CHECK-NEXT:    [[TMP8:%.*]] = fadd <4 x float> [[WIDE_LOAD]], [[WIDE_MASKED_LOAD]]
 ;
 entry:
   br label %for.body
 
 for.body:
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.inc ]
   %arrayidx = getelementptr inbounds float, float* %input, i64 %indvars.iv
   %0 = load float, float* %arrayidx, align 4

 ; CHECK-NEXT:    [[WIDE_LOAD:%.*]] = load <4 x float>, <4 x float>* [[TMP3]], align 4
 ; CHECK-NEXT:    [[TMP4:%.*]] = fcmp une <4 x float> [[WIDE_LOAD]], <float 0x7FF0000000000000, float 0x7FF0000000000000, float 0x7FF0000000000000, float 0x7FF0000000000000>
 ; CHECK-NEXT:    [[TMP5:%.*]] = fcmp une <4 x float> [[WIDE_LOAD]], <float 0xFFF0000000000000, float 0xFFF0000000000000, float 0xFFF0000000000000, float 0xFFF0000000000000>
 ; CHECK-NEXT:    [[TMP6:%.*]] = and <4 x i1> [[TMP4]], [[TMP5]]
 ; CHECK-NEXT:    [[TMP7:%.*]] = getelementptr float, float* [[OUTPUT:%.*]], i64 [[TMP0]]
 ; CHECK-NEXT:    [[TMP8:%.*]] = getelementptr float, float* [[TMP7]], i32 0
 ; CHECK-NEXT:    [[TMP9:%.*]] = bitcast float* [[TMP8]] to <4 x float>*
 ; CHECK-NEXT:    [[WIDE_MASKED_LOAD:%.*]] = call <4 x float> @llvm.masked.load.v4f32.p0v4f32(<4 x float>* [[TMP9]], i32 4, <4 x i1> [[TMP6]], <4 x float> poison)
-; CHECK-NEXT:    [[TMP10:%.*]] = fadd ninf <4 x float> [[WIDE_LOAD]], [[WIDE_MASKED_LOAD]]
+; CHECK-NEXT:    [[TMP10:%.*]] = fadd <4 x float> [[WIDE_LOAD]], [[WIDE_MASKED_LOAD]]
 entry:
   br label %for.body
 
 for.body:
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.inc ]
   %arrayidx = getelementptr inbounds float, float* %input, i64 %indvars.iv
   %0 = load float, float* %arrayidx, align 4
   %plus = fcmp une float %0, 0x7FF0000000000000

llvm/test/Transforms/LoopVectorize/X86/imprecise-through-phis.ll

 ; AVX-NEXT:    [[INDEX:%.*]] = phi i32 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
 ; AVX-NEXT:    [[VEC_PHI:%.*]] = phi <4 x double> [ zeroinitializer, [[VECTOR_PH]] ], [ [[PREDPHI:%.*]], [[VECTOR_BODY]] ]
 ; AVX-NEXT:    [[TMP0:%.*]] = add i32 [[INDEX]], 0
 ; AVX-NEXT:    [[TMP1:%.*]] = getelementptr double, double* [[ARR:%.*]], i32 [[TMP0]]
 ; AVX-NEXT:    [[TMP2:%.*]] = getelementptr double, double* [[TMP1]], i32 0
 ; AVX-NEXT:    [[TMP3:%.*]] = bitcast double* [[TMP2]] to <4 x double>*
 ; AVX-NEXT:    [[WIDE_LOAD:%.*]] = load <4 x double>, <4 x double>* [[TMP3]], align 8
 ; AVX-NEXT:    [[TMP4:%.*]] = fcmp fast une <4 x double> [[WIDE_LOAD]], <double 4.200000e+01, double 4.200000e+01, double 4.200000e+01, double 4.200000e+01>
-; AVX-NEXT:    [[TMP5:%.*]] = fadd fast <4 x double> [[VEC_PHI]], [[WIDE_LOAD]]
+; AVX-NEXT:    [[TMP5:%.*]] = fadd reassoc nsz arcp contract afn <4 x double> [[VEC_PHI]], [[WIDE_LOAD]]
 ; AVX-NEXT:    [[TMP6:%.*]] = xor <4 x i1> [[TMP4]], <i1 true, i1 true, i1 true, i1 true>
 ; AVX-NEXT:    [[PREDPHI]] = select <4 x i1> [[TMP4]], <4 x double> [[TMP5]], <4 x double> [[VEC_PHI]]
 ; AVX-NEXT:    [[INDEX_NEXT]] = add nuw i32 [[INDEX]], 4
 ; AVX-NEXT:    [[TMP7:%.*]] = icmp eq i32 [[INDEX_NEXT]], 32
 ; AVX-NEXT:    br i1 [[TMP7]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], [[LOOP0:!llvm.loop !.*]]
 ; AVX:       middle.block:
 ; AVX-NEXT:    [[TMP8:%.*]] = call fast double @llvm.vector.reduce.fadd.v4f64(double -0.000000e+00, <4 x double> [[PREDPHI]])
 ; AVX-NEXT:    [[CMP_N:%.*]] = icmp eq i32 32, 32

llvm/test/Transforms/LoopVectorize/if-reduction.ll

 ;       sum+=3*a[i];
 ;   }
 ;   return sum;
 ; }
 
 ; CHECK-LABEL: @fcmp_multi(
 ; CHECK: %[[C1:.*]] = fcmp ogt <4 x float> %[[V0:.*]], <float 1.000000e+00,
 ; CHECK: %[[C2:.*]] = fcmp olt <4 x float> %[[V0]], <float 3.000000e+00,
-; CHECK-DAG: %[[M1:.*]] = fmul fast <4 x float> %[[V0]], <float 3.000000e+00,
-; CHECK-DAG: %[[M2:.*]] = fmul fast <4 x float> %[[V0]], <float 2.000000e+00,
+; CHECK-DAG: %[[M1:.*]] = fmul reassoc nsz arcp contract afn <4 x float> %[[V0]], <float 3.000000e+00,
+; CHECK-DAG: %[[M2:.*]] = fmul reassoc nsz arcp contract afn <4 x float> %[[V0]], <float 2.000000e+00,
 ; CHECK: %[[C11:.*]] = xor <4 x i1> %[[C1]], <i1 true,
 ; CHECK-DAG: %[[C12:.*]] = select <4 x i1> %[[C11]], <4 x i1> %[[C2]], <4 x i1> zeroinitializer
 ; CHECK-DAG: %[[C21:.*]] = xor <4 x i1> %[[C2]], <i1 true,
 ; CHECK: %[[C22:.*]] = select <4 x i1> %[[C11]], <4 x i1> %[[C21]], <4 x i1> zeroinitializer
 ; CHECK: %[[S1:.*]] = select <4 x i1> %[[C22]], <4 x float> %[[M1]], <4 x float> %[[M2]]
 ; CHECK: %[[S2:.*]] = select <4 x i1> %[[C1]], <4 x float> %[[V0]], <4 x float> %[[S1]]
 ; CHECK: fadd fast <4 x float> %[[S2]],
 define float @fcmp_multi(float* nocapture readonly %a, i32 %n) nounwind readonly {

 ;       sum-=a[i];
 ;   }
 ;   return sum;
 ; }
 
 ; CHECK-LABEL: @fcmp_fadd_fsub(
 ; CHECK: %[[C1:.*]] = fcmp ogt <4 x float> %[[V0:.*]], <float 1.000000e+00,
 ; CHECK: %[[C2:.*]] = fcmp olt <4 x float> %[[V0]], <float 3.000000e+00,
-; CHECK-DAG: %[[SUB:.*]] = fsub fast <4 x float>
-; CHECK-DAG: %[[ADD:.*]] = fadd fast <4 x float>
+; CHECK-DAG: %[[SUB:.*]] = fsub reassoc nsz arcp contract afn <4 x float>
+; CHECK-DAG: %[[ADD:.*]] = fadd reassoc nsz arcp contract afn <4 x float>
 ; CHECK: %[[C11:.*]] = xor <4 x i1> %[[C1]], <i1 true,
 ; CHECK-DAG: %[[C12:.*]] = select <4 x i1> %[[C11]], <4 x i1> %[[C2]], <4 x i1> zeroinitializer
 ; CHECK-DAG: %[[C21:.*]] = xor <4 x i1> %[[C2]], <i1 true,
 ; CHECK: %[[C22:.*]] = select <4 x i1> %[[C11]], <4 x i1> %[[C21]], <4 x i1> zeroinitializer
 ; CHECK: %[[S1:.*]] = select <4 x i1> %[[C12]], <4 x float> %[[SUB]], <4 x float> %[[ADD]]
 ; CHECK: %[[S2:.*]] = select <4 x i1> %[[C22]], {{.*}} <4 x float> %[[S1]]
 define float @fcmp_fadd_fsub(float* nocapture readonly %a, i32 %n) nounwind readonly {
 entry:

llvm/test/Transforms/LoopVectorize/reduction-inloop-pred.ll

 ; CHECK-NEXT:    [[TMP6:%.*]] = fcmp ule <4 x float> [[WIDE_LOAD1]], <float 1.000000e+00, float 1.000000e+00, float 1.000000e+00, float 1.000000e+00>
 ; CHECK-NEXT:    [[TMP7:%.*]] = fcmp ogt <4 x float> [[WIDE_LOAD]], <float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00>
 ; CHECK-NEXT:    [[TMP8:%.*]] = and <4 x i1> [[TMP5]], [[TMP6]]
 ; CHECK-NEXT:    [[TMP9:%.*]] = and <4 x i1> [[TMP8]], [[TMP7]]
 ; CHECK-NEXT:    [[TMP10:%.*]] = xor <4 x i1> [[TMP7]], <i1 true, i1 true, i1 true, i1 true>
 ; CHECK-NEXT:    [[TMP11:%.*]] = and <4 x i1> [[TMP8]], [[TMP10]]
 ; CHECK-NEXT:    [[TMP12:%.*]] = xor <4 x i1> [[TMP5]], <i1 true, i1 true, i1 true, i1 true>
 ; CHECK-NEXT:    [[PREDPHI_V:%.*]] = select <4 x i1> [[TMP9]], <4 x float> [[WIDE_LOAD1]], <4 x float> [[WIDE_LOAD]]
-; CHECK-NEXT:    [[PREDPHI:%.*]] = fadd fast <4 x float> [[VEC_PHI]], [[PREDPHI_V]]
+; CHECK-NEXT:    [[PREDPHI:%.*]] = fadd reassoc nsz arcp contract afn <4 x float> [[VEC_PHI]], [[PREDPHI_V]]
 ; CHECK-NEXT:    [[TMP13:%.*]] = select <4 x i1> [[TMP12]], <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i1> [[TMP11]]
 ; CHECK-NEXT:    [[PREDPHI3]] = select <4 x i1> [[TMP13]], <4 x float> [[VEC_PHI]], <4 x float> [[PREDPHI]]
 ; CHECK-NEXT:    [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4
 ; CHECK-NEXT:    [[TMP14:%.*]] = icmp eq i64 [[INDEX_NEXT]], 128
 ; CHECK-NEXT:    br i1 [[TMP14]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP28:![0-9]+]]
 ; CHECK:       middle.block:
 ; CHECK-NEXT:    [[TMP15:%.*]] = call fast float @llvm.vector.reduce.fadd.v4f32(float -0.000000e+00, <4 x float> [[PREDPHI3]])
 ; CHECK-NEXT:    br i1 true, label [[FOR_END:%.*]], label [[SCALAR_PH]]

llvm/test/Transforms/LoopVectorize/reduction-inloop.ll

 ; CHECK-NEXT:    [[TMP6:%.*]] = fcmp ule <4 x float> [[WIDE_LOAD1]], <float 1.000000e+00, float 1.000000e+00, float 1.000000e+00, float 1.000000e+00>
 ; CHECK-NEXT:    [[TMP7:%.*]] = fcmp ogt <4 x float> [[WIDE_LOAD]], <float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+00>
 ; CHECK-NEXT:    [[TMP8:%.*]] = and <4 x i1> [[TMP5]], [[TMP6]]
 ; CHECK-NEXT:    [[TMP9:%.*]] = and <4 x i1> [[TMP8]], [[TMP7]]
 ; CHECK-NEXT:    [[TMP10:%.*]] = xor <4 x i1> [[TMP7]], <i1 true, i1 true, i1 true, i1 true>
 ; CHECK-NEXT:    [[TMP11:%.*]] = and <4 x i1> [[TMP8]], [[TMP10]]
 ; CHECK-NEXT:    [[TMP12:%.*]] = xor <4 x i1> [[TMP5]], <i1 true, i1 true, i1 true, i1 true>
 ; CHECK-NEXT:    [[PREDPHI_V:%.*]] = select <4 x i1> [[TMP9]], <4 x float> [[WIDE_LOAD1]], <4 x float> [[WIDE_LOAD]]
-; CHECK-NEXT:    [[PREDPHI:%.*]] = fadd fast <4 x float> [[VEC_PHI]], [[PREDPHI_V]]
+; CHECK-NEXT:    [[PREDPHI:%.*]] = fadd reassoc nsz arcp contract afn <4 x float> [[VEC_PHI]], [[PREDPHI_V]]
 ; CHECK-NEXT:    [[TMP13:%.*]] = select <4 x i1> [[TMP12]], <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i1> [[TMP11]]
 ; CHECK-NEXT:    [[PREDPHI3]] = select <4 x i1> [[TMP13]], <4 x float> [[VEC_PHI]], <4 x float> [[PREDPHI]]
 ; CHECK-NEXT:    [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4
 ; CHECK-NEXT:    [[TMP14:%.*]] = icmp eq i64 [[INDEX_NEXT]], 128
 ; CHECK-NEXT:    br i1 [[TMP14]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP32:![0-9]+]]
 ; CHECK:       middle.block:
 ; CHECK-NEXT:    [[TMP15:%.*]] = call fast float @llvm.vector.reduce.fadd.v4f32(float -0.000000e+00, <4 x float> [[PREDPHI3]])
 ; CHECK-NEXT:    br i1 true, label [[FOR_END:%.*]], label [[SCALAR_PH]]

llvm/test/Transforms/LoopVectorize/reduction.ll

 
 for.end:                                          ; preds = %for.body, %entry
   %x.0.lcssa = phi i32 [ 0, %entry ], [ %sub, %for.body ]
   ret i32 %x.0.lcssa
 }
 
 ; We can vectorize conditional reductions with multi-input phis.
 ; CHECK: reduction_conditional
-; CHECK: fadd fast <4 x float>
+; CHECK: fadd reassoc nsz arcp contract afn <4 x float>
 
 define float @reduction_conditional(float* %A, float* %B, float* %C, float %S) {
 entry:
   br label %for.body
 
 for.body:
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.inc ]
   %sum.033 = phi float [ %S, %entry ], [ %sum.1, %for.inc ]

llvm/test/Transforms/PhaseOrdering/ARM/mve-floatreduce.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt < %s -O3 -S                   | FileCheck %s
 ; RUN: opt < %s -passes='default<O3>' -S | FileCheck %s
 
 target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
 target triple = "thumbv8.1m.main-none-none-eabi"
 
 define arm_aapcs_vfpcc half @vecAddAcrossF16Mve(<8 x half> %in) #0 {
 ; CHECK-LABEL: @vecAddAcrossF16Mve(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = shufflevector <8 x half> [[IN:%.*]], <8 x half> poison, <8 x i32> <i32 1, i32 0, i32 3, i32 2, i32 5, i32 4, i32 7, i32 6>
 ; CHECK-NEXT:    [[TMP1:%.*]] = fadd fast <8 x half> [[TMP0]], [[IN]]
 ; CHECK-NEXT:    [[TMP2:%.*]] = bitcast <8 x half> [[TMP1]] to <4 x i32>
 ; CHECK-NEXT:    [[TMP3:%.*]] = shufflevector <4 x i32> [[TMP2]], <4 x i32> poison, <4 x i32> <i32 1, i32 undef, i32 3, i32 undef>
 ; CHECK-NEXT:    [[TMP4:%.*]] = bitcast <4 x i32> [[TMP3]] to <8 x half>
-; CHECK-NEXT:    [[TMP5:%.*]] = fadd fast <8 x half> [[TMP1]], [[TMP4]]
+; CHECK-NEXT:    [[TMP5:%.*]] = fadd reassoc nsz arcp contract afn <8 x half> [[TMP1]], [[TMP4]]
 ; CHECK-NEXT:    [[TMP6:%.*]] = extractelement <8 x half> [[TMP5]], i32 0
 ; CHECK-NEXT:    [[TMP7:%.*]] = extractelement <8 x half> [[TMP5]], i32 4
 ; CHECK-NEXT:    [[ADD:%.*]] = fadd fast half [[TMP6]], [[TMP7]]
 ; CHECK-NEXT:    ret half [[ADD]]
 ;
 entry:
   %0 = shufflevector <8 x half> %in, <8 x half> poison, <8 x i32> <i32 1, i32 0, i32 3, i32 2, i32 5, i32 4, i32 7, i32 6>
   %1 = fadd fast <8 x half> %0, %in

llvm/test/Transforms/PhaseOrdering/X86/vector-reductions-expanded.ll

 }
 
 define float @fadd_v4i32(float* %p) #0 {
 ; CHECK-LABEL: @fadd_v4i32(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = bitcast float* [[P:%.*]] to <4 x float>*
 ; CHECK-NEXT:    [[TMP1:%.*]] = load <4 x float>, <4 x float>* [[TMP0]], align 4, !tbaa [[TBAA7:![0-9]+]]
 ; CHECK-NEXT:    [[RDX_SHUF:%.*]] = shufflevector <4 x float> [[TMP1]], <4 x float> poison, <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX:%.*]] = fadd fast <4 x float> [[TMP1]], [[RDX_SHUF]]
+; CHECK-NEXT:    [[BIN_RDX:%.*]] = fadd reassoc nsz arcp contract afn <4 x float> [[TMP1]], [[RDX_SHUF]]
 ; CHECK-NEXT:    [[RDX_SHUF3:%.*]] = shufflevector <4 x float> [[BIN_RDX]], <4 x float> poison, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX4:%.*]] = fadd fast <4 x float> [[BIN_RDX]], [[RDX_SHUF3]]
+; CHECK-NEXT:    [[BIN_RDX4:%.*]] = fadd reassoc nsz arcp contract afn <4 x float> [[BIN_RDX]], [[RDX_SHUF3]]
 ; CHECK-NEXT:    [[TMP2:%.*]] = extractelement <4 x float> [[BIN_RDX4]], i32 0
 ; CHECK-NEXT:    [[BIN_RDX5:%.*]] = fadd fast float 4.200000e+01, [[TMP2]]
 ; CHECK-NEXT:    ret float [[BIN_RDX5]]
 ;
 entry:
   br label %for.cond
 
 for.cond:

 }
 
 define float @fmul_v4i32(float* %p) #0 {
 ; CHECK-LABEL: @fmul_v4i32(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = bitcast float* [[P:%.*]] to <4 x float>*
 ; CHECK-NEXT:    [[TMP1:%.*]] = load <4 x float>, <4 x float>* [[TMP0]], align 4, !tbaa [[TBAA7]]
 ; CHECK-NEXT:    [[RDX_SHUF:%.*]] = shufflevector <4 x float> [[TMP1]], <4 x float> poison, <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX:%.*]] = fmul fast <4 x float> [[TMP1]], [[RDX_SHUF]]
+; CHECK-NEXT:    [[BIN_RDX:%.*]] = fmul reassoc nsz arcp contract afn <4 x float> [[TMP1]], [[RDX_SHUF]]
 ; CHECK-NEXT:    [[RDX_SHUF3:%.*]] = shufflevector <4 x float> [[BIN_RDX]], <4 x float> poison, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
-; CHECK-NEXT:    [[BIN_RDX4:%.*]] = fmul fast <4 x float> [[BIN_RDX]], [[RDX_SHUF3]]
+; CHECK-NEXT:    [[BIN_RDX4:%.*]] = fmul reassoc nsz arcp contract afn <4 x float> [[BIN_RDX]], [[RDX_SHUF3]]
 ; CHECK-NEXT:    [[TMP2:%.*]] = extractelement <4 x float> [[BIN_RDX4]], i32 0
 ; CHECK-NEXT:    [[BIN_RDX5:%.*]] = fmul fast float 1.000000e+00, [[TMP2]]
 ; CHECK-NEXT:    [[OP_EXTRA:%.*]] = fmul fast float [[BIN_RDX5]], 4.200000e+01
 ; CHECK-NEXT:    ret float [[OP_EXTRA]]
 ;
 entry:
   br label %for.cond
 

 
 define float @fmin_v4f32(float* %p) #0 {
 ; CHECK-LABEL: @fmin_v4f32(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = bitcast float* [[P:%.*]] to <4 x float>*
 ; CHECK-NEXT:    [[TMP1:%.*]] = load <4 x float>, <4 x float>* [[TMP0]], align 4, !tbaa [[TBAA7]]
 ; CHECK-NEXT:    [[RDX_SHUF:%.*]] = shufflevector <4 x float> [[TMP1]], <4 x float> poison, <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
 ; CHECK-NEXT:    [[RDX_MINMAX_CMP:%.*]] = fcmp fast olt <4 x float> [[TMP1]], [[RDX_SHUF]]
-; CHECK-NEXT:    [[RDX_MINMAX_SELECT:%.*]] = select fast <4 x i1> [[RDX_MINMAX_CMP]], <4 x float> [[TMP1]], <4 x float> [[RDX_SHUF]]
+; CHECK-NEXT:    [[RDX_MINMAX_SELECT:%.*]] = select reassoc nsz arcp contract afn <4 x i1> [[RDX_MINMAX_CMP]], <4 x float> [[TMP1]], <4 x float> [[RDX_SHUF]]
 ; CHECK-NEXT:    [[RDX_SHUF3:%.*]] = shufflevector <4 x float> [[RDX_MINMAX_SELECT]], <4 x float> poison, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
 ; CHECK-NEXT:    [[RDX_MINMAX_CMP4:%.*]] = fcmp fast olt <4 x float> [[RDX_MINMAX_SELECT]], [[RDX_SHUF3]]
-; CHECK-NEXT:    [[RDX_MINMAX_SELECT5:%.*]] = select fast <4 x i1> [[RDX_MINMAX_CMP4]], <4 x float> [[RDX_MINMAX_SELECT]], <4 x float> [[RDX_SHUF3]]
+; CHECK-NEXT:    [[RDX_MINMAX_SELECT5:%.*]] = select reassoc nsz arcp contract afn <4 x i1> [[RDX_MINMAX_CMP4]], <4 x float> [[RDX_MINMAX_SELECT]], <4 x float> [[RDX_SHUF3]]
 ; CHECK-NEXT:    [[TMP2:%.*]] = extractelement <4 x float> [[RDX_MINMAX_SELECT5]], i32 0
 ; CHECK-NEXT:    ret float [[TMP2]]
 ;
 entry:
   br label %for.cond
 
 for.cond:
   %r.0 = phi float [  0x47EFFFFFE0000000, %entry ], [ %cond, %for.inc ]

 ; PR23116
 
 define float @findMax(<8 x float>* byval(<8 x float>) align 16 %0) {
 ; CHECK-LABEL: @findMax(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[V:%.*]] = load <8 x float>, <8 x float>* [[TMP0:%.*]], align 16, !tbaa [[TBAA0]]
 ; CHECK-NEXT:    [[RDX_SHUF:%.*]] = shufflevector <8 x float> [[V]], <8 x float> poison, <8 x i32> <i32 4, i32 5, i32 6, i32 7, i32 undef, i32 undef, i32 undef, i32 undef>
 ; CHECK-NEXT:    [[RDX_MINMAX_CMP:%.*]] = fcmp nnan ninf nsz ogt <8 x float> [[V]], [[RDX_SHUF]]
-; CHECK-NEXT:    [[RDX_MINMAX_SELECT:%.*]] = select nnan ninf nsz <8 x i1> [[RDX_MINMAX_CMP]], <8 x float> [[V]], <8 x float> [[RDX_SHUF]]
+; CHECK-NEXT:    [[RDX_MINMAX_SELECT:%.*]] = select nsz <8 x i1> [[RDX_MINMAX_CMP]], <8 x float> [[V]], <8 x float> [[RDX_SHUF]]
 ; CHECK-NEXT:    [[RDX_SHUF8:%.*]] = shufflevector <8 x float> [[RDX_MINMAX_SELECT]], <8 x float> poison, <8 x i32> <i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
 ; CHECK-NEXT:    [[RDX_MINMAX_CMP9:%.*]] = fcmp nnan ninf nsz ogt <8 x float> [[RDX_MINMAX_SELECT]], [[RDX_SHUF8]]
-; CHECK-NEXT:    [[RDX_MINMAX_SELECT10:%.*]] = select nnan ninf nsz <8 x i1> [[RDX_MINMAX_CMP9]], <8 x float> [[RDX_MINMAX_SELECT]], <8 x float> [[RDX_SHUF8]]
+; CHECK-NEXT:    [[RDX_MINMAX_SELECT10:%.*]] = select nsz <8 x i1> [[RDX_MINMAX_CMP9]], <8 x float> [[RDX_MINMAX_SELECT]], <8 x float> [[RDX_SHUF8]]
 ; CHECK-NEXT:    [[RDX_SHUF11:%.*]] = shufflevector <8 x float> [[RDX_MINMAX_SELECT10]], <8 x float> poison, <8 x i32> <i32 1, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
 ; CHECK-NEXT:    [[RDX_MINMAX_CMP12:%.*]] = fcmp nnan ninf nsz ogt <8 x float> [[RDX_MINMAX_SELECT10]], [[RDX_SHUF11]]
-; CHECK-NEXT:    [[RDX_MINMAX_SELECT13:%.*]] = select nnan ninf nsz <8 x i1> [[RDX_MINMAX_CMP12]], <8 x float> [[RDX_MINMAX_SELECT10]], <8 x float> [[RDX_SHUF11]]
+; CHECK-NEXT:    [[RDX_MINMAX_SELECT13:%.*]] = select nsz <8 x i1> [[RDX_MINMAX_CMP12]], <8 x float> [[RDX_MINMAX_SELECT10]], <8 x float> [[RDX_SHUF11]]
 ; CHECK-NEXT:    [[TMP1:%.*]] = extractelement <8 x float> [[RDX_MINMAX_SELECT13]], i32 0
 ; CHECK-NEXT:    ret float [[TMP1]]
 ;
 entry:
   %v.addr = alloca <8 x float>, align 32
   %v = load <8 x float>, <8 x float>* %0, align 16, !tbaa !3
   store <8 x float> %v, <8 x float>* %v.addr, align 32, !tbaa !3
   %1 = load <8 x float>, <8 x float>* %v.addr, align 32, !tbaa !3