[InstCombine] canonicalize fneg before fmul/fdiv

[spatel]

Summary

I'm proposing to reverse the canonicalization of fneg relative to fmul/fdiv. That makes it easier to implement the transforms (and possibly other fneg transforms) in 1 place because we can always start the pattern match from fneg (either the legacy binop or the new unop).

There's a secondary practical benefit seen in PR21914 and PR42681:
https://bugs.llvm.org/show_bug.cgi?id=21914
https://bugs.llvm.org/show_bug.cgi?id=42681
...hoisting fneg rather than sinking seems to play nicer with LICM in IR (although this change may expose analysis holes in the other direction).

1. The instcombine test changes show the expected neutral IR diffs from reversing the order.

2. The reassociation tests show that we were missing an optimization opportunity to fold away fneg-of-fneg. My reading of IEEE-754 says that all of these transforms are allowed (regardless of binop/unop fneg version) because:

"For all other operations [besides copy/abs/negate/copysign], this standard does not specify the sign bit of a NaN result."
In all of these transforms, we always have some other binop (fadd/fsub/fmul/fdiv), so we are free to flip the sign bit of a potential intermediate NaN operand.
(If that interpretation is wrong, then we must already have a bug in the existing transforms?)

3. The clang tests shouldn't exist as-is, but that's effectively a revert of rL367149 (the test broke with an extension of the pre-existing fneg canonicalization in rL367146).

[xbolva00]

llvm/test/Transforms/InstCombine/fadd.ll

163 ↗

(On Diff #212169)

Comment needs to be fixed

[spatel]

llvm/test/Transforms/InstCombine/fadd.ll

163 ↗

(On Diff #212169)

Ah, right. And some other tests have assumed the transform in the test name. Fix coming up.

[spatel]

Patch updated:
Fixed test comments/names to reflect change in transformations.

[mcberg2017]

General comment: For us internally at Apple this does cause regressions, its small but there. Mostly because fneg becomes a side effect for fp arithmetic ops for us. I would request a target info guard which we can turn off to avoid the drift. If not, this is small enough to live with.

[spatel]

In D65399#1604816, @mcberg2017 wrote:

General comment: For us internally at Apple this does cause regressions, its small but there. Mostly because fneg becomes a side effect for fp arithmetic ops for us. I would request a target info guard which we can turn off to avoid the drift. If not, this is small enough to live with.

This is target-independent instcombine, so I don't think we want to add hooks at this level. If you see regressions transforming in this direction, do you also see regressions from the current code (ie, does disabling all fneg transforms make things even better)? That would suggest that we remove these transforms from instcombine altogether and create a dedicated target-dependent IR pass for fneg folds that behaves more like what we have in DAGCombiner.

[mcberg2017]

Turning off inst combining fneg opts does not make things better for us however. It seems to consistently cause additional regressions. We can live with the new change then.

[cameron.mcinally]

2. The reassociation tests show that we were missing an optimization opportunity to fold away fneg-of-fneg. My reading of IEEE-754 says that all of these transforms are allowed (regardless of binop/unop fneg version) because:
   
   "For all other operations [besides copy/abs/negate/copysign], this standard does not specify the sign bit of a NaN result."
   
   In all of these transforms, we always have some other binop (fadd/fsub/fmul/fdiv), so we are free to flip the sign bit of a potential intermediate NaN operand. (If that interpretation is wrong, then we must already have a bug in the existing transforms?)

Oh, that's an interesting thought...

There's one edge-case I've been playing with that is worth mentioning. This transform could cause problems with denormals and Flush-To-Zero (FTZ). For example:

-0.0 - D           -->  0.0
negate(D)          -->  -D

AFAICT, FTZ breaks IEEE-754 compatibility already, so all bets are off at that point. My certainty-level is low about this though. I still need to work through it.

[cameron.mcinally]

Saw that this was Accepted while I was writing my last comment...

I don't think my comment should hold up this Diff. I'm just thinking aloud and it's something that could be easily fixed later.

[mcberg2017]

How about we add a TODO comment regarding Cameron's topic? Would that be sufficient?

[cameron.mcinally]

How about we add a TODO comment regarding Cameron's topic? Would that be sufficient?

To be fair, this is a larger open question. There are a number of transforms that may need to be addressed in the future from it.

It's probably fine to submit without a comment.

[spatel]

In D65399#1604944, @cameron.mcinally wrote:

How about we add a TODO comment regarding Cameron's topic? Would that be sufficient?

To be fair, this is a larger open question. There are a number of transforms that may need to be addressed in the future from it.

It's probably fine to submit without a comment.

Thanks for the quick reviews! I agree that there are potential pitfalls in instcombine for targets with FTZ/DAZ. If you grep around in here, you can find some transforms that are partially limited to avoid that, but I can't see how those limitations work in the general case where we don't know some value is a denorm at compile-time. I suspect we need to expand our fast-math-flags and/or use of function attributes to account for targets that are not IEEE-754 compliant even though LLVM IR assumes that model by default.

cfe/trunk/test/CodeGen/complex-math.c

float _Complex div_float_rc(float a, float _Complex b) {
   // BD = 0
   // ACpBD = AC
   //
   // AARCH64-FASTMATH: [[CC:%.*]] = fmul fast float [[C]], [[C]]
   // AARCH64-FASTMATH: [[DD:%.*]] = fmul fast float [[D]], [[D]]
   // AARCH64-FASTMATH: [[CCpDD:%.*]] = fadd fast float [[CC]], [[DD]]
   //
   // BC = 0
-  // AARCH64-FASTMATH: [[AD:%.*]] = fmul fast float [[D]], %a
-  // AARCH64-FASTMATH: [[BCmAD:%.*]] = fdiv fast float [[AC]], [[CCpDD]]
-  // AARCH64-FASTMATH: [[DIV:%.*]] = fdiv fast float [[AD]], [[CCpDD]]
-  // AARCH64-FASTMATH: fsub fast float -0.000000e+00, [[DIV]]
+  // AARCH64-FASTMATH: [[NEGA:%.*]] = fsub fast float -0.000000e+00, %a
+  // AARCH64-FASTMATH: [[AD:%.*]] = fmul fast float  [[D]], [[NEGA]]
+  //
+  // AARCH64-FASTMATH: fdiv fast float [[AC]], [[CCpDD]]
+  // AARCH64-FASTMATH: fdiv fast float [[AD]], [[CCpDD]]
   // AARCH64-FASTMATH: ret
   return a / b;
 }
 float _Complex div_float_cc(float _Complex a, float _Complex b) {
   // X86-LABEL: @div_float_cc(
   // X86-NOT: fdiv
   // X86: call {{.*}} @__divsc3(
   // X86: ret
double _Complex div_double_rc(double a, double _Complex b) {
   // BD = 0
   // ACpBD = AC
   //
   // AARCH64-FASTMATH: [[CC:%.*]] = fmul fast double [[C]], [[C]]
   // AARCH64-FASTMATH: [[DD:%.*]] = fmul fast double [[D]], [[D]]
   // AARCH64-FASTMATH: [[CCpDD:%.*]] = fadd fast double [[CC]], [[DD]]
   //
   // BC = 0
-  // AARCH64-FASTMATH: [[AD:%.*]] = fmul fast double [[D]], %a
-  // AARCH64-FASTMATH: [[BCmAD:%.*]] = fdiv fast double [[AC]], [[CCpDD]]
-  // AARCH64-FASTMATH: [[DIV:%.*]] = fdiv fast double [[AD]], [[CCpDD]]
-  // AARCH64-FASTMATH: fsub fast double -0.000000e+00, [[DIV]]
+  // AARCH64-FASTMATH: [[NEGA:%.*]] = fsub fast double -0.000000e+00, %a
+  // AARCH64-FASTMATH: [[AD:%.*]] = fmul fast double [[D]], [[NEGA]]
+  //
+  // AARCH64-FASTMATH: fdiv fast double [[AC]], [[CCpDD]]
+  // AARCH64-FASTMATH: fdiv fast double [[AD]], [[CCpDD]]
   // AARCH64-FASTMATH: ret
   return a / b;
 }
 double _Complex div_double_cc(double _Complex a, double _Complex b) {
   // X86-LABEL: @div_double_cc(
   // X86-NOT: fdiv
   // X86: call {{.*}} @__divdc3(
   // X86: ret
long double _Complex div_long_double_rc(long double a, long double _Complex b) {
   // BD = 0
   // ACpBD = AC
   //
   // AARCH64-FASTMATH: [[CC:%.*]] = fmul fast fp128 [[C]], [[C]]
   // AARCH64-FASTMATH: [[DD:%.*]] = fmul fast fp128 [[D]], [[D]]
   // AARCH64-FASTMATH: [[CCpDD:%.*]] = fadd fast fp128 [[CC]], [[DD]]
   //
   // BC = 0
-  // AARCH64-FASTMATH: [[AD:%.*]] = fmul fast fp128 [[D]], %a
-  // AARCH64-FASTMATH: [[BCmAD:%.*]] = fdiv fast fp128 [[AC]], [[CCpDD]]
-  // AARCH64-FASTMATH: [[DIV:%.*]] = fdiv fast fp128 [[AD]], [[CCpDD]]
-  // AARCH64-FASTMATH: fsub fast fp128 0xL00000000000000008000000000000000, [[DIV]]
+  // AARCH64-FASTMATH: [[NEGA:%.*]] = fsub fast fp128 0xL00000000000000008000000000000000, %a
+  // AARCH64-FASTMATH: [[AD:%.*]] = fmul fast fp128 [[D]], [[NEGA]]
+  //
+  // AARCH64-FASTMATH: fdiv fast fp128 [[AC]], [[CCpDD]]
+  // AARCH64-FASTMATH: fdiv fast fp128 [[AD]], [[CCpDD]]
   // AARCH64-FASTMATH: ret
   return a / b;
 }
 long double _Complex div_long_double_cc(long double _Complex a, long double _Complex b) {
   // X86-LABEL: @div_long_double_cc(
   // X86-NOT: fdiv
   // X86: call {{.*}} @__divxc3(
   // X86: ret

llvm/trunk/lib/Transforms/InstCombine/InstCombineAddSub.cpp

if (match(&I, m_FNeg(m_OneUse(m_FDiv(m_Value(X), m_Constant(C))))))
     return BinaryOperator::CreateFDivFMF(X, ConstantExpr::getFNeg(C), &I);
   // -(C / X) --> (-C) / X
   if (match(&I, m_FNeg(m_OneUse(m_FDiv(m_Constant(C), m_Value(X))))))
     return BinaryOperator::CreateFDivFMF(ConstantExpr::getFNeg(C), X, &I);
 
   return nullptr;
 }
 
+static Instruction *hoistFNegAboveFMulFDiv(Instruction &I,
+                                           InstCombiner::BuilderTy &Builder) {
+  Value *FNeg;
+  if (!match(&I, m_FNeg(m_Value(FNeg))))
+    return nullptr;
+
+  Value *X, *Y;
+  if (match(FNeg, m_OneUse(m_FMul(m_Value(X), m_Value(Y)))))
+    return BinaryOperator::CreateFMulFMF(Builder.CreateFNegFMF(X, &I), Y, &I);
+
+  if (match(FNeg, m_OneUse(m_FDiv(m_Value(X), m_Value(Y)))))
+    return BinaryOperator::CreateFDivFMF(Builder.CreateFNegFMF(X, &I), Y, &I);
+
+  return nullptr;
+}
+
 Instruction *InstCombiner::visitFNeg(UnaryOperator &I) {
   Value *Op = I.getOperand(0);
 
   if (Value *V = SimplifyFNegInst(Op, I.getFastMathFlags(),
                                   SQ.getWithInstruction(&I)))
     return replaceInstUsesWith(I, V);
 
   if (Instruction *X = foldFNegIntoConstant(I))
     return X;
 
   Value *X, *Y;
 
   // If we can ignore the sign of zeros: -(X - Y) --> (Y - X)
   if (I.hasNoSignedZeros() &&
       match(Op, m_OneUse(m_FSub(m_Value(X), m_Value(Y)))))
     return BinaryOperator::CreateFSubFMF(Y, X, &I);
 
+  if (Instruction *R = hoistFNegAboveFMulFDiv(I, Builder))
+    return R;
+
   return nullptr;
 }
 
 Instruction *InstCombiner::visitFSub(BinaryOperator &I) {
   if (Value *V = SimplifyFSubInst(I.getOperand(0), I.getOperand(1),
                                   I.getFastMathFlags(),
                                   SQ.getWithInstruction(&I)))
     return replaceInstUsesWith(I, V);
 
   if (Instruction *X = foldVectorBinop(I))
     return X;
 
   // Subtraction from -0.0 is the canonical form of fneg.
   // fsub nsz 0, X ==> fsub nsz -0.0, X
   Value *Op0 = I.getOperand(0), *Op1 = I.getOperand(1);
   if (I.hasNoSignedZeros() && match(Op0, m_PosZeroFP()))
     return BinaryOperator::CreateFNegFMF(Op1, &I);
 
   if (Instruction *X = foldFNegIntoConstant(I))
     return X;
 
+  if (Instruction *R = hoistFNegAboveFMulFDiv(I, Builder))
+    return R;
+
   Value *X, *Y;
   Constant *C;
 
   // If Op0 is not -0.0 or we can ignore -0.0: Z - (X - Y) --> Z + (Y - X)
   // Canonicalize to fadd to make analysis easier.
   // This can also help codegen because fadd is commutative.
   // Note that if this fsub was really an fneg, the fadd with -0.0 will get
   // killed later. We still limit that particular transform with 'hasOneUse'

llvm/trunk/lib/Transforms/InstCombine/InstCombineMulDivRem.cpp

Instruction *InstCombiner::visitFMul(BinaryOperator &I) {
   if (match(Op0, m_FNeg(m_Value(X))) && match(Op1, m_FNeg(m_Value(Y))))
     return BinaryOperator::CreateFMulFMF(X, Y, &I);
 
   // -X * C --> X * -C
   Constant *C;
   if (match(Op0, m_FNeg(m_Value(X))) && match(Op1, m_Constant(C)))
     return BinaryOperator::CreateFMulFMF(X, ConstantExpr::getFNeg(C), &I);
 
-  // Sink negation: -X * Y --> -(X * Y)
-  // But don't transform constant expressions because there's an inverse fold.
-  if (match(Op0, m_OneUse(m_FNeg(m_Value(X)))) && !isa<ConstantExpr>(Op0))
-    return BinaryOperator::CreateFNegFMF(Builder.CreateFMulFMF(X, Op1, &I), &I);
-
-  // Sink negation: Y * -X --> -(X * Y)
-  // But don't transform constant expressions because there's an inverse fold.
-  if (match(Op1, m_OneUse(m_FNeg(m_Value(X)))) && !isa<ConstantExpr>(Op1))
-    return BinaryOperator::CreateFNegFMF(Builder.CreateFMulFMF(X, Op0, &I), &I);
-
   // fabs(X) * fabs(X) -> X * X
   if (Op0 == Op1 && match(Op0, m_Intrinsic<Intrinsic::fabs>(m_Value(X))))
     return BinaryOperator::CreateFMulFMF(X, X, &I);
 
   // (select A, B, C) * (select A, D, E) --> select A, (B*D), (C*E)
   if (Value *V = SimplifySelectsFeedingBinaryOp(I, Op0, Op1))
     return replaceInstUsesWith(I, V);
 
Instruction *InstCombiner::visitFDiv(BinaryOperator &I) {
   // -X / -Y -> X / Y
   Value *X, *Y;
   if (match(Op0, m_FNeg(m_Value(X))) && match(Op1, m_FNeg(m_Value(Y)))) {
     I.setOperand(0, X);
     I.setOperand(1, Y);
     return &I;
   }
 
-  // Sink negation: -X / Y --> -(X / Y)
-  // But don't transform constant expressions because there's an inverse fold.
-  if (match(Op0, m_OneUse(m_FNeg(m_Value(X)))) && !isa<ConstantExpr>(Op0))
-    return BinaryOperator::CreateFNegFMF(Builder.CreateFDivFMF(X, Op1, &I), &I);
-
-  // Sink negation: Y / -X --> -(Y / X)
-  // But don't transform constant expressions because there's an inverse fold.
-  if (match(Op1, m_OneUse(m_FNeg(m_Value(X)))) && !isa<ConstantExpr>(Op1))
-    return BinaryOperator::CreateFNegFMF(Builder.CreateFDivFMF(Op0, X, &I), &I);
-
   // X / (X * Y) --> 1.0 / Y
   // Reassociate to (X / X -> 1.0) is legal when NaNs are not allowed.
   // We can ignore the possibility that X is infinity because INF/INF is NaN.
   if (I.hasNoNaNs() && I.hasAllowReassoc() &&
       match(Op1, m_c_FMul(m_Specific(Op0), m_Value(Y)))) {
     I.setOperand(0, ConstantFP::get(I.getType(), 1.0));
     I.setOperand(1, Y);
     return &I;

llvm/trunk/test/Transforms/InstCombine/fadd.ll

;
   %y = frem double 41.0, %py ; thwart complexity-based canonicalization
   %z = frem double 42.0, %pz ; thwart complexity-based canonicalization
   %neg = fsub double -0.000000e+00, %x
   %mul = fmul double %y, %neg
   %r = fadd double %mul, %z
   ret double %r
 }
 
-; Z + (-X / Y) --> Z - (X / Y)
+; Z + (-X / Y) - extra use means we can't transform to fsub without an extra instruction
 
 define float @fdiv_fneg1_extra_use(float %x, float %y, float %pz) {
 ; CHECK-LABEL: @fdiv_fneg1_extra_use(
 ; CHECK-NEXT:    [[Z:%.*]] = frem float 4.200000e+01, [[PZ:%.*]]
-; CHECK-NEXT:    [[TMP1:%.*]] = fdiv float [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[DIV:%.*]] = fsub float -0.000000e+00, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub float -0.000000e+00, [[X:%.*]]
+; CHECK-NEXT:    [[DIV:%.*]] = fdiv float [[NEG]], [[Y:%.*]]
 ; CHECK-NEXT:    call void @use(float [[DIV]])
-; CHECK-NEXT:    [[R:%.*]] = fsub float [[Z]], [[TMP1]]
+; CHECK-NEXT:    [[R:%.*]] = fadd float [[Z]], [[DIV]]
 ; CHECK-NEXT:    ret float [[R]]
 ;
   %z = frem float 42.0, %pz ; thwart complexity-based canonicalization
   %neg = fsub float -0.000000e+00, %x
   %div = fdiv float %neg, %y
   call void @use(float %div)
   %r = fadd float %z, %div
   ret float %r
 }
 
-; Z + (Y / -X) --> Z - (Y / X)
+; Z + (Y / -X) - extra use means we can't transform to fsub without an extra instruction
 
 define float @fdiv_fneg2_extra_use(float %x, float %py, float %pz) {
 ; CHECK-LABEL: @fdiv_fneg2_extra_use(
 ; CHECK-NEXT:    [[Y:%.*]] = frem float -4.200000e+01, [[PY:%.*]]
 ; CHECK-NEXT:    [[Z:%.*]] = frem float 4.200000e+01, [[PZ:%.*]]
-; CHECK-NEXT:    [[TMP1:%.*]] = fdiv float [[Y]], [[X:%.*]]
-; CHECK-NEXT:    [[DIV:%.*]] = fsub float -0.000000e+00, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub float -0.000000e+00, [[X:%.*]]
+; CHECK-NEXT:    [[DIV:%.*]] = fdiv float [[Y]], [[NEG]]
 ; CHECK-NEXT:    call void @use(float [[DIV]])
-; CHECK-NEXT:    [[R:%.*]] = fsub float [[Z]], [[TMP1]]
+; CHECK-NEXT:    [[R:%.*]] = fadd float [[Z]], [[DIV]]
 ; CHECK-NEXT:    ret float [[R]]
 ;
   %y = frem float -42.0, %py ; thwart complexity-based canonicalization
   %z = frem float 42.0, %pz ; thwart complexity-based canonicalization
   %neg = fsub float -0.000000e+00, %x
   %div = fdiv float %y, %neg
   call void @use(float %div)
   %r = fadd float %z, %div
   ret float %r
 }
 
-; Z + (-X * Y) --> Z - (X * Y)
+; Z + (-X * Y) - extra use means we can't transform to fsub without an extra instruction
 
 define <2 x float> @fmul_fneg1_extra_use(<2 x float> %x, <2 x float> %y, <2 x float> %pz) {
 ; CHECK-LABEL: @fmul_fneg1_extra_use(
 ; CHECK-NEXT:    [[Z:%.*]] = frem <2 x float> <float 4.200000e+01, float -1.000000e+00>, [[PZ:%.*]]
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul <2 x float> [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul <2 x float> [[NEG]], [[Y:%.*]]
 ; CHECK-NEXT:    call void @use_vec(<2 x float> [[MUL]])
-; CHECK-NEXT:    [[R:%.*]] = fsub <2 x float> [[Z]], [[TMP1]]
+; CHECK-NEXT:    [[R:%.*]] = fadd <2 x float> [[Z]], [[MUL]]
 ; CHECK-NEXT:    ret <2 x float> [[R]]
 ;
   %z = frem <2 x float> <float 42.0, float -1.0>, %pz ; thwart complexity-based canonicalization
   %neg = fsub <2 x float> <float -0.0, float -0.0>, %x
   %mul = fmul <2 x float> %neg, %y
   call void @use_vec(<2 x float> %mul)
   %r = fadd <2 x float> %z, %mul
   ret <2 x float> %r
 }
 
-; Z + (Y * -X) --> Z - (Y * X)
+; Z + (Y * -X) - extra use means we can't transform to fsub without an extra instruction
 
 define float @fmul_fneg2_extra_use(float %x, float %py, float %pz) {
 ; CHECK-LABEL: @fmul_fneg2_extra_use(
 ; CHECK-NEXT:    [[Y:%.*]] = frem float -4.200000e+01, [[PY:%.*]]
 ; CHECK-NEXT:    [[Z:%.*]] = frem float 4.200000e+01, [[PZ:%.*]]
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul float [[Y]], [[X:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub float -0.000000e+00, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub float -0.000000e+00, [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul float [[Y]], [[NEG]]
 ; CHECK-NEXT:    call void @use(float [[MUL]])
-; CHECK-NEXT:    [[R:%.*]] = fsub float [[Z]], [[TMP1]]
+; CHECK-NEXT:    [[R:%.*]] = fadd float [[Z]], [[MUL]]
 ; CHECK-NEXT:    ret float [[R]]
 ;
   %y = frem float -42.0, %py ; thwart complexity-based canonicalization
   %z = frem float 42.0, %pz ; thwart complexity-based canonicalization
   %neg = fsub float -0.000000e+00, %x
   %mul = fmul float %y, %neg
   call void @use(float %mul)
   %r = fadd float %z, %mul

llvm/trunk/test/Transforms/InstCombine/fdiv.ll

 ;
   %t1 = fdiv <2 x float> <float 3.0e0, float 7.0e0>, %x
   %t2 = fdiv arcp reassoc <2 x float> <float 15.0e0, float -7.0e0>, %t1
   ret <2 x float> %t2
 }
 
 define double @fdiv_fneg1(double %x, double %y) {
 ; CHECK-LABEL: @fdiv_fneg1(
-; CHECK-NEXT:    [[TMP1:%.*]] = fdiv double [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[DIV:%.*]] = fsub double -0.000000e+00, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub double -0.000000e+00, [[X:%.*]]
+; CHECK-NEXT:    [[DIV:%.*]] = fdiv double [[NEG]], [[Y:%.*]]
 ; CHECK-NEXT:    ret double [[DIV]]
 ;
   %neg = fsub double -0.0, %x
   %div = fdiv double %neg, %y
   ret double %div
 }
 
 define double @fdiv_unary_fneg1(double %x, double %y) {
 ; CHECK-LABEL: @fdiv_unary_fneg1(
-; CHECK-NEXT:    [[TMP1:%.*]] = fdiv double [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[DIV:%.*]] = fsub double -0.000000e+00, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fneg double [[X:%.*]]
+; CHECK-NEXT:    [[DIV:%.*]] = fdiv double [[NEG]], [[Y:%.*]]
 ; CHECK-NEXT:    ret double [[DIV]]
 ;
   %neg = fneg double %x
   %div = fdiv double %neg, %y
   ret double %div
 }
 
 define <2 x float> @fdiv_fneg2(<2 x float> %x, <2 x float> %y) {
 ; CHECK-LABEL: @fdiv_fneg2(
-; CHECK-NEXT:    [[TMP1:%.*]] = fdiv <2 x float> [[Y:%.*]], [[X:%.*]]
-; CHECK-NEXT:    [[DIV:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[DIV:%.*]] = fdiv <2 x float> [[Y:%.*]], [[NEG]]
 ; CHECK-NEXT:    ret <2 x float> [[DIV]]
 ;
   %neg = fsub <2 x float> <float -0.0, float -0.0>, %x
   %div = fdiv <2 x float> %y, %neg
   ret <2 x float> %div
 }
 
 define <2 x float> @fdiv_unary_fneg2(<2 x float> %x, <2 x float> %y) {
 ; CHECK-LABEL: @fdiv_unary_fneg2(
-; CHECK-NEXT:    [[TMP1:%.*]] = fdiv <2 x float> [[Y:%.*]], [[X:%.*]]
-; CHECK-NEXT:    [[DIV:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fneg <2 x float> [[X:%.*]]
+; CHECK-NEXT:    [[DIV:%.*]] = fdiv <2 x float> [[Y:%.*]], [[NEG]]
 ; CHECK-NEXT:    ret <2 x float> [[DIV]]
 ;
   %neg = fneg <2 x float> %x
   %div = fdiv <2 x float> %y, %neg
   ret <2 x float> %div
 }
 
 define float @fdiv_fneg1_extra_use(float %x, float %y) {

llvm/trunk/test/Transforms/InstCombine/fmul.ll

;
   %nx = fsub float -0.0, %x
   %ny = fneg float %y
   %mul = fmul afn float %nx, %ny
   call void @use_f32(float %nx)
   call void @use_f32(float %ny)
   ret float %mul
 }
 
-; (-0.0 - X) * Y => -0.0 - (X * Y)
-define float @neg_sink(float %x, float %y) {
-; CHECK-LABEL: @neg_sink(
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul float [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub float -0.000000e+00, [[TMP1]]
+; (-0.0 - X) * Y
+define float @neg_mul(float %x, float %y) {
+; CHECK-LABEL: @neg_mul(
+; CHECK-NEXT:    [[SUB:%.*]] = fsub float -0.000000e+00, [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul float [[SUB]], [[Y:%.*]]
 ; CHECK-NEXT:    ret float [[MUL]]
 ;
   %sub = fsub float -0.0, %x
   %mul = fmul float %sub, %y
   ret float %mul
 }
 
-define float @unary_neg_sink(float %x, float %y) {
-; CHECK-LABEL: @unary_neg_sink(
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul float [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub float -0.000000e+00, [[TMP1]]
+define float @unary_neg_mul(float %x, float %y) {
+; CHECK-LABEL: @unary_neg_mul(
+; CHECK-NEXT:    [[NEG:%.*]] = fneg float [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul float [[NEG]], [[Y:%.*]]
 ; CHECK-NEXT:    ret float [[MUL]]
 ;
   %neg = fneg float %x
   %mul = fmul float %neg, %y
   ret float %mul
 }
 
-define <2 x float> @neg_sink_vec(<2 x float> %x, <2 x float> %y) {
-; CHECK-LABEL: @neg_sink_vec(
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul <2 x float> [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[TMP1]]
+define <2 x float> @neg_mul_vec(<2 x float> %x, <2 x float> %y) {
+; CHECK-LABEL: @neg_mul_vec(
+; CHECK-NEXT:    [[SUB:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul <2 x float> [[SUB]], [[Y:%.*]]
 ; CHECK-NEXT:    ret <2 x float> [[MUL]]
 ;
   %sub = fsub <2 x float> <float -0.0, float -0.0>, %x
   %mul = fmul <2 x float> %sub, %y
   ret <2 x float> %mul
 }
 
-; FIXME: Should generate a unary FNeg.
-define <2 x float> @unary_neg_sink_vec(<2 x float> %x, <2 x float> %y) {
-; CHECK-LABEL: @unary_neg_sink_vec(
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul <2 x float> [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[TMP1]]
+define <2 x float> @unary_neg_mul_vec(<2 x float> %x, <2 x float> %y) {
+; CHECK-LABEL: @unary_neg_mul_vec(
+; CHECK-NEXT:    [[SUB:%.*]] = fneg <2 x float> [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul <2 x float> [[SUB]], [[Y:%.*]]
 ; CHECK-NEXT:    ret <2 x float> [[MUL]]
 ;
   %sub = fneg <2 x float> %x
   %mul = fmul <2 x float> %sub, %y
   ret <2 x float> %mul
 }
 
-define <2 x float> @neg_sink_vec_undef(<2 x float> %x, <2 x float> %y) {
-; CHECK-LABEL: @neg_sink_vec_undef(
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul <2 x float> [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[TMP1]]
+define <2 x float> @neg_mul_vec_undef(<2 x float> %x, <2 x float> %y) {
+; CHECK-LABEL: @neg_mul_vec_undef(
+; CHECK-NEXT:    [[SUB:%.*]] = fsub <2 x float> <float undef, float -0.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul <2 x float> [[SUB]], [[Y:%.*]]
 ; CHECK-NEXT:    ret <2 x float> [[MUL]]
 ;
   %sub = fsub <2 x float> <float undef, float -0.0>, %x
   %mul = fmul <2 x float> %sub, %y
   ret <2 x float> %mul
 }
 
-; (0.0 - X) * Y => 0.0 - (X * Y)
+; (0.0 - X) * Y
 define float @neg_sink_nsz(float %x, float %y) {
 ; CHECK-LABEL: @neg_sink_nsz(
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul float [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub float -0.000000e+00, [[TMP1]]
+; CHECK-NEXT:    [[SUB1:%.*]] = fsub nsz float -0.000000e+00, [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul float [[SUB1]], [[Y:%.*]]
 ; CHECK-NEXT:    ret float [[MUL]]
 ;
   %sub1 = fsub nsz float 0.0, %x
   %mul = fmul float %sub1, %y
   ret float %mul
 }
 
-; "(-0.0 - X) * Y => -0.0 - (X * Y)" is disabled if expression "-0.0 - X"
-; has multiple uses.
 define float @neg_sink_multi_use(float %x, float %y) {
 ; CHECK-LABEL: @neg_sink_multi_use(
 ; CHECK-NEXT:    [[SUB1:%.*]] = fsub float -0.000000e+00, [[X:%.*]]
 ; CHECK-NEXT:    [[MUL:%.*]] = fmul float [[SUB1]], [[Y:%.*]]
 ; CHECK-NEXT:    [[MUL2:%.*]] = fmul float [[MUL]], [[SUB1]]
 ; CHECK-NEXT:    ret float [[MUL2]]
 ;
   %sub1 = fsub float -0.0, %x
   %mul = fmul float %sub1, %y
   %mul2 = fmul float %mul, %sub1
   ret float %mul2
 }
 
-define float @unary_neg_sink_multi_use(float %x, float %y) {
-; CHECK-LABEL: @unary_neg_sink_multi_use(
+define float @unary_neg_mul_multi_use(float %x, float %y) {
+; CHECK-LABEL: @unary_neg_mul_multi_use(
 ; CHECK-NEXT:    [[SUB1:%.*]] = fneg float [[X:%.*]]
 ; CHECK-NEXT:    [[MUL:%.*]] = fmul float [[SUB1]], [[Y:%.*]]
 ; CHECK-NEXT:    [[MUL2:%.*]] = fmul float [[MUL]], [[SUB1]]
 ; CHECK-NEXT:    ret float [[MUL2]]
 ;
   %sub1 = fneg float %x
   %mul = fmul float %sub1, %y
   %mul2 = fmul float %mul, %sub1

llvm/trunk/test/Transforms/InstCombine/fsub.ll

;
   %neg = fsub double -0.000000e+00, %x
   %mul = fmul double %y, %neg
   %r = fsub double %z, %mul
   ret double %r
 }
 
 define float @fsub_fdiv_fneg1_extra_use(float %x, float %y, float %z) {
 ; CHECK-LABEL: @fsub_fdiv_fneg1_extra_use(
-; CHECK-NEXT:    [[TMP1:%.*]] = fdiv float [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[DIV:%.*]] = fsub float -0.000000e+00, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub float -0.000000e+00, [[X:%.*]]
+; CHECK-NEXT:    [[DIV:%.*]] = fdiv float [[NEG]], [[Y:%.*]]
 ; CHECK-NEXT:    call void @use(float [[DIV]])
-; CHECK-NEXT:    [[R:%.*]] = fadd float [[TMP1]], [[Z:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = fsub float [[Z:%.*]], [[DIV]]
 ; CHECK-NEXT:    ret float [[R]]
 ;
   %neg = fsub float -0.000000e+00, %x
   %div = fdiv float %neg, %y
   call void @use(float %div)
   %r = fsub float %z, %div
   ret float %r
 }
 
 define float @fsub_fdiv_fneg2_extra_use(float %x, float %y, float %z) {
 ; CHECK-LABEL: @fsub_fdiv_fneg2_extra_use(
-; CHECK-NEXT:    [[TMP1:%.*]] = fdiv float [[Y:%.*]], [[X:%.*]]
-; CHECK-NEXT:    [[DIV:%.*]] = fsub float -0.000000e+00, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub float -0.000000e+00, [[X:%.*]]
+; CHECK-NEXT:    [[DIV:%.*]] = fdiv float [[Y:%.*]], [[NEG]]
 ; CHECK-NEXT:    call void @use(float [[DIV]])
-; CHECK-NEXT:    [[R:%.*]] = fadd float [[TMP1]], [[Z:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = fsub float [[Z:%.*]], [[DIV]]
 ; CHECK-NEXT:    ret float [[R]]
 ;
   %neg = fsub float -0.000000e+00, %x
   %div = fdiv float %y, %neg
   call void @use(float %div)
   %r = fsub float %z, %div
   ret float %r
 }
 
 declare void @use_vec(<2 x float>)
 
 define <2 x float> @fsub_fmul_fneg1_extra_use(<2 x float> %x, <2 x float> %y, <2 x float> %z) {
 ; CHECK-LABEL: @fsub_fmul_fneg1_extra_use(
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul <2 x float> [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub <2 x float> <float -0.000000e+00, float -0.000000e+00>, [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul <2 x float> [[NEG]], [[Y:%.*]]
 ; CHECK-NEXT:    call void @use_vec(<2 x float> [[MUL]])
-; CHECK-NEXT:    [[R:%.*]] = fadd <2 x float> [[TMP1]], [[Z:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = fsub <2 x float> [[Z:%.*]], [[MUL]]
 ; CHECK-NEXT:    ret <2 x float> [[R]]
 ;
   %neg = fsub <2 x float> <float -0.0, float -0.0>, %x
   %mul = fmul <2 x float> %neg, %y
   call void @use_vec(<2 x float> %mul)
   %r = fsub <2 x float> %z, %mul
   ret <2 x float> %r
 }
 
 define float @fsub_fmul_fneg2_extra_use(float %x, float %y, float %z) {
 ; CHECK-LABEL: @fsub_fmul_fneg2_extra_use(
-; CHECK-NEXT:    [[TMP1:%.*]] = fmul float [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[MUL:%.*]] = fsub float -0.000000e+00, [[TMP1]]
+; CHECK-NEXT:    [[NEG:%.*]] = fsub float -0.000000e+00, [[X:%.*]]
+; CHECK-NEXT:    [[MUL:%.*]] = fmul float [[NEG]], [[Y:%.*]]
 ; CHECK-NEXT:    call void @use(float [[MUL]])
-; CHECK-NEXT:    [[R:%.*]] = fadd float [[TMP1]], [[Z:%.*]]
+; CHECK-NEXT:    [[R:%.*]] = fsub float [[Z:%.*]], [[MUL]]
 ; CHECK-NEXT:    ret float [[R]]
 ;
   %neg = fsub float -0.000000e+00, %x
   %mul = fmul float %y, %neg
   call void @use(float %mul)
   %r = fsub float %z, %mul
   ret float %r
 }

llvm/trunk/test/Transforms/Reassociate/fast-basictest.ll

 ;
   %d = fmul reassoc float %z, 4.000000e+01
   %c = fsub reassoc float 0.000000e+00, %d
   %e = fmul reassoc float %a, %c
   %f = fsub reassoc float 0.000000e+00, %e
   ret float %f
 }
 
-; It is not safe to reassociate unary fneg without nnan.
+; fneg of fneg is an identity operation, so no FMF are needed to remove those instructions.
+
+define float @test18_unary_fneg_no_FMF(float %a, float %b, float %z) {
+; CHECK-LABEL: @test18_unary_fneg_no_FMF(
+; CHECK-NEXT:    [[TMP1:%.*]] = fmul float [[Z:%.*]], 4.000000e+01
+; CHECK-NEXT:    [[F:%.*]] = fmul float [[TMP1]], [[A:%.*]]
+; CHECK-NEXT:    ret float [[F]]
+;
+  %d = fmul float %z, 4.000000e+01
+  %c = fneg float %d
+  %e = fmul float %a, %c
+  %f = fneg float %e
+  ret float %f
+}
+
 define float @test18_reassoc_unary_fneg(float %a, float %b, float %z) {
 ; CHECK-LABEL: @test18_reassoc_unary_fneg(
-; CHECK-NEXT:    [[C:%.*]] = fmul reassoc float [[Z:%.*]], -4.000000e+01
-; CHECK-NEXT:    [[E:%.*]] = fmul reassoc float [[C]], [[A:%.*]]
-; CHECK-NEXT:    [[F:%.*]] = fneg reassoc float [[E]]
+; CHECK-NEXT:    [[TMP1:%.*]] = fmul reassoc float [[Z:%.*]], 4.000000e+01
+; CHECK-NEXT:    [[F:%.*]] = fmul reassoc float [[TMP1]], [[A:%.*]]
 ; CHECK-NEXT:    ret float [[F]]
 ;
   %d = fmul reassoc float %z, 4.000000e+01
   %c = fneg reassoc float %d
   %e = fmul reassoc float %a, %c
   %f = fneg reassoc float %e
   ret float %f
 }