[InstCombine] fma x, y, 0 -> fmul x, y

[dmgreen]

Summary

If the addend of the fma is zero, common sense would suggest that we can convert fma x, y, 0.0 to fmul x, y. This comes up with some user code that was expecting the first fma in an unrolled loop to simplify to a fmul.

Floating point often does not follow naive common sense though. Alive suggests that this should be guarded by nsz (as fadd -0.0, 0.0 = 0.0). However it also did not complete running, so I have only validated this against fadd nsz (fmul(x,y), 0) -> fmul nsz (x,y), not with fma nsz (x, y, 0.0) -> fmul nsz (x, y).

[arsenm]

I don't think I see why this needs an nsz check (but think it could also fold -0 in the 3rd operand with one). The fma should only be able to produce a negative 0 if the 3rd operand is also a -0

fadd(+0, -0) -> +0
fadd(-0, -0) -> -0

fmul(-0, +x) -> -0
fmul(+0, -x) -> -0
fmul(-0, -x) -> 0

fma(+x, -0, 0) -> +0
fma(-x, +0, 0) -> +0
fma(-x, -0, 0) -> +0
fma(+x, -0, -0) -> -0

[dmgreen]

I think it's because of these ones:

fma(+0, -x, 0) -> +0

fmul(+0, -x) -> -0
fadd(+0, -0) -> +0

Removing the fadd would leave it at -0. This is what alive2 comes up with:

Processing fma.txt..                          
                                              
----------------------------------------      
  %ys = fma half %x, half %y, half 0.000000   
  ret half %ys                                
=>                                            
  %ys = fmul half %x, %y                      
  ret half %ys                                
                                              
ERROR: Value mismatch for half %ys            
                                              
Example:                                      
half %x = #xf461 (-17936)                     
half %y = #x0000 (+0.0)                       
Source value: #x0000 (+0.0)                   
Target value: #x8000 (-0.0)

[spatel]

In D82778#2120456, @dmgreen wrote:

I think it's because of these ones:

fma(+0, -x, 0) -> +0

fmul(+0, -x) -> -0
fadd(+0, -0) -> +0

Removing the fadd would leave it at -0.

Right - we need 'nsz' to fold adding +0.0 (I think this patch is correct as-is).

We can also always fold fma(x, y, -0.0) --> fmul x, y because (this is in InstSimplify):
fadd X, -0 ==> X

But I can't get the online version of Alive2 to confirm that (time-out). Local version should verify it?
If we don't have tests for the add of -0.0 variant, it would be good to add them for the likely follow-on patch.

llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp

2385–2387

Can reduce this to something like:

return BinaryOperator::CreateFMulFMF(Src0, Src1, II);

[dmgreen]

Thanks for the extra info. I ran things overnight and it said this for -0.0

Processing fma.txt..

----------------------------------------
  %ys = fma half %x, half %y, half -0.000000
  ret half %ys
=>
  %ys = fmul half %x, %y
  ret half %ys

Done: 1
Transformation seems to be correct!

real    136m22.357s
user    117m17.118s
sys     0m7.495s

Which looks good. I've extended the pattern to match and added some tests.

Unfortunately still this for +0.0:

Processing fma.txt..

----------------------------------------
  %ys = fma nsz half %x, half %y, half 0.000000
  ret half %ys
=>
  %ys = fmul half %x, %y
  ret half %ys

ERROR: SMT Error: smt tactic failed to show goal to be sat/unsat memout

real    152m39.828s
user    147m53.980s
sys     0m6.067s

But I'm fairly confident it's OK given nsz and the result for -0.0.

[spatel]

LGTM

llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp

case Intrinsic::fma: {
     if (Value *V = SimplifyFMAFMul(II->getArgOperand(0), II->getArgOperand(1),
                                    II->getFastMathFlags(),
                                    SQ.getWithInstruction(II))) {
       auto *FAdd = BinaryOperator::CreateFAdd(V, II->getArgOperand(2));
       FAdd->copyFastMathFlags(II);
       return FAdd;
     }
 
+    // fma x, y, 0 -> fmul x, y
+    // This is always valid for -0.0, but requires nsz for +0.0 as
+    // -0.0 + 0.0 = 0.0, which would not be the same as the fmul on its own.
+    if (match(II->getArgOperand(2), m_NegZeroFP()) ||
+        (match(II->getArgOperand(2), m_PosZeroFP()) &&
+         II->getFastMathFlags().noSignedZeros()))
+      return BinaryOperator::CreateFMulFMF(Src0, Src1, II);
+
     break;
   }
   case Intrinsic::copysign: {
     if (SignBitMustBeZero(II->getArgOperand(1), &TLI)) {
       // If we know that the sign argument is positive, reduce to FABS:
       // copysign X, Pos --> fabs X
       Value *Fabs = Builder.CreateUnaryIntrinsic(Intrinsic::fabs,
                                                  II->getArgOperand(0), II);

llvm/test/Transforms/InstCombine/fma.ll

 ; CHECK-NEXT:    ret float [[FMA]]
 ;
   %fma = call float @llvm.fma.f32(float %x, float %y, float 0.0)
   ret float %fma
 }
 
 define float @fma_x_y_0_nsz(float %x, float %y) {
 ; CHECK-LABEL: @fma_x_y_0_nsz(
-; CHECK-NEXT:    [[FMA:%.*]] = call nsz float @llvm.fma.f32(float [[X:%.*]], float [[Y:%.*]], float 0.000000e+00)
+; CHECK-NEXT:    [[FMA:%.*]] = fmul nsz float [[X:%.*]], [[Y:%.*]]
 ; CHECK-NEXT:    ret float [[FMA]]
 ;
   %fma = call nsz float @llvm.fma.f32(float %x, float %y, float 0.0)
   ret float %fma
 }
 
 define <8 x half> @fma_x_y_0_v(<8 x half> %x, <8 x half> %y) {
 ; CHECK-LABEL: @fma_x_y_0_v(
 ; CHECK-NEXT:    [[FMA:%.*]] = call <8 x half> @llvm.fma.v8f16(<8 x half> [[X:%.*]], <8 x half> [[Y:%.*]], <8 x half> zeroinitializer)
 ; CHECK-NEXT:    ret <8 x half> [[FMA]]
 ;
   %fma = call <8 x half> @llvm.fma.v8f16(<8 x half> %x, <8 x half> %y, <8 x half> zeroinitializer)
   ret <8 x half> %fma
 }
 
 define <8 x half> @fma_x_y_0_nsz_v(<8 x half> %x, <8 x half> %y) {
 ; CHECK-LABEL: @fma_x_y_0_nsz_v(
-; CHECK-NEXT:    [[FMA:%.*]] = call nsz <8 x half> @llvm.fma.v8f16(<8 x half> [[X:%.*]], <8 x half> [[Y:%.*]], <8 x half> zeroinitializer)
+; CHECK-NEXT:    [[FMA:%.*]] = fmul nsz <8 x half> [[X:%.*]], [[Y:%.*]]
 ; CHECK-NEXT:    ret <8 x half> [[FMA]]
 ;
   %fma = call nsz <8 x half> @llvm.fma.v8f16(<8 x half> %x, <8 x half> %y, <8 x half> zeroinitializer)
   ret <8 x half> %fma
 }
 
 define float @fmuladd_x_y_0(float %x, float %y) {
 ; CHECK-LABEL: @fmuladd_x_y_0(
 ; CHECK-NEXT:    [[FMA:%.*]] = call float @llvm.fmuladd.f32(float [[X:%.*]], float [[Y:%.*]], float 0.000000e+00)
 ; CHECK-NEXT:    ret float [[FMA]]
 ;
   %fma = call float @llvm.fmuladd.f32(float %x, float %y, float 0.0)
   ret float %fma
 }
 
 define float @fmuladd_x_y_0_nsz(float %x, float %y) {
 ; CHECK-LABEL: @fmuladd_x_y_0_nsz(
-; CHECK-NEXT:    [[FMA:%.*]] = call nsz float @llvm.fmuladd.f32(float [[X:%.*]], float [[Y:%.*]], float 0.000000e+00)
+; CHECK-NEXT:    [[FMA:%.*]] = fmul nsz float [[X:%.*]], [[Y:%.*]]
 ; CHECK-NEXT:    ret float [[FMA]]
 ;
   %fma = call nsz float @llvm.fmuladd.f32(float %x, float %y, float 0.0)
   ret float %fma
 }
 
 define float @fma_x_y_m0(float %x, float %y) {
 ; CHECK-LABEL: @fma_x_y_m0(
-; CHECK-NEXT:    [[FMA:%.*]] = call float @llvm.fma.f32(float [[X:%.*]], float [[Y:%.*]], float -0.000000e+00)
+; CHECK-NEXT:    [[FMA:%.*]] = fmul float [[X:%.*]], [[Y:%.*]]
 ; CHECK-NEXT:    ret float [[FMA]]
 ;
   %fma = call float @llvm.fma.f32(float %x, float %y, float -0.0)
   ret float %fma
 }
 
 define <8 x half> @fma_x_y_m0_v(<8 x half> %x, <8 x half> %y) {
 ; CHECK-LABEL: @fma_x_y_m0_v(
-; CHECK-NEXT:    [[FMA:%.*]] = call <8 x half> @llvm.fma.v8f16(<8 x half> [[X:%.*]], <8 x half> [[Y:%.*]], <8 x half> <half 0xH8000, half 0xH8000, half 0xH8000, half 0xH8000, half 0xH8000, half 0xH8000, half 0xH8000, half 0xH8000>)
+; CHECK-NEXT:    [[FMA:%.*]] = fmul <8 x half> [[X:%.*]], [[Y:%.*]]
 ; CHECK-NEXT:    ret <8 x half> [[FMA]]
 ;
   %fma = call <8 x half> @llvm.fma.v8f16(<8 x half> %x, <8 x half> %y, <8 x half> <half -0.0, half -0.0, half -0.0, half -0.0, half -0.0, half -0.0, half -0.0, half -0.0>)
   ret <8 x half> %fma
 }
 
 define float @fmuladd_x_y_m0(float %x, float %y) {
 ; CHECK-LABEL: @fmuladd_x_y_m0(
-; CHECK-NEXT:    [[FMA:%.*]] = call float @llvm.fmuladd.f32(float [[X:%.*]], float [[Y:%.*]], float -0.000000e+00)
+; CHECK-NEXT:    [[FMA:%.*]] = fmul float [[X:%.*]], [[Y:%.*]]
 ; CHECK-NEXT:    ret float [[FMA]]
 ;
   %fma = call float @llvm.fmuladd.f32(float %x, float %y, float -0.0)
   ret float %fma
 }
 
 define float @fmuladd_x_1_z_fast(float %x, float %z) {
 ; CHECK-LABEL: @fmuladd_x_1_z_fast(