Fix APFloat mod

[simonbyrne]

Summary

The existing code for APFloat mod was incorrect in two ways:

• intermediate rounding gave incorrect results in some cases (e.g. frem double 0.3 0.01)
• the method of integer truncation could cause intermediate overflow

This caused subsequent errors in the constant propagation pass, see https://llvm.org/bugs/show_bug.cgi?id=3316

[simonbyrne]

Use fusedMultiplyAdd.

[scanon]

Numerically this is sound, and something that I've wanted to fix for a while but haven't had time to deal with. Thanks for doing it! Please add some test cases that would detect the old error (@gottesmm can likely point you in the right direction).

[simonbyrne]

Thanks. I'm not really that familiar with the layout of the repository: where should tests go?

[Meinersbur]

In D29346#664495, @simonbyrne wrote:

Thanks. I'm not really that familiar with the layout of the repository: where should tests go?

/llvm/unittests/ADT/APFloatTest.cpp

[simonbyrne]

Changed to use subtraction to avoid intermediate overflow, added tests.

[scanon]

Great, thanks Simon! This looks fine numerically. @gottesmm, can you double-check for style, etc?

[gottesmm]

Did you run this through git-clang-format?

[simonbyrne]

Updated to use git clang-format

[gottesmm]

Thanks for running it through git-clang-format. LGTM!

[skatkov]

This patch causes incorrect functional behavior, specifically, IEEE specification requires the sign of the remainder is the same as numerator in case remainder is zero.
It is incorrect with this patch. The following tests can be used to show the issue:

{
  APFloat f1(APFloat::IEEEdouble(), "-4.0");
  APFloat f2(APFloat::IEEEdouble(), "-2.0");
  APFloat expected(APFloat::IEEEdouble(), "-0.0");
  EXPECT_EQ(f1.mod(f2), APFloat::opOK);
  EXPECT_TRUE(f1.bitwiseIsEqual(expected));
}
{
  APFloat f1(APFloat::IEEEdouble(), "-4.0");
  APFloat f2(APFloat::IEEEdouble(), "2.0");
  APFloat expected(APFloat::IEEEdouble(), "-0.0");
  EXPECT_EQ(f1.mod(f2), APFloat::opOK);
  EXPECT_TRUE(f1.bitwiseIsEqual(expected));
}

I plan to revert this patch and add mentioned tests...

[skatkov]

Actually, I've upload a fix, please review it.

lib/Support/APFloat.cpp

IEEEFloat::opStatus IEEEFloat::remainder(const IEEEFloat &rhs) {
   assert(fs==opOK || fs==opInexact);   // likewise
 
   if (isZero())
     sign = origSign;    // IEEE754 requires this
   delete[] x;
   return fs;
 }
 
-/* Normalized llvm frem (C fmod).
-   This is not currently correct in all cases.  */
+/* Normalized llvm frem (C fmod).  */
 IEEEFloat::opStatus IEEEFloat::mod(const IEEEFloat &rhs) {
   opStatus fs;
   fs = modSpecials(rhs);
 
   if (isFiniteNonZero() && rhs.isFiniteNonZero()) {
+    while (true) {
       IEEEFloat V = *this;
-    unsigned int origSign = sign;
 
-    fs = V.divide(rhs, rmNearestTiesToEven);
+      fs = V.divide(rhs, rmTowardZero);
       if (fs == opDivByZero)
         return fs;
+      V.roundToIntegral(rmTowardZero);
+      if (V.isZero())
+        break;
 
-    int parts = partCount();
-    integerPart *x = new integerPart[parts];
-    bool ignored;
-    fs = V.convertToInteger(x, parts * integerPartWidth, true,
-                            rmTowardZero, &ignored);
-    if (fs==opInvalidOp) {
-      delete[] x;
-      return fs;
+      /* *this = *this - V*rhs     */
+      V.sign = !V.sign;
+      lostFraction lost_fraction;
+      lost_fraction = V.multiplySignificand(rhs, this);
+      fs = normalize(rmTowardZero, lost_fraction);
+      assert(fs==opOK);   // should always be exact
+      *this = V;
     }
-
-    fs = V.convertFromZeroExtendedInteger(x, parts * integerPartWidth, true,
-                                          rmNearestTiesToEven);
-    assert(fs==opOK);   // should always work
-
-    fs = V.multiply(rhs, rmNearestTiesToEven);
-    assert(fs==opOK || fs==opInexact);   // should not overflow or underflow
-
-    fs = subtract(V, rmNearestTiesToEven);
-    assert(fs==opOK || fs==opInexact);   // likewise
-
-    if (isZero())
-      sign = origSign;    // IEEE754 requires this
-    delete[] x;
   }
   return fs;
 }
 
 /* Normalized fused-multiply-add.  */
 IEEEFloat::opStatus IEEEFloat::fusedMultiplyAdd(const IEEEFloat &multiplicand,
                                                 const IEEEFloat &addend,
                                                 roundingMode rounding_mode) {