[compiler-rt] Add AArch64 to CMake configuration and several missing builtins

[sdmitrouk]

Summary

Currently CMake doesn't build builtins for AArch64 and if one does this anyway
it's likely that at least __multc3, __floatditf and __floatunditf will be
missing. There is actually more builtins to add, but these come from
different libc implementations, thus providing them makes compiler-rt for
AArch64 good enough at least for basic usage.

Builtins implementation were originally taken from FreeBSD project:

• __multc3
• __floatditf and __floatunditf

Until they have been tested to find mistakes in __float* functions.
__floatditf was based on __floatsitf, which had the same mistakes
(fixed it in r243746).

Version of the builtins in this patch are fixed and complemented with basic
tests. Additionally they were tested via GCC's torture (this is what revealed
these issues).

P.S. Ed (author of FreeBSD patches) asked for feedback on the list some time ago (here here)
and got no response, but it seems to be worth adding these builtins as is and
extracting common part later.

[rengolin]

I have to say I didn't delve into the IEEE-ness of the algorithms, but I welcome some builtins in AArch64.

I'm adding some AArch64 folks to have a look at the code, but generally if Ed is happy, I'm happy. :)

Where have you tested this? Does "make check-all" pass when compiling RT with Clang? That's what the buildbot will do, so it needs to be green.

cheers,
--renato

[sdmitrouk]

Renato,

Where have you tested this?

On Qemu, don't have usable AArch64 board.

Does "make check-all" pass when compiling RT with Clang? That's what the buildbot will do, so it needs to be green.

I should get used to look up build instructions in www/, ran tests in a weird way. There are seven failures of existing builtins (by the way, these and one more of tests don't even compile out of the box due to missing include, which is obviously easy to fix):

compiler-rt :: builtins :: Unit/divxc3_test.c
compiler-rt :: builtins :: Unit/fixtfsi_test.c
compiler-rt :: builtins :: Unit/fixunsxfti_test.c
compiler-rt :: builtins :: Unit/fixxfti_test.c
compiler-rt :: builtins :: Unit/floattixf_test.c
compiler-rt :: builtins :: Unit/floatuntixf_test.c
compiler-rt :: builtins :: Unit/multf3_test.c

So looks like adding AArch64 target is too early. Does it still makes sense to add missing builtins without updating configuration? It's not that easy to fix broken functions of this kind and I can't promise to do it soon, I wonder if these are actually used by someone at the moment.

Thanks,
Sergey

[rengolin]

Hi Sergey,

I'd rather leave the first implementation to someone that can test on real hardware and turn the architecture on, even if that means XFAILing a few tests. But QEMU is not enough, and without a proper build, it's hard to put something in, even if it "looks ok".

cheers,
--renato

[rengolin]

So, this is not an endorsement, nor a certificate of achievement, but I just built Compiler-RT on an XGene, building the sanitizers, builtins, profiles, etc. and the build finishes and all tests pass.

However, this is not the same architecture we have for the buildbots (which is a Juno), nor it's a vanilla ARM core, so we can't assume it's kosher.

I'm adding Gabor (the Juno bot owner) to see if he can run some tests on that machine, and if that pass, we can go forward with the patch.

cheers,
--renato

[rengolin]

Adding Adhemerval to have a look.

Now that we have ASAN on and the bots are green, it's a good time to add this one before the more controversial TSAN changes.

lib/builtins/multc3.c

22

nitpick: is it necessary to have __ac and others, since they're all local context?

[zatrazz]

LGTM with the tests comment. I also have tested it with some random inputs (2^26) to check against GCC implementation and everything seems ok.

test/builtins/Unit/floatunditf_test.c

26

This should be 'unsigned long long a'.

28

Same as before.

46

I think it is better to explicit use ULL constants for this all these tests.

[rengolin]

LGTM too. Thanks for the review.

[sdmitrouk]

@zatrazz, thanks for the review and tests.

@rengolin, thank you for checks, review and finding reviewers!

Regards,
Sergey

lib/builtins/multc3.c

22

I was wondering the same and checked other files, some of which contain __ prefixes, so I decided to leave as is unless it will bother anyone. I'll remove __.

[rengolin]

lib/builtins/multc3.c

22

No worries, if that's the style.

[sdmitrouk]

Address review comments.

lib/builtins/CMakeLists.txt

set(arm_SOURCES
   arm/truncdfsf2vfp.S
   arm/udivmodsi4.S
   arm/udivsi3.S
   arm/umodsi3.S
   arm/unorddf2vfp.S
   arm/unordsf2vfp.S
   ${GENERIC_SOURCES})
 
+set(aarch64_SOURCES
+  comparetf2.c
+  extenddftf2.c
+  extendsftf2.c
+  fixtfdi.c
+  fixtfsi.c
+  fixtfti.c
+  fixunstfdi.c
+  fixunstfsi.c
+  fixunstfti.c
+  floatditf.c
+  floatsitf.c
+  floatunditf.c
+  floatunsitf.c
+  multc3.c
+  trunctfdf2.c
+  trunctfsf2.c
+  ${GENERIC_SOURCES})
+
 add_custom_target(builtins)
 
 if (NOT WIN32 OR MINGW)
-  foreach (arch x86_64 i386 i686 arm)
+  foreach (arch x86_64 i386 i686 arm aarch64)
     if (CAN_TARGET_${arch})
       # Filter out generic versions of routines that are re-implemented in
       # architecture specific manner.  This prevents multiple definitions of the
       # same symbols, making the symbol selection non-deterministic.
       foreach (_file ${${arch}_SOURCES})
         if (${_file} MATCHES ${arch}/*)
           get_filename_component(_name ${_file} NAME)
           string(REPLACE ".S" ".c" _cname "${_name}")

lib/builtins/floatditf.c

+//===-- lib/floatditf.c - integer -> quad-precision conversion ----*- C -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements di_int to quad-precision conversion for the
+// compiler-rt library in the IEEE-754 default round-to-nearest, ties-to-even
+// mode.
+//
+//===----------------------------------------------------------------------===//
+
+#define QUAD_PRECISION
+#include "fp_lib.h"
+
+#if defined(CRT_HAS_128BIT) && defined(CRT_LDBL_128BIT)
+COMPILER_RT_ABI fp_t __floatditf(di_int a) {
+
+    const int aWidth = sizeof a * CHAR_BIT;
+
+    // Handle zero as a special case to protect clz
+    if (a == 0)
+        return fromRep(0);
+
+    // All other cases begin by extracting the sign and absolute value of a
+    rep_t sign = 0;
+    du_int aAbs = (du_int)a;
+    if (a < 0) {
+        sign = signBit;
+        aAbs = ~(du_int)a + 1U;
+    }
+
+    // Exponent of (fp_t)a is the width of abs(a).
+    const int exponent = (aWidth - 1) - __builtin_clzll(aAbs);
+    rep_t result;
+
+    // Shift a into the significand field, rounding if it is a right-shift
+    const int shift = significandBits - exponent;
+    result = (rep_t)aAbs << shift ^ implicitBit;
+
+    // Insert the exponent
+    result += (rep_t)(exponent + exponentBias) << significandBits;
+    // Insert the sign bit and return
+    return fromRep(result | sign);
+}
+
+#endif

lib/builtins/floatunditf.c

+//===-- lib/floatunditf.c - uint -> quad-precision conversion -----*- C -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements du_int to quad-precision conversion for the
+// compiler-rt library in the IEEE-754 default round-to-nearest, ties-to-even
+// mode.
+//
+//===----------------------------------------------------------------------===//
+
+#define QUAD_PRECISION
+#include "fp_lib.h"
+
+#if defined(CRT_HAS_128BIT) && defined(CRT_LDBL_128BIT)
+COMPILER_RT_ABI fp_t __floatunditf(du_int a) {
+
+    const int aWidth = sizeof a * CHAR_BIT;
+
+    // Handle zero as a special case to protect clz
+    if (a == 0) return fromRep(0);
+
+    // Exponent of (fp_t)a is the width of abs(a).
+    const int exponent = (aWidth - 1) - __builtin_clzll(a);
+    rep_t result;
+
+    // Shift a into the significand field and clear the implicit bit.
+    const int shift = significandBits - exponent;
+    result = (rep_t)a << shift ^ implicitBit;
+
+    // Insert the exponent
+    result += (rep_t)(exponent + exponentBias) << significandBits;
+    return fromRep(result);
+}
+
+#endif

lib/builtins/multc3.c

+/* ===-- multc3.c - Implement __multc3 -------------------------------------===
+ *
+ *                     The LLVM Compiler Infrastructure
+ *
+ * This file is dual licensed under the MIT and the University of Illinois Open
+ * Source Licenses. See LICENSE.TXT for details.
+ *
+ * ===----------------------------------------------------------------------===
+ *
+ * This file implements __multc3 for the compiler_rt library.
+ *
+ * ===----------------------------------------------------------------------===
+ */
+
+#include "int_lib.h"
+#include "int_math.h"
+
+/* Returns: the product of a + ib and c + id */
+
+COMPILER_RT_ABI long double _Complex
+__multc3(long double a, long double b, long double c, long double d)
+{
+    long double ac = a * c;
+    long double bd = b * d;
+    long double ad = a * d;
+    long double bc = b * c;
+    long double _Complex z;
+    __real__ z = ac - bd;
+    __imag__ z = ad + bc;
+    if (crt_isnan(__real__ z) && crt_isnan(__imag__ z)) {
+        int recalc = 0;
+        if (crt_isinf(a) || crt_isinf(b)) {
+            a = crt_copysignl(crt_isinf(a) ? 1 : 0, a);
+            b = crt_copysignl(crt_isinf(b) ? 1 : 0, b);
+            if (crt_isnan(c))
+                c = crt_copysignl(0, c);
+            if (crt_isnan(d))
+                d = crt_copysignl(0, d);
+            recalc = 1;
+        }
+        if (crt_isinf(c) || crt_isinf(d)) {
+            c = crt_copysignl(crt_isinf(c) ? 1 : 0, c);
+            d = crt_copysignl(crt_isinf(d) ? 1 : 0, d);
+            if (crt_isnan(a))
+                a = crt_copysignl(0, a);
+            if (crt_isnan(b))
+                b = crt_copysignl(0, b);
+            recalc = 1;
+        }
+        if (!recalc && (crt_isinf(ac) || crt_isinf(bd) ||
+                          crt_isinf(ad) || crt_isinf(bc))) {
+            if (crt_isnan(a))
+                a = crt_copysignl(0, a);
+            if (crt_isnan(b))
+                b = crt_copysignl(0, b);
+            if (crt_isnan(c))
+                c = crt_copysignl(0, c);
+            if (crt_isnan(d))
+                d = crt_copysignl(0, d);
+            recalc = 1;
+        }
+        if (recalc) {
+            __real__ z = CRT_INFINITY * (a * c - b * d);
+            __imag__ z = CRT_INFINITY * (a * d + b * c);
+        }
+    }
+    return z;
+}

test/builtins/Unit/floatditf_test.c

+//===-- floatditf_test.c - Test __floatditf -------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file tests __floatditf for the compiler_rt library.
+//
+//===----------------------------------------------------------------------===//
+
+#include "int_lib.h"
+#include <math.h>
+#include <complex.h>
+#include <stdio.h>
+
+#if __LDBL_MANT_DIG__ == 113
+
+#include "fp_test.h"
+
+// Returns: long integer converted to long double
+
+COMPILER_RT_ABI long double __floatditf(long long a);
+
+int test__floatditf(long long a, uint64_t expectedHi, uint64_t expectedLo)
+{
+    long double x = __floatditf(a);
+    int ret = compareResultLD(x, expectedHi, expectedLo);
+
+    if (ret)
+        printf("error in __floatditf(%Ld) = %.20Lf, "
+               "expected %.20Lf\n", a, x, fromRep128(expectedHi, expectedLo));
+    return ret;
+}
+
+char assumption_1[sizeof(long double) * CHAR_BIT == 128] = {0};
+
+#endif
+
+int main()
+{
+#if __LDBL_MANT_DIG__ == 113
+    if (test__floatditf(0x7fffffffffffffff, UINT64_C(0x403dffffffffffff), UINT64_C(0xfffc000000000000)))
+        return 1;
+    if (test__floatditf(0x123456789abcdef1, UINT64_C(0x403b23456789abcd), UINT64_C(0xef10000000000000)))
+        return 1;
+    if (test__floatditf(0x2, UINT64_C(0x4000000000000000), UINT64_C(0x0)))
+        return 1;
+    if (test__floatditf(0x1, UINT64_C(0x3fff000000000000), UINT64_C(0x0)))
+        return 1;
+    if (test__floatditf(0x0, UINT64_C(0x0), UINT64_C(0x0)))
+        return 1;
+    if (test__floatditf(0xffffffffffffffff, UINT64_C(0xbfff000000000000), UINT64_C(0x0)))
+        return 1;
+    if (test__floatditf(0xfffffffffffffffe, UINT64_C(0xc000000000000000), UINT64_C(0x0)))
+        return 1;
+    if (test__floatditf(-0x123456789abcdef1, UINT64_C(0xc03b23456789abcd), UINT64_C(0xef10000000000000)))
+        return 1;
+    if (test__floatditf(0x8000000000000000, UINT64_C(0xc03e000000000000), UINT64_C(0x0)))
+        return 1;
+
+#else
+    printf("skipped\n");
+
+#endif
+    return 0;
+}

test/builtins/Unit/floatunditf_test.c

+//===-- floatunditf_test.c - Test __floatunditf -------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file tests __floatunditf for the compiler_rt library.
+//
+//===----------------------------------------------------------------------===//
+
+#include "int_lib.h"
+#include <math.h>
+#include <complex.h>
+#include <stdio.h>
+
+#if __LDBL_MANT_DIG__ == 113
+
+#include "fp_test.h"
+
+// Returns: long integer converted to long double
+
+COMPILER_RT_ABI long double __floatunditf(unsigned long long a);
+
+int test__floatunditf(unsigned long long a, uint64_t expectedHi, uint64_t expectedLo)
+{
+    long double x = __floatunditf(a);
+    int ret = compareResultLD(x, expectedHi, expectedLo);
+
+    if (ret)
+        printf("error in __floatunditf(%Lu) = %.20Lf, "
+               "expected %.20Lf\n", a, x, fromRep128(expectedHi, expectedLo));
+    return ret;
+}
+
+char assumption_1[sizeof(long double) * CHAR_BIT == 128] = {0};
+
+#endif
+
+int main()
+{
+#if __LDBL_MANT_DIG__ == 113
+    if (test__floatunditf(0xffffffffffffffffULL, UINT64_C(0x403effffffffffff), UINT64_C(0xfffe000000000000)))
+        return 1;
+    if (test__floatunditf(0xfffffffffffffffeULL, UINT64_C(0x403effffffffffff), UINT64_C(0xfffc000000000000)))
+        return 1;
+    if (test__floatunditf(0x8000000000000000ULL, UINT64_C(0x403e000000000000), UINT64_C(0x0)))
+        return 1;
+    if (test__floatunditf(0x7fffffffffffffffULL, UINT64_C(0x403dffffffffffff), UINT64_C(0xfffc000000000000)))
+        return 1;
+    if (test__floatunditf(0x123456789abcdef1ULL, UINT64_C(0x403b23456789abcd), UINT64_C(0xef10000000000000)))
+        return 1;
+    if (test__floatunditf(0x2ULL, UINT64_C(0x4000000000000000), UINT64_C(0x0)))
+        return 1;
+    if (test__floatunditf(0x1ULL, UINT64_C(0x3fff000000000000), UINT64_C(0x0)))
+        return 1;
+    if (test__floatunditf(0x0ULL, UINT64_C(0x0), UINT64_C(0x0)))
+        return 1;
+
+#else
+    printf("skipped\n");
+
+#endif
+    return 0;
+}

test/builtins/Unit/multc3_test.c

 //===-- multc3_test.c - Test __multc3 -------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file tests __multc3 for the compiler_rt library.
 //
 //===----------------------------------------------------------------------===//
 
 #include <stdio.h>
 
-#if _ARCH_PPC
+#if _ARCH_PPC || __aarch64__
 
 #include "int_lib.h"
 #include <math.h>
 #include <complex.h>
 
 // Returns: the product of a + ib and c + id
 
 COMPILER_RT_ABI long double _Complex
long double x[][2] =
     {INFINITY, INFINITY}
 
 };
 
 #endif
 
 int main()
 {
-#if _ARCH_PPC
+#if _ARCH_PPC || __aarch64__
     const unsigned N = sizeof(x) / sizeof(x[0]);
     unsigned i, j;
     for (i = 0; i < N; ++i)
     {
         for (j = 0; j < N; ++j)
         {
             if (test__multc3(x[i][0], x[i][1], x[j][0], x[j][1]))
                 return 1;
         }
     }
 #else
     printf("skipped\n");
 #endif
     return 0;
 }