Enable builtins necessary for SLEEF [AArch64] vectorized trigonometry libm functions

[steleman]

Summary

This changeset adds the required Builtins for enabling AArch64 vectorization of libm trigonometry functions via SLEEF: http://sleef.org/.

• A new argument is added to -fveclib=<X>: SLEEF.
• A number of Builtins that were previously unimplemented are now enabled.

This changeset depends on https://reviews.llvm.org/D53927.

List of SLEEF vectorized trigonometry functions:

acos
asin
atan
atanh
cos
cosh
exp
exp2
exp10
lgamma
log10
log2
log
sin
sinh
sqrt
tan
tanh
tgamma

[steleman]

• changed the -fveclib=<X> argument value to 'sleefgnuabi'.
• added atan2 and pow.
• spreadsheet with comparison between libm and sleef is here:

https://docs.google.com/spreadsheets/d/1lcpESCnuzEoTl_XHBqE9FLL0tXJB_tZGR8yciCx1yjg/edit?usp=sharing

• comprehensive test case in C with timings is here:

https://drive.google.com/open?id=1PGKRUdL29_ANoYebOo3Q59syhKp_mNSj

[steleman]

Updated version of this changeset/patch, as per Renato's latest comments from D53927 (https://reviews.llvm.org/D53927).

New benchmark comparing SLEEF performance when built with GCC 8.2 vs. clang ToT from 12/15/2018:

https://docs.google.com/spreadsheets/d/1QgdJjgCTFA9I2_5MvJU0uR4JT4knoW5xu-ujxszRbd0/edit?usp=sharing

[rengolin]

Adding clang/omp developers for proper review. Please feel free to add more.

FYI, there are four main ongoing discussions on the LLVM thread (D53927):

1. There is collusion between clang pragmas (clang vectorise), OpenMP pragmas (omp simd) and newly proposed veclib pragmas, which IMHO is too much. We need more OpenMP people looking at them. My proposal is to not add new ones and use OpenMP SIMD without creating library dependencies with OMP, but the interpretation of other pragmas in the same source needs sorting.

2. How we're going to handle the naming scheme, currently proposed to have a local math.h and mask the SLEEF includes with macros. My proposal is to have a special header that the compiler only includes upon -mveclibabi instead. Library includes end up as -lsleef in the linker command line.

3. Who controls the header file. The proposal is Clang control the headers for all vector libraries (all in math.h with guards), which means the evolution of Clang and each library is now tied. My proposal is for the library to have their own headers in a standard place (/usr/include/arch/something') and get Clang to agree on the right place, so that it can include the header from $default_path/$library_name.h`.

4. The option was called fveclib but GCC calls it mveclibabi. There doesn't seem to be any behavioural differences, so I think we should follow the same name. We may be forced to create an alias, because this is not new in LLVM (already existed for SVML). This can go on a separate patch. But it's good to be aware of that. Making it an alias, we are fixing the implementation to be at least compatible with GCC, which is a good thing.

cheers,
--renato

lib/Index/CommentToXML.cpp

230 ↗

(On Diff #178477)

Spurious change and unrelated to the patch. Please revert this part.

[steleman]

Removed spurious patch for an unrelated change.

[steleman]

Ping!

Yes, I know, everyone was away for the holidays. :-)

Could someone please take a look at this. We'd like to make sure everything is OK on this side before D53927 can proceed.

Yes, it's a bit of a circular dependency in that sense.

This changeset doesn't cover any of the problems with OpenMP and conflicting pragmas and semantics described above by Renato. This is just the clang side of straightforward mapping of SLEEF vectorized function calls.

[hfinkel]

LGTM

Please, in the future, make sure you post full-context patches.

[steleman]

This is a small - but signifcant - update to the original changeset.

What's new in this update:

${top_srcdir}/lib/Frontend/CompilerInvocation.cpp:

When -fveclib=SLEEF was passed on compile-line, neither clang nor LLVM emit any warnings if SLEEF is not supported on the Target architecture. SLEEF is currently supported only on AArch64. So, for other ISA's, -fveclib=SLEEF would be quietly ignored, without any hints.

This new version of the changeset adds a diagnostic if -fveclib=SLEEF is called on an ISA that doesn't (yet) support SLEEF.

[rengolin]

Since the previous version was approved already, I'm "requesting changes" so that we can look at it again, together with D53927 to make sure it's still the right way to go.

[rengolin]

@hfinkel There are changes in this patch, as well as its LLVM counterpart D53927. Please, re-review.

[rengolin]

Funny thing is, SVML is also only supported, AFAIK, for Intel. I agree that we should emit errors, but we should also emit a similar error on SVML.

I know it's not entirely relevant to this patch, but we should keep the behaviour consistent on all veclibs.

Also, can you add a test for the new error messages, please?

The new changes look good to me and don't substantially deviate from the previous, approved, patch, so LGTM after addressing the comments.

Thanks!

lib/Frontend/CompilerInvocation.cpp

678

This is not really invalid value for the flag, it's invalid architecture for the value.

I think there's already a string for that somewhere in Clang.

[steleman]

Hi Renato,

Thank you very much for the comments.

I will create a new changeset incorporating your comments, and then re-submit.

I will also add a Diagnostic for SVML and X86/X86_64.

[steleman]

Addressed comments from Renato:

• Diagnostic for unsupported argument to -fveclib=<N> is now:

error: unsupported option '<X>' for target '<Y>'

• Added check and diagnostic for SVML on non-Intel ISA's.

[rengolin]

Thanks, LGTM!

include/clang/Basic/Builtins.def

 BUILTIN(__builtin_erfcf, "ff", "Fne")
 BUILTIN(__builtin_erfcl, "LdLd", "Fne")
 BUILTIN(__builtin_exp , "dd"  , "Fne")
 BUILTIN(__builtin_expf, "ff"  , "Fne")
 BUILTIN(__builtin_expl, "LdLd", "Fne")
 BUILTIN(__builtin_exp2 , "dd"  , "Fne")
 BUILTIN(__builtin_exp2f, "ff"  , "Fne")
 BUILTIN(__builtin_exp2l, "LdLd", "Fne")
+BUILTIN(__builtin_exp10 , "dd"  , "Fne")
+BUILTIN(__builtin_exp10f, "ff"  , "Fne")
+BUILTIN(__builtin_exp10l, "LdLd", "Fne")
 BUILTIN(__builtin_expm1 , "dd", "Fne")
 BUILTIN(__builtin_expm1f, "ff", "Fne")
 BUILTIN(__builtin_expm1l, "LdLd", "Fne")
 BUILTIN(__builtin_fdim, "ddd", "Fne")
 BUILTIN(__builtin_fdimf, "fff", "Fne")
 BUILTIN(__builtin_fdiml, "LdLdLd", "Fne")
 BUILTIN(__builtin_floor , "dd"  , "Fnc")
 BUILTIN(__builtin_floorf, "ff"  , "Fnc")

 LIBBUILTIN(exp2, "dd", "fne", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(exp2f, "ff", "fne", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(exp2l, "LdLd", "fne", "math.h", ALL_LANGUAGES)
 
 LIBBUILTIN(expm1, "dd", "fne", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(expm1f, "ff", "fne", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(expm1l, "LdLd", "fne", "math.h", ALL_LANGUAGES)
 
+LIBBUILTIN(exp10, "dd", "fne", "math.h", ALL_LANGUAGES)
+LIBBUILTIN(exp10f, "ff", "fne", "math.h", ALL_LANGUAGES)
+LIBBUILTIN(exp10l, "LdLd", "fne", "math.h", ALL_LANGUAGES)
+
 LIBBUILTIN(fdim, "ddd", "fne", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(fdimf, "fff", "fne", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(fdiml, "LdLdLd", "fne", "math.h", ALL_LANGUAGES)
 
 LIBBUILTIN(floor, "dd", "fnc", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(floorf, "ff", "fnc", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(floorl, "LdLd", "fnc", "math.h", ALL_LANGUAGES)
 

 LIBBUILTIN(__sinpif, "ff", "fne", "math.h", ALL_LANGUAGES)
 
 LIBBUILTIN(__cospi, "dd", "fne", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(__cospif, "ff", "fne", "math.h", ALL_LANGUAGES)
 
 LIBBUILTIN(__tanpi, "dd", "fne", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(__tanpif, "ff", "fne", "math.h", ALL_LANGUAGES)
 
-// Similarly, __exp10 is OS X only
-LIBBUILTIN(__exp10, "dd", "fne", "math.h", ALL_LANGUAGES)
-LIBBUILTIN(__exp10f, "ff", "fne", "math.h", ALL_LANGUAGES)
-
 // Blocks runtime Builtin math library functions
 LIBBUILTIN(_Block_object_assign, "vv*vC*iC", "f", "Blocks.h", ALL_LANGUAGES)
 LIBBUILTIN(_Block_object_dispose, "vvC*iC", "f", "Blocks.h", ALL_LANGUAGES)
 // FIXME: Also declare NSConcreteGlobalBlock and NSConcreteStackBlock.
 
 // Annotation function
 BUILTIN(__builtin_annotation, "v.", "tn")
 

include/clang/Basic/CodeGenOptions.h

     NormalInlining,     // Use the standard function inlining pass.
     OnlyHintInlining,   // Inline only (implicitly) hinted functions.
     OnlyAlwaysInlining  // Only run the always inlining pass.
   };
 
   enum VectorLibrary {
     NoLibrary,  // Don't use any vector library.
     Accelerate, // Use the Accelerate framework.
-    SVML        // Intel short vector math library.
+    SVML,       // Intel short vector math library.
+    SLEEF       // SLEEF - SIMD Library for Evaluating Elementary Functions.
+                // (Experimental).
   };
 
 
   enum ObjCDispatchMethodKind {
     Legacy = 0,
     NonLegacy = 1,
     Mixed = 2
   };

include/clang/Driver/Options.td

 def fno_inline_functions : Flag<["-"], "fno-inline-functions">, Group<f_clang_Group>, Flags<[CC1Option]>;
 def fno_inline : Flag<["-"], "fno-inline">, Group<f_clang_Group>, Flags<[CC1Option]>;
 def fno_experimental_isel : Flag<["-"], "fno-experimental-isel">, Group<f_clang_Group>,
   HelpText<"Disables the experimental global instruction selector">;
 def fno_experimental_new_pass_manager : Flag<["-"], "fno-experimental-new-pass-manager">,
   Group<f_clang_Group>, Flags<[CC1Option]>,
   HelpText<"Disables an experimental new pass manager in LLVM.">;
 def fveclib : Joined<["-"], "fveclib=">, Group<f_Group>, Flags<[CC1Option]>,
-    HelpText<"Use the given vector functions library">, Values<"Accelerate,SVML,none">;
+    HelpText<"Use the given vector functions library">, Values<"Accelerate,SVML,SLEEF,none">;
 def fno_lax_vector_conversions : Flag<["-"], "fno-lax-vector-conversions">, Group<f_Group>,
   HelpText<"Disallow implicit conversions between vectors with a different number of elements or different element types">, Flags<[CC1Option]>;
 def fno_merge_all_constants : Flag<["-"], "fno-merge-all-constants">, Group<f_Group>,
   HelpText<"Disallow merging of constants">;
 def fno_modules : Flag <["-"], "fno-modules">, Group<f_Group>,
   Flags<[DriverOption]>;
 def fno_implicit_module_maps : Flag <["-"], "fno-implicit-module-maps">, Group<f_Group>,
   Flags<[DriverOption]>;

lib/CodeGen/BackendUtil.cpp

 
   switch (CodeGenOpts.getVecLib()) {
   case CodeGenOptions::Accelerate:
     TLII->addVectorizableFunctionsFromVecLib(TargetLibraryInfoImpl::Accelerate);
     break;
   case CodeGenOptions::SVML:
     TLII->addVectorizableFunctionsFromVecLib(TargetLibraryInfoImpl::SVML);
     break;
+  case CodeGenOptions::SLEEF:
+    TLII->addVectorizableFunctionsFromVecLib(TargetLibraryInfoImpl::SLEEF);
+    break;
   default:
     break;
   }
   return TLII;
 }
 
 static void addSymbolRewriterPass(const CodeGenOptions &Opts,
                                   legacy::PassManager *MPM) {

lib/CodeGen/CGBuiltin.cpp

     case Builtin::BIcopysignf:
     case Builtin::BIcopysignl:
     case Builtin::BI__builtin_copysign:
     case Builtin::BI__builtin_copysignf:
     case Builtin::BI__builtin_copysignl:
     case Builtin::BI__builtin_copysignf128:
       return RValue::get(emitBinaryBuiltin(*this, E, Intrinsic::copysign));
 
+    case Builtin::BIacos:
+    case Builtin::BIacosf:
+    case Builtin::BIacosl:
+    case Builtin::BI__builtin_acos:
+    case Builtin::BI__builtin_acosf:
+    case Builtin::BI__builtin_acosl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::acos));
+
+    case Builtin::BIacosh:
+    case Builtin::BIacoshf:
+    case Builtin::BIacoshl:
+    case Builtin::BI__builtin_acosh:
+    case Builtin::BI__builtin_acoshf:
+    case Builtin::BI__builtin_acoshl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::acosh));
+
+    case Builtin::BIasin:
+    case Builtin::BIasinf:
+    case Builtin::BIasinl:
+    case Builtin::BI__builtin_asin:
+    case Builtin::BI__builtin_asinf:
+    case Builtin::BI__builtin_asinl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::asin));
+
+    case Builtin::BIasinh:
+    case Builtin::BIasinhf:
+    case Builtin::BIasinhl:
+    case Builtin::BI__builtin_asinh:
+    case Builtin::BI__builtin_asinhf:
+    case Builtin::BI__builtin_asinhl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::asinh));
+
+    case Builtin::BIatan:
+    case Builtin::BIatanf:
+    case Builtin::BIatanl:
+    case Builtin::BI__builtin_atan:
+    case Builtin::BI__builtin_atanf:
+    case Builtin::BI__builtin_atanl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::atan));
+
+    case Builtin::BIatan2:
+    case Builtin::BIatan2f:
+    case Builtin::BIatan2l:
+    case Builtin::BI__builtin_atan2:
+    case Builtin::BI__builtin_atan2f:
+    case Builtin::BI__builtin_atan2l:
+      return RValue::get(emitBinaryBuiltin(*this, E, Intrinsic::atan2));
+
+    case Builtin::BIatanh:
+    case Builtin::BIatanhf:
+    case Builtin::BIatanhl:
+    case Builtin::BI__builtin_atanh:
+    case Builtin::BI__builtin_atanhf:
+    case Builtin::BI__builtin_atanhl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::atanh));
+
     case Builtin::BIcos:
     case Builtin::BIcosf:
     case Builtin::BIcosl:
     case Builtin::BI__builtin_cos:
     case Builtin::BI__builtin_cosf:
     case Builtin::BI__builtin_cosl:
       return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::cos));
 
+    case Builtin::BIcosh:
+    case Builtin::BIcoshf:
+    case Builtin::BIcoshl:
+    case Builtin::BI__builtin_cosh:
+    case Builtin::BI__builtin_coshf:
+    case Builtin::BI__builtin_coshl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::cosh));
+
     case Builtin::BIexp:
     case Builtin::BIexpf:
     case Builtin::BIexpl:
     case Builtin::BI__builtin_exp:
     case Builtin::BI__builtin_expf:
     case Builtin::BI__builtin_expl:
       return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::exp));
 
     case Builtin::BIexp2:
     case Builtin::BIexp2f:
     case Builtin::BIexp2l:
     case Builtin::BI__builtin_exp2:
     case Builtin::BI__builtin_exp2f:
     case Builtin::BI__builtin_exp2l:
       return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::exp2));
 
+    case Builtin::BIexp10:
+    case Builtin::BIexp10f:
+    case Builtin::BIexp10l:
+    case Builtin::BI__builtin_exp10:
+    case Builtin::BI__builtin_exp10f:
+    case Builtin::BI__builtin_exp10l:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::exp10));
+
     case Builtin::BIfabs:
     case Builtin::BIfabsf:
     case Builtin::BIfabsl:
     case Builtin::BI__builtin_fabs:
     case Builtin::BI__builtin_fabsf:
     case Builtin::BI__builtin_fabsl:
     case Builtin::BI__builtin_fabsf128:
       return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::fabs));

     case Builtin::BI__builtin_fmod:
     case Builtin::BI__builtin_fmodf:
     case Builtin::BI__builtin_fmodl: {
       Value *Arg1 = EmitScalarExpr(E->getArg(0));
       Value *Arg2 = EmitScalarExpr(E->getArg(1));
       return RValue::get(Builder.CreateFRem(Arg1, Arg2, "fmod"));
     }
 
+    case Builtin::BIlgamma:
+    case Builtin::BIlgammaf:
+    case Builtin::BIlgammal:
+    case Builtin::BI__builtin_lgamma:
+    case Builtin::BI__builtin_lgammaf:
+    case Builtin::BI__builtin_lgammal:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::lgamma));
+
     case Builtin::BIlog:
     case Builtin::BIlogf:
     case Builtin::BIlogl:
     case Builtin::BI__builtin_log:
     case Builtin::BI__builtin_logf:
     case Builtin::BI__builtin_logl:
       return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::log));
 

     case Builtin::BIsin:
     case Builtin::BIsinf:
     case Builtin::BIsinl:
     case Builtin::BI__builtin_sin:
     case Builtin::BI__builtin_sinf:
     case Builtin::BI__builtin_sinl:
       return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::sin));
 
+    case Builtin::BIsinh:
+    case Builtin::BIsinhf:
+    case Builtin::BIsinhl:
+    case Builtin::BI__builtin_sinh:
+    case Builtin::BI__builtin_sinhf:
+    case Builtin::BI__builtin_sinhl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::sinh));
+
     case Builtin::BIsqrt:
     case Builtin::BIsqrtf:
     case Builtin::BIsqrtl:
     case Builtin::BI__builtin_sqrt:
     case Builtin::BI__builtin_sqrtf:
     case Builtin::BI__builtin_sqrtl:
       return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::sqrt));
 
+    case Builtin::BItan:
+    case Builtin::BItanf:
+    case Builtin::BItanl:
+    case Builtin::BI__builtin_tan:
+    case Builtin::BI__builtin_tanf:
+    case Builtin::BI__builtin_tanl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::tan));
+
+    case Builtin::BItanh:
+    case Builtin::BItanhf:
+    case Builtin::BItanhl:
+    case Builtin::BI__builtin_tanh:
+    case Builtin::BI__builtin_tanhf:
+    case Builtin::BI__builtin_tanhl:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::tanh));
+
+    case Builtin::BItgamma:
+    case Builtin::BItgammaf:
+    case Builtin::BItgammal:
+    case Builtin::BI__builtin_tgamma:
+    case Builtin::BI__builtin_tgammaf:
+    case Builtin::BI__builtin_tgammal:
+      return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::tgamma));
+
     case Builtin::BItrunc:
     case Builtin::BItruncf:
     case Builtin::BItruncl:
     case Builtin::BI__builtin_trunc:
     case Builtin::BI__builtin_truncf:
     case Builtin::BI__builtin_truncl:
       return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::trunc));
 

lib/Frontend/CompilerInvocation.cpp

   Opts.DebugPassManager =
       Args.hasFlag(OPT_fdebug_pass_manager, OPT_fno_debug_pass_manager,
                    /* Default */ false);
 
   if (Arg *A = Args.getLastArg(OPT_fveclib)) {
     StringRef Name = A->getValue();
     if (Name == "Accelerate")
       Opts.setVecLib(CodeGenOptions::Accelerate);
-    else if (Name == "SVML")
-      Opts.setVecLib(CodeGenOptions::SVML);
-    else if (Name == "none")
+    else if (Name == "SVML") {
+      if (Triple.getArch() == llvm::Triple::x86 ||
+          Triple.getArch() == llvm::Triple::x86_64)
+        Opts.setVecLib(CodeGenOptions::SVML);
+      else
+        Diags.Report(diag::err_drv_unsupported_opt_for_target)
+          << Name << Triple.getArchName();
+    } else if (Name == "SLEEF") {
+      if (Triple.getArch() == llvm::Triple::aarch64 ||
+          Triple.getArch() == llvm::Triple::aarch64_be)
+        Opts.setVecLib(CodeGenOptions::SLEEF);
+      else
+        Diags.Report(diag::err_drv_unsupported_opt_for_target)
+          << Name << Triple.getArchName();
+    } else if (Name == "none")
       Opts.setVecLib(CodeGenOptions::NoLibrary);
     else
       Diags.Report(diag::err_drv_invalid_value) << A->getAsString(Args) << Name;
   }
 
   if (Arg *A = Args.getLastArg(OPT_debug_info_kind_EQ)) {
     unsigned Val =
         llvm::StringSwitch<unsigned>(A->getValue())