[RFC] Add new intrinsics and attribute to control accuracy of FP calls

[andrew.w.kaylor]

Summary

This patch adds a new set of fpbuiltin intrinsics to represent operations that are equivalent to common math library functions and basic operations, and adds a new call site attribute ("fp-max-error") to specify the required accuracy of these calls.

The purpose of these new IR constructs is to support alternate math library implementations and provide a general mechanism for selecting among multiple implementations based on specific requirements.

This is a follow-up to discussions here: https://discourse.llvm.org/t/rfc-floating-point-accuracy-control/66018

[andrew.w.kaylor]

Add Language Reference documentation for the new intrinsics and attribute.

[kpn]

llvm/docs/LangRef.rst

22593

Is there any way to enforce this? If the constrained intrinsics are merged in with the non-constrained then we lose the safe-by-default property.

The paragraph above would be stronger if it said "must not" instead of using the word "should".

[andrew.w.kaylor]

llvm/docs/LangRef.rst

22593

The safe-by-default property only ever came from the fact that existing optimizations didn't recognize the intrinsics at all. Initially we'd get that same benefit with these new intrinsics.

I was thinking I could add a function like FPBuiltinIntrinsic::hasUnrecognizedFPAttrs() that would take a list of IDs of FP attributes that the caller did know about. Then as we teach a pass to use these intrinsics, we can also bake in the list of attributes that the pass knows how to check for. I'm imagining a pattern something like this:

RecognizedFPAttrs.push_back(FPBuiltinIntrinsic::FP_MAX_ERROR);
if (FPI->hasUnrecognizedFPAttrs(RecognizedFPAttrs)
  return false;
if (FPI->getRequiredAccuracy() != None)
  return false;
// Do the transformation

How does that sound?

[kpn]

llvm/docs/LangRef.rst

22593

I'm a little worried that that much code would get "refactored" into a function out of sight of authors and reviewers and therefore encourage mistakes. Maybe it'll be OK.

It's going to require more vigilance watching for changes to be certain that nobody accidentally slips in a change that breaks the strictfp support. It's also going to be work to be sure that anyone making changes understands all of the different fp attributes. Cross-training is a good thing, but to be clear it is also a cost.

Does it make sense to include builtins that replace LLVM instructions? Add, subtract, etc? Are there going to be different add implementations that have the different precision issues seen with library calls?

I admit I'm a little nervous. and I'm sorry I can't be more concrete about my concerns.

The IR matchers will have to be dealt with at some point. Granted, this is also true without your proposal.

[andrew.w.kaylor]

This doesn't seem to have drawn a lot of support, perhaps because the use cases for the accuracy control are rather abstract right now. I'm going to move this over to the repository we use for SYCL development (https://github.com/intel/llvm) because I need to make progress with the implementation there. Hopefully once we have some open source libraries that leverage the interfaces I'm adding I'll be able to bring it back here with a bit of momentum behind it. I'll post a link here once I have a pull request up.

In the meantime, I hope we'll consider this as a model for any additional fp-related intrinsics we may think are needed.

llvm/docs/LangRef.rst

22593

Sorry for the long silence.

Does it make sense to include builtins that replace LLVM instructions? Add, subtract, etc? Are there going to be different add implementations that have the different precision issues seen with library calls?

The inclusion of the operations like add and subtract was in response to a request from an FPGA compiler developer I talked to. Apparently in the FPGA world there is a potential use case for less accurate add and subtract. I don't know how that works, but I'm taking his word for it.

[andrew.w.kaylor]

Here is a link to the pull request in the SYCL development fork: https://github.com/intel/llvm/pull/8134

llvm/docs/LangRef.rst

 
 Semantics:
 """"""""""
 
 Returns another pointer that aliases its argument but which has no associated
 ``invariant.group`` metadata.
 It does not read any memory and can be speculated.
 
+.. _fpbuiltin:
+
+Floating-Point Builtin Intrinsics
+-------------------------------------
+
+These intrinsics are used to represent common floating-point operations with
+the explicit expectation that the semantics of the operation may be modified
+by call-site attributes that are specific to these intrinsics. Although many
+of these operations correspond directly to functions defined by the standard
+C math library, these intrinsics are intended to allow replacement of the
+intrinsic with implementation outside the standard library, such as vector
+implementations of the operation or alternate implementations to satisfy
+different accuracy requirements.
+
+The following call-site attributes are currently recognized as being associated
+with the floating-point builtin intrinsics:
+
+``"fp-max-error"="<ulp>"``
+    This attribute specifies the required accuracy for the operation in ULPs.
+    The accuracy value must be a non-negative floating-point number. A value
+    of 0.5 or less indicates that the result is required to be correctly
+    rounded according to IEEE-754 rules. The default rounding mode
+    (round-to-nearest) may be assumed.
+
+    If this attribute is absent, basic operations (fadd, fsub, fmul, fdiv,
+    frem, and sqrt) are assumed to provide correctly rounded result. The
+    accuracy of other operations is target-dependent, corresponding to the
+    accuracy of the target-default implementation of the operation (usually
+    the implementation provided by the standard math library). If this
+    attribute is present, the intrinsic may only be replaced with
+    implementations which are known to provide at least the accuracy described.
+    An implementation which is more accurate than required by this attribute
+    may be used.
+
+The semantics of the fpbuiltin intrinsics may be further constrained by defining
+new callsite attributes beginning with "fp-". All such string attribute
+identifiers are considered reserved for use with fpbuiltin intrinsics.
+
+No transformation should be performed on any fpbuiltin intrinsic if the
+intrinsic has any callsite attributes begining with "fp-" that that code
+performing the transformation does not recognize.
+
+Unless otherwise specified using callsite attributes, the fpbuiltin intrinsics
+do not set ``errno`` or and may be assumed not to trap or raise floating-point
+exceptions.
+
+All fpbuiltin intrinsics are overloaded intrinsics which may operate on any
+scalar or vector floating-point type. Not all targets support all types.
+
+'``llvm.fpbuiltin.fadd``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.fadd(<type> <op1>, <type> <op2>)
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuiltin.fadd``' intrinsic returns the sum of its two operands.
+
+Arguments:
+""""""""""
+
+The arguments to the '``llvm.fpbuiltin.fadd``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. Both arguments must have identical types.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point sum of the two value operands and has
+the same type as the operands. Unless modified by the "fp-max-error" callsite
+attribute, the result is assumed to be correctly rounded.
+
+
+'``llvm.fpbuiltin.fsub``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.fsub(<type> <op1>, <type> <op2>)
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuiltin.fsub``' intrinsic returns the difference of its two
+operands.
+
+Arguments:
+""""""""""
+
+The arguments to the '``llvm.fpbuiltin.fsub``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. Both arguments must have identical types.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point difference of the two value operands
+and has the same type as the operands. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to be correctly rounded.
+
+
+'``llvm.fpbuiltin.fmul``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.fmul(<type> <op1>, <type> <op2>)
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuiltin.fmul``' intrinsic returns the product of its two operands.
+
+Arguments:
+""""""""""
+
+The arguments to the '``llvm.fpbuiltin.fmul``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. Both arguments must have identical types.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point product of the two value operands and
+has the same type as the operands. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to be correctly rounded.
+
+
+'``llvm.fpbuiltin.fdiv``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.fdiv(<type> <op1>, <type> <op2>)
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuiltin.fdiv``' intrinsic returns the quotient of its two
+operands.
+
+Arguments:
+""""""""""
+
+The arguments to the '``llvm.fpbuiltin.fdiv``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. Both arguments must have identical types.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point quotient of the two value operands and
+has the same type as the operands. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to be correctly rounded.
+
+
+'``llvm.fpbuiltin.frem``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.frem(<type> <op1>, <type> <op2>)
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuiltin.frem``' intrinsic returns the remainder from the division
+of its two operands.
+
+
+Arguments:
+""""""""""
+
+The arguments to the '``llvm.fpbuiltin.frem``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. Both arguments must have identical types.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point remainder from the division of the two
+value operands and has the same type as the operands.  The remainder has the
+same sign as the dividend. Unless modified by the "fp-max-error" callsite
+attribute, the result is assumed to be correctly rounded.
+
+
+'``llvm.fpbuilt.sin``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.sin(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.sin``' intrinsics return the sine of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.sin``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point sine of the operand and
+has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the sine operation for the input type.
+
+
+'``llvm.fpbuilt.cos``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.cos(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.cos``' intrinsics return the cosine of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.cos``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point cosine of the operand and
+has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the cosine operation for the input type.
+
+
+'``llvm.fpbuilt.tan``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.tan(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.tan``' intrinsics return the tangent of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.tan``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point tangent of the operand and
+has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the tangent operation for the input type.
+
+
+'``llvm.fpbuilt.sinh^``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.sinh(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.sinh``' intrinsics return the hyperbolic sine of the
+operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.sinh``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point hyperbolic sine of the operand
+and has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the hyperbolic sine operation for the input
+type.
+
+
+'``llvm.fpbuilt.cosh``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.cosh(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.cosh``' intrinsics return the hyperbolic cosine of the
+operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.cosh``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point hyperbolic cosine of the operand
+and has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the hyperbolic cosine operation for the input
+type.
+
+
+'``llvm.fpbuilt.tanh``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.tanh(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.tanh``' intrinsics return the hyperbolic tangent of the
+operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.tanh``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point hyperbolic tangent of the operand
+and has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the hyperbolic tangent operation for the
+input type.
+
+
+'``llvm.fpbuilt.asin``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.asin(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.asin``' intrinsics return the principal value of the
+arc sine of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.asin``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the principal value of the floating-point arc sine of
+the operand and has the same type as the operand. Unless modified by the
+"fp-max-error" callsite attribute, the result is assumed to have the accuracy
+of the target-default implementation of the arc sine operation for the input
+type.
+
+
+'``llvm.fpbuilt.acos``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.acos(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.acos``' intrinsics return the principal value of the
+arc cosine of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.acos``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the principal value of the floating-point arc cosine
+of the operand and has the same type as the operand. Unless modified by the
+"fp-max-error" callsite attribute, the result is assumed to have the accuracy
+of the target-default implementation of the arc cosine operation for the input
+type.
+
+
+'``llvm.fpbuilt.atan``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.atan(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.atan``' intrinsics return the principal value of the
+arc tangent of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.atan``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the principal value of the floating-point arc tangent
+of the operand and has the same type as the operand. Unless modified by the
+"fp-max-error" callsite attribute, the result is assumed to have the accuracy
+of the target-default implementation of the arc tangent operation for the
+input type.
+
+
+'``llvm.fpbuilt.atan2``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.atan2(<type> <op1>, <type> <op2>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.atan2``' intrinsics return the principal value of the
+arc tangent of op1/op2, expressed in radians.
+
+Arguments:
+""""""""""
+
+The arguments to the '``llvm.fpbuiltin.atan2``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. Both arguments must have identical types.
+
+Semantics:
+""""""""""
+
+The value produced is the principal value of the floating-point arc tangent
+of the quotient of the operands, expressed in radians, and has the same type
+as the operands. Unless modified by the "fp-max-error" callsite attribute,
+the result is assumed to have the accuracy of the target-default
+implementation of the atan2 operation for the input type.
+
+
+'``llvm.fpbuilt.asinh^``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.asinh(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.asinh``' intrinsics return the area hyperbolic sine of the
+operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.asinh``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point area hyperbolic sine of the operand
+and has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the area hyperbolic sine operation for the
+input type.
+
+
+'``llvm.fpbuilt.acosh``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.acosh(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.acosh``' intrinsics return the area hyperbolic cosine of
+the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.acosh``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point area hyperbolic cosine of the operand
+and has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the area hyperbolic cosine operation for the
+input type.
+
+
+'``llvm.fpbuilt.atanh``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.atanh(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.tanh``' intrinsics return the area hyperbolic tangent of
+the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.atanh``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point area hyperbolic tangent of the operand
+and has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the area hyperbolic tangent operation for the
+input type.
+
+
+'``llvm.fpbuilt.exp``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.exp(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.exp``' intrinsics return the base-e exponential function
+of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.exp``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point base-e exponential function of the
+operand and has the same type as the operand. Unless modified by the
+"fp-max-error" callsite attribute, the result is assumed to have the accuracy
+of the target-default implementation of the exp operation for the input type.
+
+
+'``llvm.fpbuilt.exp2``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.exp2(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.exp2``' intrinsics return the base-2 exponential function
+of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.exp2``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point base-2 exponential function of the
+operand and has the same type as the operand. Unless modified by the
+"fp-max-error" callsite attribute, the result is assumed to have the accuracy
+of the target-default implementation of the exp2 operation for the input type.
+
+
+'``llvm.fpbuilt.exp10``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.exp10(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.exp10``' intrinsics return the base-10 exponential function
+of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.exp10``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point base-10 exponential function of the
+operand and has the same type as the operand. Unless modified by the
+"fp-max-error" callsite attribute, the result is assumed to have the accuracy
+of the target-default implementation of the exp10 operation for the input type.
+
+
+'``llvm.fpbuilt.expm1``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.expm1(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.expm1``' intrinsics return e raised to the power of the
+operand minus one.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.expm1``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point value of e raised to the power the
+operand minus one and has the same type as the operand. Unless modified by the
+"fp-max-error" callsite attribute, the result is assumed to have the accuracy
+of the target-default implementation of the expm1 operation for the input type.
+
+
+'``llvm.fpbuilt.log``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.log(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.log``' intrinsics return the natural logarithm of the
+operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.log``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point natural logarithm of the operand and
+has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the log operation for the input type.
+
+
+'``llvm.fpbuilt.log2``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.log2(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.log2``' intrinsics return the base-2 logarithm of the
+operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.log2``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point base-2 logarithm of the operand and
+has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the log2 operation for the input type.
+
+
+'``llvm.fpbuilt.log10``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.log10(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.log10``' intrinsics return the base-10 logarithm of the
+operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.log10``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point base-10 logarithm of the operand and
+has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the log10 operation for the input type.
+
+
+'``llvm.fpbuilt.log1p``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.log(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.log``' intrinsics return the natural logarithm of
+one plus the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.log1p``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point natural logarithm of one plus
+the operand and has the same type as the operand. Unless modified by the
+"fp-max-error" callsite attribute, the result is assumed to have the accuracy
+of the target-default implementation of the log1p operation for the input
+type.
+
+
+'``llvm.fpbuilt.hypot``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.hypot(<type> <op1>, <type> <op2>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.hypot``' intrinsics return the hypotenuse of a
+right triangle whose legs are op1 and op2.
+
+Arguments:
+""""""""""
+
+The arguments to the '``llvm.fpbuiltin.hypot``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. Both arguments must have identical types.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point hypotenuse of a right triangle
+whose legs are the operands and has the same type as the operands. Unless
+modified by the "fp-max-error" callsite attribute, the result is assumed
+to have the accuracy of the target-default implementation of the hypot
+operation for the input type.
+
+
+'``llvm.fpbuilt.pow``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.pow(<type> <op1>, <type> <op2>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.pow``' intrinsics return the value of op1 raised
+to the power of op2.
+
+Arguments:
+""""""""""
+
+The arguments to the '``llvm.fpbuiltin.pow``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. Both arguments must have identical types.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point value of the first operand raised
+to the power of the second operand and has the same type as the operands.
+Unless modified by the "fp-max-error" callsite attribute, the result is
+assumed to have the accuracy of the target-default implementation of the pow
+operation for the input type.
+
+
+'``llvm.fpbuilt.ldexp``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.ldexp(<type> <op1>, <type> <op2>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.ldexp``' intrinsics return the value of op1 multiplied by
+by two raised to the power of op2.
+
+Arguments:
+""""""""""
+
+The first argument to the '``llvm.fpbuiltin.ldexp``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. The second argument must be a 32-bit integer value
+or a :ref:`vector <t_vector>` of 32-bit integers with the same number of
+elements as the first operand.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point value of the first operand multiplied
+by two raised to the power of the second operand and has the same type as the
+operands. Unless modified by the "fp-max-error" callsite attribute, the result
+is assumed to have the accuracy of the target-default implementation of the
+ldexp operation for the input type.
+
+
+'``llvm.fpbuilt.sqrt``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.sqrt(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.sqrt``' intrinsics return the square root of the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.sqrt``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point square root the operand and
+has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to be correctly rounded.
+
+
+'``llvm.fpbuilt.rsqrt``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.rsqrt(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.sqrt``' intrinsics return the inverse square root of the
+operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.rsqrt``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point inverse square root the operand
+and has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the rsqrt operation for the input type.
+
+
+'``llvm.fpbuilt.erf``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.erf(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.erf``' intrinsics return the error function value for
+the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.erf``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point error function value for the operand
+and has the same type as the operand. Unless modified by the "fp-max-error"
+callsite attribute, the result is assumed to have the accuracy of the
+target-default implementation of the erf operation for the input type.
+
+
+'``llvm.fpbuilt.erfc``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.erfc(<type> <op>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.erfc``' intrinsics return the error function value for
+the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.fpbuiltin.erfc``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values.
+
+Semantics:
+""""""""""
+
+The value produced is the floating-point complementary error function value
+for the operand and has the same type as the operand. Unless modified by the
+"fp-max-error" callsite attribute, the result is assumed to have the accuracy
+of the target-default implementation of the erf operation for the input type.
+
+
+'``llvm.fpbuilt.sincos``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <type> @llvm.fpbuiltin.sincos(<type> <op1>, ptr <sin>, ptr <cos>)
+
+
+Overview:
+"""""""""
+
+The '``llvm.fpbuilt.sincos``' intrinsics compute the sine and cosine of the
+first operand and returns the results via the pointers passed as the second
+and third operands.
+
+Arguments:
+""""""""""
+
+The first argument to the '``llvm.fpbuiltin.sincos``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
+floating-point values. The second and third arguments must be dereferenceable
+pointers to memory which can hold a value of the first operand's type.
+
+Semantics:
+""""""""""
+
+The values produced are the floating-point sine and cosine of the first
+operand and are stored using the same type as the first operand. Unless
+modified by the "fp-max-error" callsite attribute, the result is assumed to
+have the accuracy of the target-default implementation of the sincos operation
+for the input type.
+
 
 
 .. _constrainedfp:
 
 Constrained Floating-Point Intrinsics
 -------------------------------------
 
 These intrinsics are used to provide special handling of floating-point

llvm/include/llvm/Analysis/AltMathLibFuncs.def

+//===-- AltMathLibFuncs.def - Library information ---------*- C++ -*-------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// This .def file will create descriptions of available fpbuilt math library
+// function implementations and their constraining attributes. The current
+// support is limited to a fake test library for verifying the infrastructure.
+// The fake implementation can be removed when a real implementation is
+// available.
+
+// An accuracy of 0.5 indicates that the result is exact or correctly rounded.
+
+#define FIXED(NL) ElementCount::getFixed(NL)
+#define SCALABLE(NL) ElementCount::getScalable(NL)
+
+#if !(defined(TLI_DEFINE_ALTMATHFUNC))
+#define TLI_DEFINE_ALTMATHFUNC(IID, TYPE, VECSIZE, NAME, ACCURACY)  \
+    {IID, TYPE, VECSIZE, NAME, ACCURACY},
+#endif
+
+
+#if defined(TLI_DEFINE_TEST_ALTMATHFUNCS)
+
+// Just define a few examples to test the infrastructure
+
+// TEST_ALTMATH_LIB Half precision implementations
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_fdiv, Type::HalfTyID, FIXED(1), "__test_altmath_fdivh_med", 2.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sin, Type::HalfTyID, FIXED(1), "__test_altmath_sinh_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_cos, Type::HalfTyID, FIXED(1), "__test_altmath_cosh_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_cos, Type::HalfTyID, FIXED(1), "__test_altmath_cosh_med", 4.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sqrt, Type::HalfTyID, FIXED(1), "__test_altmath_sqrth_cr", 0.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_rsqrt, Type::HalfTyID, FIXED(1), "__test_altmath_rsqrth_cr", 0.5)
+
+// TEST_ALTMATH_LIB Single precision implementations
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_fdiv, Type::FloatTyID, FIXED(1), "__test_altmath_fdivf_med", 2.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sin, Type::FloatTyID, FIXED(1), "__test_altmath_sinf_cr", 0.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sin, Type::FloatTyID, FIXED(1), "__test_altmath_sinf_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_cos, Type::FloatTyID, FIXED(1), "__test_altmath_cosf_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_cos, Type::FloatTyID, FIXED(1), "__test_altmath_cosf_med", 4.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_tan, Type::FloatTyID, FIXED(1), "__test_altmath_tanf_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sqrt, Type::FloatTyID, FIXED(1), "__test_altmath_sqrtf_cr", 0.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sqrt, Type::FloatTyID, FIXED(1), "__test_altmath_sqrtf_med", 2.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_rsqrt, Type::FloatTyID, FIXED(1), "__test_altmath_rsqrtf_cr", 0.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_rsqrt, Type::FloatTyID, FIXED(1), "__test_altmath_rsqrtf_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_rsqrt, Type::FloatTyID, FIXED(1), "__test_altmath_rsqrtf_low", 4096.0)
+
+// TEST_ALTMATH_LIB Double precision implementations
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_fdiv, Type::DoubleTyID, FIXED(1), "__test_altmath_fdiv_med", 2.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sin, Type::DoubleTyID, FIXED(1), "__test_altmath_sin_cr", 0.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sin, Type::DoubleTyID, FIXED(1), "__test_altmath_sin_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_cos, Type::DoubleTyID, FIXED(1), "__test_altmath_cos_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_cos, Type::DoubleTyID, FIXED(1), "__test_altmath_cos_med", 4.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_tan, Type::DoubleTyID, FIXED(1), "__test_altmath_tan_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sqrt, Type::DoubleTyID, FIXED(1), "__test_altmath_sqrt_cr", 0.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sqrt, Type::DoubleTyID, FIXED(1), "__test_altmath_sqrt_med", 2.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_rsqrt, Type::DoubleTyID, FIXED(1), "__test_altmath_rsqrt_cr", 0.5)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_rsqrt, Type::DoubleTyID, FIXED(1), "__test_altmath_rsqrt_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_rsqrt, Type::DoubleTyID, FIXED(1), "__test_altmath_rsqrt_low", 4096.0)
+
+// TEST_ALTMATH_LIB 4 x float implementations
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sin, Type::FloatTyID, FIXED(4), "__test_altmath_sinf4_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_cos, Type::FloatTyID, FIXED(4), "__test_altmath_cosf4_high", 1.0)
+
+// TEST_ALTMATH_LIB 8 x float implementations
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sin, Type::FloatTyID, FIXED(8), "__test_altmath_sinf8_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_cos, Type::FloatTyID, FIXED(8), "__test_altmath_cosf8_high", 1.0)
+
+// TEST_ALTMATH_LIB 2 x double implementations
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_sin, Type::DoubleTyID, FIXED(2), "__test_altmath_sin2_high", 1.0)
+TLI_DEFINE_ALTMATHFUNC(Intrinsic::fpbuiltin_cos, Type::DoubleTyID, FIXED(2), "__test_altmath_cos2_high", 1.0)
+
+
+#endif
+
+
+
+#undef TLI_DEFINE_ALTMATHFUNC
+#undef TLI_DEFINE_TEST_ALTMATHFUNCS

llvm/include/llvm/Analysis/TargetLibraryInfo.h

 //===-- TargetLibraryInfo.h - Library information ---------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_ANALYSIS_TARGETLIBRARYINFO_H
 #define LLVM_ANALYSIS_TARGETLIBRARYINFO_H
 
 #include "llvm/ADT/BitVector.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/IR/InstrTypes.h"
+#include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/Pass.h"
 
 namespace llvm {
 
 template <typename T> class ArrayRef;
 class Function;
 class Module;
 class Triple;
 
+/// Describes a possible implementation of a floating point builtin operation
+struct AltMathDesc {
+  Intrinsic::ID IntrinID;
+  Type::TypeID BaseFPType;
+  ElementCount VectorizationFactor;
+  StringRef FnImplName;
+  float Accuracy;
+};
+
 /// Describes a possible vectorization of a function.
 /// Function 'VectorFnName' is equivalent to 'ScalarFnName' vectorized
 /// by a factor 'VectorizationFactor'.
 struct VecDesc {
   StringRef ScalarFnName;
   StringRef VectorFnName;
   ElementCount VectorizationFactor;
 };
class TargetLibraryInfoImpl {
   void setState(LibFunc F, AvailabilityState State) {
     AvailableArray[F/4] &= ~(3 << 2*(F&3));
     AvailableArray[F/4] |= State << 2*(F&3);
   }
   AvailabilityState getState(LibFunc F) const {
     return static_cast<AvailabilityState>((AvailableArray[F/4] >> 2*(F&3)) & 3);
   }
 
+  /// Alternate math library functions - sorted by intrinsic ID, then type,
+  ///   then vector size, then accuracy
+  std::vector<AltMathDesc> AltMathFuncDescs;
+
   /// Vectorization descriptors - sorted by ScalarFnName.
   std::vector<VecDesc> VectorDescs;
   /// Scalarization descriptors - same content as VectorDescs but sorted based
   /// on VectorFnName rather than ScalarFnName.
   std::vector<VecDesc> ScalarDescs;
 
   /// Return true if the function type FTy is valid for the library function
   /// F, regardless of whether the function is available.
enum VectorLibrary {
     NoLibrary,        // Don't use any vector library.
     Accelerate,       // Use Accelerate framework.
     DarwinLibSystemM, // Use Darwin's libsystem_m.
     LIBMVEC_X86,      // GLIBC Vector Math library.
     MASSV,            // IBM MASS vector library.
     SVML              // Intel short vector math library.
   };
 
+  /// List of known alternate math libraries.
+  ///
+  /// The alternate math library provides a set of functions that can ve used
+  /// to replace llvm.fpbuiltin intrinsic calls when one or more constraining
+  /// attributes are specified.
+  /// The library can be specified by either frontend or a commandline option,
+  /// and then used by addAltMathFunctionsFromLib for populating the tables of
+  /// math function implementations.
+  enum AltMathLibrary {
+    NoAltMathLibrary,   // Don't use any alternate math library
+    TestAltMathLibrary  // Use a fake alternate math library for testing
+  };
+
   TargetLibraryInfoImpl();
   explicit TargetLibraryInfoImpl(const Triple &T);
 
   // Provide value semantics.
   TargetLibraryInfoImpl(const TargetLibraryInfoImpl &TLI);
   TargetLibraryInfoImpl(TargetLibraryInfoImpl &&TLI);
   TargetLibraryInfoImpl &operator=(const TargetLibraryInfoImpl &TLI);
   TargetLibraryInfoImpl &operator=(TargetLibraryInfoImpl &&TLI);
void setAvailableWithName(LibFunc F, StringRef Name) {
     }
   }
 
   /// Disables all builtins.
   ///
   /// This can be used for options like -fno-builtin.
   void disableAllFunctions();
 
+  /// Add a set of alternate math library function implementations with
+  /// attributes that can be used to select an implementation for an
+  /// llvm.fpbuiltin intrinsic
+  void addAltMathFunctions(ArrayRef<AltMathDesc> Fns);
+
+  /// Calls addAltMathFunctions with a known preset of functions for the
+  /// given alternate math library.
+  void addAltMathFunctionsFromLib(enum AltMathLibrary AltLib);
+
+  /// Select an alternate math library implementation that meets the criteria
+  /// described by an FPBuiltinIntrinsic call.
+  StringRef selectFPBuiltinImplementation(FPBuiltinIntrinsic *Builtin) const;
+
   /// Add a set of scalar -> vector mappings, queryable via
   /// getVectorizedFunction and getScalarizedFunction.
   void addVectorizableFunctions(ArrayRef<VecDesc> Fns);
 
   /// Calls addVectorizableFunctions with a known preset of functions for the
   /// given vector library.
   void addVectorizableFunctionsFromVecLib(enum VectorLibrary VecLib);
 
bool has(LibFunc F) const {
     return getState(F) != TargetLibraryInfoImpl::Unavailable;
   }
   bool isFunctionVectorizable(StringRef F, const ElementCount &VF) const {
     return Impl->isFunctionVectorizable(F, VF);
   }
   bool isFunctionVectorizable(StringRef F) const {
     return Impl->isFunctionVectorizable(F);
   }
+  StringRef selectFPBuiltinImplementation(FPBuiltinIntrinsic *Builtin) const {
+    return Impl->selectFPBuiltinImplementation(Builtin);
+  }
   StringRef getVectorizedFunction(StringRef F, const ElementCount &VF) const {
     return Impl->getVectorizedFunction(F, VF);
   }
 
   /// Tests if the function is both available and a candidate for optimized code
   /// generation.
   bool hasOptimizedCodeGen(LibFunc F) const {
     if (getState(F) == TargetLibraryInfoImpl::Unavailable)

llvm/include/llvm/CodeGen/CodeGenPassBuilder.h

 #include "llvm/Analysis/AliasAnalysis.h"
 #include "llvm/Analysis/BasicAliasAnalysis.h"
 #include "llvm/Analysis/CFLAndersAliasAnalysis.h"
 #include "llvm/Analysis/CFLSteensAliasAnalysis.h"
 #include "llvm/Analysis/ScopedNoAliasAA.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Analysis/TypeBasedAliasAnalysis.h"
 #include "llvm/CodeGen/ExpandReductions.h"
+#include "llvm/CodeGen/FPBuiltinFnSelection.h"
 #include "llvm/CodeGen/MachinePassManager.h"
 #include "llvm/CodeGen/PreISelIntrinsicLowering.h"
 #include "llvm/CodeGen/ReplaceWithVeclib.h"
 #include "llvm/CodeGen/UnreachableBlockElim.h"
 #include "llvm/IR/IRPrintingPasses.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/MC/MCAsmInfo.h"

 template <typename Derived>
 void CodeGenPassBuilder<Derived>::addISelPasses(AddIRPass &addPass) const {
   if (TM.useEmulatedTLS())
     addPass(LowerEmuTLSPass());
 
   addPass(PreISelIntrinsicLoweringPass());
 
   derived().addIRPasses(addPass);
+  addPass(FPBuiltinFnSelectionPass());
   derived().addCodeGenPrepare(addPass);
   addPassesToHandleExceptions(addPass);
   derived().addISelPrepare(addPass);
 }
 
 /// Add common target configurable passes that perform LLVM IR to IR transforms
 /// following machine independent optimization.
 template <typename Derived>

llvm/include/llvm/CodeGen/FPBuiltinFnSelection.h

+//===- FPBuiltinFnSelection.h - Pre-ISel intrinsic lowering pass ----------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This pass implements alternate math library implementation selection for
+// llvm.fpbuiltin.* intrinsics.
+//
+//===----------------------------------------------------------------------===//
+#ifndef LLVM_CODEGEN_FPBUILTINFNSELECTION_H
+#define LLVM_CODEGEN_FPBUILTINFNSELECTION_H
+
+#include "llvm/IR/PassManager.h"
+
+namespace llvm {
+
+class Module;
+
+struct FPBuiltinFnSelectionPass
+    : PassInfoMixin<FPBuiltinFnSelectionPass> {
+  PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM);
+};
+
+} // end namespace llvm
+
+#endif // LLVM_CODEGEN_FPBUILTINFNSELECTION_H

llvm/include/llvm/CodeGen/MachinePassRegistry.def

 
 #ifndef FUNCTION_PASS
 #define FUNCTION_PASS(NAME, PASS_NAME, CONSTRUCTOR)
 #endif
 FUNCTION_PASS("mergeicmps", MergeICmpsPass, ())
 FUNCTION_PASS("lower-constant-intrinsics", LowerConstantIntrinsicsPass, ())
 FUNCTION_PASS("unreachableblockelim", UnreachableBlockElimPass, ())
 FUNCTION_PASS("consthoist", ConstantHoistingPass, ())
+FUNCTION_PASS("fpbuiltin-fn-selection", FPBuiltinFnSelectionPass, ())
 FUNCTION_PASS("replace-with-veclib", ReplaceWithVeclib, ())
 FUNCTION_PASS("partially-inline-libcalls", PartiallyInlineLibCallsPass, ())
 FUNCTION_PASS("ee-instrument", EntryExitInstrumenterPass, (false))
 FUNCTION_PASS("post-inline-ee-instrument", EntryExitInstrumenterPass, (true))
 FUNCTION_PASS("expand-large-div-rem", ExpandLargeDivRemPass, ())
 FUNCTION_PASS("expand-reductions", ExpandReductionsPass, ())
 FUNCTION_PASS("expandvp", ExpandVectorPredicationPass, ())
 FUNCTION_PASS("lowerinvoke", LowerInvokePass, ())

llvm/include/llvm/CodeGen/Passes.h

namespace llvm {
   ModulePass *createLowerEmuTLSPass();
 
   /// This pass lowers the \@llvm.load.relative and \@llvm.objc.* intrinsics to
   /// instructions.  This is unsafe to do earlier because a pass may combine the
   /// constant initializer into the load, which may result in an overflowing
   /// evaluation.
   ModulePass *createPreISelIntrinsicLoweringPass();
 
+  /// This pass lowers the \@llvm.fpbuiltin.{operation} intrinsics to
+  /// matching library function calls based on call site attributes.
+  FunctionPass *createFPBuiltinFnSelectionPass();
+
   /// GlobalMerge - This pass merges internal (by default) globals into structs
   /// to enable reuse of a base pointer by indexed addressing modes.
   /// It can also be configured to focus on size optimizations only.
   ///
   Pass *createGlobalMergePass(const TargetMachine *TM, unsigned MaximalOffset,
                               bool OnlyOptimizeForSize = false,
                               bool MergeExternalByDefault = false);
 

llvm/include/llvm/IR/FPBuiltinOps.def

+//===--- llvm/IR/FPBuiltinOps.def - Constrained intrinsics ------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Defines properties of floating point builtin intrinsics.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef OPERATION
+#define OPERATION(N,I)
+#endif
+
+// Arguments of the entries are:
+// - operation name.
+// - name of the fpbuiltin intrinsic to represent this operation.
+
+// These are definitions for instructions, that are converted into constrained
+// intrinsics.
+//
+OPERATION(FAdd,   fpbuiltin_fadd)
+OPERATION(FSub,   fpbuiltin_fsub)
+OPERATION(FMul,   fpbuiltin_fmul)
+OPERATION(FDiv,   fpbuiltin_fdiv)
+OPERATION(FRem,   fpbuiltin_frem)
+OPERATION(Sin,    fpbuiltin_sin)
+OPERATION(Cos,    fpbuiltin_cos)
+OPERATION(Tan,    fpbuiltin_tan)
+OPERATION(Sinh,   fpbuiltin_sinh)
+OPERATION(Cosh,   fpbuiltin_cosh)
+OPERATION(Tanh,   fpbuiltin_tanh)
+OPERATION(Asin,   fpbuiltin_asin)
+OPERATION(Acos,   fpbuiltin_acos)
+OPERATION(Atan,   fpbuiltin_atan)
+OPERATION(Atan2,  fpbuiltin_atan2)
+OPERATION(Asinh,  fpbuiltin_asinh)
+OPERATION(Acosh,  fpbuiltin_acosh)
+OPERATION(Atanh,  fpbuiltin_atanh)
+OPERATION(Exp,    fpbuiltin_exp)
+OPERATION(Exp2,   fpbuiltin_exp2)
+OPERATION(Exp10,  fpbuiltin_exp10)
+OPERATION(Expm1,  fpbuiltin_expm1)
+OPERATION(Log,    fpbuiltin_log)
+OPERATION(Log2,   fpbuiltin_log2)
+OPERATION(Log10,  fpbuiltin_log10)
+OPERATION(Log1p,  fpbuiltin_log1p)
+OPERATION(Hypot,  fpbuiltin_hypot)
+OPERATION(Pow,    fpbuiltin_pow)
+OPERATION(Ldexp,  fpbuiltin_ldexp)
+OPERATION(Sqrt,   fpbuiltin_sqrt)
+OPERATION(Rsqrt,  fpbuiltin_rsqrt)
+OPERATION(Erf,    fpbuiltin_erf)
+OPERATION(Erfc,   fpbuiltin_erfc)
+OPERATION(Sincos, fpbuiltin_sincos)
+
+#undef OPERATION

llvm/include/llvm/IR/IntrinsicInst.h

static bool classof(const IntrinsicInst *I) {
     return VPCmpIntrinsic::isVPCmp(I->getIntrinsicID());
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
   /// @}
 };
 
+/// This is the common base class for floating point builtin intrinsics.
+class FPBuiltinIntrinsic : public IntrinsicInst {
+public:
+  Optional<float> getRequiredAccuracy() const;
+
+  Type::TypeID getBaseTypeID() const;
+  ElementCount getElementCount() const;
+
+  // Methods for support type inquiry through isa, cast, and dyn_cast:
+  static bool classof(const IntrinsicInst *I);
+  static bool classof(const Value *V) {
+    return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
+  }
+};
+
 /// This is the common base class for constrained floating point intrinsics.
 class ConstrainedFPIntrinsic : public IntrinsicInst {
 public:
   bool isUnaryOp() const;
   bool isTernaryOp() const;
   Optional<RoundingMode> getRoundingMode() const;
   Optional<fp::ExceptionBehavior> getExceptionBehavior() const;
   bool isDefaultFPEnvironment() const;

llvm/include/llvm/IR/Intrinsics.td

 //===--------------- Floating Point Properties ----------------------------===//
 //
 
 def int_is_fpclass
     : DefaultAttrsIntrinsic<[LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>],
                             [llvm_anyfloat_ty, llvm_i32_ty],
                             [IntrNoMem, IntrWillReturn, ImmArg<ArgIndex<1>>]>;
 
+//===----------------- Floating Point Builtin Intrinsics ------------------===//
+//
+// These intrinsics are intended as explicitly replaceable versions of common
+// floating point math operations. Passes must check for call site attributes
+// that constrain the behavior of these intrinsics before transforming them in
+// any way.
+//
+// While many of these operations correspond to functions in the standard C
+// math library, these intrinsics are explicitly intended to be replaceable by
+// by alternate implementations.
+//
+
+let IntrProperties = [IntrNoMem, IntrWillReturn] in {
+  def int_fpbuiltin_fadd : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0>,
+                                                      LLVMMatchType<0> ]>;
+  def int_fpbuiltin_fsub : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0>,
+                                                      LLVMMatchType<0> ]>;
+  def int_fpbuiltin_fmul : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0>,
+                                                      LLVMMatchType<0> ]>;
+  def int_fpbuiltin_fdiv : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0>,
+                                                      LLVMMatchType<0> ]>;
+  def int_fpbuiltin_frem : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0>,
+                                                      LLVMMatchType<0> ]>;
+
+  def int_fpbuiltin_sin : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_cos : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_tan : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_sinh : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_cosh : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_tanh : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_asin : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_acos : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_atan : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_atan2 : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0>,
+                                                      LLVMMatchType<0> ]>;
+  def int_fpbuiltin_asinh : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_acosh : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_atanh : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+
+  def int_fpbuiltin_exp : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_exp2 : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_exp10 : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_expm1 : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_log : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_log2 : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_log10 : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_log1p : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+
+  def int_fpbuiltin_hypot : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0>,
+                                                      LLVMMatchType<0> ]>;
+  def int_fpbuiltin_pow : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0>,
+                                                      LLVMMatchType<0> ]>;
+  def int_fpbuiltin_ldexp : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                               [ LLVMMatchType<0>,
+                                 LLVMScalarOrSameVectorWidth<0, llvm_i32_ty> ]>;
+
+  def int_fpbuiltin_sqrt : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_rsqrt : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+
+  def int_fpbuiltin_erf : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+  def int_fpbuiltin_erfc : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                                    [ LLVMMatchType<0> ]>;
+}
+
+let IntrProperties = [IntrArgMemOnly, IntrWillReturn] in {
+ def int_fpbuiltin_sincos : DefaultAttrsIntrinsic<[],
+                                                    [ llvm_anyfloat_ty,
+                                                      llvm_ptr_ty,
+                                                      llvm_ptr_ty ]>;
+}
+
 //===--------------- Constrained Floating Point Intrinsics ----------------===//
 //
 
 let IntrProperties = [IntrInaccessibleMemOnly, IntrWillReturn] in {
   def int_experimental_constrained_fadd : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
                                                     [ LLVMMatchType<0>,
                                                       LLVMMatchType<0>,
                                                       llvm_metadata_ty,

llvm/include/llvm/InitializePasses.h

 void initializeExpandLargeDivRemLegacyPassPass(PassRegistry&);
 void initializeExpandMemCmpPassPass(PassRegistry&);
 void initializeExpandPostRAPass(PassRegistry&);
 void initializeExpandReductionsPass(PassRegistry&);
 void initializeExpandVectorPredicationPass(PassRegistry &);
 void initializeMakeGuardsExplicitLegacyPassPass(PassRegistry&);
 void initializeExternalAAWrapperPassPass(PassRegistry&);
 void initializeFEntryInserterPass(PassRegistry&);
+void initializeFPBuiltinFnSelectionLegacyPassPass(PassRegistry&);
 void initializeFinalizeISelPass(PassRegistry&);
 void initializeFinalizeMachineBundlesPass(PassRegistry&);
 void initializeFixIrreduciblePass(PassRegistry &);
 void initializeFixupStatepointCallerSavedPass(PassRegistry&);
 void initializeFlattenCFGLegacyPassPass(PassRegistry &);
 void initializeFloat2IntLegacyPassPass(PassRegistry&);
 void initializeForceFunctionAttrsLegacyPassPass(PassRegistry&);
 void initializeForwardControlFlowIntegrityPass(PassRegistry&);

llvm/lib/Analysis/TargetLibraryInfo.cpp

 
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Support/CommandLine.h"
 using namespace llvm;
 
+static cl::opt<TargetLibraryInfoImpl::AltMathLibrary> ClAltMathLibrary(
+    "alt-math-library", cl::Hidden,
+    cl::desc("Alternate floating point math library"),
+    cl::init(TargetLibraryInfoImpl::NoAltMathLibrary),
+    cl::values(clEnumValN(TargetLibraryInfoImpl::NoAltMathLibrary, "none",
+                          "No alternate math library"),
+               clEnumValN(TargetLibraryInfoImpl::TestAltMathLibrary, "test",
+                          "Fake library used for testing")));
+
 static cl::opt<TargetLibraryInfoImpl::VectorLibrary> ClVectorLibrary(
     "vector-library", cl::Hidden, cl::desc("Vector functions library"),
     cl::init(TargetLibraryInfoImpl::NoLibrary),
     cl::values(clEnumValN(TargetLibraryInfoImpl::NoLibrary, "none",
                           "No vector functions library"),
                clEnumValN(TargetLibraryInfoImpl::Accelerate, "Accelerate",
                           "Accelerate framework"),
                clEnumValN(TargetLibraryInfoImpl::DarwinLibSystemM,
static void initialize(TargetLibraryInfoImpl &TLI, const Triple &T,
   if (!T.isOSAIX()) {
     TLI.setUnavailable(LibFunc_vec_calloc);
     TLI.setUnavailable(LibFunc_vec_malloc);
     TLI.setUnavailable(LibFunc_vec_realloc);
     TLI.setUnavailable(LibFunc_vec_free);
   }
 
   TLI.addVectorizableFunctionsFromVecLib(ClVectorLibrary);
+  TLI.addAltMathFunctionsFromLib(ClAltMathLibrary);
 }
 
 TargetLibraryInfoImpl::TargetLibraryInfoImpl() {
   // Default to everything being available.
   memset(AvailableArray, -1, sizeof(AvailableArray));
 
   initialize(*this, Triple(), StandardNames);
 }
 
 TargetLibraryInfoImpl::TargetLibraryInfoImpl(const Triple &T) {
   // Default to everything being available.
   memset(AvailableArray, -1, sizeof(AvailableArray));
 
   initialize(*this, T, StandardNames);
 }
 
 TargetLibraryInfoImpl::TargetLibraryInfoImpl(const TargetLibraryInfoImpl &TLI)
     : CustomNames(TLI.CustomNames), ShouldExtI32Param(TLI.ShouldExtI32Param),
       ShouldExtI32Return(TLI.ShouldExtI32Return),
       ShouldSignExtI32Param(TLI.ShouldSignExtI32Param),
       SizeOfInt(TLI.SizeOfInt) {
   memcpy(AvailableArray, TLI.AvailableArray, sizeof(AvailableArray));
   VectorDescs = TLI.VectorDescs;
   ScalarDescs = TLI.ScalarDescs;
+  AltMathFuncDescs = TLI.AltMathFuncDescs;
 }
 
 TargetLibraryInfoImpl::TargetLibraryInfoImpl(TargetLibraryInfoImpl &&TLI)
     : CustomNames(std::move(TLI.CustomNames)),
       ShouldExtI32Param(TLI.ShouldExtI32Param),
       ShouldExtI32Return(TLI.ShouldExtI32Return),
       ShouldSignExtI32Param(TLI.ShouldSignExtI32Param),
       SizeOfInt(TLI.SizeOfInt) {
   std::move(std::begin(TLI.AvailableArray), std::end(TLI.AvailableArray),
             AvailableArray);
   VectorDescs = TLI.VectorDescs;
   ScalarDescs = TLI.ScalarDescs;
+  AltMathFuncDescs = TLI.AltMathFuncDescs;
 }
 
 TargetLibraryInfoImpl &TargetLibraryInfoImpl::operator=(const TargetLibraryInfoImpl &TLI) {
   CustomNames = TLI.CustomNames;
   ShouldExtI32Param = TLI.ShouldExtI32Param;
   ShouldExtI32Return = TLI.ShouldExtI32Return;
   ShouldSignExtI32Param = TLI.ShouldSignExtI32Param;
   SizeOfInt = TLI.SizeOfInt;
   memcpy(AvailableArray, TLI.AvailableArray, sizeof(AvailableArray));
+  VectorDescs = TLI.VectorDescs;
+  ScalarDescs = TLI.ScalarDescs;
+  AltMathFuncDescs = TLI.AltMathFuncDescs;
   return *this;
 }
 
 TargetLibraryInfoImpl &TargetLibraryInfoImpl::operator=(TargetLibraryInfoImpl &&TLI) {
   CustomNames = std::move(TLI.CustomNames);
   ShouldExtI32Param = TLI.ShouldExtI32Param;
   ShouldExtI32Return = TLI.ShouldExtI32Return;
   ShouldSignExtI32Param = TLI.ShouldSignExtI32Param;
   SizeOfInt = TLI.SizeOfInt;
   std::move(std::begin(TLI.AvailableArray), std::end(TLI.AvailableArray),
             AvailableArray);
+  VectorDescs = TLI.VectorDescs;
+  ScalarDescs = TLI.ScalarDescs;
+  AltMathFuncDescs = TLI.AltMathFuncDescs;
   return *this;
 }
 
 static StringRef sanitizeFunctionName(StringRef funcName) {
   // Filter out empty names and names containing null bytes, those can't be in
   // our table.
   if (funcName.empty() || funcName.contains('\0'))
     return StringRef();
bool TargetLibraryInfoImpl::getLibFunc(const Function &FDecl,
   return getLibFunc(FDecl.getName(), F) &&
          isValidProtoForLibFunc(*FDecl.getFunctionType(), F, *M);
 }
 
 void TargetLibraryInfoImpl::disableAllFunctions() {
   memset(AvailableArray, 0, sizeof(AvailableArray));
 }
 
+static bool compareAltMathDescs(const AltMathDesc &LHS,
+                                const AltMathDesc &RHS) {
+  if (LHS.IntrinID != RHS.IntrinID)
+    return LHS.IntrinID < RHS.IntrinID;
+  if (LHS.BaseFPType != RHS.BaseFPType)
+    return LHS.BaseFPType < RHS.BaseFPType;
+  if (LHS.VectorizationFactor != RHS.VectorizationFactor) {
+    // Sort scalar types ahead of vector types
+    if (LHS.VectorizationFactor.isScalar() !=
+        RHS.VectorizationFactor.isScalar())
+      return LHS.VectorizationFactor.isScalar() >
+             RHS.VectorizationFactor.isScalar();
+    assert((LHS.VectorizationFactor.isVector() &&
+            RHS.VectorizationFactor.isVector()) &&
+           "Unexpected vectorization factor in alt math fn desc");
+    // Sort scaleable vector types ahead of fixed vector types
+    if (LHS.VectorizationFactor.isScalable() !=
+        RHS.VectorizationFactor.isScalable())
+      return LHS.VectorizationFactor.isScalable() >
+             RHS.VectorizationFactor
+                 .isScalable();
+    // For non-scaleable vectors, this will be the fixed size
+    // For scaleable vectors, it's the size that's multiplied by the vscale
+    return LHS.VectorizationFactor.getKnownMinValue() <
+           RHS.VectorizationFactor.getKnownMinValue();
+  }
+  // Sort in order of descending accuracy
+  return LHS.Accuracy > RHS.Accuracy;
+}
+
+void TargetLibraryInfoImpl::addAltMathFunctions(ArrayRef<AltMathDesc> Fns) {
+  llvm::append_range(AltMathFuncDescs, Fns);
+  llvm::sort(AltMathFuncDescs, compareAltMathDescs);
+}
+
+void TargetLibraryInfoImpl::addAltMathFunctionsFromLib(
+    enum AltMathLibrary AltLib) {
+  switch (AltLib) {
+  case TestAltMathLibrary: {
+    const AltMathDesc AltMathFuncs[] = {
+    #define TLI_DEFINE_TEST_ALTMATHFUNCS
+    #include "llvm/Analysis/AltMathLibFuncs.def"
+    };
+    addAltMathFunctions(AltMathFuncs);
+    break;
+  }
+  case NoAltMathLibrary:
+    break;
+  }
+}
+
+/// Select an alternate math library implementation that meets the criteria
+/// described by an FPBuiltinIntrinsic call.
+StringRef TargetLibraryInfoImpl::selectFPBuiltinImplementation(
+    FPBuiltinIntrinsic *Builtin) const {
+  // TODO: Handle the case of no specified accuracy.
+  if (Builtin->getRequiredAccuracy() == None)
+    return StringRef();
+  AltMathDesc RequiredDesc = {Builtin->getIntrinsicID(),
+                              Builtin->getBaseTypeID(),
+                              Builtin->getElementCount(),
+                              "", Builtin->getRequiredAccuracy().value()};
+  std::vector<AltMathDesc>::const_iterator I =
+      llvm::lower_bound(AltMathFuncDescs, RequiredDesc, compareAltMathDescs);
+  if (I == AltMathFuncDescs.end())
+      return StringRef(); // TODO: Report fatal error?
+  // No match found
+  if (I->IntrinID != Builtin->getIntrinsicID())
+    return StringRef(); // TODO: Report fatal error?
+  return I->FnImplName;
+}
+
 static bool compareByScalarFnName(const VecDesc &LHS, const VecDesc &RHS) {
   return LHS.ScalarFnName < RHS.ScalarFnName;
 }
 
 static bool compareByVectorFnName(const VecDesc &LHS, const VecDesc &RHS) {
   return LHS.VectorFnName < RHS.VectorFnName;
 }
 

llvm/lib/CodeGen/CMakeLists.txt

add_llvm_component_library(LLVMCodeGen
   ExecutionDomainFix.cpp
   ExpandLargeDivRem.cpp
   ExpandMemCmp.cpp
   ExpandPostRAPseudos.cpp
   ExpandReductions.cpp
   ExpandVectorPredication.cpp
   FaultMaps.cpp
   FEntryInserter.cpp
+  FPBuiltinFnSelection.cpp
   FinalizeISel.cpp
   FixupStatepointCallerSaved.cpp
   FuncletLayout.cpp
   GCMetadata.cpp
   GCMetadataPrinter.cpp
   GCRootLowering.cpp
   GlobalMerge.cpp
   HardwareLoops.cpp

llvm/lib/CodeGen/CodeGen.cpp

void llvm::initializeCodeGen(PassRegistry &Registry) {
   initializeEarlyIfConverterPass(Registry);
   initializeEarlyIfPredicatorPass(Registry);
   initializeEarlyMachineLICMPass(Registry);
   initializeEarlyTailDuplicatePass(Registry);
   initializeExpandLargeDivRemLegacyPassPass(Registry);
   initializeExpandMemCmpPassPass(Registry);
   initializeExpandPostRAPass(Registry);
   initializeFEntryInserterPass(Registry);
+  initializeFPBuiltinFnSelectionLegacyPassPass(Registry);
   initializeFinalizeISelPass(Registry);
   initializeFinalizeMachineBundlesPass(Registry);
   initializeFixupStatepointCallerSavedPass(Registry);
   initializeFuncletLayoutPass(Registry);
   initializeGCMachineCodeAnalysisPass(Registry);
   initializeGCModuleInfoPass(Registry);
   initializeHardwareLoopsPass(Registry);
   initializeIfConverterPass(Registry);

llvm/lib/CodeGen/FPBuiltinFnSelection.cpp

+//===- FPBuiltinFnSelection.cpp - Pre-ISel intrinsic lowering pass --------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This pass implements alternate math library implementation selection for
+// llvm.fpbuiltin.* intrinsics.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/CodeGen/FPBuiltinFnSelection.h"
+#include "llvm/Analysis/TargetLibraryInfo.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/InstIterator.h"
+#include "llvm/IR/IntrinsicInst.h"
+#include "llvm/InitializePasses.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "fpbuiltin-fn-selection"
+
+static bool replaceWithAltMathFunction(FPBuiltinIntrinsic &BuiltinCall,
+                                       const StringRef ImplName) {
+  Module *M = BuiltinCall.getModule();
+
+  Function *OldFunc = BuiltinCall.getCalledFunction();
+
+  // Check if the alt math library function is already declared in this module,
+  // otherwise insert it.
+  Function *ImplFunc = M->getFunction(ImplName);
+  if (!ImplFunc) {
+    ImplFunc = Function::Create(OldFunc->getFunctionType(),
+                                Function::ExternalLinkage, ImplName, *M);
+    // TODO: Copy non-builtin attributes ImplFunc->copyAttributesFrom(OldFunc);
+  }
+
+  // Replace the call to the fpbuiltin intrinsic with a call
+  // to the corresponding function from the alternate math library.
+  IRBuilder<> IRBuilder(&BuiltinCall);
+  SmallVector<Value *> Args(BuiltinCall.args());
+  // Preserve the operand bundles.
+  SmallVector<OperandBundleDef, 1> OpBundles;
+  BuiltinCall.getOperandBundlesAsDefs(OpBundles);
+  CallInst *Replacement = IRBuilder.CreateCall(ImplFunc, Args, OpBundles);
+  assert(OldFunc->getFunctionType() == ImplFunc->getFunctionType() &&
+         "Expecting function types to be identical");
+  BuiltinCall.replaceAllUsesWith(Replacement);
+  // TODO: fpbuiltin.sincos won't be reported as an FPMathOperator
+  //       Do we need to do anything about that?
+  if (isa<FPMathOperator>(Replacement)) {
+    // Preserve fast math flags for FP math.
+    Replacement->copyFastMathFlags(&BuiltinCall);
+  }
+
+  LLVM_DEBUG(dbgs() << DEBUG_TYPE << ": Replaced call to `"
+                    << OldFunc->getName() << "` with call to `" << ImplName
+                    << "`.\n");
+  return true;
+}
+
+static bool selectFnForFPBuiltinCalls(const TargetLibraryInfo &TLI,
+                                      FPBuiltinIntrinsic &BuiltinCall) {
+  LLVM_DEBUG({
+    dbgs() << "Selecting an implementation for "
+           << BuiltinCall.getCalledFunction()->getName()
+           << " with accuracy = ";
+    if (BuiltinCall.getRequiredAccuracy() == None)
+      dbgs() << "(none)\n";
+    else
+      dbgs() << BuiltinCall.getRequiredAccuracy().value() << "\n";
+  });
+
+  /// Call TLI to select a function implementation to call
+  StringRef ImplName = TLI.selectFPBuiltinImplementation(&BuiltinCall);
+  if (ImplName.empty()) {
+    // TODO: Report an error
+    LLVM_DEBUG(dbgs() << "No matching implementation found!\n");
+    return false;
+  }
+
+  LLVM_DEBUG(dbgs() << "Selected " << ImplName << "\n");
+
+  return replaceWithAltMathFunction(BuiltinCall, ImplName);
+}
+
+static bool runImpl(const TargetLibraryInfo &TLI, Function &F) {
+  bool Changed = false;
+  SmallVector<FPBuiltinIntrinsic *> ReplacedCalls;
+  for (auto &I : instructions(F)) {
+    if (auto *CI = dyn_cast<FPBuiltinIntrinsic>(&I)) {
+      if (selectFnForFPBuiltinCalls(TLI, *CI)) {
+        ReplacedCalls.push_back(CI);
+        Changed = true;
+      }
+    }
+  }
+  // Erase the calls to the intrinsics that have been replaced
+  // with calls to the alternate math library.
+  for (auto *CI : ReplacedCalls) {
+    CI->eraseFromParent();
+  }
+  return Changed;
+}
+
+
+namespace {
+
+class FPBuiltinFnSelectionLegacyPass : public FunctionPass {
+public:
+  static char ID;
+
+  FPBuiltinFnSelectionLegacyPass() : FunctionPass(ID) {}
+
+  bool runOnFunction(Function &F) override {
+    const TargetLibraryInfo *TLI =
+        &getAnalysis<TargetLibraryInfoWrapperPass>().getTLI(F);
+
+    return runImpl(*TLI, F);
+  }
+
+  void getAnalysisUsage(AnalysisUsage &AU) const {
+    AU.setPreservesCFG();
+    AU.addRequired<TargetLibraryInfoWrapperPass>();
+    AU.addPreserved<TargetLibraryInfoWrapperPass>();
+  }
+};
+
+} // end anonymous namespace
+
+char FPBuiltinFnSelectionLegacyPass::ID;
+
+INITIALIZE_PASS_BEGIN(FPBuiltinFnSelectionLegacyPass,
+                DEBUG_TYPE, "FPBuiltin Function Selection",
+                false, false)
+INITIALIZE_PASS_DEPENDENCY(TargetLibraryInfoWrapperPass)
+INITIALIZE_PASS_END(FPBuiltinFnSelectionLegacyPass,
+                DEBUG_TYPE, "FPBuiltin Function Selection",
+                false, false)
+
+FunctionPass *llvm::createFPBuiltinFnSelectionPass() {
+  return new FPBuiltinFnSelectionLegacyPass;
+}
+
+PreservedAnalyses FPBuiltinFnSelectionPass::run(Function &F,
+                                                FunctionAnalysisManager &AM) {
+  const TargetLibraryInfo &TLI = AM.getResult<TargetLibraryAnalysis>(F);
+  bool Changed = runImpl(TLI, F);
+  if (Changed) {
+    PreservedAnalyses PA;
+    PA.preserveSet<CFGAnalyses>();
+    PA.preserve<TargetLibraryAnalysis>();
+    return PA;
+  } else {
+    // The pass did not replace any calls, hence it preserves all analyses.
+    return PreservedAnalyses::all();
+  }
+
+}

llvm/lib/CodeGen/TargetPassConfig.cpp

 bool TargetPassConfig::addISelPasses() {
   if (TM->useEmulatedTLS())
     addPass(createLowerEmuTLSPass());
 
   addPass(createPreISelIntrinsicLoweringPass());
   PM->add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
   addPass(createExpandLargeDivRemPass());
   addIRPasses();
+  addPass(createFPBuiltinFnSelectionPass());
   addCodeGenPrepare();
   addPassesToHandleExceptions();
   addISelPrepare();
 
   return addCoreISelPasses();
 }
 
 /// -regalloc=... command line option.

llvm/lib/IR/IntrinsicInst.cpp

Value *InstrProfIncrementInst::getStep() const {
   if (InstrProfIncrementInstStep::classof(this)) {
     return const_cast<Value *>(getArgOperand(4));
   }
   const Module *M = getModule();
   LLVMContext &Context = M->getContext();
   return ConstantInt::get(Type::getInt64Ty(Context), 1);
 }
 
+Type::TypeID FPBuiltinIntrinsic::getBaseTypeID() const {
+  // All currently supported FP builtins are characterized by the type of their
+  // first argument. Since llvm.fpbuiltin.sincos doesn't return a value, using
+  // the type of the first argument is the most consistent technique.
+  Type *OperandTy = getArgOperand(0)->getType();
+  assert((OperandTy->isFloatingPointTy() ||
+          (OperandTy->isVectorTy() &&
+           OperandTy->getScalarType()->isFloatingPointTy())) &&
+             "Unexpected type for floating point builtin intrinsic!");
+  return OperandTy->getScalarType()->getTypeID();
+}
+
+ElementCount FPBuiltinIntrinsic::getElementCount() const {
+  Type *OperandTy = getArgOperand(0)->getType();
+  assert((OperandTy->isFloatingPointTy() ||
+          (OperandTy->isVectorTy() &&
+           OperandTy->getScalarType()->isFloatingPointTy())) &&
+             "Unexpected type for floating point builtin intrinsic!");
+  if (auto *VecTy = dyn_cast<VectorType>(OperandTy))
+    return VecTy->getElementCount();
+  return ElementCount::getFixed(1);
+}
+
+Optional<float> FPBuiltinIntrinsic::getRequiredAccuracy() const {
+  if (!hasFnAttr("fp-max-error"))
+    return None;
+  // This should be a string attribute with a floating-point value
+  // If it isn't the IR verifier should report the problem. Here
+  // we handle that as if the attribute were absent.
+  // TODO: Create Attribute::getValueAsDouble()?
+  double Accuracy;
+  // getAsDouble returns false if it succeeds
+  if (getFnAttr("fp-max-error").getValueAsString().getAsDouble(Accuracy))
+    return None;
+  return (float)Accuracy;
+}
+
+bool FPBuiltinIntrinsic::classof(const IntrinsicInst *I) {
+  switch (I->getIntrinsicID()) {
+#define OPERATION(NAME, INTRINSIC)                        \
+  case Intrinsic::INTRINSIC:
+#include "llvm/IR/FPBuiltinOps.def"
+    return true;
+  default:
+    return false;
+  }
+}
+
+
 Optional<RoundingMode> ConstrainedFPIntrinsic::getRoundingMode() const {
   unsigned NumOperands = arg_size();
   Metadata *MD = nullptr;
   auto *MAV = dyn_cast<MetadataAsValue>(getArgOperand(NumOperands - 2));
   if (MAV)
     MD = MAV->getMetadata();
   if (!MD || !isa<MDString>(MD))
     return None;

llvm/test/CodeGen/AArch64/O0-pipeline.ll

 ; CHECK-NEXT:       Shadow Stack GC Lowering
 ; CHECK-NEXT:       Lower constant intrinsics
 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Expand vector predication intrinsics
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       AArch64 Stack Tagging
 ; CHECK-NEXT:       SME ABI Pass
+; CHECK-NEXT:       FPBuiltin Function Selection
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       Analysis containing CSE Info
 ; CHECK-NEXT:       IRTranslator
 ; CHECK-NEXT:       Analysis for ComputingKnownBits
 ; CHECK-NEXT:       AArch64O0PreLegalizerCombiner

llvm/test/CodeGen/AArch64/O3-pipeline.ll

 ; CHECK-NEXT:       Function Alias Analysis Results
 ; CHECK-NEXT:       AArch64 Stack Tagging
 ; CHECK-NEXT:       Function Alias Analysis Results
 ; CHECK-NEXT:       Memory SSA
 ; CHECK-NEXT:       Interleaved Load Combine Pass
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Interleaved Access Pass
 ; CHECK-NEXT:       SME ABI Pass
+; CHECK-NEXT:       FPBuiltin Function Selection
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       Type Promotion
 ; CHECK-NEXT:       CodeGen Prepare
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:     AArch64 Promote Constant
 ; CHECK-NEXT:       FunctionPass Manager

llvm/test/CodeGen/AMDGPU/llc-pipeline.ll

 ; GCN-O0-NEXT:    Lower uses of LDS variables from non-kernel functions
 ; GCN-O0-NEXT:    FunctionPass Manager
 ; GCN-O0-NEXT:      Expand Atomic instructions
 ; GCN-O0-NEXT:      Lower constant intrinsics
 ; GCN-O0-NEXT:      Remove unreachable blocks from the CFG
 ; GCN-O0-NEXT:      Expand vector predication intrinsics
 ; GCN-O0-NEXT:      Scalarize Masked Memory Intrinsics
 ; GCN-O0-NEXT:      Expand reduction intrinsics
+; GCN-O0-NEXT:      FPBuiltin Function Selection
 ; GCN-O0-NEXT:    AMDGPU Attributor
 ; GCN-O0-NEXT:    CallGraph Construction
 ; GCN-O0-NEXT:    Call Graph SCC Pass Manager
 ; GCN-O0-NEXT:      AMDGPU Annotate Kernel Features
 ; GCN-O0-NEXT:      FunctionPass Manager
 ; GCN-O0-NEXT:        AMDGPU Lower Kernel Arguments
 ; GCN-O0-NEXT:        Lazy Value Information Analysis
 ; GCN-O0-NEXT:        Lower SwitchInst's to branches

 ; GCN-O1-NEXT:      Constant Hoisting
 ; GCN-O1-NEXT:      Replace intrinsics with calls to vector library
 ; GCN-O1-NEXT:      Partially inline calls to library functions
 ; GCN-O1-NEXT:      Expand vector predication intrinsics
 ; GCN-O1-NEXT:      Scalarize Masked Memory Intrinsics
 ; GCN-O1-NEXT:      Expand reduction intrinsics
 ; GCN-O1-NEXT:      Natural Loop Information
 ; GCN-O1-NEXT:      TLS Variable Hoist
+; GCN-O1-NEXT:      FPBuiltin Function Selection
 ; GCN-O1-NEXT:    AMDGPU Attributor
 ; GCN-O1-NEXT:    CallGraph Construction
 ; GCN-O1-NEXT:    Call Graph SCC Pass Manager
 ; GCN-O1-NEXT:      AMDGPU Annotate Kernel Features
 ; GCN-O1-NEXT:      FunctionPass Manager
 ; GCN-O1-NEXT:        AMDGPU Lower Kernel Arguments
 ; GCN-O1-NEXT:        Dominator Tree Construction
 ; GCN-O1-NEXT:        Natural Loop Information

 ; GCN-O1-OPTS-NEXT:      Replace intrinsics with calls to vector library
 ; GCN-O1-OPTS-NEXT:      Partially inline calls to library functions
 ; GCN-O1-OPTS-NEXT:      Expand vector predication intrinsics
 ; GCN-O1-OPTS-NEXT:      Scalarize Masked Memory Intrinsics
 ; GCN-O1-OPTS-NEXT:      Expand reduction intrinsics
 ; GCN-O1-OPTS-NEXT:      Natural Loop Information
 ; GCN-O1-OPTS-NEXT:      TLS Variable Hoist
 ; GCN-O1-OPTS-NEXT:      Early CSE
+; GCN-O1-OPTS-NEXT:      FPBuiltin Function Selection
 ; GCN-O1-OPTS-NEXT:    AMDGPU Attributor
 ; GCN-O1-OPTS-NEXT:    CallGraph Construction
 ; GCN-O1-OPTS-NEXT:    Call Graph SCC Pass Manager
 ; GCN-O1-OPTS-NEXT:      AMDGPU Annotate Kernel Features
 ; GCN-O1-OPTS-NEXT:      FunctionPass Manager
 ; GCN-O1-OPTS-NEXT:        AMDGPU Lower Kernel Arguments
 ; GCN-O1-OPTS-NEXT:        Dominator Tree Construction
 ; GCN-O1-OPTS-NEXT:        Natural Loop Information

 ; GCN-O2-NEXT:      Replace intrinsics with calls to vector library
 ; GCN-O2-NEXT:      Partially inline calls to library functions
 ; GCN-O2-NEXT:      Expand vector predication intrinsics
 ; GCN-O2-NEXT:      Scalarize Masked Memory Intrinsics
 ; GCN-O2-NEXT:      Expand reduction intrinsics
 ; GCN-O2-NEXT:      Natural Loop Information
 ; GCN-O2-NEXT:      TLS Variable Hoist
 ; GCN-O2-NEXT:      Early CSE
+; GCN-O2-NEXT:      FPBuiltin Function Selection
 ; GCN-O2-NEXT:    AMDGPU Attributor
 ; GCN-O2-NEXT:    CallGraph Construction
 ; GCN-O2-NEXT:    Call Graph SCC Pass Manager
 ; GCN-O2-NEXT:      AMDGPU Annotate Kernel Features
 ; GCN-O2-NEXT:      FunctionPass Manager
 ; GCN-O2-NEXT:        AMDGPU Lower Kernel Arguments
 ; GCN-O2-NEXT:        Dominator Tree Construction
 ; GCN-O2-NEXT:        Natural Loop Information

 ; GCN-O3-NEXT:      Phi Values Analysis
 ; GCN-O3-NEXT:      Basic Alias Analysis (stateless AA impl)
 ; GCN-O3-NEXT:      Function Alias Analysis Results
 ; GCN-O3-NEXT:      Memory Dependence Analysis
 ; GCN-O3-NEXT:      Lazy Branch Probability Analysis
 ; GCN-O3-NEXT:      Lazy Block Frequency Analysis
 ; GCN-O3-NEXT:      Optimization Remark Emitter
 ; GCN-O3-NEXT:      Global Value Numbering
+; GCN-O3-NEXT:      FPBuiltin Function Selection
 ; GCN-O3-NEXT:    AMDGPU Attributor
 ; GCN-O3-NEXT:    CallGraph Construction
 ; GCN-O3-NEXT:    Call Graph SCC Pass Manager
 ; GCN-O3-NEXT:      AMDGPU Annotate Kernel Features
 ; GCN-O3-NEXT:      FunctionPass Manager
 ; GCN-O3-NEXT:        AMDGPU Lower Kernel Arguments
 ; GCN-O3-NEXT:        Dominator Tree Construction
 ; GCN-O3-NEXT:        Natural Loop Information

llvm/test/CodeGen/ARM/O3-pipeline.ll

 ; CHECK-NEXT:      Natural Loop Information
 ; CHECK-NEXT:      TLS Variable Hoist
 ; CHECK-NEXT:      Scalar Evolution Analysis
 ; CHECK-NEXT:      Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:      Function Alias Analysis Results
 ; CHECK-NEXT:      Transform functions to use DSP intrinsics
 ; CHECK-NEXT:      Complex Deinterleaving Pass
 ; CHECK-NEXT:      Interleaved Access Pass
+; CHECK-NEXT:      FPBuiltin Function Selection
 ; CHECK-NEXT:      Type Promotion
 ; CHECK-NEXT:      CodeGen Prepare
 ; CHECK-NEXT:      Dominator Tree Construction
 ; CHECK-NEXT:      Exception handling preparation
 ; CHECK-NEXT:      Merge internal globals
 ; CHECK-NEXT:      Natural Loop Information
 ; CHECK-NEXT:      Scalar Evolution Analysis
 ; CHECK-NEXT:      Lazy Branch Probability Analysis

llvm/test/CodeGen/Generic/fp-builtin-intrinsics.ll

+; RUN: opt -alt-math-library=test -fpbuiltin-fn-selection -S < %s | FileCheck %s
+
+; Basic argument tests for fp-builtin intrinsics.
+; Only a few representative functions are tested.
+
+; CHECK-LABEL: @test_scalar_cr
+; CHECK: call half @__test_altmath_sqrth_cr
+; CHECK: call half @__test_altmath_rsqrth_cr
+; CHECK: call float @__test_altmath_sinf_cr
+; CHECK: call float @__test_altmath_sqrtf_cr
+; CHECK: call float @__test_altmath_rsqrtf_cr
+; CHECK: call double @__test_altmath_sin_cr
+; CHECK: call double @__test_altmath_sqrt_cr
+; CHECK: call double @__test_altmath_rsqrt_cr
+define void @test_scalar_cr(half %h, float %f, double %d) {
+entry:
+  %t1 = call half @llvm.fpbuiltin.sqrt.f16(half %h) #0
+  %t2 = call half @llvm.fpbuiltin.rsqrt.f16(half %h) #0
+  %t3 = call float @llvm.fpbuiltin.sin.f32(float %f) #0
+  %t4 = call float @llvm.fpbuiltin.sqrt.f32(float %f) #0
+  %t5 = call float @llvm.fpbuiltin.rsqrt.f32(float %f) #0
+  %t6 = call double @llvm.fpbuiltin.sin.f64(double %d) #0
+  %t7 = call double @llvm.fpbuiltin.sqrt.f64(double %d) #0
+  %t8 = call double @llvm.fpbuiltin.rsqrt.f64(double %d) #0
+  ret void
+}
+
+; CHECK-LABEL: @test_scalar_1_0
+; CHECK: call half @__test_altmath_sinh_high
+; CHECK: call half @__test_altmath_cosh_high
+; CHECK: call float @__test_altmath_sinf_high
+; CHECK: call float @__test_altmath_cosf_high
+; CHECK: call float @__test_altmath_tanf_high
+; CHECK: call float @__test_altmath_rsqrtf_high
+; CHECK: call double @__test_altmath_sin_high
+; CHECK: call double @__test_altmath_cos_high
+; CHECK: call double @__test_altmath_tan_high
+; CHECK: call double @__test_altmath_rsqrt_high
+define void @test_scalar_1_0(half %h, float %f, double %d) {
+entry:
+  %t1 = call half @llvm.fpbuiltin.sin.f16(half %h) #1
+  %t2 = call half @llvm.fpbuiltin.cos.f16(half %h) #1
+  %t3 = call float @llvm.fpbuiltin.sin.f32(float %f) #1
+  %t4 = call float @llvm.fpbuiltin.cos.f32(float %f) #1
+  %t5 = call float @llvm.fpbuiltin.tan.f32(float %f) #1
+  %t6 = call float @llvm.fpbuiltin.rsqrt.f32(float %f) #1
+  %t7 = call double @llvm.fpbuiltin.sin.f64(double %d) #1
+  %t8 = call double @llvm.fpbuiltin.cos.f64(double %d) #1
+  %t9 = call double @llvm.fpbuiltin.tan.f64(double %d) #1
+  %t10 = call double @llvm.fpbuiltin.rsqrt.f64(double %d) #1
+  ret void
+}
+
+; CHECK-LABEL: @test_scalar_2_5
+; CHECK: call half @__test_altmath_fdivh_med
+; CHECK: call float @__test_altmath_fdivf_med
+; CHECK: call float @__test_altmath_sqrtf_med
+; CHECK: call double @__test_altmath_fdiv_med
+; CHECK: call double @__test_altmath_sqrt_med
+define void @test_scalar_2_5(half %h1, half %h2, float %f1, float %f2,
+                            double %d1, double %d2) {
+entry:
+  %t1 = call half @llvm.fpbuiltin.fdiv.f16(half %h1, half %h2) #2
+  %t2 = call float @llvm.fpbuiltin.fdiv.f32(float %f1, float %f2) #2
+  %t3 = call float @llvm.fpbuiltin.sqrt.f32(float %f1) #2
+  %t4 = call double @llvm.fpbuiltin.fdiv.f64(double %d1, double %d2) #2
+  %t5 = call double @llvm.fpbuiltin.sqrt.f64(double %d1) #2
+  ret void
+}
+
+; CHECK-LABEL: @test_scalar_4_0
+; CHECK: call half @__test_altmath_cosh_med
+; CHECK: call float @__test_altmath_cosf_med
+; CHECK: call double @__test_altmath_cos_med
+define void @test_scalar_4_0(half %h, float %f, double %d) {
+entry:
+  %t1 = call half @llvm.fpbuiltin.cos.f16(half %h) #3
+  %t2 = call float @llvm.fpbuiltin.cos.f32(float %f) #3
+  %t3 = call double @llvm.fpbuiltin.cos.f64(double %d) #3
+  ret void
+}
+
+; CHECK-LABEL: @test_scalar_4096
+; CHECK: call float @__test_altmath_rsqrtf_low
+; CHECK: call double @__test_altmath_rsqrt_low
+define void @test_scalar_4096(float %f, double %d) {
+entry:
+  %t6 = call float @llvm.fpbuiltin.rsqrt.f32(float %f) #4
+  %t10 = call double @llvm.fpbuiltin.rsqrt.f64(double %d) #4
+  ret void
+}
+
+; CHECK-LABEL: @test_vector_1_0
+; CHECK: call <4 x float> @__test_altmath_sinf4_high
+; CHECK: call <4 x float> @__test_altmath_cosf4_high
+; CHECK: call <8 x float> @__test_altmath_sinf8_high
+; CHECK: call <8 x float> @__test_altmath_cosf8_high
+; CHECK: call <2 x double> @__test_altmath_sin2_high
+; CHECK: call <2 x double> @__test_altmath_cos2_high
+define void @test_vector_1_0(<4 x float> %v4f, <8 x float> %v8f, <2 x double> %vd) {
+entry:
+  %t1 = call <4 x float> @llvm.fpbuiltin.sin.v4f32(<4 x float> %v4f) #1
+  %t2 = call <4 x float> @llvm.fpbuiltin.cos.v4f32(<4 x float> %v4f) #1
+  %t3 = call <8 x float> @llvm.fpbuiltin.sin.v8f32(<8 x float> %v8f) #1
+  %t4 = call <8 x float> @llvm.fpbuiltin.cos.v8f32(<8 x float> %v8f) #1
+  %t5 = call <2 x double> @llvm.fpbuiltin.sin.v2f64(<2 x double> %vd) #1
+  %t6 = call <2 x double> @llvm.fpbuiltin.cos.v2f64(<2 x double> %vd) #1
+  ret void
+}
+
+; TODO: Add a test with different vector sizes of the same base type
+
+
+; Test cases where the only available implementations are more accurate than
+;   the required accuracy (3.5)
+; CHECK-LABEL: @test_scalar_inexact
+; CHECK: call half @__test_altmath_fdivh_med
+; CHECK: call half @__test_altmath_sinh_high
+; CHECK: call half @__test_altmath_cosh_high
+; CHECK: call half @__test_altmath_sqrth_cr
+; CHECK: call half @__test_altmath_rsqrth_cr
+; CHECK: call float @__test_altmath_fdivf_med
+; CHECK: call float @__test_altmath_sinf_high
+; CHECK: call float @__test_altmath_cosf_high
+; CHECK: call float @__test_altmath_tanf_high
+; CHECK: call float @__test_altmath_sqrtf_med
+; CHECK: call float @__test_altmath_rsqrtf_high
+; CHECK: call double @__test_altmath_fdiv_med
+; CHECK: call double @__test_altmath_sin_high
+; CHECK: call double @__test_altmath_cos_high
+; CHECK: call double @__test_altmath_tan_high
+; CHECK: call double @__test_altmath_sqrt_med
+; CHECK: call double @__test_altmath_rsqrt_high
+define void @test_scalar_inexact(half %h1, half %h2, float %f1, float %f2,
+                                 double %d1, double %d2) {
+entry:
+  %t1 = call half @llvm.fpbuiltin.fdiv.f16(half %h1, half %h2) #5
+  %t2 = call half @llvm.fpbuiltin.sin.f16(half %h1) #5
+  %t3 = call half @llvm.fpbuiltin.cos.f16(half %h1) #5
+  %t4 = call half @llvm.fpbuiltin.sqrt.f16(half %h1) #5
+  %t5 = call half @llvm.fpbuiltin.rsqrt.f16(half %h1) #5
+  %t6 = call float @llvm.fpbuiltin.fdiv.f32(float %f1, float %f2) #5
+  %t7 = call float @llvm.fpbuiltin.sin.f32(float %f1) #5
+  %t8 = call float @llvm.fpbuiltin.cos.f32(float %f1) #5
+  %t9 = call float @llvm.fpbuiltin.tan.f32(float %f1) #5
+  %t10 = call float @llvm.fpbuiltin.sqrt.f32(float %f1) #5
+  %t11 = call float @llvm.fpbuiltin.rsqrt.f32(float %f1) #5
+  %t12 = call double @llvm.fpbuiltin.fdiv.f64(double %d1, double %d2) #5
+  %t13 = call double @llvm.fpbuiltin.sin.f64(double %d1) #5
+  %t14 = call double @llvm.fpbuiltin.cos.f64(double %d1) #5
+  %t15 = call double @llvm.fpbuiltin.tan.f64(double %d1) #5
+  %t16 = call double @llvm.fpbuiltin.sqrt.f64(double %d1) #5
+  %t17 = call double @llvm.fpbuiltin.rsqrt.f64(double %d1) #5
+  ret void
+}
+
+declare half @llvm.fpbuiltin.fdiv.f16(half, half)
+declare half @llvm.fpbuiltin.sin.f16(half)
+declare half @llvm.fpbuiltin.cos.f16(half)
+declare half @llvm.fpbuiltin.sqrt.f16(half)
+declare half @llvm.fpbuiltin.rsqrt.f16(half)
+declare float @llvm.fpbuiltin.fdiv.f32(float, float)
+declare float @llvm.fpbuiltin.sin.f32(float)
+declare float @llvm.fpbuiltin.cos.f32(float)
+declare float @llvm.fpbuiltin.tan.f32(float)
+declare float @llvm.fpbuiltin.sqrt.f32(float)
+declare float @llvm.fpbuiltin.rsqrt.f32(float)
+declare double @llvm.fpbuiltin.fdiv.f64(double, double)
+declare double @llvm.fpbuiltin.sin.f64(double)
+declare double @llvm.fpbuiltin.cos.f64(double)
+declare double @llvm.fpbuiltin.tan.f64(double)
+declare double @llvm.fpbuiltin.sqrt.f64(double)
+declare double @llvm.fpbuiltin.rsqrt.f64(double)
+declare <4 x float> @llvm.fpbuiltin.sin.v4f32(<4 x float>)
+declare <4 x float> @llvm.fpbuiltin.cos.v4f32(<4 x float>)
+declare <8 x float> @llvm.fpbuiltin.sin.v8f32(<8 x float>)
+declare <8 x float> @llvm.fpbuiltin.cos.v8f32(<8 x float>)
+declare <2 x double> @llvm.fpbuiltin.sin.v2f64(<2 x double>)
+declare <2 x double> @llvm.fpbuiltin.cos.v2f64(<2 x double>)
+
+attributes #0 = { "fp-max-error"="0.5" }
+attributes #1 = { "fp-max-error"="1.0" }
+attributes #2 = { "fp-max-error"="2.5" }
+attributes #3 = { "fp-max-error"="4.0" }
+attributes #4 = { "fp-max-error"="4096.0" }
+attributes #5 = { "fp-max-error"="3.0" }

llvm/test/CodeGen/PowerPC/O3-pipeline.ll

 ; CHECK-NEXT:       Constant Hoisting
 ; CHECK-NEXT:       Replace intrinsics with calls to vector library
 ; CHECK-NEXT:       Partially inline calls to library functions
 ; CHECK-NEXT:       Expand vector predication intrinsics
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       TLS Variable Hoist
+; CHECK-NEXT:       FPBuiltin Function Selection
 ; CHECK-NEXT:       CodeGen Prepare
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       Scalar Evolution Analysis
 ; CHECK-NEXT:       Prepare loop for ppc preferred instruction forms
 ; CHECK-NEXT:       Scalar Evolution Analysis
 ; CHECK-NEXT:       Lazy Branch Probability Analysis

llvm/test/CodeGen/RISCV/O0-pipeline.ll

 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       Lower Garbage Collection Instructions
 ; CHECK-NEXT:       Shadow Stack GC Lowering
 ; CHECK-NEXT:       Lower constant intrinsics
 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Expand vector predication intrinsics
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
+; CHECK-NEXT:       FPBuiltin Function Selection
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       RISCV DAG->DAG Pattern Instruction Selection
 ; CHECK-NEXT:       Finalize ISel and expand pseudo-instructions
 ; CHECK-NEXT:       Local Stack Slot Allocation
 ; CHECK-NEXT:       RISCV Pre-RA pseudo instruction expansion pass

llvm/test/CodeGen/RISCV/O3-pipeline.ll

 ; CHECK-NEXT:       Constant Hoisting
 ; CHECK-NEXT:       Replace intrinsics with calls to vector library
 ; CHECK-NEXT:       Partially inline calls to library functions
 ; CHECK-NEXT:       Expand vector predication intrinsics
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       TLS Variable Hoist
+; CHECK-NEXT:       FPBuiltin Function Selection
 ; CHECK-NEXT:       CodeGen Prepare
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:     A No-Op Barrier Pass
 ; CHECK-NEXT:     FunctionPass Manager
 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
 ; CHECK-NEXT:       Module Verifier

llvm/test/CodeGen/X86/O0-pipeline.ll

 ; CHECK-NEXT:       Lower Garbage Collection Instructions
 ; CHECK-NEXT:       Shadow Stack GC Lowering
 ; CHECK-NEXT:       Lower constant intrinsics
 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Expand vector predication intrinsics
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       Expand indirectbr instructions
+; CHECK-NEXT:       FPBuiltin Function Selection
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       X86 DAG->DAG Instruction Selection
 ; CHECK-NEXT:       X86 PIC Global Base Reg Initialization
 ; CHECK-NEXT:       Finalize ISel and expand pseudo-instructions
 ; CHECK-NEXT:       Local Stack Slot Allocation

llvm/test/CodeGen/X86/opt-pipeline.ll

 ; CHECK-NEXT:       Expand vector predication intrinsics
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       TLS Variable Hoist
 ; CHECK-NEXT:       Interleaved Access Pass
 ; CHECK-NEXT:       X86 Partial Reduction
 ; CHECK-NEXT:       Expand indirectbr instructions
+; CHECK-NEXT:       FPBuiltin Function Selection
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       CodeGen Prepare
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)

llvm/tools/opt/opt.cpp

std::vector<StringRef> PassNameExact = {
       "generic-to-nvvm",      "expandmemcmp",
       "loop-reduce",          "lower-amx-type",
       "pre-amx-config",       "lower-amx-intrinsics",
       "polyhedral-info",      "print-polyhedral-info",
       "replace-with-veclib",  "jmc-instrument",
       "dot-regions",          "dot-regions-only",
       "view-regions",         "view-regions-only",
       "select-optimize",      "expand-large-div-rem",
-      "structurizecfg",       "fix-irreducible"};
+      "structurizecfg",       "fix-irreducible",
+      "fpbuiltin-fn-selection"};
   for (const auto &P : PassNamePrefix)
     if (Pass.startswith(P))
       return true;
   for (const auto &P : PassNameContain)
     if (Pass.contains(P))
       return true;
   return llvm::is_contained(PassNameExact, Pass);
 }
int main(int argc, char **argv) {
   initializeExpandReductionsPass(Registry);
   initializeExpandVectorPredicationPass(Registry);
   initializeWasmEHPreparePass(Registry);
   initializeWriteBitcodePassPass(Registry);
   initializeHardwareLoopsPass(Registry);
   initializeTypePromotionPass(Registry);
   initializeReplaceWithVeclibLegacyPass(Registry);
   initializeJMCInstrumenterPass(Registry);
+  initializeFPBuiltinFnSelectionLegacyPassPass(Registry);
 
 #ifdef BUILD_EXAMPLES
   initializeExampleIRTransforms(Registry);
 #endif
 
   SmallVector<PassPlugin, 1> PluginList;
   PassPlugins.setCallback([&](const std::string &PluginPath) {
     auto Plugin = PassPlugin::Load(PluginPath);