Introduce intrinsic llvm.isnan

[sepavloff]

Summary

Clang has builtin function '__builtin_isnan', which implements C
library function 'isnan'. This function now is implemented entirely in
clang codegen, which expands the function into set of IR operations.
There are three mechanisms by which the expansion can be made.

• The most common mechanism is using an unordered comparison made by instruction 'fcmp uno'. This simple solution is target-independent and works well in most cases. It however is not suitable if floating point exceptions are tracked. Corresponding IEEE 754 operation and C function must never raise FP exception, even if the argument is a signaling NaN. Compare instructions usually does not have such property, they raise 'invalid' exception in such case. So this mechanism is unsuitable when exception behavior is strict. In particular it could result in unexpected trapping if argument is SNaN.

• Another solution was implemented in https://reviews.llvm.org/D95948. It is used in the cases when raising FP exceptions by 'isnan' is not allowed. This solution implements 'isnan' using integer operations. It solves the problem of exceptions, but offers one solution for all targets, however some can do the check in more efficient way.

• Solution implemented by https://reviews.llvm.org/D96568 introduced a hook 'clang::TargetCodeGenInfo::testFPKind', which injects target specific code into IR. Now only SystemZ implements this hook and it generates a call to target specific intrinsic function.

Although these mechanisms allow to implement 'isnan' with enough
efficiency, expanding 'isnan' in clang has drawbacks:

• The operation 'isnan' is hidden behind generic integer operations or target-specific intrinsics. It complicates analysis and can prevent some optimizations.

• IR can be created by tools other than clang, in this case treatment of 'isnan' has to be duplicated in that tool.

Another issue with the current implementation of 'isnan' comes from the
use of options '-ffast-math' or '-fno-honor-nans'. If such option is
specified, 'fcmp uno' may be optimized to 'false'. It is valid
optimization in general, but it results in 'isnan' always returning
'false'. For example, in some libc++ implementations the following code
returns 'false':

std::isnan(std::numeric_limits<float>::quiet_NaN())

The options '-ffast-math' and '-fno-honor-nans' imply that FP operation
operands are never NaNs. This assumption however should not be applied
to the functions that check FP number properties, including 'isnan'. If
such function returns expected result instead of actually making
checks, it becomes useless in many cases. The option '-ffast-math' is
often used for performance critical code, as it can speed up execution
by the expense of manual treatment of corner cases. If 'isnan' returns
assumed result, a user cannot use it in the manual treatment of NaNs
and has to invent replacements, like making the check using integer
operations. There is a discussion in https://reviews.llvm.org/D18513#387418,
which also expresses the opinion, that limitations imposed by
'-ffast-math' should be applied only to 'math' functions but not to
'tests'.

To overcome these drawbacks, this change introduces a new IR intrinsic
function 'llvm.isnan', which realises the check as specified by IEEE-754
and C standards in target-agnostic way. During IR transformations it
does not undergo undesirable optimizations. It reaches instruction
selection, where is lowered in target-dependent way. The lowering can
vary depending on options like '-ffast-math' or '-ffp-model' so the
resulting code satisfies requested semantics.

[thopre]

Are you planning to do this for the other FP test builtin (isinf, isfinite, isinf_sign, isnormal)?

[sepavloff]

In D104854#2838468, @thopre wrote:

Are you planning to do this for the other FP test builtin (isinf, isfinite, isinf_sign, isnormal)?

Yes, they have similar problems.
If someone would like to implement them it would be nice.

[jdoerfert]

In D104854#2838495, @sepavloff wrote:

In D104854#2838468, @thopre wrote:

Are you planning to do this for the other FP test builtin (isinf, isfinite, isinf_sign, isnormal)?

Yes, they have similar problems.
If someone would like to implement them it would be nice.

From a user perspective, it seems sub-optimal to postpone the others "untile someone wants to implement them".
Once isnan behaves differently than the rest, I can see users being confused and rightfully so.
I don't have a strong opinion but I'd prefer we switch them over together.

[sepavloff]

In D104854#2838659, @jdoerfert wrote:

In D104854#2838495, @sepavloff wrote:

In D104854#2838468, @thopre wrote:

Are you planning to do this for the other FP test builtin (isinf, isfinite, isinf_sign, isnormal)?

Yes, they have similar problems.
If someone would like to implement them it would be nice.

From a user perspective, it seems sub-optimal to postpone the others "untile someone wants to implement them".
Once isnan behaves differently than the rest, I can see users being confused and rightfully so.
I don't have a strong opinion but I'd prefer we switch them over together.

Sure. I Just want to say that if someone wants or has plans to implement these functions, I appreciate these efforts. This functionality is in my plans. Considering only one function in this patch must facilitatу the review process.

[craig.topper]

Doesn't gcc also fold isnan to false under fast math? If we diverge here that means your code would only work correctly with clang.

[sepavloff]

In D104854#2838754, @craig.topper wrote:

Doesn't gcc also fold isnan to false under fast math? If we diverge here that means your code would only work correctly with clang.

GCC does the same transformation, ICC and MSVC do not: https://godbolt.org/z/ovboWqPeb. Both clang and GCC do this transformation only with optimization level > 0. With -O0 both produce code that does real check, so semantic of the produced code is different depending on optimization level, it looks more like a bug rather than feature.

There is a GCC ticket: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84949, which refers to the similar thing. It is created against to libstdc++ but the reason is in the compiler. In one on the comments there (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84949#c8) an opinion is expressed:

... My conclusion: std::numeric_limits means "has NaN bitpattern" and "has IEC559 bit layout" not "has NaNs with NaN behavior" and "has IEC559 behavior".

So this behavior is considered as incorrect.

[thopre]

LGTM (besides comment fix) but I'm not too familiar with the vector side of things

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

6987

I seem to have made a mistake when I wrote this.

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

665

Don't you net to preserve the NoFPExcept flag? Same with all the other type legalization functions

llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp

601

If this is ResultVT then the Extend created next is always a NOP. Should this be MVT::i1?

4730

I wonder if we should be using getSetCCResultType here like WidenVecOp_SETCC?

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6422

Why not pass flags to getNode?

6426

This breaks if we add constant folding for ISD::ISNAN to getNode.

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

7001

Why can't we just check < 0 here? Why do we need to shift?

[craig.topper]

llvm/lib/Target/X86/X86ISelLowering.cpp

22183

The code you copied this form was overly complicated. You can output Glue instead of MVT::i16 from XAM node and then pass that directly to FNSTSW16r in place of FPSW, FPSW.getValue(1). I have made this change to X86ISelDAGToDAG.cpp

[efriedma]

A few of the AArch64 sequences don't look ideal, but that's not the fault of your patch.

I'd like to see some test coverage for all the floating-point types (half, bfloat16, ppc_fp128).

[sepavloff]

Addressed reviewer's notes

• Math flags are set when ISNAN node is transformed,
• They are set in calls to getNode,
• WidenVecOp_ISNAN is made similar to WidenVecOp_SETCC,
• Building ISNAN node was changed,
• Fixed error in comment,
• Removed extra shift.

[sepavloff]

In D104854#2841505, @efriedma wrote:

I'd like to see some test coverage for all the floating-point types (half, bfloat16, ppc_fp128).

Tests for half are added to aarch64-fpclass.ll, new file was created to test ppc_fp128. As for bfloat16, tests are added to aarch64-fpclass.ll but only in strictfp mode. Without this attribute codegen creates unordered compare operation, which is not supported for bfloat16.

[sepavloff]

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

665

Yes, it is more correct way. Updated functions.

llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp

601

Indeed. Thank you!

4730

Rewritten the function in this way.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6422

Yes, it should be set via getNode.

6426

Now expansion occurs before getNode.

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

6987

Thank you! I updated the comment.

7001

It seems the shift is not needed.

[sepavloff]

Missed optimization in X86 codegen proposed by Craig

[sepavloff]

Rebased

[efriedma]

The options '-ffast-math' and '-fno-honor-nans' imply that FP operation
operands are never NaNs. This assumption however should not be applied
to the functions that check FP number properties, including 'isnan'. If
such function returns expected result instead of actually making
checks, it becomes useless in many cases.

This doesn't work the way you want it to, at least given the way nnan/ninf are currently defined in LangRef. It's possible to end up in a situation where isnan(x) == isnan(x) evaluates to false at runtime. It doesn't matter how you compute isnan; the problem is that the input is poisoned.

I think the right solution to this sort of issue is to insert a "freeze" in the LLVM IR, or something like that. Not sure how we'd expect users to write this in C. Suggestions welcome.

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

6981

Maybe we want to consider falling back to the integer path if SETCC isn't legal for the given operand type? We could do that as a followup, though.

7001

Instead of emitting ExpMaskV - AbsV != 0, can we just emit ExpMaskV != AbsV?

[sepavloff]

Updated patch

• Rebased,
• Applied small enhancement to integer implementation.

[tschuett]

llvm/docs/LangRef.rst

20991

[sepavloff]

In D104854#2886328, @efriedma wrote:

The options '-ffast-math' and '-fno-honor-nans' imply that FP operation
operands are never NaNs. This assumption however should not be applied
to the functions that check FP number properties, including 'isnan'. If
such function returns expected result instead of actually making
checks, it becomes useless in many cases.

This doesn't work the way you want it to, at least given the way nnan/ninf are currently defined in LangRef. It's possible to end up in a situation where isnan(x) == isnan(x) evaluates to false at runtime. It doesn't matter how you compute isnan; the problem is that the input is poisoned.

I think the right solution to this sort of issue is to insert a "freeze" in the LLVM IR, or something like that. Not sure how we'd expect users to write this in C. Suggestions welcome.

According to the documentation, nnan/ninf may be applied to fneg, fadd, fsub, fmul, fdiv, frem, fcmp, phi, select and call. We can ignore this flag for calls of isnan and similar functions. Of course, if conditions of using -ffast-math are broken, we have undefined behavior and isnan(x) != isnan(x) becomes possible, like in this code:

%c = fadd %a, nan
%r = call llvm.isnan.f32(%c)

Similarly, it is legitimate to optimize isnan in the code:

%c = fadd %a, %b
%r = call llvm.isnan.f32(%c)

In this case the result of fadd cannot be NaN, otherwise contract of -ffast-math is broken. So isnan in this case may be optimized to false.

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

6981

It makes sense, it could be beneficial for targets that have limited set of floating point comparisons. However straightforward check like:

if (Flags.hasNoFPExcept() && isOperationLegalOrCustom(ISD::SETCC, OperandVT))

results in worse code, mainly for vector types. It should be more complex check.

7001

Instead of emitting ExpMaskV - AbsV != 0, can we just emit ExpMaskV != AbsV?

Implemented.

[sepavloff]

Fixed documentation

[sepavloff]

llvm/docs/LangRef.rst

20991

Fixed, thank you.

[efriedma]

In D104854#2904826, @sepavloff wrote:

In D104854#2886328, @efriedma wrote:

The options '-ffast-math' and '-fno-honor-nans' imply that FP operation
operands are never NaNs. This assumption however should not be applied
to the functions that check FP number properties, including 'isnan'. If
such function returns expected result instead of actually making
checks, it becomes useless in many cases.

This doesn't work the way you want it to, at least given the way nnan/ninf are currently defined in LangRef. It's possible to end up in a situation where isnan(x) == isnan(x) evaluates to false at runtime. It doesn't matter how you compute isnan; the problem is that the input is poisoned.

I think the right solution to this sort of issue is to insert a "freeze" in the LLVM IR, or something like that. Not sure how we'd expect users to write this in C. Suggestions welcome.

According to the documentation, nnan/ninf may be applied to fneg, fadd, fsub, fmul, fdiv, frem, fcmp, phi, select and call. We can ignore this flag for calls of isnan and similar functions. Of course, if conditions of using -ffast-math are broken, we have undefined behavior and isnan(x) != isnan(x) becomes possible, like in this code:

Right... so how can you produce a NaN in these circumstances? You could load one from memory, I guess?

It would probably be a good idea to have an instcombine that combines away isnan on a value produced by an operation marked nnan, so we don't confuse people reading assembly into assuming isnan is actually reliable in that context.

[sepavloff]

Add InstCombine optimization for nnan operations

[sepavloff]

In D104854#2905430, @efriedma wrote:

In D104854#2904826, @sepavloff wrote:

In D104854#2886328, @efriedma wrote:

The options '-ffast-math' and '-fno-honor-nans' imply that FP operation
operands are never NaNs. This assumption however should not be applied
to the functions that check FP number properties, including 'isnan'. If
such function returns expected result instead of actually making
checks, it becomes useless in many cases.

This doesn't work the way you want it to, at least given the way nnan/ninf are currently defined in LangRef. It's possible to end up in a situation where isnan(x) == isnan(x) evaluates to false at runtime. It doesn't matter how you compute isnan; the problem is that the input is poisoned.

I think the right solution to this sort of issue is to insert a "freeze" in the LLVM IR, or something like that. Not sure how we'd expect users to write this in C. Suggestions welcome.

According to the documentation, nnan/ninf may be applied to fneg, fadd, fsub, fmul, fdiv, frem, fcmp, phi, select and call. We can ignore this flag for calls of isnan and similar functions. Of course, if conditions of using -ffast-math are broken, we have undefined behavior and isnan(x) != isnan(x) becomes possible, like in this code:

Right... so how can you produce a NaN in these circumstances? You could load one from memory, I guess?

Yes, they come from structures in memory. I think they can also come from function arguments, if some source files are compiled with option -ffast-math and some without.

It would probably be a good idea to have an instcombine that combines away isnan on a value produced by an operation marked nnan, so we don't confuse people reading assembly into assuming isnan is actually reliable in that context.

Added such transformation (file llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp, test in llvm/test/Transforms/InstSimplify/ConstProp/fpclassify.ll).

[RKSimon]

llvm/test/CodeGen/X86/x86-fpclass.ll

174

add nounwind to reduce cfi noise (other tests would benefit as well)?

llvm/test/Transforms/InstSimplify/ConstProp/fpclassify.ll

2

Use update_test_checks.py?

[sepavloff]

Updated tests

• Use update_test_checks to generate assertions in one file,
• Use nounwrap to get rid of .cfi directives.

[sepavloff]

llvm/test/CodeGen/X86/x86-fpclass.ll

174

Good hint, thank you!

llvm/test/Transforms/InstSimplify/ConstProp/fpclassify.ll

2

Done.

[RKSimon]

llvm/test/Transforms/InstCombine/fpclass.ll

30

You probably need some negative tests (no flags, ninf instead of nnan etc.)?

[sepavloff]

Added tests and changed check for legality of SETCC

• Added tests for InstCombine,
• Modified condition that checks for lowering of SETCC. Now the check only analyze SETCC for scalar type. It results in better code in most cases.

[sepavloff]

llvm/test/Transforms/InstCombine/fpclass.ll

30

Added few such tests.

[efriedma]

LGTM. (Since there have been a bunch of reviewers involved, please give a few days before you merge.)

[sepavloff]

Thanks!

[nemanjai]

This appears to have caused some failures on PPC buildbots. For example: https://lab.llvm.org/buildbot/#/builders/105/builds/13446
We are investigating this. Can you please pull this to bring the bots back to green until we track down the reason for the problem and can provide a fix?

[nemanjai]

This comment was removed by nemanjai.

[nemanjai]

Rather than reverting this commit again, I pushed 62fe3dcf98d1 to use the same expansion as before (using unordered comparison) for ppc_fp128. I am not sure if there are other types that suffer from the same problem (perhaps the Intel 80-bit long double).

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

6981

Out of curiosity, why was this added when you recognized that it results in worse code? This is certainly part of the reason for the regression for ppc_fp128.

It would appear that before this patch, we would emit a comparison for all types that are not IEEE FP types (such as ppc_fp128). Those semantics do not seem to have carried over.

[sepavloff]

In D104854#2932444, @nemanjai wrote:

Rather than reverting this commit again, I pushed 62fe3dcf98d1 to use the same expansion as before (using unordered comparison) for ppc_fp128.

Thank you very much for fixing that!

I am not sure if there are other types that suffer from the same problem (perhaps the Intel 80-bit long double).

Intel fp80 is also non-IEEE type but it got custom lowering in this patch. There is little chance for such type to work properly without custom lowering.

I am working on patch that would add custom lowering of llvm.isnan to PowerPC.

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

6981

Out of curiosity, why was this added when you recognized that it results in worse code? This is certainly part of the reason for the regression for ppc_fp128.

It is my mistake. After experiments I forgot to remove this change. I am sorry.

For x86 and AArch64 I used modified test-suite, with changes from D106804. Without proper tests it is hard to reveal why one intrinsic starts to fail.

It would appear that before this patch, we would emit a comparison for all types that are not IEEE FP types (such as ppc_fp128). Those semantics do not seem to have carried over.

The previous behavior is not correct in non-default FP environment. Unordered comparison raises Invalid exception if an operand is signaling NaN. On the other hand, isnan must never raise exceptions.

[nemanjai]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

6981

Well, if the must never raise exceptions is an IEEE-754 requirement (i.e. as noted in 5.7.2), I think it is reasonable that operations on types that do not conform to IEEE-754 are not bound by it.

[sepavloff]

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

6981

C standard defines macro isnan, of which the recent draft (http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2596.pdf, F.3p6) states:

The C classification macros fpclassify, iscanonical, isfinite, isinf, isnan, isnormal,
issignaling, issubnormal, and iszero provide the IEC 60559 operations indicated in the table above 
provided their arguments are in the format of their semantic type. Then these macros
raise no floating-point exceptions, even if an argument is a signaling NaN.

This statement is not restricted to IEEE-compatible types, so any floating point type must behave according to this statement.

[sivachandra]

llvm/lib/Target/X86/X86ISelLowering.cpp

22193

While I do not understand the code mechanics of this patch, I am mostly in agreement with the general direction of this patch. However, it has lead to a change in behavior wrt 80-bit x86 floating point numbers. Unlike the 32-bit and 64-bit floating point numbers, 80-bit numbers have an additional class of "Unsupported Numbers". Those numbers were previously treated as NaNs. Since this change uses the fxam instruction to classify the input number, that is not the case any more as the fxam instruction distinguishes between unsupported numbers and NaNs. So, to restore the previous behavior, can we extend this patch to treat unsupported numbers as NaNs? At a high level, what I am effectively saying is that we should implement isnan this way:

bool isnan(long double x) {
  uint16_t status;
  __asm__ __volatile__("fldt %0" : : "m"(x));
  __asm__ __volatile__("fxam");
  __asm__ __volatile__("fnstsw %0": "=m"(status):);
  uint16_t c0c2c3 = (status >> 8) & 0x45;
  return c0c2c3 <= 1; // This patch seems to be only doing c0c2c3 == 1 check.
}

[sepavloff]

Patch D108037 must make lowering of llvm.isnan more close to the code produced by __builtin_isnan earlier.

[spatel]

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

[sepavloff]

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

[lebedev.ri]

In D104854#2957471, @sepavloff wrote:

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

Eeek. Was there an RFC about this?
This does not sound good to me at all,
much like "let's not apply fast-math flags to x86 vector intrinsics".

[spatel]

In D104854#2957471, @sepavloff wrote:

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

I understand that the codegen was supposed to be no worse, but the difference in IR causes optimizer regressions like:
https://llvm.org/PR51556

If we want this intrinsic (and its siblings that haven't been created yet) to survive through IR, then we have to enhance IR passes to recognize the new patterns.
It would be easier to do this in steps: (1) create the intrinsic only if not in the default FP env, (2) update IR analysis/passes to recognize the intrinsic, (3) create the intrinsic in the default FP env with no FMF, (4) create the intrinsic always.

[xbolva00]

In D104854#2957529, @spatel wrote:

In D104854#2957471, @sepavloff wrote:

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

I understand that the codegen was supposed to be no worse, but the difference in IR causes optimizer regressions like:
https://llvm.org/PR51556

If we want this intrinsic (and its siblings that haven't been created yet) to survive through IR, then we have to enhance IR passes to recognize the new patterns.
It would be easier to do this in steps: (1) create the intrinsic only if not in the default FP env, (2) update IR analysis/passes to recognize the intrinsic, (3) create the intrinsic in the default FP env with no FMF, (4) create the intrinsic always.

Meanwhile this should be reverted.. also possibly in llvm 13? (Regressions vs 12)

[lebedev.ri]

In D104854#2957529, @spatel wrote:

In D104854#2957471, @sepavloff wrote:

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

I understand that the codegen was supposed to be no worse, but the difference in IR causes optimizer regressions like:
https://llvm.org/PR51556

If we want this intrinsic (and its siblings that haven't been created yet) to survive through IR, then we have to enhance IR passes to recognize the new patterns.
It would be easier to do this in steps: (1) create the intrinsic only if not in the default FP env, (2) update IR analysis/passes to recognize the intrinsic, (3) create the intrinsic in the default FP env with no FMF, (4) create the intrinsic always.

+1, but right now i'm not sold on the behavior of not optimizing away NaN checks in no-NaN's mode.
At least that part should be reconciled now. It *might* be an improvement, but it caters to expectations
of one group while catering away from the documentation and existing expectations of other groups.
This shouldn't be decided in a review, it should be driven by an RFC.

[spatel]

In D104854#2957582, @lebedev.ri wrote:

In D104854#2957529, @spatel wrote:

In D104854#2957471, @sepavloff wrote:

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

I understand that the codegen was supposed to be no worse, but the difference in IR causes optimizer regressions like:
https://llvm.org/PR51556

If we want this intrinsic (and its siblings that haven't been created yet) to survive through IR, then we have to enhance IR passes to recognize the new patterns.
It would be easier to do this in steps: (1) create the intrinsic only if not in the default FP env, (2) update IR analysis/passes to recognize the intrinsic, (3) create the intrinsic in the default FP env with no FMF, (4) create the intrinsic always.

+1, but right now i'm not sold on the behavior of not optimizing away NaN checks in no-NaN's mode.
At least that part should be reconciled now. It *might* be an improvement, but it caters to expectations
of one group while catering away from the documentation and existing expectations of other groups.
This shouldn't be decided in a review, it should be driven by an RFC.

Agree. I think a revert followed by RFC to make sure there is consensus on semantics is needed.

[kpn]

In D104854#2957490, @lebedev.ri wrote:

In D104854#2957471, @sepavloff wrote:

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

Eeek. Was there an RFC about this?
This does not sound good to me at all,
much like "let's not apply fast-math flags to x86 vector intrinsics".

We can switch into and out of the default FP environment inside a single function. If we want different behavior based on the FP environment then this should be a constrained intrinsic. Then the intrinsic would know the FP environment, or at least enough about it to know if traps and FP status bits are relevant.

I think the distinction is constrained vs non-constrained because FMF can optionally be used in both cases.

[thopre]

In D104854#2957735, @kpn wrote:

In D104854#2957490, @lebedev.ri wrote:

In D104854#2957471, @sepavloff wrote:

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

Eeek. Was there an RFC about this?
This does not sound good to me at all,
much like "let's not apply fast-math flags to x86 vector intrinsics".

We can switch into and out of the default FP environment inside a single function.

Really? The constrained intrinsic documentation claims the reverse (https://llvm.org/docs/LangRef.html#constrainedfp):

If any FP operation in a function is constrained then they all must be constrained. This is required for correct LLVM IR. Optimizations that move code around can create miscompiles if mixing of constrained and normal operations is done. The correct way to mix constrained and less constrained operations is to use the rounding mode and exception handling metadata to mark constrained intrinsics as having LLVM’s default behavior.

[lebedev.ri]

In D104854#2959680, @thopre wrote:

In D104854#2957735, @kpn wrote:

In D104854#2957490, @lebedev.ri wrote:

In D104854#2957471, @sepavloff wrote:

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

Eeek. Was there an RFC about this?
This does not sound good to me at all,
much like "let's not apply fast-math flags to x86 vector intrinsics".

We can switch into and out of the default FP environment inside a single function.

Really? The constrained intrinsic documentation claims the reverse (https://llvm.org/docs/LangRef.html#constrainedfp):

If any FP operation in a function is constrained then they all must be constrained. This is required for correct LLVM IR. Optimizations that move code around can create miscompiles if mixing of constrained and normal operations is done. The correct way to mix constrained and less constrained operations is to use the rounding mode and exception handling metadata to mark constrained intrinsics as having LLVM’s default behavior.

@sepavloff please can you undo the clang part of this change (+@hans) and post an RFC to further hash out the design here?

[sepavloff]

I posted the RFC: https://lists.llvm.org/pipermail/llvm-dev/2021-August/152257.html

Depending on the feedback I'll revert the check or modify the implementation.

[kpn]

In D104854#2959680, @thopre wrote:

In D104854#2957735, @kpn wrote:

In D104854#2957490, @lebedev.ri wrote:

In D104854#2957471, @sepavloff wrote:

In D104854#2957423, @spatel wrote:

Is it intentional that we are not canonicalizing the intrinsic call back to fcmp uno in the default FP environment?

It is lowered to unordered comparison by default. Changing llvm.isnan to fcmp uno somewhere in IR would make it possible to optimize out the latter if fast-math mode is on. Preserving semantics of isnan when fast-math is in effect was one of the goals of this change.

Eeek. Was there an RFC about this?
This does not sound good to me at all,
much like "let's not apply fast-math flags to x86 vector intrinsics".

We can switch into and out of the default FP environment inside a single function.

Really? The constrained intrinsic documentation claims the reverse (https://llvm.org/docs/LangRef.html#constrainedfp):

If any FP operation in a function is constrained then they all must be constrained. This is required for correct LLVM IR. Optimizations that move code around can create miscompiles if mixing of constrained and normal operations is done. The correct way to mix constrained and less constrained operations is to use the rounding mode and exception handling metadata to mark constrained intrinsics as having LLVM’s default behavior.

Use of constrained intrinsics does not mean that we are automatically in an alternate FP environment.

When constrained intrinsics are used and the metadata says the rounding mode is "tonearest" with exceptions set to "ignore" then that's the default FP environment. If, for example, #pragma STDC FENV_ACCESS is used in only a part of a function then the constrained intrinsics will be used in the entire function but the metadata will specify different exception or rounding behavior in the part covered by the FENV_ACCESS.

That the constrained intrinsics can state that they are in the default FP environment is what makes it safe for EarlyCSE to treat them the same as a normal FP instruction (which is assumed to be in the default FP environment). For example.

[thopre]

Ah fair enough. Thanks.

clang/lib/CodeGen/CGBuiltin.cpp

case Builtin::BI__builtin_islessgreater:
       break;
     case Builtin::BI__builtin_isunordered:
       LHS = Builder.CreateFCmpUNO(LHS, RHS, "cmp");
       break;
     }
     // ZExt bool to int type.
     return RValue::get(Builder.CreateZExt(LHS, ConvertType(E->getType())));
   }
+
   case Builtin::BI__builtin_isnan: {
     CodeGenFunction::CGFPOptionsRAII FPOptsRAII(*this, E);
     Value *V = EmitScalarExpr(E->getArg(0));
-    llvm::Type *Ty = V->getType();
-    const llvm::fltSemantics &Semantics = Ty->getFltSemantics();
-    if (!Builder.getIsFPConstrained() ||
-        Builder.getDefaultConstrainedExcept() == fp::ebIgnore ||
-        !Ty->isIEEE()) {
-      V = Builder.CreateFCmpUNO(V, V, "cmp");
-      return RValue::get(Builder.CreateZExt(V, ConvertType(E->getType())));
-    }
 
     if (Value *Result = getTargetHooks().testFPKind(V, BuiltinID, Builder, CGM))
       return RValue::get(Result);
 
-    // NaN has all exp bits set and a non zero significand. Therefore:
-    // isnan(V) == ((exp mask - (abs(V) & exp mask)) < 0)
-    unsigned bitsize = Ty->getScalarSizeInBits();
-    llvm::IntegerType *IntTy = Builder.getIntNTy(bitsize);
-    Value *IntV = Builder.CreateBitCast(V, IntTy);
-    APInt AndMask = APInt::getSignedMaxValue(bitsize);
-    Value *AbsV =
-        Builder.CreateAnd(IntV, llvm::ConstantInt::get(IntTy, AndMask));
-    APInt ExpMask = APFloat::getInf(Semantics).bitcastToAPInt();
-    Value *Sub =
-        Builder.CreateSub(llvm::ConstantInt::get(IntTy, ExpMask), AbsV);
-    // V = sign bit (Sub) <=> V = (Sub < 0)
-    V = Builder.CreateLShr(Sub, llvm::ConstantInt::get(IntTy, bitsize - 1));
-    if (bitsize > 32)
-      V = Builder.CreateTrunc(V, ConvertType(E->getType()));
-    return RValue::get(V);
+    Function *F = CGM.getIntrinsic(Intrinsic::isnan, V->getType());
+    Value *Call = Builder.CreateCall(F, V);
+    return RValue::get(Builder.CreateZExt(Call, ConvertType(E->getType())));
   }
 
   case Builtin::BI__builtin_matrix_transpose: {
     const auto *MatrixTy = E->getArg(0)->getType()->getAs<ConstantMatrixType>();
     Value *MatValue = EmitScalarExpr(E->getArg(0));
     MatrixBuilder<CGBuilderTy> MB(Builder);
     Value *Result = MB.CreateMatrixTranspose(MatValue, MatrixTy->getNumRows(),
                                              MatrixTy->getNumColumns());

clang/test/CodeGen/X86/strictfp_builtins.c

 // CHECK-LABEL: @p(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[STR_ADDR:%.*]] = alloca i8*, align 8
 // CHECK-NEXT:    [[X_ADDR:%.*]] = alloca i32, align 4
 // CHECK-NEXT:    store i8* [[STR:%.*]], i8** [[STR_ADDR]], align 8
 // CHECK-NEXT:    store i32 [[X:%.*]], i32* [[X_ADDR]], align 4
 // CHECK-NEXT:    [[TMP0:%.*]] = load i8*, i8** [[STR_ADDR]], align 8
 // CHECK-NEXT:    [[TMP1:%.*]] = load i32, i32* [[X_ADDR]], align 4
-// CHECK-NEXT:    [[CALL:%.*]] = call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([8 x i8], [8 x i8]* @.str, i64 0, i64 0), i8* [[TMP0]], i32 [[TMP1]]) [[ATTR4:#.*]]
+// CHECK-NEXT:    [[CALL:%.*]] = call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([8 x i8], [8 x i8]* @.str, i64 0, i64 0), i8* [[TMP0]], i32 [[TMP1]]) #[[ATTR3:[0-9]+]]
 // CHECK-NEXT:    ret void
 //
 void p(char *str, int x) {
   printf("%s: %d\n", str, x);
 }
 
 #define P(n,args) p(#n #args, __builtin_##n args)
 
 // CHECK-LABEL: @test_long_double_isinf(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca x86_fp80, align 16
-// CHECK-NEXT:    store x86_fp80 [[D:%.*]], x86_fp80* [[LD_ADDR]], align 16
+// CHECK-NEXT:    store x86_fp80 [[LD:%.*]], x86_fp80* [[LD_ADDR]], align 16
 // CHECK-NEXT:    [[TMP0:%.*]] = load x86_fp80, x86_fp80* [[LD_ADDR]], align 16
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast x86_fp80 [[TMP0]] to i80
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i80 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp eq i80 [[SHL1]], -18446744073709551616
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @.str.[[#STRID:1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast x86_fp80 [[TMP0]] to i80
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i80 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp eq i80 [[TMP2]], -18446744073709551616
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @.str.1, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR3]]
 // CHECK-NEXT:    ret void
 //
 void test_long_double_isinf(long double ld) {
   P(isinf, (ld));
 
   return;
 }
 
 // CHECK-LABEL: @test_long_double_isfinite(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca x86_fp80, align 16
-// CHECK-NEXT:    store x86_fp80 [[D:%.*]], x86_fp80* [[LD_ADDR]], align 16
+// CHECK-NEXT:    store x86_fp80 [[LD:%.*]], x86_fp80* [[LD_ADDR]], align 16
 // CHECK-NEXT:    [[TMP0:%.*]] = load x86_fp80, x86_fp80* [[LD_ADDR]], align 16
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast x86_fp80 [[TMP0]] to i80
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i80 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp ult i80 [[SHL1]], -18446744073709551616
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([13 x i8], [13 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast x86_fp80 [[TMP0]] to i80
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i80 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp ult i80 [[TMP2]], -18446744073709551616
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([13 x i8], [13 x i8]* @.str.2, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR3]]
 // CHECK-NEXT:    ret void
 //
 void test_long_double_isfinite(long double ld) {
   P(isfinite, (ld));
 
   return;
 }
 
 // CHECK-LABEL: @test_long_double_isnan(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca x86_fp80, align 16
-// CHECK-NEXT:    store x86_fp80 [[D:%.*]], x86_fp80* [[LD_ADDR]], align 16
+// CHECK-NEXT:    store x86_fp80 [[LD:%.*]], x86_fp80* [[LD_ADDR]], align 16
 // CHECK-NEXT:    [[TMP0:%.*]] = load x86_fp80, x86_fp80* [[LD_ADDR]], align 16
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast x86_fp80 [[TMP0]] to i80
-// CHECK-NEXT:    [[ABS:%.*]] = and i80 [[BITCAST]], 604462909807314587353087
-// CHECK-NEXT:    [[TMP1:%.*]] = sub i80 604453686435277732577280, [[ABS]]
-// CHECK-NEXT:    [[ISNAN:%.*]] = lshr i80 [[TMP1]], 79
-// CHECK-NEXT:    [[RES:%.*]] = trunc i80 [[ISNAN]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = call i1 @llvm.isnan.f80(x86_fp80 [[TMP0]]) #[[ATTR3]]
+// CHECK-NEXT:    [[TMP2:%.*]] = zext i1 [[TMP1]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @.str.3, i64 0, i64 0), i32 [[TMP2]]) #[[ATTR3]]
 // CHECK-NEXT:    ret void
 //
 void test_long_double_isnan(long double ld) {
   P(isnan, (ld));
 
   return;
 }

clang/test/CodeGen/aarch64-strictfp-builtins.c

+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
 // RUN: %clang_cc1 %s -emit-llvm -ffp-exception-behavior=maytrap -fexperimental-strict-floating-point -o - -triple arm64-none-linux-gnu | FileCheck %s
 
 // Test that the constrained intrinsics are picking up the exception
 // metadata from the AST instead of the global default from the command line.
 
 #pragma float_control(except, on)
 
 int printf(const char *, ...);
 
 // CHECK-LABEL: @p(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[STR_ADDR:%.*]] = alloca i8*, align 8
 // CHECK-NEXT:    [[X_ADDR:%.*]] = alloca i32, align 4
 // CHECK-NEXT:    store i8* [[STR:%.*]], i8** [[STR_ADDR]], align 8
 // CHECK-NEXT:    store i32 [[X:%.*]], i32* [[X_ADDR]], align 4
 // CHECK-NEXT:    [[TMP0:%.*]] = load i8*, i8** [[STR_ADDR]], align 8
 // CHECK-NEXT:    [[TMP1:%.*]] = load i32, i32* [[X_ADDR]], align 4
-// CHECK-NEXT:    [[CALL:%.*]] = call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([8 x i8], [8 x i8]* @.str, i64 0, i64 0), i8* [[TMP0]], i32 [[TMP1]])  [[ATTR4:#.*]]
+// CHECK-NEXT:    [[CALL:%.*]] = call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([8 x i8], [8 x i8]* @.str, i64 0, i64 0), i8* [[TMP0]], i32 [[TMP1]]) #[[ATTR3:[0-9]+]]
 // CHECK-NEXT:    ret void
 //
 void p(char *str, int x) {
   printf("%s: %d\n", str, x);
 }
 
 #define P(n,args) p(#n #args, __builtin_##n args)
 
 // CHECK-LABEL: @test_long_double_isinf(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca fp128, align 16
-// CHECK-NEXT:    store fp128 [[D:%.*]], fp128* [[LD_ADDR]], align 16
+// CHECK-NEXT:    store fp128 [[LD:%.*]], fp128* [[LD_ADDR]], align 16
 // CHECK-NEXT:    [[TMP0:%.*]] = load fp128, fp128* [[LD_ADDR]], align 16
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast fp128 [[TMP0]] to i128
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i128 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp eq i128 [[SHL1]], -10384593717069655257060992658440192
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @.str.[[#STRID:1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast fp128 [[TMP0]] to i128
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i128 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp eq i128 [[TMP2]], -10384593717069655257060992658440192
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @.str.1, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR3]]
 // CHECK-NEXT:    ret void
 //
 void test_long_double_isinf(long double ld) {
   P(isinf, (ld));
 
   return;
 }
 
 // CHECK-LABEL: @test_long_double_isfinite(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca fp128, align 16
-// CHECK-NEXT:    store fp128 [[D:%.*]], fp128* [[LD_ADDR]], align 16
+// CHECK-NEXT:    store fp128 [[LD:%.*]], fp128* [[LD_ADDR]], align 16
 // CHECK-NEXT:    [[TMP0:%.*]] = load fp128, fp128* [[LD_ADDR]], align 16
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast fp128 [[TMP0]] to i128
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i128 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp ult i128 [[SHL1]], -10384593717069655257060992658440192
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([13 x i8], [13 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast fp128 [[TMP0]] to i128
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i128 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp ult i128 [[TMP2]], -10384593717069655257060992658440192
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([13 x i8], [13 x i8]* @.str.2, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR3]]
 // CHECK-NEXT:    ret void
 //
 void test_long_double_isfinite(long double ld) {
   P(isfinite, (ld));
 
   return;
 }
 
 // CHECK-LABEL: @test_long_double_isnan(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca fp128, align 16
-// CHECK-NEXT:    store fp128 [[D:%.*]], fp128* [[LD_ADDR]], align 16
+// CHECK-NEXT:    store fp128 [[LD:%.*]], fp128* [[LD_ADDR]], align 16
 // CHECK-NEXT:    [[TMP0:%.*]] = load fp128, fp128* [[LD_ADDR]], align 16
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast fp128 [[TMP0]] to i128
-// CHECK-NEXT:    [[ABS:%.*]] = and i128 [[BITCAST]], 170141183460469231731687303715884105727
-// CHECK-NEXT:    [[TMP1:%.*]] = sub i128 170135991163610696904058773219554885632, [[ABS]]
-// CHECK-NEXT:    [[ISNAN:%.*]] = lshr i128 [[TMP1]], 127
-// CHECK-NEXT:    [[RES:%.*]] = trunc i128 [[ISNAN]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]])
+// CHECK-NEXT:    [[TMP1:%.*]] = call i1 @llvm.isnan.f128(fp128 [[TMP0]]) #[[ATTR3]]
+// CHECK-NEXT:    [[TMP2:%.*]] = zext i1 [[TMP1]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @.str.3, i64 0, i64 0), i32 [[TMP2]]) #[[ATTR3]]
 // CHECK-NEXT:    ret void
 //
 void test_long_double_isnan(long double ld) {
   P(isnan, (ld));
 
   return;
 }

clang/test/CodeGen/strictfp_builtins.c

 // CHECK-LABEL: @p(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[STR_ADDR:%.*]] = alloca i8*, align 8
 // CHECK-NEXT:    [[X_ADDR:%.*]] = alloca i32, align 4
 // CHECK-NEXT:    store i8* [[STR:%.*]], i8** [[STR_ADDR]], align 8
 // CHECK-NEXT:    store i32 [[X:%.*]], i32* [[X_ADDR]], align 4
 // CHECK-NEXT:    [[TMP0:%.*]] = load i8*, i8** [[STR_ADDR]], align 8
 // CHECK-NEXT:    [[TMP1:%.*]] = load i32, i32* [[X_ADDR]], align 4
-// CHECK-NEXT:    [[CALL:%.*]] = call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([8 x i8], [8 x i8]* @.str, i64 0, i64 0), i8* [[TMP0]], i32 [[TMP1]]) [[ATTR4:#.*]]
+// CHECK-NEXT:    [[CALL:%.*]] = call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([8 x i8], [8 x i8]* @.str, i64 0, i64 0), i8* [[TMP0]], i32 [[TMP1]]) #[[ATTR5:[0-9]+]]
 // CHECK-NEXT:    ret void
 //
 void p(char *str, int x) {
   printf("%s: %d\n", str, x);
 }
 
 #define P(n,args) p(#n #args, __builtin_##n args)
 
 // CHECK-LABEL: @test_fpclassify(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[D_ADDR:%.*]] = alloca double, align 8
 // CHECK-NEXT:    store double [[D:%.*]], double* [[D_ADDR]], align 8
 // CHECK-NEXT:    [[TMP0:%.*]] = load double, double* [[D_ADDR]], align 8
-// CHECK-NEXT:    [[ISZERO:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP0]], double 0.000000e+00, metadata !"oeq", metadata !"fpexcept.strict") [[ATTR4]]
+// CHECK-NEXT:    [[ISZERO:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP0]], double 0.000000e+00, metadata !"oeq", metadata !"fpexcept.strict") #[[ATTR5]]
 // CHECK-NEXT:    br i1 [[ISZERO]], label [[FPCLASSIFY_END:%.*]], label [[FPCLASSIFY_NOT_ZERO:%.*]]
 // CHECK:       fpclassify_end:
 // CHECK-NEXT:    [[FPCLASSIFY_RESULT:%.*]] = phi i32 [ 4, [[ENTRY:%.*]] ], [ 0, [[FPCLASSIFY_NOT_ZERO]] ], [ 1, [[FPCLASSIFY_NOT_NAN:%.*]] ], [ [[TMP2:%.*]], [[FPCLASSIFY_NOT_INF:%.*]] ]
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([29 x i8], [29 x i8]* @.str.1, i64 0, i64 0), i32 [[FPCLASSIFY_RESULT]]) [[ATTR4]]
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([29 x i8], [29 x i8]* @.str.1, i64 0, i64 0), i32 [[FPCLASSIFY_RESULT]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 // CHECK:       fpclassify_not_zero:
-// CHECK-NEXT:    [[CMP:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP0]], double [[TMP0]], metadata !"uno", metadata !"fpexcept.strict") [[ATTR4]]
+// CHECK-NEXT:    [[CMP:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP0]], double [[TMP0]], metadata !"uno", metadata !"fpexcept.strict") #[[ATTR5]]
 // CHECK-NEXT:    br i1 [[CMP]], label [[FPCLASSIFY_END]], label [[FPCLASSIFY_NOT_NAN]]
 // CHECK:       fpclassify_not_nan:
-// CHECK-NEXT:    [[TMP1:%.*]] = call double @llvm.fabs.f64(double [[TMP0]]) [[ATTR5:#.*]]
-// CHECK-NEXT:    [[ISINF:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x7FF0000000000000, metadata !"oeq", metadata !"fpexcept.strict") [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = call double @llvm.fabs.f64(double [[TMP0]]) #[[ATTR6:[0-9]+]]
+// CHECK-NEXT:    [[ISINF:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x7FF0000000000000, metadata !"oeq", metadata !"fpexcept.strict") #[[ATTR5]]
 // CHECK-NEXT:    br i1 [[ISINF]], label [[FPCLASSIFY_END]], label [[FPCLASSIFY_NOT_INF]]
 // CHECK:       fpclassify_not_inf:
-// CHECK-NEXT:    [[ISNORMAL:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x10000000000000, metadata !"uge", metadata !"fpexcept.strict") [[ATTR4]]
+// CHECK-NEXT:    [[ISNORMAL:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x10000000000000, metadata !"uge", metadata !"fpexcept.strict") #[[ATTR5]]
 // CHECK-NEXT:    [[TMP2]] = select i1 [[ISNORMAL]], i32 2, i32 3
 // CHECK-NEXT:    br label [[FPCLASSIFY_END]]
 //
 void test_fpclassify(double d) {
   P(fpclassify, (0, 1, 2, 3, 4, d));
 
   return;
 }
 
 // CHECK-LABEL: @test_fp16_isinf(
 // CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca half, align 2
-// CHECK-NEXT:    store half [[H:%.*]], half* [[LD_ADDR]], align 2
-// CHECK-NEXT:    [[TMP0:%.*]] = load half, half* [[LD_ADDR]], align 2
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast half [[TMP0]] to i16
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i16 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp eq i16 [[SHL1]], -2048
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.[[#STRID:2]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[H_ADDR:%.*]] = alloca half, align 2
+// CHECK-NEXT:    store half [[H:%.*]], half* [[H_ADDR]], align 2
+// CHECK-NEXT:    [[TMP0:%.*]] = load half, half* [[H_ADDR]], align 2
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast half [[TMP0]] to i16
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i16 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp eq i16 [[TMP2]], -2048
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.2, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_fp16_isinf(__fp16 h) {
   P(isinf, (h));
 
   return;
 }
 
 // CHECK-LABEL: @test_float_isinf(
 // CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca float, align 4
-// CHECK-NEXT:    store float [[F:%.*]], float* [[LD_ADDR]], align 4
-// CHECK-NEXT:    [[TMP0:%.*]] = load float, float* [[LD_ADDR]], align 4
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast float [[TMP0]] to i32
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i32 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp eq i32 [[SHL1]], -16777216
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[F_ADDR:%.*]] = alloca float, align 4
+// CHECK-NEXT:    store float [[F:%.*]], float* [[F_ADDR]], align 4
+// CHECK-NEXT:    [[TMP0:%.*]] = load float, float* [[F_ADDR]], align 4
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast float [[TMP0]] to i32
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i32 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp eq i32 [[TMP2]], -16777216
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.3, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_float_isinf(float f) {
   P(isinf, (f));
 
   return;
 }
 
 // CHECK-LABEL: @test_double_isinf(
 // CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca double, align 8
-// CHECK-NEXT:    store double [[D:%.*]], double* [[LD_ADDR]], align 8
-// CHECK-NEXT:    [[TMP0:%.*]] = load double, double* [[LD_ADDR]], align 8
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast double [[TMP0]] to i64
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i64 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp eq i64 [[SHL1]], -9007199254740992
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[D_ADDR:%.*]] = alloca double, align 8
+// CHECK-NEXT:    store double [[D:%.*]], double* [[D_ADDR]], align 8
+// CHECK-NEXT:    [[TMP0:%.*]] = load double, double* [[D_ADDR]], align 8
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast double [[TMP0]] to i64
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i64 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp eq i64 [[TMP2]], -9007199254740992
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.4, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_double_isinf(double d) {
   P(isinf, (d));
 
   return;
 }
 
 // CHECK-LABEL: @test_fp16_isfinite(
 // CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca half, align 2
-// CHECK-NEXT:    store half [[H:%.*]], half* [[LD_ADDR]], align 2
-// CHECK-NEXT:    [[TMP0:%.*]] = load half, half* [[LD_ADDR]], align 2
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast half [[TMP0]] to i16
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i16 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp ult i16 [[SHL1]], -2048
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([12 x i8], [12 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[H_ADDR:%.*]] = alloca half, align 2
+// CHECK-NEXT:    store half [[H:%.*]], half* [[H_ADDR]], align 2
+// CHECK-NEXT:    [[TMP0:%.*]] = load half, half* [[H_ADDR]], align 2
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast half [[TMP0]] to i16
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i16 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp ult i16 [[TMP2]], -2048
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([12 x i8], [12 x i8]* @.str.5, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_fp16_isfinite(__fp16 h) {
   P(isfinite, (h));
 
   return;
 }
 
 // CHECK-LABEL: @test_float_isfinite(
 // CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca float, align 4
-// CHECK-NEXT:    store float [[F:%.*]], float* [[LD_ADDR]], align 4
-// CHECK-NEXT:    [[TMP0:%.*]] = load float, float* [[LD_ADDR]], align 4
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast float [[TMP0]] to i32
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i32 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp ult i32 [[SHL1]], -16777216
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([12 x i8], [12 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[F_ADDR:%.*]] = alloca float, align 4
+// CHECK-NEXT:    store float [[F:%.*]], float* [[F_ADDR]], align 4
+// CHECK-NEXT:    [[TMP0:%.*]] = load float, float* [[F_ADDR]], align 4
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast float [[TMP0]] to i32
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i32 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp ult i32 [[TMP2]], -16777216
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([12 x i8], [12 x i8]* @.str.6, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_float_isfinite(float f) {
   P(isfinite, (f));
 
   return;
 }
 
 // CHECK-LABEL: @test_double_isfinite(
 // CHECK-NEXT:  entry:
-// CHECK-NEXT:    [[LD_ADDR:%.*]] = alloca double, align 8
-// CHECK-NEXT:    store double [[D:%.*]], double* [[LD_ADDR]], align 8
-// CHECK-NEXT:    [[TMP0:%.*]] = load double, double* [[LD_ADDR]], align 8
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast double [[TMP0]] to i64
-// CHECK-NEXT:    [[SHL1:%.*]] = shl i64 [[BITCAST]], 1
-// CHECK-NEXT:    [[CMP:%.*]] = icmp ult i64 [[SHL1]], -9007199254740992
-// CHECK-NEXT:    [[RES:%.*]] = zext i1 [[CMP]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([12 x i8], [12 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[D_ADDR:%.*]] = alloca double, align 8
+// CHECK-NEXT:    store double [[D:%.*]], double* [[D_ADDR]], align 8
+// CHECK-NEXT:    [[TMP0:%.*]] = load double, double* [[D_ADDR]], align 8
+// CHECK-NEXT:    [[TMP1:%.*]] = bitcast double [[TMP0]] to i64
+// CHECK-NEXT:    [[TMP2:%.*]] = shl i64 [[TMP1]], 1
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp ult i64 [[TMP2]], -9007199254740992
+// CHECK-NEXT:    [[TMP4:%.*]] = zext i1 [[TMP3]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([12 x i8], [12 x i8]* @.str.7, i64 0, i64 0), i32 [[TMP4]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_double_isfinite(double d) {
   P(isfinite, (d));
 
   return;
 }
 
 // CHECK-LABEL: @test_isinf_sign(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[D_ADDR:%.*]] = alloca double, align 8
 // CHECK-NEXT:    store double [[D:%.*]], double* [[D_ADDR]], align 8
 // CHECK-NEXT:    [[TMP0:%.*]] = load double, double* [[D_ADDR]], align 8
-// CHECK-NEXT:    [[TMP1:%.*]] = call double @llvm.fabs.f64(double [[TMP0]]) [[ATTR5]]
-// CHECK-NEXT:    [[ISINF:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x7FF0000000000000, metadata !"oeq", metadata !"fpexcept.strict") [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = call double @llvm.fabs.f64(double [[TMP0]]) #[[ATTR6]]
+// CHECK-NEXT:    [[ISINF:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x7FF0000000000000, metadata !"oeq", metadata !"fpexcept.strict") #[[ATTR5]]
 // CHECK-NEXT:    [[TMP2:%.*]] = bitcast double [[TMP0]] to i64
 // CHECK-NEXT:    [[TMP3:%.*]] = icmp slt i64 [[TMP2]], 0
 // CHECK-NEXT:    [[TMP4:%.*]] = select i1 [[TMP3]], i32 -1, i32 1
 // CHECK-NEXT:    [[TMP5:%.*]] = select i1 [[ISINF]], i32 [[TMP4]], i32 0
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([14 x i8], [14 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[TMP5]]) [[ATTR4]]
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([14 x i8], [14 x i8]* @.str.8, i64 0, i64 0), i32 [[TMP5]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_isinf_sign(double d) {
   P(isinf_sign, (d));
 
   return;
 }
 
 // CHECK-LABEL: @test_fp16_isnan(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[H_ADDR:%.*]] = alloca half, align 2
 // CHECK-NEXT:    store half [[H:%.*]], half* [[H_ADDR]], align 2
 // CHECK-NEXT:    [[TMP0:%.*]] = load half, half* [[H_ADDR]], align 2
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast half [[TMP0]] to i16
-// CHECK-NEXT:    [[ABS:%.*]] = and i16 [[BITCAST]], [[#%u,0x7FFF]]
-// CHECK-NEXT:    [[TMP1:%.*]] = sub i16 [[#%u,0x7C00]], [[ABS]]
-// CHECK-NEXT:    [[ISNAN:%.*]] = lshr i16 [[TMP1]], 15
-// CHECK-NEXT:    [[RES:%.*]] = zext i16 [[ISNAN]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = call i1 @llvm.isnan.f16(half [[TMP0]]) #[[ATTR5]]
+// CHECK-NEXT:    [[TMP2:%.*]] = zext i1 [[TMP1]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.9, i64 0, i64 0), i32 [[TMP2]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_fp16_isnan(__fp16 h) {
   P(isnan, (h));
 
   return;
 }
 
 // CHECK-LABEL: @test_float_isnan(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[F_ADDR:%.*]] = alloca float, align 4
 // CHECK-NEXT:    store float [[F:%.*]], float* [[F_ADDR]], align 4
 // CHECK-NEXT:    [[TMP0:%.*]] = load float, float* [[F_ADDR]], align 4
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast float [[TMP0]] to i32
-// CHECK-NEXT:    [[ABS:%.*]] = and i32 [[BITCAST]], [[#%u,0x7FFFFFFF]]
-// CHECK-NEXT:    [[TMP1:%.*]] = sub i32 [[#%u,0x7F800000]], [[ABS]]
-// CHECK-NEXT:    [[ISNAN:%.*]] = lshr i32 [[TMP1]], 31
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[ISNAN]]) [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = call i1 @llvm.isnan.f32(float [[TMP0]]) #[[ATTR5]]
+// CHECK-NEXT:    [[TMP2:%.*]] = zext i1 [[TMP1]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.10, i64 0, i64 0), i32 [[TMP2]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_float_isnan(float f) {
   P(isnan, (f));
 
   return;
 }
 
 // CHECK-LABEL: @test_double_isnan(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[D_ADDR:%.*]] = alloca double, align 8
 // CHECK-NEXT:    store double [[D:%.*]], double* [[D_ADDR]], align 8
 // CHECK-NEXT:    [[TMP0:%.*]] = load double, double* [[D_ADDR]], align 8
-// CHECK-NEXT:    [[BITCAST:%.*]] = bitcast double [[TMP0]] to i64
-// CHECK-NEXT:    [[ABS:%.*]] = and i64 [[BITCAST]], [[#%u,0x7FFFFFFFFFFFFFFF]]
-// CHECK-NEXT:    [[TMP1:%.*]] = sub i64 [[#%u,0x7FF0000000000000]], [[ABS]]
-// CHECK-NEXT:    [[ISNAN:%.*]] = lshr i64 [[TMP1]], 63
-// CHECK-NEXT:    [[RES:%.*]] = trunc i64 [[ISNAN]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[RES]]) [[ATTR4]]
+// CHECK-NEXT:    [[TMP1:%.*]] = call i1 @llvm.isnan.f64(double [[TMP0]]) #[[ATTR5]]
+// CHECK-NEXT:    [[TMP2:%.*]] = zext i1 [[TMP1]] to i32
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str.11, i64 0, i64 0), i32 [[TMP2]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_double_isnan(double d) {
   P(isnan, (d));
 
   return;
 }
 
 // CHECK-LABEL: @test_isnormal(
 // CHECK-NEXT:  entry:
 // CHECK-NEXT:    [[D_ADDR:%.*]] = alloca double, align 8
 // CHECK-NEXT:    store double [[D:%.*]], double* [[D_ADDR]], align 8
 // CHECK-NEXT:    [[TMP0:%.*]] = load double, double* [[D_ADDR]], align 8
-// CHECK-NEXT:    [[ISEQ:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP0]], double [[TMP0]], metadata !"oeq", metadata !"fpexcept.strict") [[ATTR4]]
-// CHECK-NEXT:    [[TMP1:%.*]] = call double @llvm.fabs.f64(double [[TMP0]]) [[ATTR5]]
-// CHECK-NEXT:    [[ISINF:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x7FF0000000000000, metadata !"ult", metadata !"fpexcept.strict") [[ATTR4]]
-// CHECK-NEXT:    [[ISNORMAL:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x10000000000000, metadata !"uge", metadata !"fpexcept.strict") [[ATTR4]]
+// CHECK-NEXT:    [[ISEQ:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP0]], double [[TMP0]], metadata !"oeq", metadata !"fpexcept.strict") #[[ATTR5]]
+// CHECK-NEXT:    [[TMP1:%.*]] = call double @llvm.fabs.f64(double [[TMP0]]) #[[ATTR6]]
+// CHECK-NEXT:    [[ISINF:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x7FF0000000000000, metadata !"ult", metadata !"fpexcept.strict") #[[ATTR5]]
+// CHECK-NEXT:    [[ISNORMAL:%.*]] = call i1 @llvm.experimental.constrained.fcmp.f64(double [[TMP1]], double 0x10000000000000, metadata !"uge", metadata !"fpexcept.strict") #[[ATTR5]]
 // CHECK-NEXT:    [[AND:%.*]] = and i1 [[ISEQ]], [[ISINF]]
 // CHECK-NEXT:    [[AND1:%.*]] = and i1 [[AND]], [[ISNORMAL]]
 // CHECK-NEXT:    [[TMP2:%.*]] = zext i1 [[AND1]] to i32
-// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([12 x i8], [12 x i8]* @.str.[[#STRID:STRID+1]], i64 0, i64 0), i32 [[TMP2]]) [[ATTR4]]
+// CHECK-NEXT:    call void @p(i8* getelementptr inbounds ([12 x i8], [12 x i8]* @.str.12, i64 0, i64 0), i32 [[TMP2]]) #[[ATTR5]]
 // CHECK-NEXT:    ret void
 //
 void test_isnormal(double d) {
   P(isnormal, (d));
 
   return;
 }

llvm/docs/LangRef.rst

 """"""""""
 
 The '``llvm.set.rounding``' intrinsic sets the current rounding mode. It is
 similar to C library function 'fesetround', however this intrinsic does not
 return any value and uses platform-independent representation of IEEE rounding
 modes.
 
 
+Floating Point Test Intrinsics
+------------------------------
+
+These functions get properties of floating point values.
+
+
+'``llvm.isnan``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare i1 @llvm.isnan(<fptype> <op>)
+      declare <N x i1> @llvm.isnan(<vector-fptype> <op>)
+
+Overview:
+"""""""""
+
+The '``llvm.isnan``' intrinsic returns a boolean value or vector of boolean
+values depending on whether the value is NaN.
+
+If the operand is a floating-point scalar, then the result type is a
+boolean (:ref:`i1 <t_integer>`).
+
+If the operand is a floating-point vector, then the result type is a
+vector of boolean with the same number of elements as the operand.
+
+Arguments:
+""""""""""
+
+The argument to the '``llvm.isnan``' intrinsic must be
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>`
+of floating-point values.
+
+
+Semantics:
+""""""""""
+
+The function tests if ``op`` is NaN. If ``op`` is a vector, then the
+check is made element by element. Each test yields an :ref:`i1 <t_integer>`
+result, which is ``true``, if the value is NaN. The function never raises
+floating point exceptions.
+
+
 General Intrinsics
 ------------------
 
 This class of intrinsics is designed to be generic and has no specific
 purpose.
 
 '``llvm.var.annotation``' Intrinsic
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

llvm/include/llvm/CodeGen/ISDOpcodes.h

enum NodeType {
 
   /// INT = FGETSIGN(FP) - Return the sign bit of the specified floating point
   /// value as an integer 0/1 value.
   FGETSIGN,
 
   /// Returns platform specific canonical encoding of a floating point number.
   FCANONICALIZE,
 
+  /// Performs check of floating point number property, defined by IEEE-754. The
+  /// only operand is the floating point value to check. Returns boolean value.
+  ISNAN,
+
   /// BUILD_VECTOR(ELT0, ELT1, ELT2, ELT3,...) - Return a fixed-width vector
   /// with the specified, possibly variable, elements. The types of the
   /// operands must match the vector element type, except that integer types
   /// are allowed to be larger than the element type, in which case the
   /// operands are implicitly truncated. The types of the operands must all
   /// be the same.
   BUILD_VECTOR,
 

llvm/include/llvm/CodeGen/TargetLowering.h

   /// Expand fminnum/fmaxnum into fminnum_ieee/fmaxnum_ieee with quieted inputs.
   SDValue expandFMINNUM_FMAXNUM(SDNode *N, SelectionDAG &DAG) const;
 
   /// Expand FP_TO_[US]INT_SAT into FP_TO_[US]INT and selects or min/max.
   /// \param N Node to expand
   /// \returns The expansion result
   SDValue expandFP_TO_INT_SAT(SDNode *N, SelectionDAG &DAG) const;
 
+  /// Expand isnan depending on function attributes.
+  SDValue expandISNAN(EVT ResultVT, SDValue Op, SDNodeFlags Flags,
+                      const SDLoc &DL, SelectionDAG &DAG) const;
+
   /// Expand CTPOP nodes. Expands vector/scalar CTPOP nodes,
   /// vector nodes can only succeed if all operations are legal/custom.
   /// \param N Node to expand
   /// \param Result output after conversion
   /// \returns True, if the expansion was successful, false otherwise
   bool expandCTPOP(SDNode *N, SDValue &Result, SelectionDAG &DAG) const;
 
   /// Expand CTLZ/CTLZ_ZERO_UNDEF nodes. Expands vector/scalar CTLZ nodes,

llvm/include/llvm/IR/Intrinsics.td

 //===--------------- Access to Floating Point Environment -----------------===//
 //
 
 let IntrProperties = [IntrInaccessibleMemOnly, IntrWillReturn] in {
   def int_flt_rounds    : DefaultAttrsIntrinsic<[llvm_i32_ty], []>;
   def int_set_rounding  : DefaultAttrsIntrinsic<[], [llvm_i32_ty]>;
 }
 
+//===--------------- Floating Point Test Intrinsics -----------------------===//
+//
+
+def int_isnan
+    : DefaultAttrsIntrinsic<[LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>],
+                            [llvm_anyfloat_ty],
+                            [IntrNoMem, IntrWillReturn]>;
+
 //===--------------- Constrained Floating Point Intrinsics ----------------===//
 //
 
 let IntrProperties = [IntrInaccessibleMemOnly, IntrWillReturn] in {
   def int_experimental_constrained_fadd : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
                                                     [ LLVMMatchType<0>,
                                                       LLVMMatchType<0>,
                                                       llvm_metadata_ty,

llvm/lib/Analysis/ConstantFolding.cpp

bool llvm::canConstantFoldCallTo(const CallBase *Call, const Function *F) {
   case Intrinsic::x86_avx512_cvttss2usi64:
   case Intrinsic::x86_avx512_vcvtsd2usi32:
   case Intrinsic::x86_avx512_vcvtsd2usi64:
   case Intrinsic::x86_avx512_cvttsd2usi:
   case Intrinsic::x86_avx512_cvttsd2usi64:
     return !Call->isStrictFP();
 
   // Sign operations are actually bitwise operations, they do not raise
-  // exceptions even for SNANs.
+  // exceptions even for SNANs. The same applies to classification functions.
   case Intrinsic::fabs:
   case Intrinsic::copysign:
+  case Intrinsic::isnan:
   // Non-constrained variants of rounding operations means default FP
   // environment, they can be folded in any case.
   case Intrinsic::ceil:
   case Intrinsic::floor:
   case Intrinsic::round:
   case Intrinsic::roundeven:
   case Intrinsic::trunc:
   case Intrinsic::nearbyint:
if (IntrinsicID == Intrinsic::fptoui_sat ||
       // convertToInteger() already has the desired saturation semantics.
       APSInt Int(Ty->getIntegerBitWidth(),
                  IntrinsicID == Intrinsic::fptoui_sat);
       bool IsExact;
       U.convertToInteger(Int, APFloat::rmTowardZero, &IsExact);
       return ConstantInt::get(Ty, Int);
     }
 
+    if (IntrinsicID == Intrinsic::isnan)
+      return ConstantInt::get(Ty, U.isNaN());
+
     if (!Ty->isHalfTy() && !Ty->isFloatTy() && !Ty->isDoubleTy())
       return nullptr;
 
     // Use internal versions of these intrinsics.
 
     if (IntrinsicID == Intrinsic::nearbyint || IntrinsicID == Intrinsic::rint) {
       U.roundToIntegral(APFloat::rmNearestTiesToEven);
       return ConstantFP::get(Ty->getContext(), U);

llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

#endif
   case ISD::VECREDUCE_FMIN:
     Action = TLI.getOperationAction(
         Node->getOpcode(), Node->getOperand(0).getValueType());
     break;
   case ISD::VECREDUCE_SEQ_FADD:
     Action = TLI.getOperationAction(
         Node->getOpcode(), Node->getOperand(1).getValueType());
     break;
+  case ISD::ISNAN:
+    Action = TLI.getOperationAction(Node->getOpcode(),
+                                    Node->getOperand(0).getValueType());
+    break;
   default:
     if (Node->getOpcode() >= ISD::BUILTIN_OP_END) {
       Action = TargetLowering::Legal;
     } else {
       Action = TLI.getOperationAction(Node->getOpcode(), Node->getValueType(0));
     }
     break;
   }
case ISD::STACKRESTORE:
     break;
   case ISD::GET_DYNAMIC_AREA_OFFSET:
     Results.push_back(DAG.getConstant(0, dl, Node->getValueType(0)));
     Results.push_back(Results[0].getValue(0));
     break;
   case ISD::FCOPYSIGN:
     Results.push_back(ExpandFCOPYSIGN(Node));
     break;
+  case ISD::ISNAN:
+    if (SDValue Expanded =
+            TLI.expandISNAN(Node->getValueType(0), Node->getOperand(0),
+                            Node->getFlags(), SDLoc(Node), DAG))
+      Results.push_back(Expanded);
+    break;
   case ISD::FNEG:
     Results.push_back(ExpandFNEG(Node));
     break;
   case ISD::FABS:
     Results.push_back(ExpandFABS(Node));
     break;
   case ISD::SMIN:
   case ISD::SMAX:

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

#endif
   case ISD::FP_TO_SINT_SAT:
   case ISD::FP_TO_UINT_SAT:
                          Res = PromoteIntRes_FP_TO_XINT_SAT(N); break;
 
   case ISD::FP_TO_FP16:  Res = PromoteIntRes_FP_TO_FP16(N); break;
 
   case ISD::FLT_ROUNDS_: Res = PromoteIntRes_FLT_ROUNDS(N); break;
 
+  case ISD::ISNAN:       Res = PromoteIntRes_ISNAN(N); break;
+
   case ISD::AND:
   case ISD::OR:
   case ISD::XOR:
   case ISD::ADD:
   case ISD::SUB:
   case ISD::MUL:         Res = PromoteIntRes_SimpleIntBinOp(N); break;
 
   case ISD::SDIV:
SDValue Res =
       DAG.getNode(N->getOpcode(), dl, {NVT, MVT::Other}, N->getOperand(0));
 
   // Legalize the chain result - switch anything that used the old chain to
   // use the new one.
   ReplaceValueWith(SDValue(N, 1), Res.getValue(1));
   return Res;
 }
 
+SDValue DAGTypeLegalizer::PromoteIntRes_ISNAN(SDNode *N) {
+  SDLoc DL(N);
+  EVT ResultVT = N->getValueType(0);
+  EVT NewResultVT = TLI.getTypeToTransformTo(*DAG.getContext(), ResultVT);
+  return DAG.getNode(N->getOpcode(), DL, NewResultVT, N->getOperand(0),
+                     N->getFlags());
+}
+
 SDValue DAGTypeLegalizer::PromoteIntRes_INT_EXTEND(SDNode *N) {
   EVT NVT = TLI.getTypeToTransformTo(*DAG.getContext(), N->getValueType(0));
   SDLoc dl(N);
 
   if (getTypeAction(N->getOperand(0).getValueType())
       == TargetLowering::TypePromoteInteger) {
     SDValue Res = GetPromotedInteger(N->getOperand(0));
     assert(Res.getValueType().bitsLE(NVT) && "Extension doesn't make sense!");

llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h

private:
   SDValue PromoteIntRes_UNDEF(SDNode *N);
   SDValue PromoteIntRes_VAARG(SDNode *N);
   SDValue PromoteIntRes_VSCALE(SDNode *N);
   SDValue PromoteIntRes_XMULO(SDNode *N, unsigned ResNo);
   SDValue PromoteIntRes_ADDSUBSHLSAT(SDNode *N);
   SDValue PromoteIntRes_MULFIX(SDNode *N);
   SDValue PromoteIntRes_DIVFIX(SDNode *N);
   SDValue PromoteIntRes_FLT_ROUNDS(SDNode *N);
+  SDValue PromoteIntRes_ISNAN(SDNode *N);
   SDValue PromoteIntRes_VECREDUCE(SDNode *N);
   SDValue PromoteIntRes_ABS(SDNode *N);
   SDValue PromoteIntRes_Rotate(SDNode *N);
   SDValue PromoteIntRes_FunnelShift(SDNode *N);
 
   // Integer Operand Promotion.
   bool PromoteIntegerOperand(SDNode *N, unsigned OpNo);
   SDValue PromoteIntOp_ANY_EXTEND(SDNode *N);
private:
   SDValue ScalarizeVecRes_SCALAR_TO_VECTOR(SDNode *N);
   SDValue ScalarizeVecRes_VSELECT(SDNode *N);
   SDValue ScalarizeVecRes_SELECT(SDNode *N);
   SDValue ScalarizeVecRes_SELECT_CC(SDNode *N);
   SDValue ScalarizeVecRes_SETCC(SDNode *N);
   SDValue ScalarizeVecRes_UNDEF(SDNode *N);
   SDValue ScalarizeVecRes_VECTOR_SHUFFLE(SDNode *N);
   SDValue ScalarizeVecRes_FP_TO_XINT_SAT(SDNode *N);
+  SDValue ScalarizeVecRes_ISNAN(SDNode *N);
 
   SDValue ScalarizeVecRes_FIX(SDNode *N);
 
   // Vector Operand Scalarization: <1 x ty> -> ty.
   bool ScalarizeVectorOperand(SDNode *N, unsigned OpNo);
   SDValue ScalarizeVecOp_BITCAST(SDNode *N);
   SDValue ScalarizeVecOp_UnaryOp(SDNode *N);
   SDValue ScalarizeVecOp_UnaryOp_StrictFP(SDNode *N);
private:
 
   void SplitVecRes_BITCAST(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_BUILD_VECTOR(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_CONCAT_VECTORS(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_EXTRACT_SUBVECTOR(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_INSERT_SUBVECTOR(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_FPOWI(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_FCOPYSIGN(SDNode *N, SDValue &Lo, SDValue &Hi);
+  void SplitVecRes_ISNAN(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_INSERT_VECTOR_ELT(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_LOAD(LoadSDNode *LD, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_MLOAD(MaskedLoadSDNode *MLD, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_MGATHER(MaskedGatherSDNode *MGT, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_ScalarOp(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_STEP_VECTOR(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_SETCC(SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitVecRes_VECTOR_REVERSE(SDNode *N, SDValue &Lo, SDValue &Hi);
private:
   SDValue WidenVecOp_MSCATTER(SDNode* N, unsigned OpNo);
   SDValue WidenVecOp_SETCC(SDNode* N);
   SDValue WidenVecOp_STRICT_FSETCC(SDNode* N);
   SDValue WidenVecOp_VSELECT(SDNode *N);
 
   SDValue WidenVecOp_Convert(SDNode *N);
   SDValue WidenVecOp_FP_TO_XINT_SAT(SDNode *N);
   SDValue WidenVecOp_FCOPYSIGN(SDNode *N);
+  SDValue WidenVecOp_ISNAN(SDNode *N);
   SDValue WidenVecOp_VECREDUCE(SDNode *N);
   SDValue WidenVecOp_VECREDUCE_SEQ(SDNode *N);
 
   /// Helper function to generate a set of operations to perform
   /// a vector operation for a wider type.
   ///
   SDValue UnrollVectorOp_StrictFP(SDNode *N, unsigned ResNE);
 

llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp

#endif
   case ISD::SCALAR_TO_VECTOR:  R = ScalarizeVecRes_SCALAR_TO_VECTOR(N); break;
   case ISD::SIGN_EXTEND_INREG: R = ScalarizeVecRes_InregOp(N); break;
   case ISD::VSELECT:           R = ScalarizeVecRes_VSELECT(N); break;
   case ISD::SELECT:            R = ScalarizeVecRes_SELECT(N); break;
   case ISD::SELECT_CC:         R = ScalarizeVecRes_SELECT_CC(N); break;
   case ISD::SETCC:             R = ScalarizeVecRes_SETCC(N); break;
   case ISD::UNDEF:             R = ScalarizeVecRes_UNDEF(N); break;
   case ISD::VECTOR_SHUFFLE:    R = ScalarizeVecRes_VECTOR_SHUFFLE(N); break;
+  case ISD::ISNAN:             R = ScalarizeVecRes_ISNAN(N); break;
   case ISD::ANY_EXTEND_VECTOR_INREG:
   case ISD::SIGN_EXTEND_VECTOR_INREG:
   case ISD::ZERO_EXTEND_VECTOR_INREG:
     R = ScalarizeVecRes_VecInregOp(N);
     break;
   case ISD::ABS:
   case ISD::ANY_EXTEND:
   case ISD::BITREVERSE:
SDValue Res = DAG.getNode(ISD::SETCC, DL, MVT::i1, LHS, RHS,
                             N->getOperand(2));
   // Vectors may have a different boolean contents to scalars.  Promote the
   // value appropriately.
   ISD::NodeType ExtendCode =
       TargetLowering::getExtendForContent(TLI.getBooleanContents(OpVT));
   return DAG.getNode(ExtendCode, DL, NVT, Res);
 }
 
+SDValue DAGTypeLegalizer::ScalarizeVecRes_ISNAN(SDNode *N) {
+  SDLoc DL(N);
+  SDValue Arg = N->getOperand(0);
+  EVT ArgVT = Arg.getValueType();
+  EVT ResultVT = N->getValueType(0).getVectorElementType();
+
+  // Handle case where result is scalarized but operand is not.
+  if (getTypeAction(ArgVT) == TargetLowering::TypeScalarizeVector) {
+    Arg = GetScalarizedVector(Arg);
+  } else {
+    EVT VT = ArgVT.getVectorElementType();
+    Arg = DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, VT, Arg,
+                      DAG.getVectorIdxConstant(0, DL));
+  }
+
+  SDValue Res = DAG.getNode(ISD::ISNAN, DL, MVT::i1, Arg, N->getFlags());
+  // Vectors may have a different boolean contents to scalars.  Promote the
+  // value appropriately.
+  ISD::NodeType ExtendCode =
+      TargetLowering::getExtendForContent(TLI.getBooleanContents(ArgVT));
+  return DAG.getNode(ExtendCode, DL, ResultVT, Res);
+}
 
 //===----------------------------------------------------------------------===//
 //  Operand Vector Scalarization <1 x ty> -> ty.
 //===----------------------------------------------------------------------===//
 
 bool DAGTypeLegalizer::ScalarizeVectorOperand(SDNode *N, unsigned OpNo) {
   LLVM_DEBUG(dbgs() << "Scalarize node operand " << OpNo << ": "; N->dump(&DAG);
              dbgs() << "\n");
#endif
   case ISD::UNDEF:        SplitRes_UNDEF(N, Lo, Hi); break;
   case ISD::BITCAST:           SplitVecRes_BITCAST(N, Lo, Hi); break;
   case ISD::BUILD_VECTOR:      SplitVecRes_BUILD_VECTOR(N, Lo, Hi); break;
   case ISD::CONCAT_VECTORS:    SplitVecRes_CONCAT_VECTORS(N, Lo, Hi); break;
   case ISD::EXTRACT_SUBVECTOR: SplitVecRes_EXTRACT_SUBVECTOR(N, Lo, Hi); break;
   case ISD::INSERT_SUBVECTOR:  SplitVecRes_INSERT_SUBVECTOR(N, Lo, Hi); break;
   case ISD::FPOWI:             SplitVecRes_FPOWI(N, Lo, Hi); break;
   case ISD::FCOPYSIGN:         SplitVecRes_FCOPYSIGN(N, Lo, Hi); break;
+  case ISD::ISNAN:             SplitVecRes_ISNAN(N, Lo, Hi); break;
   case ISD::INSERT_VECTOR_ELT: SplitVecRes_INSERT_VECTOR_ELT(N, Lo, Hi); break;
   case ISD::SPLAT_VECTOR:
   case ISD::SCALAR_TO_VECTOR:
     SplitVecRes_ScalarOp(N, Lo, Hi);
     break;
   case ISD::STEP_VECTOR:
     SplitVecRes_STEP_VECTOR(N, Lo, Hi);
     break;
void DAGTypeLegalizer::SplitVecRes_FCOPYSIGN(SDNode *N, SDValue &Lo,
   else
     std::tie(RHSLo, RHSHi) = DAG.SplitVector(RHS, SDLoc(RHS));
 
 
   Lo = DAG.getNode(ISD::FCOPYSIGN, DL, LHSLo.getValueType(), LHSLo, RHSLo);
   Hi = DAG.getNode(ISD::FCOPYSIGN, DL, LHSHi.getValueType(), LHSHi, RHSHi);
 }
 
+void DAGTypeLegalizer::SplitVecRes_ISNAN(SDNode *N, SDValue &Lo, SDValue &Hi) {
+  SDLoc DL(N);
+  SDValue ArgLo, ArgHi;
+  GetSplitVector(N->getOperand(0), ArgLo, ArgHi);
+  EVT LoVT, HiVT;
+  std::tie(LoVT, HiVT) = DAG.GetSplitDestVTs(N->getValueType(0));
+
+  Lo = DAG.getNode(ISD::ISNAN, DL, LoVT, ArgLo, N->getFlags());
+  Hi = DAG.getNode(ISD::ISNAN, DL, HiVT, ArgHi, N->getFlags());
+}
+
 void DAGTypeLegalizer::SplitVecRes_InregOp(SDNode *N, SDValue &Lo,
                                            SDValue &Hi) {
   SDValue LHSLo, LHSHi;
   GetSplitVector(N->getOperand(0), LHSLo, LHSHi);
   SDLoc dl(N);
 
   EVT LoVT, HiVT;
   std::tie(LoVT, HiVT) =
#endif
   case ISD::MSTORE:             Res = WidenVecOp_MSTORE(N, OpNo); break;
   case ISD::MGATHER:            Res = WidenVecOp_MGATHER(N, OpNo); break;
   case ISD::MSCATTER:           Res = WidenVecOp_MSCATTER(N, OpNo); break;
   case ISD::SETCC:              Res = WidenVecOp_SETCC(N); break;
   case ISD::STRICT_FSETCC:
   case ISD::STRICT_FSETCCS:     Res = WidenVecOp_STRICT_FSETCC(N); break;
   case ISD::VSELECT:            Res = WidenVecOp_VSELECT(N); break;
   case ISD::FCOPYSIGN:          Res = WidenVecOp_FCOPYSIGN(N); break;
+  case ISD::ISNAN:              Res = WidenVecOp_ISNAN(N); break;
 
   case ISD::ANY_EXTEND:
   case ISD::SIGN_EXTEND:
   case ISD::ZERO_EXTEND:
     Res = WidenVecOp_EXTEND(N);
     break;
 
   case ISD::FP_EXTEND:

 
 SDValue DAGTypeLegalizer::WidenVecOp_FCOPYSIGN(SDNode *N) {
   // The result (and first input) is legal, but the second input is illegal.
   // We can't do much to fix that, so just unroll and let the extracts off of
   // the second input be widened as needed later.
   return DAG.UnrollVectorOp(N);
 }
 
+SDValue DAGTypeLegalizer::WidenVecOp_ISNAN(SDNode *N) {
+  SDLoc DL(N);
+  EVT ResultVT = N->getValueType(0);
+  SDValue WideArg = GetWidenedVector(N->getOperand(0));
+
+  // Process this node similarly to SETCC.
+  EVT WideResultVT = getSetCCResultType(WideArg.getValueType());
+  if (ResultVT.getScalarType() == MVT::i1)
+    WideResultVT = EVT::getVectorVT(*DAG.getContext(), MVT::i1,
+                                    WideResultVT.getVectorNumElements());
+
+  SDValue WideNode =
+      DAG.getNode(ISD::ISNAN, DL, WideResultVT, WideArg, N->getFlags());
+
+  // Extract the needed results from the result vector.
+  EVT ResVT =
+      EVT::getVectorVT(*DAG.getContext(), WideResultVT.getVectorElementType(),
+                       ResultVT.getVectorNumElements());
+  SDValue CC = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, ResVT, WideNode,
+                           DAG.getVectorIdxConstant(0, DL));
+
+  EVT OpVT = N->getOperand(0).getValueType();
+  ISD::NodeType ExtendCode =
+      TargetLowering::getExtendForContent(TLI.getBooleanContents(OpVT));
+  return DAG.getNode(ExtendCode, DL, ResultVT, CC);
+}
+
 SDValue DAGTypeLegalizer::WidenVecOp_Convert(SDNode *N) {
   // Since the result is legal and the input is illegal.
   EVT VT = N->getValueType(0);
   EVT EltVT = VT.getVectorElementType();
   SDLoc dl(N);
   unsigned NumElts = VT.getVectorNumElements();
   SDValue InOp = N->getOperand(N->isStrictFPOpcode() ? 1 : 0);
   assert(getTypeAction(InOp.getValueType()) ==

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

case Intrinsic::set_rounding:
     setValue(&I, Res);
     DAG.setRoot(Res.getValue(0));
     return;
   case Intrinsic::pcmarker: {
     SDValue Tmp = getValue(I.getArgOperand(0));
     DAG.setRoot(DAG.getNode(ISD::PCMARKER, sdl, MVT::Other, getRoot(), Tmp));
     return;
   }
+  case Intrinsic::isnan: {
+    const DataLayout DLayout = DAG.getDataLayout();
+    EVT DestVT = TLI.getValueType(DLayout, I.getType());
+    EVT ArgVT = TLI.getValueType(DLayout, I.getArgOperand(0)->getType());
+    MachineFunction &MF = DAG.getMachineFunction();
+    const Function &F = MF.getFunction();
+    SDValue Op = getValue(I.getArgOperand(0));
+    SDNodeFlags Flags;
+    Flags.setNoFPExcept(
+        !F.getAttributes().hasFnAttribute(llvm::Attribute::StrictFP));
+
+    // If ISD::ISNAN should be expanded, do it right now, because the expansion
+    // can use illegal types. Making expansion early allows to legalize these
+    // types prior to selection.
+    if (!TLI.isOperationLegalOrCustom(ISD::ISNAN, ArgVT)) {
+      SDValue Result = TLI.expandISNAN(DestVT, Op, Flags, sdl, DAG);
+      setValue(&I, Result);
+      return;
+    }
+
+    SDValue V = DAG.getNode(ISD::ISNAN, sdl, DestVT, Op, Flags);
+    setValue(&I, V);
+    return;
+  }
   case Intrinsic::readcyclecounter: {
     SDValue Op = getRoot();
     Res = DAG.getNode(ISD::READCYCLECOUNTER, sdl,
                       DAG.getVTList(MVT::i64, MVT::Other), Op);
     setValue(&I, Res);
     DAG.setRoot(Res.getValue(1));
     return;
   }

llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp

#endif
   case ISD::FMA:                        return "fma";
   case ISD::STRICT_FMA:                 return "strict_fma";
   case ISD::FMAD:                       return "fmad";
   case ISD::FREM:                       return "frem";
   case ISD::STRICT_FREM:                return "strict_frem";
   case ISD::FCOPYSIGN:                  return "fcopysign";
   case ISD::FGETSIGN:                   return "fgetsign";
   case ISD::FCANONICALIZE:              return "fcanonicalize";
+  case ISD::ISNAN:                      return "isnan";
   case ISD::FPOW:                       return "fpow";
   case ISD::STRICT_FPOW:                return "strict_fpow";
   case ISD::SMIN:                       return "smin";
   case ISD::SMAX:                       return "smax";
   case ISD::UMIN:                       return "umin";
   case ISD::UMAX:                       return "umax";
 
   case ISD::FPOWI:                      return "fpowi";

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

if (Node->getFlags().hasNoNaNs()) {
     Flags.setNoSignedZeros(true);
     SelCC->setFlags(Flags);
     return SelCC;
   }
 
   return SDValue();
 }
 
+SDValue TargetLowering::expandISNAN(EVT ResultVT, SDValue Op, SDNodeFlags Flags,
+                                    const SDLoc &DL, SelectionDAG &DAG) const {
+  EVT OperandVT = Op.getValueType();
+  assert(OperandVT.isFloatingPoint());
+
+  // If floating point exceptions are ignored, expand to unordered comparison.
+  if (Flags.hasNoFPExcept() &&
+      isOperationLegalOrCustom(ISD::SETCC, OperandVT.getScalarType()))
+    return DAG.getSetCC(DL, ResultVT, Op, DAG.getConstantFP(0.0, DL, OperandVT),
+                        ISD::SETUO);
+
+  // In general case use integer operations to avoid traps if argument is SNaN.
+
+  // NaN has all exp bits set and a non zero significand. Therefore:
+  // isnan(V) == exp mask < abs(V)
+  unsigned BitSize = OperandVT.getScalarSizeInBits();
+  EVT IntVT = OperandVT.changeTypeToInteger();
+  SDValue ArgV = DAG.getBitcast(IntVT, Op);
+  APInt AndMask = APInt::getSignedMaxValue(BitSize);
+  SDValue AndMaskV = DAG.getConstant(AndMask, DL, IntVT);
+  SDValue AbsV = DAG.getNode(ISD::AND, DL, IntVT, ArgV, AndMaskV);
+  EVT ScalarFloatVT = OperandVT.getScalarType();
+  const Type *FloatTy = ScalarFloatVT.getTypeForEVT(*DAG.getContext());
+  const llvm::fltSemantics &Semantics = FloatTy->getFltSemantics();
+  APInt ExpMask = APFloat::getInf(Semantics).bitcastToAPInt();
+  SDValue ExpMaskV = DAG.getConstant(ExpMask, DL, IntVT);
+  return DAG.getSetCC(DL, ResultVT, ExpMaskV, AbsV, ISD::SETLT);
+}
+
 bool TargetLowering::expandCTPOP(SDNode *Node, SDValue &Result,
                                  SelectionDAG &DAG) const {
   SDLoc dl(Node);
   EVT VT = Node->getValueType(0);
   EVT ShVT = getShiftAmountTy(VT, DAG.getDataLayout());
   SDValue Op = Node->getOperand(0);
   unsigned Len = VT.getScalarSizeInBits();
   assert(VT.isInteger() && "CTPOP not implemented for this type.");

llvm/lib/CodeGen/TargetLoweringBase.cpp

for (unsigned IM = (unsigned)ISD::PRE_INC;
       setIndexedMaskedStoreAction(IM, VT, Expand);
     }
 
     // Most backends expect to see the node which just returns the value loaded.
     setOperationAction(ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS, VT, Expand);
 
     // These operations default to expand.
     setOperationAction(ISD::FGETSIGN, VT, Expand);
+    setOperationAction(ISD::ISNAN, VT, Expand);
     setOperationAction(ISD::CONCAT_VECTORS, VT, Expand);
     setOperationAction(ISD::FMINNUM, VT, Expand);
     setOperationAction(ISD::FMAXNUM, VT, Expand);
     setOperationAction(ISD::FMINNUM_IEEE, VT, Expand);
     setOperationAction(ISD::FMAXNUM_IEEE, VT, Expand);
     setOperationAction(ISD::FMINIMUM, VT, Expand);
     setOperationAction(ISD::FMAXIMUM, VT, Expand);
     setOperationAction(ISD::FMAD, VT, Expand);

llvm/lib/Target/X86/X86ISelLowering.cpp

if (UseX87) {
     setOperationAction(ISD::FTRUNC, MVT::f80, Expand);
     setOperationAction(ISD::FRINT,  MVT::f80, Expand);
     setOperationAction(ISD::FNEARBYINT, MVT::f80, Expand);
     setOperationAction(ISD::FMA, MVT::f80, Expand);
     setOperationAction(ISD::LROUND, MVT::f80, Expand);
     setOperationAction(ISD::LLROUND, MVT::f80, Expand);
     setOperationAction(ISD::LRINT, MVT::f80, Custom);
     setOperationAction(ISD::LLRINT, MVT::f80, Custom);
+    setOperationAction(ISD::ISNAN, MVT::f80, Custom);
 
     // Handle constrained floating-point operations of scalar.
     setOperationAction(ISD::STRICT_FADD     , MVT::f80, Legal);
     setOperationAction(ISD::STRICT_FSUB     , MVT::f80, Legal);
     setOperationAction(ISD::STRICT_FMUL     , MVT::f80, Legal);
     setOperationAction(ISD::STRICT_FDIV     , MVT::f80, Legal);
     setOperationAction(ISD::STRICT_FSQRT    , MVT::f80, Legal);
     setOperationAction(ISD::STRICT_FP_EXTEND, MVT::f80, Legal);
static SDValue LowerFGETSIGN(SDValue Op, SelectionDAG &DAG) {
   MVT VecVT = (OpVT == MVT::f32 ? MVT::v4f32 : MVT::v2f64);
   SDValue Res = DAG.getNode(ISD::SCALAR_TO_VECTOR, dl, VecVT, N0);
   Res = DAG.getNode(X86ISD::MOVMSK, dl, MVT::i32, Res);
   Res = DAG.getZExtOrTrunc(Res, dl, VT);
   Res = DAG.getNode(ISD::AND, dl, VT, Res, DAG.getConstant(1, dl, VT));
   return Res;
 }
 
+static SDValue lowerISNAN(SDValue Op, SelectionDAG &DAG) {
+  SDLoc DL(Op);
+  SDValue Arg = Op.getOperand(0);
+  MVT ArgVT = Arg.getSimpleValueType();
+  MVT ResultVT = Op.getSimpleValueType();
+
+  // Determine classification of argument using instruction FXAM.
+  unsigned Opc;
+  switch (ArgVT.SimpleTy) {
+  default:
+    llvm_unreachable("Unexpected type!");
+  case MVT::f32:
+    Opc = X86::XAM_Fp32;
+    break;
+  case MVT::f64:
+    Opc = X86::XAM_Fp64;
+    break;
+  case MVT::f80:
+    Opc = X86::XAM_Fp80;
+    break;
+  }
+  SDValue Test(DAG.getMachineNode(Opc, DL, MVT::Glue, Arg), 0);
+
+  // Move FPSW to AX.
+  SDValue FNSTSW =
+      SDValue(DAG.getMachineNode(X86::FNSTSW16r, DL, MVT::i16, Test), 0);
+
+  // Extract upper 8-bits of AX.
+  SDValue Extract =
+      DAG.getTargetExtractSubreg(X86::sub_8bit_hi, DL, MVT::i8, FNSTSW);
+
+  // Mask all bits but C3, C2, C0.
+  Extract = DAG.getNode(ISD::AND, DL, MVT::i8, Extract,
+                        DAG.getConstant(0x45, DL, MVT::i8));
+
+  return DAG.getSetCC(DL, ResultVT, Extract, DAG.getConstant(1, DL, MVT::i8),
+                      ISD::CondCode::SETEQ);
+}
+
 /// Helper for creating a X86ISD::SETCC node.
 static SDValue getSETCC(X86::CondCode Cond, SDValue EFLAGS, const SDLoc &dl,
                         SelectionDAG &DAG) {
   return DAG.getNode(X86ISD::SETCC, dl, MVT::i8,
                      DAG.getTargetConstant(Cond, dl, MVT::i8), EFLAGS);
 }
 
 /// Helper for matching OR(EXTRACTELT(X,0),OR(EXTRACTELT(X,1),...))
SDValue X86TargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   case ISD::STORE:              return LowerStore(Op, Subtarget, DAG);
   case ISD::FADD:
   case ISD::FSUB:               return lowerFaddFsub(Op, DAG);
   case ISD::FROUND:             return LowerFROUND(Op, DAG);
   case ISD::FABS:
   case ISD::FNEG:               return LowerFABSorFNEG(Op, DAG);
   case ISD::FCOPYSIGN:          return LowerFCOPYSIGN(Op, DAG);
   case ISD::FGETSIGN:           return LowerFGETSIGN(Op, DAG);
+  case ISD::ISNAN:              return lowerISNAN(Op, DAG);
   case ISD::LRINT:
   case ISD::LLRINT:             return LowerLRINT_LLRINT(Op, DAG);
   case ISD::SETCC:
   case ISD::STRICT_FSETCC:
   case ISD::STRICT_FSETCCS:     return LowerSETCC(Op, DAG);
   case ISD::SETCCCARRY:         return LowerSETCCCARRY(Op, DAG);
   case ISD::SELECT:             return LowerSELECT(Op, DAG);
   case ISD::BRCOND:             return LowerBRCOND(Op, DAG);

llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp

case Intrinsic::fma: {
     // -0.0 + 0.0 = 0.0, which would not be the same as the fmul on its own.
     if (match(II->getArgOperand(2), m_NegZeroFP()) ||
         (match(II->getArgOperand(2), m_PosZeroFP()) &&
          II->getFastMathFlags().noSignedZeros()))
       return BinaryOperator::CreateFMulFMF(Src0, Src1, II);
 
     break;
   }
+  case Intrinsic::isnan: {
+    Value *Arg = II->getArgOperand(0);
+    if (const auto *Inst = dyn_cast<Instruction>(Arg)) {
+      // If argument of this intrinsic call is an instruction that has 'nnan'
+      // flag, we can assume that NaN cannot be produced, otherwise it is
+      // undefined behavior.
+      if (Inst->getFastMathFlags().noNaNs())
+        return replaceInstUsesWith(
+            *II, ConstantInt::get(II->getType(), APInt::getNullValue(1)));
+    }
+    break;
+  }
   case Intrinsic::copysign: {
     Value *Mag = II->getArgOperand(0), *Sign = II->getArgOperand(1);
     if (SignBitMustBeZero(Sign, &TLI)) {
       // If we know that the sign argument is positive, reduce to FABS:
       // copysign Mag, +Sign --> fabs Mag
       Value *Fabs = Builder.CreateUnaryIntrinsic(Intrinsic::fabs, Mag, II);
       return replaceInstUsesWith(*II, Fabs);
     }

llvm/test/CodeGen/AArch64/aarch64-fpclass.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=aarch64-none-linux-gnu -mattr=+bf16 | FileCheck %s -check-prefix=CHECK
+
+define i1 @isnan_half(half %x) nounwind {
+; CHECK-LABEL: isnan_half:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $h0 killed $h0 def $s0
+; CHECK-NEXT:    fmov w8, s0
+; CHECK-NEXT:    and w8, w8, #0x7fff
+; CHECK-NEXT:    mov w9, #31744
+; CHECK-NEXT:    cmp w8, w9
+; CHECK-NEXT:    cset w0, gt
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call i1 @llvm.isnan.f16(half %x)
+  ret i1 %0
+}
+
+define i1 @isnan_float(float %x) nounwind {
+; CHECK-LABEL: isnan_float:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    fcmp s0, s0
+; CHECK-NEXT:    cset w0, vs
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call i1 @llvm.isnan.f32(float %x)
+  ret i1 %0
+}
+
+define i1 @isnan_double(double %x) nounwind {
+; CHECK-LABEL: isnan_double:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    fcmp d0, d0
+; CHECK-NEXT:    cset w0, vs
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call i1 @llvm.isnan.f64(double %x)
+  ret i1 %0
+}
+
+define i1 @isnan_ldouble(fp128 %x) nounwind {
+; CHECK-LABEL: isnan_ldouble:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
+; CHECK-NEXT:    mov v1.16b, v0.16b
+; CHECK-NEXT:    bl __unordtf2
+; CHECK-NEXT:    cmp w0, #0
+; CHECK-NEXT:    cset w0, ne
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call i1 @llvm.isnan.f128(fp128 %x)
+  ret i1 %0
+}
+
+
+define i1 @isnan_half_strictfp(half %x) strictfp nounwind {
+; CHECK-LABEL: isnan_half_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $h0 killed $h0 def $s0
+; CHECK-NEXT:    fmov w8, s0
+; CHECK-NEXT:    and w8, w8, #0x7fff
+; CHECK-NEXT:    mov w9, #31744
+; CHECK-NEXT:    cmp w8, w9
+; CHECK-NEXT:    cset w0, gt
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call i1 @llvm.isnan.f16(half %x)
+  ret i1 %0
+}
+
+define i1 @isnan_bfloat_strictfp(bfloat %x) strictfp nounwind {
+; CHECK-LABEL: isnan_bfloat_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $h0 killed $h0 def $s0
+; CHECK-NEXT:    fmov w8, s0
+; CHECK-NEXT:    and w8, w8, #0x7fff
+; CHECK-NEXT:    mov w9, #32640
+; CHECK-NEXT:    cmp w8, w9
+; CHECK-NEXT:    cset w0, gt
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call i1 @llvm.isnan.bf16(bfloat %x)
+  ret i1 %0
+}
+
+define i1 @isnan_float_strictfp(float %x) strictfp nounwind {
+; CHECK-LABEL: isnan_float_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    fmov w8, s0
+; CHECK-NEXT:    and w8, w8, #0x7fffffff
+; CHECK-NEXT:    mov w9, #2139095040
+; CHECK-NEXT:    cmp w8, w9
+; CHECK-NEXT:    cset w0, gt
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call i1 @llvm.isnan.f32(float %x)
+  ret i1 %0
+}
+
+define i1 @isnan_double_strictfp(double %x) strictfp nounwind {
+; CHECK-LABEL: isnan_double_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    fmov x8, d0
+; CHECK-NEXT:    and x8, x8, #0x7fffffffffffffff
+; CHECK-NEXT:    mov x9, #9218868437227405312
+; CHECK-NEXT:    cmp x8, x9
+; CHECK-NEXT:    cset w0, gt
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call i1 @llvm.isnan.f64(double %x)
+  ret i1 %0
+}
+
+define i1 @isnan_ldouble_strictfp(fp128 %x) strictfp nounwind {
+; CHECK-LABEL: isnan_ldouble_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    str q0, [sp, #-16]!
+; CHECK-NEXT:    ldp x8, x9, [sp], #16
+; CHECK-NEXT:    mov x10, #9223090561878065152
+; CHECK-NEXT:    cmp x8, #0
+; CHECK-NEXT:    and x8, x9, #0x7fffffffffffffff
+; CHECK-NEXT:    cset w9, ne
+; CHECK-NEXT:    cmp x8, x10
+; CHECK-NEXT:    cset w8, gt
+; CHECK-NEXT:    csel w0, w9, w8, eq
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call i1 @llvm.isnan.f128(fp128 %x)
+  ret i1 %0
+}
+
+
+define <1 x i1> @isnan_half_vec1(<1 x half> %x) nounwind {
+; CHECK-LABEL: isnan_half_vec1:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $h0 killed $h0 def $q0
+; CHECK-NEXT:    umov w8, v0.h[0]
+; CHECK-NEXT:    and w8, w8, #0x7fff
+; CHECK-NEXT:    mov w9, #31744
+; CHECK-NEXT:    cmp w8, w9
+; CHECK-NEXT:    cset w0, gt
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f16(<1 x half> %x)
+  ret <1 x i1> %0
+}
+
+define <1 x i1> @isnan_float_vec1(<1 x float> %x) nounwind {
+; CHECK-LABEL: isnan_float_vec1:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $d0 killed $d0 def $q0
+; CHECK-NEXT:    fcmp s0, s0
+; CHECK-NEXT:    cset w0, vs
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f32(<1 x float> %x)
+  ret <1 x i1> %0
+}
+
+define <1 x i1> @isnan_double_vec1(<1 x double> %x) nounwind {
+; CHECK-LABEL: isnan_double_vec1:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    fcmp d0, d0
+; CHECK-NEXT:    cset w0, vs
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f64(<1 x double> %x)
+  ret <1 x i1> %0
+}
+
+define <1 x i1> @isnan_ldouble_vec1(<1 x fp128> %x) nounwind {
+; CHECK-LABEL: isnan_ldouble_vec1:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
+; CHECK-NEXT:    mov v1.16b, v0.16b
+; CHECK-NEXT:    bl __unordtf2
+; CHECK-NEXT:    cmp w0, #0
+; CHECK-NEXT:    cset w0, ne
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f128(<1 x fp128> %x)
+  ret <1 x i1> %0
+}
+
+
+define <2 x i1> @isnan_half_vec2(<2 x half> %x) nounwind {
+; CHECK-LABEL: isnan_half_vec2:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $d0 killed $d0 def $q0
+; CHECK-NEXT:    umov w8, v0.h[0]
+; CHECK-NEXT:    umov w9, v0.h[1]
+; CHECK-NEXT:    fmov s1, w8
+; CHECK-NEXT:    movi v0.2s, #127, msl #8
+; CHECK-NEXT:    mov v1.s[1], w9
+; CHECK-NEXT:    and v0.8b, v1.8b, v0.8b
+; CHECK-NEXT:    movi v1.2s, #124, lsl #8
+; CHECK-NEXT:    cmgt v0.2s, v0.2s, v1.2s
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f16(<2 x half> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_float_vec2(<2 x float> %x) nounwind {
+; CHECK-LABEL: isnan_float_vec2:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    fcmge v1.2s, v0.2s, #0.0
+; CHECK-NEXT:    fcmlt v0.2s, v0.2s, #0.0
+; CHECK-NEXT:    orr v0.8b, v0.8b, v1.8b
+; CHECK-NEXT:    mvn v0.8b, v0.8b
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f32(<2 x float> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_double_vec2(<2 x double> %x) nounwind {
+; CHECK-LABEL: isnan_double_vec2:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    fcmge v1.2d, v0.2d, #0.0
+; CHECK-NEXT:    fcmlt v0.2d, v0.2d, #0.0
+; CHECK-NEXT:    orr v0.16b, v0.16b, v1.16b
+; CHECK-NEXT:    mvn v0.16b, v0.16b
+; CHECK-NEXT:    xtn v0.2s, v0.2d
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f64(<2 x double> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_ldouble_vec2(<2 x fp128> %x) nounwind {
+; CHECK-LABEL: isnan_ldouble_vec2:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    sub sp, sp, #48
+; CHECK-NEXT:    str q0, [sp, #16] // 16-byte Folded Spill
+; CHECK-NEXT:    mov v0.16b, v1.16b
+; CHECK-NEXT:    str x30, [sp, #32] // 8-byte Folded Spill
+; CHECK-NEXT:    bl __unordtf2
+; CHECK-NEXT:    cmp w0, #0
+; CHECK-NEXT:    cset w8, ne
+; CHECK-NEXT:    sbfx x8, x8, #0, #1
+; CHECK-NEXT:    dup v0.2d, x8
+; CHECK-NEXT:    str q0, [sp] // 16-byte Folded Spill
+; CHECK-NEXT:    ldr q0, [sp, #16] // 16-byte Folded Reload
+; CHECK-NEXT:    mov v1.16b, v0.16b
+; CHECK-NEXT:    bl __unordtf2
+; CHECK-NEXT:    cmp w0, #0
+; CHECK-NEXT:    ldr q1, [sp] // 16-byte Folded Reload
+; CHECK-NEXT:    cset w8, ne
+; CHECK-NEXT:    sbfx x8, x8, #0, #1
+; CHECK-NEXT:    ldr x30, [sp, #32] // 8-byte Folded Reload
+; CHECK-NEXT:    dup v0.2d, x8
+; CHECK-NEXT:    zip1 v0.4s, v0.4s, v1.4s
+; CHECK-NEXT:    // kill: def $d0 killed $d0 killed $q0
+; CHECK-NEXT:    add sp, sp, #48
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f128(<2 x fp128> %x)
+  ret <2 x i1> %0
+}
+
+
+define <2 x i1> @isnan_half_vec2_strictfp(<2 x half> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_half_vec2_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $d0 killed $d0 def $q0
+; CHECK-NEXT:    umov w8, v0.h[0]
+; CHECK-NEXT:    umov w9, v0.h[1]
+; CHECK-NEXT:    fmov s1, w8
+; CHECK-NEXT:    movi v0.2s, #127, msl #8
+; CHECK-NEXT:    mov v1.s[1], w9
+; CHECK-NEXT:    and v0.8b, v1.8b, v0.8b
+; CHECK-NEXT:    movi v1.2s, #124, lsl #8
+; CHECK-NEXT:    cmgt v0.2s, v0.2s, v1.2s
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f16(<2 x half> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_bfloat_vec2_strictfp(<2 x bfloat> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_bfloat_vec2_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $d0 killed $d0 def $q0
+; CHECK-NEXT:    umov w8, v0.h[0]
+; CHECK-NEXT:    umov w9, v0.h[1]
+; CHECK-NEXT:    fmov s1, w8
+; CHECK-NEXT:    movi v0.2s, #127, msl #8
+; CHECK-NEXT:    mov w10, #32640
+; CHECK-NEXT:    mov v1.s[1], w9
+; CHECK-NEXT:    and v0.8b, v1.8b, v0.8b
+; CHECK-NEXT:    dup v1.2s, w10
+; CHECK-NEXT:    cmgt v0.2s, v0.2s, v1.2s
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2bf16(<2 x bfloat> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_float_vec2_strictfp(<2 x float> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_float_vec2_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov w8, #2139095040
+; CHECK-NEXT:    dup v1.2s, w8
+; CHECK-NEXT:    bic v0.2s, #128, lsl #24
+; CHECK-NEXT:    cmgt v0.2s, v0.2s, v1.2s
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f32(<2 x float> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_double_vec2_strictfp(<2 x double> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_double_vec2_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov x8, #9223372036854775807
+; CHECK-NEXT:    mov x9, #9218868437227405312
+; CHECK-NEXT:    dup v1.2d, x8
+; CHECK-NEXT:    and v0.16b, v0.16b, v1.16b
+; CHECK-NEXT:    dup v1.2d, x9
+; CHECK-NEXT:    cmgt v0.2d, v0.2d, v1.2d
+; CHECK-NEXT:    xtn v0.2s, v0.2d
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f64(<2 x double> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_ldouble_vec2_strictfp(<2 x fp128> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_ldouble_vec2_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    stp q0, q1, [sp, #-32]!
+; CHECK-NEXT:    ldp x11, x10, [sp, #16]
+; CHECK-NEXT:    ldp x8, x9, [sp]
+; CHECK-NEXT:    mov x12, #9223090561878065152
+; CHECK-NEXT:    and x10, x10, #0x7fffffffffffffff
+; CHECK-NEXT:    cmp x11, #0
+; CHECK-NEXT:    cset w11, ne
+; CHECK-NEXT:    cmp x10, x12
+; CHECK-NEXT:    cset w10, gt
+; CHECK-NEXT:    and x9, x9, #0x7fffffffffffffff
+; CHECK-NEXT:    csel w10, w11, w10, eq
+; CHECK-NEXT:    cmp x8, #0
+; CHECK-NEXT:    sbfx x8, x10, #0, #1
+; CHECK-NEXT:    cset w10, ne
+; CHECK-NEXT:    cmp x9, x12
+; CHECK-NEXT:    dup v0.2d, x8
+; CHECK-NEXT:    cset w8, gt
+; CHECK-NEXT:    csel w8, w10, w8, eq
+; CHECK-NEXT:    sbfx x8, x8, #0, #1
+; CHECK-NEXT:    dup v1.2d, x8
+; CHECK-NEXT:    zip1 v0.4s, v1.4s, v0.4s
+; CHECK-NEXT:    // kill: def $d0 killed $d0 killed $q0
+; CHECK-NEXT:    add sp, sp, #32
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f128(<2 x fp128> %x)
+  ret <2 x i1> %0
+}
+
+
+define <4 x i1> @isnan_half_vec4(<4 x half> %x) nounwind {
+; CHECK-LABEL: isnan_half_vec4:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    movi v1.4h, #124, lsl #8
+; CHECK-NEXT:    bic v0.4h, #128, lsl #8
+; CHECK-NEXT:    cmgt v0.4h, v0.4h, v1.4h
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f16(<4 x half> %x)
+  ret <4 x i1> %0
+}
+
+define <4 x i1> @isnan_float_vec4(<4 x float> %x) nounwind {
+; CHECK-LABEL: isnan_float_vec4:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    fcmge v1.4s, v0.4s, #0.0
+; CHECK-NEXT:    fcmlt v0.4s, v0.4s, #0.0
+; CHECK-NEXT:    orr v0.16b, v0.16b, v1.16b
+; CHECK-NEXT:    mvn v0.16b, v0.16b
+; CHECK-NEXT:    xtn v0.4h, v0.4s
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f32(<4 x float> %x)
+  ret <4 x i1> %0
+}
+
+define <4 x i1> @isnan_double_vec4(<4 x double> %x) nounwind {
+; CHECK-LABEL: isnan_double_vec4:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    fcmge v2.2d, v0.2d, #0.0
+; CHECK-NEXT:    fcmlt v0.2d, v0.2d, #0.0
+; CHECK-NEXT:    fcmge v3.2d, v1.2d, #0.0
+; CHECK-NEXT:    fcmlt v1.2d, v1.2d, #0.0
+; CHECK-NEXT:    orr v0.16b, v0.16b, v2.16b
+; CHECK-NEXT:    orr v1.16b, v1.16b, v3.16b
+; CHECK-NEXT:    mvn v0.16b, v0.16b
+; CHECK-NEXT:    xtn v0.2s, v0.2d
+; CHECK-NEXT:    mvn v1.16b, v1.16b
+; CHECK-NEXT:    xtn2 v0.4s, v1.2d
+; CHECK-NEXT:    xtn v0.4h, v0.4s
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f64(<4 x double> %x)
+  ret <4 x i1> %0
+}
+
+
+define <4 x i1> @isnan_half_vec4_strictfp(<4 x half> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_half_vec4_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    movi v1.4h, #124, lsl #8
+; CHECK-NEXT:    bic v0.4h, #128, lsl #8
+; CHECK-NEXT:    cmgt v0.4h, v0.4h, v1.4h
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f16(<4 x half> %x)
+  ret <4 x i1> %0
+}
+
+define <4 x i1> @isnan_bfloat_vec4_strictfp(<4 x bfloat> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_bfloat_vec4_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov w8, #32640
+; CHECK-NEXT:    dup v1.4h, w8
+; CHECK-NEXT:    bic v0.4h, #128, lsl #8
+; CHECK-NEXT:    cmgt v0.4h, v0.4h, v1.4h
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4bf16(<4 x bfloat> %x)
+  ret <4 x i1> %0
+}
+
+define <4 x i1> @isnan_float_vec4_strictfp(<4 x float> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_float_vec4_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov w8, #2139095040
+; CHECK-NEXT:    dup v1.4s, w8
+; CHECK-NEXT:    bic v0.4s, #128, lsl #24
+; CHECK-NEXT:    cmgt v0.4s, v0.4s, v1.4s
+; CHECK-NEXT:    xtn v0.4h, v0.4s
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f32(<4 x float> %x)
+  ret <4 x i1> %0
+}
+
+define <4 x i1> @isnan_double_vec4_strictfp(<4 x double> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_double_vec4_strictfp:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov x8, #9223372036854775807
+; CHECK-NEXT:    mov x9, #9218868437227405312
+; CHECK-NEXT:    dup v2.2d, x8
+; CHECK-NEXT:    dup v3.2d, x9
+; CHECK-NEXT:    and v0.16b, v0.16b, v2.16b
+; CHECK-NEXT:    and v1.16b, v1.16b, v2.16b
+; CHECK-NEXT:    cmgt v0.2d, v0.2d, v3.2d
+; CHECK-NEXT:    cmgt v1.2d, v1.2d, v3.2d
+; CHECK-NEXT:    xtn v0.2s, v0.2d
+; CHECK-NEXT:    xtn2 v0.4s, v1.2d
+; CHECK-NEXT:    xtn v0.4h, v0.4s
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f64(<4 x double> %x)
+  ret <4 x i1> %0
+}
+
+
+declare i1 @llvm.isnan.f16(half)
+declare i1 @llvm.isnan.bf16(bfloat)
+declare i1 @llvm.isnan.f32(float)
+declare i1 @llvm.isnan.f64(double)
+declare i1 @llvm.isnan.f128(fp128)
+declare <1 x i1> @llvm.isnan.v1f16(<1 x half>)
+declare <1 x i1> @llvm.isnan.v1bf16(<1 x bfloat>)
+declare <1 x i1> @llvm.isnan.v1f32(<1 x float>)
+declare <1 x i1> @llvm.isnan.v1f64(<1 x double>)
+declare <1 x i1> @llvm.isnan.v1f128(<1 x fp128>)
+declare <2 x i1> @llvm.isnan.v2f16(<2 x half>)
+declare <2 x i1> @llvm.isnan.v2bf16(<2 x bfloat>)
+declare <2 x i1> @llvm.isnan.v2f32(<2 x float>)
+declare <2 x i1> @llvm.isnan.v2f64(<2 x double>)
+declare <2 x i1> @llvm.isnan.v2f128(<2 x fp128>)
+declare <4 x i1> @llvm.isnan.v4f16(<4 x half>)
+declare <4 x i1> @llvm.isnan.v4bf16(<4 x bfloat>)
+declare <4 x i1> @llvm.isnan.v4f32(<4 x float>)
+declare <4 x i1> @llvm.isnan.v4f64(<4 x double>)
+declare <4 x i1> @llvm.isnan.v4f128(<4 x fp128>)

llvm/test/CodeGen/PowerPC/ppc-fpclass.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=powerpcle-unknown-linux-gnu -verify-machineinstrs -o - %s | FileCheck %s
+
+
+define i1 @isnan_float(float %x) nounwind {
+; CHECK-LABEL: isnan_float:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:    fcmpu 0, 1, 1
+; CHECK-NEXT:    li 4, 1
+; CHECK-NEXT:    bc 12, 3, .LBB0_1
+; CHECK-NEXT:    blr
+; CHECK-NEXT:  .LBB0_1: # %entry
+; CHECK-NEXT:    addi 3, 4, 0
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i1 @llvm.isnan.f32(float %x)
+  ret i1 %0
+}
+
+define i1 @isnan_double(double %x) nounwind {
+; CHECK-LABEL: isnan_double:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:    fcmpu 0, 1, 1
+; CHECK-NEXT:    li 4, 1
+; CHECK-NEXT:    bc 12, 3, .LBB1_1
+; CHECK-NEXT:    blr
+; CHECK-NEXT:  .LBB1_1: # %entry
+; CHECK-NEXT:    addi 3, 4, 0
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i1 @llvm.isnan.f64(double %x)
+  ret i1 %0
+}
+
+define i1 @isnan_ldouble(ppc_fp128 %x) nounwind {
+; CHECK-LABEL: isnan_ldouble:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stwu 1, -32(1)
+; CHECK-NEXT:    stfd 1, 16(1)
+; CHECK-NEXT:    lis 3, 32752
+; CHECK-NEXT:    lwz 4, 20(1)
+; CHECK-NEXT:    stfd 2, 24(1)
+; CHECK-NEXT:    lwz 5, 28(1)
+; CHECK-NEXT:    cmplw 1, 4, 3
+; CHECK-NEXT:    lwz 3, 24(1)
+; CHECK-NEXT:    xoris 4, 4, 32752
+; CHECK-NEXT:    lwz 6, 16(1)
+; CHECK-NEXT:    clrlwi. 5, 5, 1
+; CHECK-NEXT:    cmplwi 5, 5, 0
+; CHECK-NEXT:    crandc 24, 1, 22
+; CHECK-NEXT:    cmpwi 3, 0
+; CHECK-NEXT:    crandc 20, 22, 2
+; CHECK-NEXT:    cmpwi 6, 0
+; CHECK-NEXT:    cmplwi 7, 4, 0
+; CHECK-NEXT:    or 3, 3, 5
+; CHECK-NEXT:    crandc 21, 5, 30
+; CHECK-NEXT:    crandc 22, 30, 2
+; CHECK-NEXT:    cmplwi 3, 0
+; CHECK-NEXT:    cror 20, 20, 24
+; CHECK-NEXT:    cror 21, 22, 21
+; CHECK-NEXT:    crandc 20, 20, 2
+; CHECK-NEXT:    crand 21, 2, 21
+; CHECK-NEXT:    crnor 20, 21, 20
+; CHECK-NEXT:    li 3, 1
+; CHECK-NEXT:    bc 12, 20, .LBB2_1
+; CHECK-NEXT:    b .LBB2_2
+; CHECK-NEXT:  .LBB2_1: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:  .LBB2_2: # %entry
+; CHECK-NEXT:    addi 1, 1, 32
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i1 @llvm.isnan.ppcf128(ppc_fp128 %x)
+  ret i1 %0
+}
+
+
+define i1 @isnan_float_strictfp(float %x) strictfp nounwind {
+; CHECK-LABEL: isnan_float_strictfp:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stwu 1, -16(1)
+; CHECK-NEXT:    stfs 1, 12(1)
+; CHECK-NEXT:    lis 3, 32640
+; CHECK-NEXT:    lwz 4, 12(1)
+; CHECK-NEXT:    clrlwi 4, 4, 1
+; CHECK-NEXT:    cmpw 4, 3
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:    li 4, 1
+; CHECK-NEXT:    bc 12, 1, .LBB3_1
+; CHECK-NEXT:    b .LBB3_2
+; CHECK-NEXT:  .LBB3_1: # %entry
+; CHECK-NEXT:    addi 3, 4, 0
+; CHECK-NEXT:  .LBB3_2: # %entry
+; CHECK-NEXT:    addi 1, 1, 16
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i1 @llvm.isnan.f32(float %x)
+  ret i1 %0
+}
+
+define i1 @isnan_double_strictfp(double %x) strictfp nounwind {
+; CHECK-LABEL: isnan_double_strictfp:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stwu 1, -16(1)
+; CHECK-NEXT:    stfd 1, 8(1)
+; CHECK-NEXT:    lis 3, 32752
+; CHECK-NEXT:    lwz 4, 12(1)
+; CHECK-NEXT:    lwz 5, 8(1)
+; CHECK-NEXT:    clrlwi 4, 4, 1
+; CHECK-NEXT:    cmpw 4, 3
+; CHECK-NEXT:    xoris 3, 4, 32752
+; CHECK-NEXT:    cmplwi 1, 3, 0
+; CHECK-NEXT:    crandc 20, 1, 6
+; CHECK-NEXT:    cmpwi 5, 0
+; CHECK-NEXT:    crandc 21, 6, 2
+; CHECK-NEXT:    crnor 20, 21, 20
+; CHECK-NEXT:    li 3, 1
+; CHECK-NEXT:    bc 12, 20, .LBB4_1
+; CHECK-NEXT:    b .LBB4_2
+; CHECK-NEXT:  .LBB4_1: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:  .LBB4_2: # %entry
+; CHECK-NEXT:    addi 1, 1, 16
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i1 @llvm.isnan.f64(double %x)
+  ret i1 %0
+}
+
+define i1 @isnan_ldouble_strictfp(ppc_fp128 %x) strictfp nounwind {
+; CHECK-LABEL: isnan_ldouble_strictfp:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stwu 1, -32(1)
+; CHECK-NEXT:    stfd 1, 16(1)
+; CHECK-NEXT:    lis 3, 32752
+; CHECK-NEXT:    lwz 4, 20(1)
+; CHECK-NEXT:    stfd 2, 24(1)
+; CHECK-NEXT:    lwz 5, 28(1)
+; CHECK-NEXT:    cmplw 1, 4, 3
+; CHECK-NEXT:    lwz 3, 24(1)
+; CHECK-NEXT:    xoris 4, 4, 32752
+; CHECK-NEXT:    lwz 6, 16(1)
+; CHECK-NEXT:    clrlwi. 5, 5, 1
+; CHECK-NEXT:    cmplwi 5, 5, 0
+; CHECK-NEXT:    crandc 24, 1, 22
+; CHECK-NEXT:    cmpwi 3, 0
+; CHECK-NEXT:    crandc 20, 22, 2
+; CHECK-NEXT:    cmpwi 6, 0
+; CHECK-NEXT:    cmplwi 7, 4, 0
+; CHECK-NEXT:    or 3, 3, 5
+; CHECK-NEXT:    crandc 21, 5, 30
+; CHECK-NEXT:    crandc 22, 30, 2
+; CHECK-NEXT:    cmplwi 3, 0
+; CHECK-NEXT:    cror 20, 20, 24
+; CHECK-NEXT:    cror 21, 22, 21
+; CHECK-NEXT:    crandc 20, 20, 2
+; CHECK-NEXT:    crand 21, 2, 21
+; CHECK-NEXT:    crnor 20, 21, 20
+; CHECK-NEXT:    li 3, 1
+; CHECK-NEXT:    bc 12, 20, .LBB5_1
+; CHECK-NEXT:    b .LBB5_2
+; CHECK-NEXT:  .LBB5_1: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:  .LBB5_2: # %entry
+; CHECK-NEXT:    addi 1, 1, 32
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i1 @llvm.isnan.ppcf128(ppc_fp128 %x)
+  ret i1 %0
+}
+
+
+define <1 x i1> @isnan_float_vec1(<1 x float> %x) nounwind {
+; CHECK-LABEL: isnan_float_vec1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:    fcmpu 0, 1, 1
+; CHECK-NEXT:    li 4, 1
+; CHECK-NEXT:    bc 12, 3, .LBB6_1
+; CHECK-NEXT:    blr
+; CHECK-NEXT:  .LBB6_1: # %entry
+; CHECK-NEXT:    addi 3, 4, 0
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f32(<1 x float> %x)
+  ret <1 x i1> %0
+}
+
+define <1 x i1> @isnan_double_vec1(<1 x double> %x) nounwind {
+; CHECK-LABEL: isnan_double_vec1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:    fcmpu 0, 1, 1
+; CHECK-NEXT:    li 4, 1
+; CHECK-NEXT:    bc 12, 3, .LBB7_1
+; CHECK-NEXT:    blr
+; CHECK-NEXT:  .LBB7_1: # %entry
+; CHECK-NEXT:    addi 3, 4, 0
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f64(<1 x double> %x)
+  ret <1 x i1> %0
+}
+
+define <1 x i1> @isnan_ldouble_vec1(<1 x ppc_fp128> %x) nounwind {
+; CHECK-LABEL: isnan_ldouble_vec1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stwu 1, -32(1)
+; CHECK-NEXT:    stfd 1, 16(1)
+; CHECK-NEXT:    lis 3, 32752
+; CHECK-NEXT:    lwz 4, 20(1)
+; CHECK-NEXT:    stfd 2, 24(1)
+; CHECK-NEXT:    lwz 5, 28(1)
+; CHECK-NEXT:    cmplw 1, 4, 3
+; CHECK-NEXT:    lwz 3, 24(1)
+; CHECK-NEXT:    xoris 4, 4, 32752
+; CHECK-NEXT:    lwz 6, 16(1)
+; CHECK-NEXT:    clrlwi. 5, 5, 1
+; CHECK-NEXT:    cmplwi 5, 5, 0
+; CHECK-NEXT:    crandc 24, 1, 22
+; CHECK-NEXT:    cmpwi 3, 0
+; CHECK-NEXT:    crandc 20, 22, 2
+; CHECK-NEXT:    cmpwi 6, 0
+; CHECK-NEXT:    cmplwi 7, 4, 0
+; CHECK-NEXT:    or 3, 3, 5
+; CHECK-NEXT:    crandc 21, 5, 30
+; CHECK-NEXT:    crandc 22, 30, 2
+; CHECK-NEXT:    cmplwi 3, 0
+; CHECK-NEXT:    cror 20, 20, 24
+; CHECK-NEXT:    cror 21, 22, 21
+; CHECK-NEXT:    crandc 20, 20, 2
+; CHECK-NEXT:    crand 21, 2, 21
+; CHECK-NEXT:    crnor 20, 21, 20
+; CHECK-NEXT:    li 3, 1
+; CHECK-NEXT:    bc 12, 20, .LBB8_1
+; CHECK-NEXT:    b .LBB8_2
+; CHECK-NEXT:  .LBB8_1: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:  .LBB8_2: # %entry
+; CHECK-NEXT:    addi 1, 1, 32
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1ppcf128(<1 x ppc_fp128> %x)
+  ret <1 x i1> %0
+}
+
+
+define <2 x i1> @isnan_float_vec2(<2 x float> %x) nounwind {
+; CHECK-LABEL: isnan_float_vec2:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    li 4, 0
+; CHECK-NEXT:    fcmpu 0, 2, 2
+; CHECK-NEXT:    fcmpu 1, 1, 1
+; CHECK-NEXT:    li 5, 1
+; CHECK-NEXT:    bc 12, 7, .LBB9_2
+; CHECK-NEXT:  # %bb.1: # %entry
+; CHECK-NEXT:    ori 3, 4, 0
+; CHECK-NEXT:    b .LBB9_3
+; CHECK-NEXT:  .LBB9_2: # %entry
+; CHECK-NEXT:    addi 3, 5, 0
+; CHECK-NEXT:  .LBB9_3: # %entry
+; CHECK-NEXT:    bc 12, 3, .LBB9_4
+; CHECK-NEXT:    blr
+; CHECK-NEXT:  .LBB9_4: # %entry
+; CHECK-NEXT:    addi 4, 5, 0
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f32(<2 x float> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_double_vec2(<2 x double> %x) nounwind {
+; CHECK-LABEL: isnan_double_vec2:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    li 4, 0
+; CHECK-NEXT:    fcmpu 0, 2, 2
+; CHECK-NEXT:    fcmpu 1, 1, 1
+; CHECK-NEXT:    li 5, 1
+; CHECK-NEXT:    bc 12, 7, .LBB10_2
+; CHECK-NEXT:  # %bb.1: # %entry
+; CHECK-NEXT:    ori 3, 4, 0
+; CHECK-NEXT:    b .LBB10_3
+; CHECK-NEXT:  .LBB10_2: # %entry
+; CHECK-NEXT:    addi 3, 5, 0
+; CHECK-NEXT:  .LBB10_3: # %entry
+; CHECK-NEXT:    bc 12, 3, .LBB10_4
+; CHECK-NEXT:    blr
+; CHECK-NEXT:  .LBB10_4: # %entry
+; CHECK-NEXT:    addi 4, 5, 0
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f64(<2 x double> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_ldouble_vec2(<2 x ppc_fp128> %x) nounwind {
+; CHECK-LABEL: isnan_ldouble_vec2:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stwu 1, -48(1)
+; CHECK-NEXT:    stfd 3, 32(1)
+; CHECK-NEXT:    lis 3, 32752
+; CHECK-NEXT:    lwz 8, 32(1)
+; CHECK-NEXT:    stfd 4, 40(1)
+; CHECK-NEXT:    lwz 9, 44(1)
+; CHECK-NEXT:    cmpwi 1, 8, 0
+; CHECK-NEXT:    lwz 10, 36(1)
+; CHECK-NEXT:    lwz 8, 40(1)
+; CHECK-NEXT:    clrlwi. 9, 9, 1
+; CHECK-NEXT:    stfd 1, 16(1)
+; CHECK-NEXT:    cmplwi 5, 9, 0
+; CHECK-NEXT:    lwz 5, 20(1)
+; CHECK-NEXT:    crandc 24, 1, 22
+; CHECK-NEXT:    stfd 2, 24(1)
+; CHECK-NEXT:    cmpwi 8, 0
+; CHECK-NEXT:    lwz 4, 16(1)
+; CHECK-NEXT:    cmplw 7, 10, 3
+; CHECK-NEXT:    lwz 7, 28(1)
+; CHECK-NEXT:    xoris 10, 10, 32752
+; CHECK-NEXT:    crandc 20, 22, 2
+; CHECK-NEXT:    cmplwi 10, 0
+; CHECK-NEXT:    lwz 6, 24(1)
+; CHECK-NEXT:    crandc 21, 29, 2
+; CHECK-NEXT:    cmplw 7, 5, 3
+; CHECK-NEXT:    xoris 3, 5, 32752
+; CHECK-NEXT:    crandc 22, 2, 6
+; CHECK-NEXT:    cmplwi 3, 0
+; CHECK-NEXT:    cmpwi 1, 4, 0
+; CHECK-NEXT:    crandc 23, 29, 2
+; CHECK-NEXT:    crandc 25, 2, 6
+; CHECK-NEXT:    clrlwi. 3, 7, 1
+; CHECK-NEXT:    cmplwi 1, 3, 0
+; CHECK-NEXT:    crandc 26, 1, 6
+; CHECK-NEXT:    cmpwi 6, 0
+; CHECK-NEXT:    or 4, 8, 9
+; CHECK-NEXT:    crandc 27, 6, 2
+; CHECK-NEXT:    cmplwi 4, 0
+; CHECK-NEXT:    or 3, 6, 3
+; CHECK-NEXT:    cror 20, 20, 24
+; CHECK-NEXT:    cror 21, 22, 21
+; CHECK-NEXT:    cmplwi 1, 3, 0
+; CHECK-NEXT:    cror 22, 25, 23
+; CHECK-NEXT:    crandc 20, 20, 2
+; CHECK-NEXT:    crand 21, 2, 21
+; CHECK-NEXT:    cror 23, 27, 26
+; CHECK-NEXT:    crand 22, 6, 22
+; CHECK-NEXT:    crnor 20, 21, 20
+; CHECK-NEXT:    crandc 21, 23, 6
+; CHECK-NEXT:    crnor 21, 22, 21
+; CHECK-NEXT:    li 4, 1
+; CHECK-NEXT:    bc 12, 21, .LBB11_2
+; CHECK-NEXT:  # %bb.1: # %entry
+; CHECK-NEXT:    ori 3, 4, 0
+; CHECK-NEXT:    b .LBB11_3
+; CHECK-NEXT:  .LBB11_2: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:  .LBB11_3: # %entry
+; CHECK-NEXT:    bc 12, 20, .LBB11_4
+; CHECK-NEXT:    b .LBB11_5
+; CHECK-NEXT:  .LBB11_4: # %entry
+; CHECK-NEXT:    li 4, 0
+; CHECK-NEXT:  .LBB11_5: # %entry
+; CHECK-NEXT:    addi 1, 1, 48
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2ppcf128(<2 x ppc_fp128> %x)
+  ret <2 x i1> %0
+}
+
+
+define <2 x i1> @isnan_float_vec2_strictfp(<2 x float> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_float_vec2_strictfp:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stwu 1, -16(1)
+; CHECK-NEXT:    stfs 1, 8(1)
+; CHECK-NEXT:    lis 3, 32640
+; CHECK-NEXT:    stfs 2, 12(1)
+; CHECK-NEXT:    lwz 4, 12(1)
+; CHECK-NEXT:    lwz 5, 8(1)
+; CHECK-NEXT:    clrlwi 4, 4, 1
+; CHECK-NEXT:    cmpw 4, 3
+; CHECK-NEXT:    clrlwi 5, 5, 1
+; CHECK-NEXT:    li 4, 0
+; CHECK-NEXT:    cmpw 1, 5, 3
+; CHECK-NEXT:    li 5, 1
+; CHECK-NEXT:    bc 12, 5, .LBB12_2
+; CHECK-NEXT:  # %bb.1: # %entry
+; CHECK-NEXT:    ori 3, 4, 0
+; CHECK-NEXT:    b .LBB12_3
+; CHECK-NEXT:  .LBB12_2: # %entry
+; CHECK-NEXT:    addi 3, 5, 0
+; CHECK-NEXT:  .LBB12_3: # %entry
+; CHECK-NEXT:    bc 12, 1, .LBB12_4
+; CHECK-NEXT:    b .LBB12_5
+; CHECK-NEXT:  .LBB12_4: # %entry
+; CHECK-NEXT:    addi 4, 5, 0
+; CHECK-NEXT:  .LBB12_5: # %entry
+; CHECK-NEXT:    addi 1, 1, 16
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f32(<2 x float> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_double_vec2_strictfp(<2 x double> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_double_vec2_strictfp:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stwu 1, -32(1)
+; CHECK-NEXT:    stfd 2, 24(1)
+; CHECK-NEXT:    lis 3, 32752
+; CHECK-NEXT:    lwz 5, 28(1)
+; CHECK-NEXT:    stfd 1, 16(1)
+; CHECK-NEXT:    lwz 6, 20(1)
+; CHECK-NEXT:    clrlwi 5, 5, 1
+; CHECK-NEXT:    lwz 7, 24(1)
+; CHECK-NEXT:    cmpw 5, 3
+; CHECK-NEXT:    xoris 5, 5, 32752
+; CHECK-NEXT:    lwz 4, 16(1)
+; CHECK-NEXT:    cmplwi 1, 5, 0
+; CHECK-NEXT:    crandc 20, 1, 6
+; CHECK-NEXT:    cmpwi 7, 0
+; CHECK-NEXT:    clrlwi 5, 6, 1
+; CHECK-NEXT:    crandc 21, 6, 2
+; CHECK-NEXT:    cmpw 5, 3
+; CHECK-NEXT:    xoris 3, 5, 32752
+; CHECK-NEXT:    cmplwi 1, 3, 0
+; CHECK-NEXT:    crandc 22, 1, 6
+; CHECK-NEXT:    cmpwi 4, 0
+; CHECK-NEXT:    crandc 23, 6, 2
+; CHECK-NEXT:    crnor 20, 21, 20
+; CHECK-NEXT:    crnor 21, 23, 22
+; CHECK-NEXT:    li 4, 1
+; CHECK-NEXT:    bc 12, 21, .LBB13_2
+; CHECK-NEXT:  # %bb.1: # %entry
+; CHECK-NEXT:    ori 3, 4, 0
+; CHECK-NEXT:    b .LBB13_3
+; CHECK-NEXT:  .LBB13_2: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:  .LBB13_3: # %entry
+; CHECK-NEXT:    bc 12, 20, .LBB13_4
+; CHECK-NEXT:    b .LBB13_5
+; CHECK-NEXT:  .LBB13_4: # %entry
+; CHECK-NEXT:    li 4, 0
+; CHECK-NEXT:  .LBB13_5: # %entry
+; CHECK-NEXT:    addi 1, 1, 32
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f64(<2 x double> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_ldouble_vec2_strictfp(<2 x ppc_fp128> %x) strictfp nounwind {
+; CHECK-LABEL: isnan_ldouble_vec2_strictfp:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stwu 1, -48(1)
+; CHECK-NEXT:    stfd 3, 32(1)
+; CHECK-NEXT:    lis 3, 32752
+; CHECK-NEXT:    lwz 8, 32(1)
+; CHECK-NEXT:    stfd 4, 40(1)
+; CHECK-NEXT:    lwz 9, 44(1)
+; CHECK-NEXT:    cmpwi 1, 8, 0
+; CHECK-NEXT:    lwz 10, 36(1)
+; CHECK-NEXT:    lwz 8, 40(1)
+; CHECK-NEXT:    clrlwi. 9, 9, 1
+; CHECK-NEXT:    stfd 1, 16(1)
+; CHECK-NEXT:    cmplwi 5, 9, 0
+; CHECK-NEXT:    lwz 5, 20(1)
+; CHECK-NEXT:    crandc 24, 1, 22
+; CHECK-NEXT:    stfd 2, 24(1)
+; CHECK-NEXT:    cmpwi 8, 0
+; CHECK-NEXT:    lwz 4, 16(1)
+; CHECK-NEXT:    cmplw 7, 10, 3
+; CHECK-NEXT:    lwz 7, 28(1)
+; CHECK-NEXT:    xoris 10, 10, 32752
+; CHECK-NEXT:    crandc 20, 22, 2
+; CHECK-NEXT:    cmplwi 10, 0
+; CHECK-NEXT:    lwz 6, 24(1)
+; CHECK-NEXT:    crandc 21, 29, 2
+; CHECK-NEXT:    cmplw 7, 5, 3
+; CHECK-NEXT:    xoris 3, 5, 32752
+; CHECK-NEXT:    crandc 22, 2, 6
+; CHECK-NEXT:    cmplwi 3, 0
+; CHECK-NEXT:    cmpwi 1, 4, 0
+; CHECK-NEXT:    crandc 23, 29, 2
+; CHECK-NEXT:    crandc 25, 2, 6
+; CHECK-NEXT:    clrlwi. 3, 7, 1
+; CHECK-NEXT:    cmplwi 1, 3, 0
+; CHECK-NEXT:    crandc 26, 1, 6
+; CHECK-NEXT:    cmpwi 6, 0
+; CHECK-NEXT:    or 4, 8, 9
+; CHECK-NEXT:    crandc 27, 6, 2
+; CHECK-NEXT:    cmplwi 4, 0
+; CHECK-NEXT:    or 3, 6, 3
+; CHECK-NEXT:    cror 20, 20, 24
+; CHECK-NEXT:    cror 21, 22, 21
+; CHECK-NEXT:    cmplwi 1, 3, 0
+; CHECK-NEXT:    cror 22, 25, 23
+; CHECK-NEXT:    crandc 20, 20, 2
+; CHECK-NEXT:    crand 21, 2, 21
+; CHECK-NEXT:    cror 23, 27, 26
+; CHECK-NEXT:    crand 22, 6, 22
+; CHECK-NEXT:    crnor 20, 21, 20
+; CHECK-NEXT:    crandc 21, 23, 6
+; CHECK-NEXT:    crnor 21, 22, 21
+; CHECK-NEXT:    li 4, 1
+; CHECK-NEXT:    bc 12, 21, .LBB14_2
+; CHECK-NEXT:  # %bb.1: # %entry
+; CHECK-NEXT:    ori 3, 4, 0
+; CHECK-NEXT:    b .LBB14_3
+; CHECK-NEXT:  .LBB14_2: # %entry
+; CHECK-NEXT:    li 3, 0
+; CHECK-NEXT:  .LBB14_3: # %entry
+; CHECK-NEXT:    bc 12, 20, .LBB14_4
+; CHECK-NEXT:    b .LBB14_5
+; CHECK-NEXT:  .LBB14_4: # %entry
+; CHECK-NEXT:    li 4, 0
+; CHECK-NEXT:  .LBB14_5: # %entry
+; CHECK-NEXT:    addi 1, 1, 48
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2ppcf128(<2 x ppc_fp128> %x)
+  ret <2 x i1> %0
+}
+
+
+declare i1 @llvm.isnan.f32(float)
+declare i1 @llvm.isnan.f64(double)
+declare i1 @llvm.isnan.ppcf128(ppc_fp128)
+declare <1 x i1> @llvm.isnan.v1f32(<1 x float>)
+declare <1 x i1> @llvm.isnan.v1f64(<1 x double>)
+declare <1 x i1> @llvm.isnan.v1ppcf128(<1 x ppc_fp128>)
+declare <2 x i1> @llvm.isnan.v2f32(<2 x float>)
+declare <2 x i1> @llvm.isnan.v2f64(<2 x double>)
+declare <2 x i1> @llvm.isnan.v2ppcf128(<2 x ppc_fp128>)

llvm/test/CodeGen/X86/x86-fpclass.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=i686 | FileCheck %s -check-prefix=CHECK-32
+; RUN: llc < %s -mtriple=x86_64 | FileCheck %s -check-prefix=CHECK-64
+
+define i1 @isnan_half(half %x) nounwind {
+; CHECK-32-LABEL: isnan_half:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %eax # imm = 0x7C01
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_half:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-64-NEXT:    cmpl $31745, %edi # imm = 0x7C01
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call i1 @llvm.isnan.f16(half %x)
+  ret i1 %0
+}
+
+define i1 @isnan_bfloat(i16 %x) nounwind {
+; CHECK-32-LABEL: isnan_bfloat:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %eax # imm = 0x7F81
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_bfloat:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-64-NEXT:    cmpl $32641, %edi # imm = 0x7F81
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = bitcast i16 %x to bfloat
+  %1 = tail call i1 @llvm.isnan.bf16(bfloat %0)
+  ret i1 %1
+}
+
+define i1 @isnan_float(float %x) nounwind {
+; CHECK-32-LABEL: isnan_float:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    flds {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_float:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    ucomiss %xmm0, %xmm0
+; CHECK-64-NEXT:    setp %al
+; CHECK-64-NEXT:    retq
+; NOSSE-32-LABEL: isnan_float:
+entry:
+  %0 = tail call i1 @llvm.isnan.f32(float %x)
+  ret i1 %0
+}
+
+define i1 @isnan_double(double %x) nounwind {
+; CHECK-32-LABEL: isnan_double:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    fldl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_double:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    ucomisd %xmm0, %xmm0
+; CHECK-64-NEXT:    setp %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call i1 @llvm.isnan.f64(double %x)
+  ret i1 %0
+}
+
+define i1 @isnan_ldouble(x86_fp80 %x) nounwind {
+; CHECK-32-LABEL: isnan_ldouble:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    fldt {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fxam
+; CHECK-32-NEXT:    fstp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    andb $69, %ah
+; CHECK-32-NEXT:    cmpb $1, %ah
+; CHECK-32-NEXT:    sete %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_ldouble:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    fldt {{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    fxam
+; CHECK-64-NEXT:    fstp %st(0)
+; CHECK-64-NEXT:    fnstsw %ax
+; CHECK-64-NEXT:    andb $69, %ah
+; CHECK-64-NEXT:    cmpb $1, %ah
+; CHECK-64-NEXT:    sete %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call i1 @llvm.isnan.f80(x86_fp80 %x)
+  ret i1 %0
+}
+
+define i1 @isnan_half_strictfp(half %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_half_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %eax # imm = 0x7C01
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_half_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-64-NEXT:    cmpl $31745, %edi # imm = 0x7C01
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call i1 @llvm.isnan.f16(half %x)
+  ret i1 %0
+}
+
+define i1 @isnan_bfloat_strict(i16 %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_bfloat_strict:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %eax # imm = 0x7F81
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_bfloat_strict:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-64-NEXT:    cmpl $32641, %edi # imm = 0x7F81
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = bitcast i16 %x to bfloat
+  %1 = tail call i1 @llvm.isnan.bf16(bfloat %0)
+  ret i1 %1
+}
+
+define i1 @isnan_float_strictfp(float %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_float_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movl $2147483647, %eax # imm = 0x7FFFFFFF
+; CHECK-32-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    cmpl $2139095041, %eax # imm = 0x7F800001
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_float_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    movd %xmm0, %eax
+; CHECK-64-NEXT:    andl $2147483647, %eax # imm = 0x7FFFFFFF
+; CHECK-64-NEXT:    cmpl $2139095041, %eax # imm = 0x7F800001
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+; NOSSE-32-LABEL: isnan_float_strictfp:
+entry:
+  %0 = tail call i1 @llvm.isnan.f32(float %x)
+  ret i1 %0
+}
+
+define i1 @isnan_double_strictfp(double %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_double_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movl $2147483647, %eax # imm = 0x7FFFFFFF
+; CHECK-32-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    xorl %ecx, %ecx
+; CHECK-32-NEXT:    cmpl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    movl $2146435072, %ecx # imm = 0x7FF00000
+; CHECK-32-NEXT:    sbbl %eax, %ecx
+; CHECK-32-NEXT:    setl %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_double_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    movq %xmm0, %rax
+; CHECK-64-NEXT:    movabsq $9223372036854775807, %rcx # imm = 0x7FFFFFFFFFFFFFFF
+; CHECK-64-NEXT:    andq %rax, %rcx
+; CHECK-64-NEXT:    movabsq $9218868437227405312, %rax # imm = 0x7FF0000000000000
+; CHECK-64-NEXT:    cmpq %rax, %rcx
+; CHECK-64-NEXT:    setg %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call i1 @llvm.isnan.f64(double %x)
+  ret i1 %0
+}
+
+define i1 @isnan_ldouble_strictfp(x86_fp80 %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_ldouble_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    fldt {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fxam
+; CHECK-32-NEXT:    fstp %st(0)
+; CHECK-32-NEXT:    wait
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    andb $69, %ah
+; CHECK-32-NEXT:    cmpb $1, %ah
+; CHECK-32-NEXT:    sete %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_ldouble_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    fldt {{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    fxam
+; CHECK-64-NEXT:    fstp %st(0)
+; CHECK-64-NEXT:    wait
+; CHECK-64-NEXT:    fnstsw %ax
+; CHECK-64-NEXT:    andb $69, %ah
+; CHECK-64-NEXT:    cmpb $1, %ah
+; CHECK-64-NEXT:    sete %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call i1 @llvm.isnan.f80(x86_fp80 %x)
+  ret i1 %0
+}
+
+define <1 x i1> @isnan_half_vec1(<1 x half> %x) nounwind {
+; CHECK-32-LABEL: isnan_half_vec1:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %eax # imm = 0x7C01
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_half_vec1:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-64-NEXT:    cmpl $31745, %edi # imm = 0x7C01
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f16(<1 x half> %x)
+  ret <1 x i1> %0
+}
+
+define <1 x i1> @isnan_bfloat_vec1(<1 x i16> %x) nounwind {
+; CHECK-32-LABEL: isnan_bfloat_vec1:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %eax # imm = 0x7F81
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_bfloat_vec1:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-64-NEXT:    cmpl $32641, %edi # imm = 0x7F81
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = bitcast <1 x i16> %x to <1 x bfloat>
+  %1 = tail call <1 x i1> @llvm.isnan.v1bf16(<1 x bfloat> %0)
+  ret <1 x i1> %1
+}
+
+define <1 x i1> @isnan_float_vec1(<1 x float> %x) nounwind {
+; CHECK-32-LABEL: isnan_float_vec1:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    flds {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_float_vec1:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    ucomiss %xmm0, %xmm0
+; CHECK-64-NEXT:    setp %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f32(<1 x float> %x)
+  ret <1 x i1> %0
+}
+
+define <1 x i1> @isnan_double_vec1(<1 x double> %x) nounwind {
+; CHECK-32-LABEL: isnan_double_vec1:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    fldl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_double_vec1:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    ucomisd %xmm0, %xmm0
+; CHECK-64-NEXT:    setp %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f64(<1 x double> %x)
+  ret <1 x i1> %0
+}
+
+define <2 x i1> @isnan_half_vec2(<2 x half> %x) nounwind {
+; CHECK-32-LABEL: isnan_half_vec2:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %eax # imm = 0x7C01
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    andl $32767, %ecx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %ecx # imm = 0x7C01
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_half_vec2:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    movw %si, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movw %di, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,0,0,0]
+; CHECK-64-NEXT:    pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,5,5,5,5]
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f16(<2 x half> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_bfloat_vec2(<2 x i16> %x) nounwind {
+; CHECK-32-LABEL: isnan_bfloat_vec2:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %eax # imm = 0x7F81
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    andl $32767, %ecx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %ecx # imm = 0x7F81
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_bfloat_vec2:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,0,0,0]
+; CHECK-64-NEXT:    pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,5,5,5,5]
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = bitcast <2 x i16> %x to <2 x bfloat>
+  %1 = tail call <2 x i1> @llvm.isnan.v2bf16(<2 x bfloat> %0)
+  ret <2 x i1> %1
+}
+
+define <2 x i1> @isnan_float_vec2(<2 x float> %x) nounwind {
+; CHECK-32-LABEL: isnan_float_vec2:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    flds {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    flds {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %cl
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %dl
+; CHECK-32-NEXT:    movl %ecx, %eax
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_float_vec2:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    xorps %xmm1, %xmm1
+; CHECK-64-NEXT:    cmpunordps %xmm0, %xmm1
+; CHECK-64-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,1,1,3]
+; CHECK-64-NEXT:    movaps %xmm1, %xmm0
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f32(<2 x float> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_double_vec2(<2 x double> %x) nounwind {
+; CHECK-32-LABEL: isnan_double_vec2:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    fldl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fldl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %cl
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %dl
+; CHECK-32-NEXT:    movl %ecx, %eax
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_double_vec2:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    xorpd %xmm1, %xmm1
+; CHECK-64-NEXT:    cmpunordpd %xmm1, %xmm0
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f64(<2 x double> %x)
+  ret <2 x i1> %0
+}
+
+define <4 x i1> @isnan_half_vec4(<4 x half> %x) nounwind {
+; CHECK-32-LABEL: isnan_half_vec4:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    pushl %ebx
+; CHECK-32-NEXT:    pushl %edi
+; CHECK-32-NEXT:    pushl %esi
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %edx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %esi
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %edi
+; CHECK-32-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %edi # imm = 0x7C01
+; CHECK-32-NEXT:    setge %bl
+; CHECK-32-NEXT:    andl $32767, %esi # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %esi # imm = 0x7C01
+; CHECK-32-NEXT:    setge %bh
+; CHECK-32-NEXT:    addb %bh, %bh
+; CHECK-32-NEXT:    orb %bl, %bh
+; CHECK-32-NEXT:    andl $32767, %edx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %edx # imm = 0x7C01
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    andl $32767, %ecx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %ecx # imm = 0x7C01
+; CHECK-32-NEXT:    setge %cl
+; CHECK-32-NEXT:    addb %cl, %cl
+; CHECK-32-NEXT:    orb %dl, %cl
+; CHECK-32-NEXT:    shlb $2, %cl
+; CHECK-32-NEXT:    orb %bh, %cl
+; CHECK-32-NEXT:    movb %cl, (%eax)
+; CHECK-32-NEXT:    popl %esi
+; CHECK-32-NEXT:    popl %edi
+; CHECK-32-NEXT:    popl %ebx
+; CHECK-32-NEXT:    retl $4
+;
+; CHECK-64-LABEL: isnan_half_vec4:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    movw %cx, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movw %si, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movw %di, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3]
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f16(<4 x half> %x)
+  ret <4 x i1> %0
+}
+
+define <4 x i1> @isnan_bfloat_vec4(<4 x i16> %x) nounwind {
+; CHECK-32-LABEL: isnan_bfloat_vec4:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    pushl %ebx
+; CHECK-32-NEXT:    pushl %edi
+; CHECK-32-NEXT:    pushl %esi
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %edx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %esi
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %edi
+; CHECK-32-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %edi # imm = 0x7F81
+; CHECK-32-NEXT:    setge %bl
+; CHECK-32-NEXT:    andl $32767, %esi # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %esi # imm = 0x7F81
+; CHECK-32-NEXT:    setge %bh
+; CHECK-32-NEXT:    addb %bh, %bh
+; CHECK-32-NEXT:    orb %bl, %bh
+; CHECK-32-NEXT:    andl $32767, %edx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %edx # imm = 0x7F81
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    andl $32767, %ecx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %ecx # imm = 0x7F81
+; CHECK-32-NEXT:    setge %cl
+; CHECK-32-NEXT:    addb %cl, %cl
+; CHECK-32-NEXT:    orb %dl, %cl
+; CHECK-32-NEXT:    shlb $2, %cl
+; CHECK-32-NEXT:    orb %bh, %cl
+; CHECK-32-NEXT:    movb %cl, (%eax)
+; CHECK-32-NEXT:    popl %esi
+; CHECK-32-NEXT:    popl %edi
+; CHECK-32-NEXT:    popl %ebx
+; CHECK-32-NEXT:    retl $4
+;
+; CHECK-64-LABEL: isnan_bfloat_vec4:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3]
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = bitcast <4 x i16> %x to <4 x bfloat>
+  %1 = tail call <4 x i1> @llvm.isnan.v4bf16(<4 x bfloat> %0)
+  ret <4 x i1> %1
+}
+
+define <4 x i1> @isnan_float_vec4(<4 x float> %x) nounwind {
+; CHECK-32-LABEL: isnan_float_vec4:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    flds {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    flds {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    flds {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    flds {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %dl
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %dh
+; CHECK-32-NEXT:    addb %dh, %dh
+; CHECK-32-NEXT:    orb %dl, %dh
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %dl
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %al
+; CHECK-32-NEXT:    addb %al, %al
+; CHECK-32-NEXT:    orb %dl, %al
+; CHECK-32-NEXT:    shlb $2, %al
+; CHECK-32-NEXT:    orb %dh, %al
+; CHECK-32-NEXT:    movb %al, (%ecx)
+; CHECK-32-NEXT:    movl %ecx, %eax
+; CHECK-32-NEXT:    retl $4
+;
+; CHECK-64-LABEL: isnan_float_vec4:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    xorps %xmm1, %xmm1
+; CHECK-64-NEXT:    cmpunordps %xmm1, %xmm0
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f32(<4 x float> %x)
+  ret <4 x i1> %0
+}
+
+define <4 x i1> @isnan_double_vec4(<4 x double> %x) nounwind {
+; CHECK-32-LABEL: isnan_double_vec4:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    fldl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fldl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fldl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fldl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %dl
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %dh
+; CHECK-32-NEXT:    addb %dh, %dh
+; CHECK-32-NEXT:    orb %dl, %dh
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %dl
+; CHECK-32-NEXT:    fucomp %st(0)
+; CHECK-32-NEXT:    fnstsw %ax
+; CHECK-32-NEXT:    # kill: def $ah killed $ah killed $ax
+; CHECK-32-NEXT:    sahf
+; CHECK-32-NEXT:    setp %al
+; CHECK-32-NEXT:    addb %al, %al
+; CHECK-32-NEXT:    orb %dl, %al
+; CHECK-32-NEXT:    shlb $2, %al
+; CHECK-32-NEXT:    orb %dh, %al
+; CHECK-32-NEXT:    movb %al, (%ecx)
+; CHECK-32-NEXT:    movl %ecx, %eax
+; CHECK-32-NEXT:    retl $4
+;
+; CHECK-64-LABEL: isnan_double_vec4:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    xorpd %xmm2, %xmm2
+; CHECK-64-NEXT:    cmpunordpd %xmm2, %xmm1
+; CHECK-64-NEXT:    cmpunordpd %xmm2, %xmm0
+; CHECK-64-NEXT:    packssdw %xmm1, %xmm0
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f64(<4 x double> %x)
+  ret <4 x i1> %0
+}
+
+
+define <1 x i1> @isnan_half_vec1_strictfp(<1 x half> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_half_vec1_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %eax # imm = 0x7C01
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_half_vec1_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-64-NEXT:    cmpl $31745, %edi # imm = 0x7C01
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f16(<1 x half> %x)
+  ret <1 x i1> %0
+}
+
+define <1 x i1> @isnan_bfloat_vec1_strictfp(<1 x i16> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_bfloat_vec1_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %eax # imm = 0x7F81
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_bfloat_vec1_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-64-NEXT:    cmpl $32641, %edi # imm = 0x7F81
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = bitcast <1 x i16> %x to <1 x bfloat>
+  %1 = tail call <1 x i1> @llvm.isnan.v1bf16(<1 x bfloat> %0)
+  ret <1 x i1> %1
+}
+
+define <1 x i1> @isnan_float_vec1_strictfp(<1 x float> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_float_vec1_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movl $2147483647, %eax # imm = 0x7FFFFFFF
+; CHECK-32-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    cmpl $2139095041, %eax # imm = 0x7F800001
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_float_vec1_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    movd %xmm0, %eax
+; CHECK-64-NEXT:    andl $2147483647, %eax # imm = 0x7FFFFFFF
+; CHECK-64-NEXT:    cmpl $2139095041, %eax # imm = 0x7F800001
+; CHECK-64-NEXT:    setge %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f32(<1 x float> %x)
+  ret <1 x i1> %0
+}
+
+define <1 x i1> @isnan_double_vec1_strictfp(<1 x double> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_double_vec1_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    pushl %ebp
+; CHECK-32-NEXT:    movl %esp, %ebp
+; CHECK-32-NEXT:    andl $-8, %esp
+; CHECK-32-NEXT:    subl $8, %esp
+; CHECK-32-NEXT:    fldl 8(%ebp)
+; CHECK-32-NEXT:    fstpl (%esp)
+; CHECK-32-NEXT:    wait
+; CHECK-32-NEXT:    movl $2147483647, %eax # imm = 0x7FFFFFFF
+; CHECK-32-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    xorl %ecx, %ecx
+; CHECK-32-NEXT:    cmpl (%esp), %ecx
+; CHECK-32-NEXT:    movl $2146435072, %ecx # imm = 0x7FF00000
+; CHECK-32-NEXT:    sbbl %eax, %ecx
+; CHECK-32-NEXT:    setl %al
+; CHECK-32-NEXT:    movl %ebp, %esp
+; CHECK-32-NEXT:    popl %ebp
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_double_vec1_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    movq %xmm0, %rax
+; CHECK-64-NEXT:    movabsq $9223372036854775807, %rcx # imm = 0x7FFFFFFFFFFFFFFF
+; CHECK-64-NEXT:    andq %rax, %rcx
+; CHECK-64-NEXT:    movabsq $9218868437227405312, %rax # imm = 0x7FF0000000000000
+; CHECK-64-NEXT:    cmpq %rax, %rcx
+; CHECK-64-NEXT:    setg %al
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <1 x i1> @llvm.isnan.v1f64(<1 x double> %x)
+  ret <1 x i1> %0
+}
+
+define <2 x i1> @isnan_half_vec2_strictfp(<2 x half> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_half_vec2_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %eax # imm = 0x7C01
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    andl $32767, %ecx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %ecx # imm = 0x7C01
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_half_vec2_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    movw %si, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movw %di, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,0,0,0]
+; CHECK-64-NEXT:    pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,5,5,5,5]
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f16(<2 x half> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_bfloat_vec2_strictfp(<2 x i16> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_bfloat_vec2_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl $32767, %eax # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %eax # imm = 0x7F81
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    andl $32767, %ecx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %ecx # imm = 0x7F81
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_bfloat_vec2_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,0,0,0]
+; CHECK-64-NEXT:    pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,5,5,5,5]
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = bitcast <2 x i16> %x to <2 x bfloat>
+  %1 = tail call <2 x i1> @llvm.isnan.v2bf16(<2 x bfloat> %0)
+  ret <2 x i1> %1
+}
+
+define <2 x i1> @isnan_float_vec2_strictfp(<2 x float> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_float_vec2_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    movl $2147483647, %ecx # imm = 0x7FFFFFFF
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl %ecx, %eax
+; CHECK-32-NEXT:    cmpl $2139095041, %eax # imm = 0x7F800001
+; CHECK-32-NEXT:    setge %al
+; CHECK-32-NEXT:    andl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    cmpl $2139095041, %ecx # imm = 0x7F800001
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_float_vec2_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,1,1,3]
+; CHECK-64-NEXT:    andps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtd {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f32(<2 x float> %x)
+  ret <2 x i1> %0
+}
+
+define <2 x i1> @isnan_double_vec2_strictfp(<2 x double> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_double_vec2_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    pushl %ebp
+; CHECK-32-NEXT:    movl %esp, %ebp
+; CHECK-32-NEXT:    pushl %edi
+; CHECK-32-NEXT:    pushl %esi
+; CHECK-32-NEXT:    andl $-8, %esp
+; CHECK-32-NEXT:    subl $16, %esp
+; CHECK-32-NEXT:    fldl 8(%ebp)
+; CHECK-32-NEXT:    fstpl (%esp)
+; CHECK-32-NEXT:    fldl 16(%ebp)
+; CHECK-32-NEXT:    fstpl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    wait
+; CHECK-32-NEXT:    movl $2147483647, %ecx # imm = 0x7FFFFFFF
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    andl %ecx, %eax
+; CHECK-32-NEXT:    xorl %edx, %edx
+; CHECK-32-NEXT:    cmpl (%esp), %edx
+; CHECK-32-NEXT:    movl $2146435072, %esi # imm = 0x7FF00000
+; CHECK-32-NEXT:    movl $2146435072, %edi # imm = 0x7FF00000
+; CHECK-32-NEXT:    sbbl %eax, %edi
+; CHECK-32-NEXT:    setl %al
+; CHECK-32-NEXT:    andl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    cmpl {{[0-9]+}}(%esp), %edx
+; CHECK-32-NEXT:    sbbl %ecx, %esi
+; CHECK-32-NEXT:    setl %dl
+; CHECK-32-NEXT:    leal -8(%ebp), %esp
+; CHECK-32-NEXT:    popl %esi
+; CHECK-32-NEXT:    popl %edi
+; CHECK-32-NEXT:    popl %ebp
+; CHECK-32-NEXT:    retl
+;
+; CHECK-64-LABEL: isnan_double_vec2_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pxor {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    movdqa {{.*#+}} xmm1 = [9218868439374888960,9218868439374888960]
+; CHECK-64-NEXT:    movdqa %xmm0, %xmm2
+; CHECK-64-NEXT:    pcmpgtd %xmm1, %xmm2
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm3 = xmm2[0,0,2,2]
+; CHECK-64-NEXT:    pcmpeqd %xmm1, %xmm0
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm1 = xmm0[1,1,3,3]
+; CHECK-64-NEXT:    pand %xmm3, %xmm1
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm0 = xmm2[1,1,3,3]
+; CHECK-64-NEXT:    por %xmm1, %xmm0
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <2 x i1> @llvm.isnan.v2f64(<2 x double> %x)
+  ret <2 x i1> %0
+}
+
+define <4 x i1> @isnan_half_vec4_strictfp(<4 x half> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_half_vec4_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    pushl %ebx
+; CHECK-32-NEXT:    pushl %edi
+; CHECK-32-NEXT:    pushl %esi
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %edx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %esi
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %edi
+; CHECK-32-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %edi # imm = 0x7C01
+; CHECK-32-NEXT:    setge %bl
+; CHECK-32-NEXT:    andl $32767, %esi # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %esi # imm = 0x7C01
+; CHECK-32-NEXT:    setge %bh
+; CHECK-32-NEXT:    addb %bh, %bh
+; CHECK-32-NEXT:    orb %bl, %bh
+; CHECK-32-NEXT:    andl $32767, %edx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %edx # imm = 0x7C01
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    andl $32767, %ecx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $31745, %ecx # imm = 0x7C01
+; CHECK-32-NEXT:    setge %cl
+; CHECK-32-NEXT:    addb %cl, %cl
+; CHECK-32-NEXT:    orb %dl, %cl
+; CHECK-32-NEXT:    shlb $2, %cl
+; CHECK-32-NEXT:    orb %bh, %cl
+; CHECK-32-NEXT:    movb %cl, (%eax)
+; CHECK-32-NEXT:    popl %esi
+; CHECK-32-NEXT:    popl %edi
+; CHECK-32-NEXT:    popl %ebx
+; CHECK-32-NEXT:    retl $4
+;
+; CHECK-64-LABEL: isnan_half_vec4_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    movw %cx, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movw %si, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movw %di, -{{[0-9]+}}(%rsp)
+; CHECK-64-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3]
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f16(<4 x half> %x)
+  ret <4 x i1> %0
+}
+
+define <4 x i1> @isnan_bfloat_vec4_strictfp(<4 x i16> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_bfloat_vec4_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    pushl %ebx
+; CHECK-32-NEXT:    pushl %edi
+; CHECK-32-NEXT:    pushl %esi
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %edx
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %esi
+; CHECK-32-NEXT:    movzwl {{[0-9]+}}(%esp), %edi
+; CHECK-32-NEXT:    andl $32767, %edi # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %edi # imm = 0x7F81
+; CHECK-32-NEXT:    setge %bl
+; CHECK-32-NEXT:    andl $32767, %esi # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %esi # imm = 0x7F81
+; CHECK-32-NEXT:    setge %bh
+; CHECK-32-NEXT:    addb %bh, %bh
+; CHECK-32-NEXT:    orb %bl, %bh
+; CHECK-32-NEXT:    andl $32767, %edx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %edx # imm = 0x7F81
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    andl $32767, %ecx # imm = 0x7FFF
+; CHECK-32-NEXT:    cmpl $32641, %ecx # imm = 0x7F81
+; CHECK-32-NEXT:    setge %cl
+; CHECK-32-NEXT:    addb %cl, %cl
+; CHECK-32-NEXT:    orb %dl, %cl
+; CHECK-32-NEXT:    shlb $2, %cl
+; CHECK-32-NEXT:    orb %bh, %cl
+; CHECK-32-NEXT:    movb %cl, (%eax)
+; CHECK-32-NEXT:    popl %esi
+; CHECK-32-NEXT:    popl %edi
+; CHECK-32-NEXT:    popl %ebx
+; CHECK-32-NEXT:    retl $4
+;
+; CHECK-64-LABEL: isnan_bfloat_vec4_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtw {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3]
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = bitcast <4 x i16> %x to <4 x bfloat>
+  %1 = tail call <4 x i1> @llvm.isnan.v4bf16(<4 x bfloat> %0)
+  ret <4 x i1> %1
+}
+
+define <4 x i1> @isnan_float_vec4_strictfp(<4 x float> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_float_vec4_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    pushl %esi
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    movl $2147483647, %ecx # imm = 0x7FFFFFFF
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; CHECK-32-NEXT:    andl %ecx, %edx
+; CHECK-32-NEXT:    cmpl $2139095041, %edx # imm = 0x7F800001
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; CHECK-32-NEXT:    andl %ecx, %esi
+; CHECK-32-NEXT:    cmpl $2139095041, %esi # imm = 0x7F800001
+; CHECK-32-NEXT:    setge %dh
+; CHECK-32-NEXT:    addb %dh, %dh
+; CHECK-32-NEXT:    orb %dl, %dh
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; CHECK-32-NEXT:    andl %ecx, %esi
+; CHECK-32-NEXT:    cmpl $2139095041, %esi # imm = 0x7F800001
+; CHECK-32-NEXT:    setge %dl
+; CHECK-32-NEXT:    andl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    cmpl $2139095041, %ecx # imm = 0x7F800001
+; CHECK-32-NEXT:    setge %cl
+; CHECK-32-NEXT:    addb %cl, %cl
+; CHECK-32-NEXT:    orb %dl, %cl
+; CHECK-32-NEXT:    shlb $2, %cl
+; CHECK-32-NEXT:    orb %dh, %cl
+; CHECK-32-NEXT:    movb %cl, (%eax)
+; CHECK-32-NEXT:    popl %esi
+; CHECK-32-NEXT:    retl $4
+;
+; CHECK-64-LABEL: isnan_float_vec4_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    pcmpgtd {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f32(<4 x float> %x)
+  ret <4 x i1> %0
+}
+
+define <4 x i1> @isnan_double_vec4_strictfp(<4 x double> %x) strictfp nounwind {
+; CHECK-32-LABEL: isnan_double_vec4_strictfp:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    pushl %ebp
+; CHECK-32-NEXT:    movl %esp, %ebp
+; CHECK-32-NEXT:    pushl %edi
+; CHECK-32-NEXT:    pushl %esi
+; CHECK-32-NEXT:    andl $-8, %esp
+; CHECK-32-NEXT:    subl $32, %esp
+; CHECK-32-NEXT:    fldl 12(%ebp)
+; CHECK-32-NEXT:    fstpl (%esp)
+; CHECK-32-NEXT:    fldl 20(%ebp)
+; CHECK-32-NEXT:    fstpl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fldl 28(%ebp)
+; CHECK-32-NEXT:    fstpl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    fldl 36(%ebp)
+; CHECK-32-NEXT:    fstpl {{[0-9]+}}(%esp)
+; CHECK-32-NEXT:    wait
+; CHECK-32-NEXT:    movl $2147483647, %eax # imm = 0x7FFFFFFF
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-32-NEXT:    andl %eax, %ecx
+; CHECK-32-NEXT:    xorl %edx, %edx
+; CHECK-32-NEXT:    cmpl (%esp), %edx
+; CHECK-32-NEXT:    movl $2146435072, %esi # imm = 0x7FF00000
+; CHECK-32-NEXT:    sbbl %ecx, %esi
+; CHECK-32-NEXT:    setl %cl
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; CHECK-32-NEXT:    andl %eax, %esi
+; CHECK-32-NEXT:    cmpl {{[0-9]+}}(%esp), %edx
+; CHECK-32-NEXT:    movl $2146435072, %edi # imm = 0x7FF00000
+; CHECK-32-NEXT:    sbbl %esi, %edi
+; CHECK-32-NEXT:    setl %ch
+; CHECK-32-NEXT:    addb %ch, %ch
+; CHECK-32-NEXT:    orb %cl, %ch
+; CHECK-32-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; CHECK-32-NEXT:    andl %eax, %esi
+; CHECK-32-NEXT:    cmpl {{[0-9]+}}(%esp), %edx
+; CHECK-32-NEXT:    movl $2146435072, %edi # imm = 0x7FF00000
+; CHECK-32-NEXT:    sbbl %esi, %edi
+; CHECK-32-NEXT:    setl %cl
+; CHECK-32-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; CHECK-32-NEXT:    cmpl {{[0-9]+}}(%esp), %edx
+; CHECK-32-NEXT:    movl $2146435072, %edx # imm = 0x7FF00000
+; CHECK-32-NEXT:    sbbl %eax, %edx
+; CHECK-32-NEXT:    setl %dl
+; CHECK-32-NEXT:    addb %dl, %dl
+; CHECK-32-NEXT:    orb %cl, %dl
+; CHECK-32-NEXT:    shlb $2, %dl
+; CHECK-32-NEXT:    orb %ch, %dl
+; CHECK-32-NEXT:    movl 8(%ebp), %eax
+; CHECK-32-NEXT:    movb %dl, (%eax)
+; CHECK-32-NEXT:    leal -8(%ebp), %esp
+; CHECK-32-NEXT:    popl %esi
+; CHECK-32-NEXT:    popl %edi
+; CHECK-32-NEXT:    popl %ebp
+; CHECK-32-NEXT:    retl $4
+;
+; CHECK-64-LABEL: isnan_double_vec4_strictfp:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    movdqa {{.*#+}} xmm2 = [9223372036854775807,9223372036854775807]
+; CHECK-64-NEXT:    pand %xmm2, %xmm0
+; CHECK-64-NEXT:    pand %xmm2, %xmm1
+; CHECK-64-NEXT:    movdqa {{.*#+}} xmm2 = [2147483648,2147483648]
+; CHECK-64-NEXT:    pxor %xmm2, %xmm1
+; CHECK-64-NEXT:    movdqa {{.*#+}} xmm3 = [9218868439374888960,9218868439374888960]
+; CHECK-64-NEXT:    movdqa %xmm1, %xmm4
+; CHECK-64-NEXT:    pcmpgtd %xmm3, %xmm4
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm5 = xmm4[0,0,2,2]
+; CHECK-64-NEXT:    pcmpeqd %xmm3, %xmm1
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[1,1,3,3]
+; CHECK-64-NEXT:    pand %xmm5, %xmm1
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm4 = xmm4[1,1,3,3]
+; CHECK-64-NEXT:    por %xmm1, %xmm4
+; CHECK-64-NEXT:    pxor %xmm2, %xmm0
+; CHECK-64-NEXT:    movdqa %xmm0, %xmm1
+; CHECK-64-NEXT:    pcmpgtd %xmm3, %xmm1
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[0,0,2,2]
+; CHECK-64-NEXT:    pcmpeqd %xmm3, %xmm0
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm3 = xmm0[1,1,3,3]
+; CHECK-64-NEXT:    pand %xmm2, %xmm3
+; CHECK-64-NEXT:    pshufd {{.*#+}} xmm0 = xmm1[1,1,3,3]
+; CHECK-64-NEXT:    por %xmm3, %xmm0
+; CHECK-64-NEXT:    packssdw %xmm4, %xmm0
+; CHECK-64-NEXT:    retq
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f64(<4 x double> %x)
+  ret <4 x i1> %0
+}
+
+
+declare i1 @llvm.isnan.f16(half)
+declare i1 @llvm.isnan.bf16(bfloat)
+declare i1 @llvm.isnan.f32(float)
+declare i1 @llvm.isnan.f64(double)
+declare i1 @llvm.isnan.f80(x86_fp80)
+declare <1 x i1> @llvm.isnan.v1f16(<1 x half>)
+declare <1 x i1> @llvm.isnan.v1bf16(<1 x bfloat>)
+declare <1 x i1> @llvm.isnan.v1f32(<1 x float>)
+declare <1 x i1> @llvm.isnan.v1f64(<1 x double>)
+declare <2 x i1> @llvm.isnan.v2f16(<2 x half>)
+declare <2 x i1> @llvm.isnan.v2bf16(<2 x bfloat>)
+declare <2 x i1> @llvm.isnan.v2f32(<2 x float>)
+declare <2 x i1> @llvm.isnan.v2f64(<2 x double>)
+declare <4 x i1> @llvm.isnan.v4f16(<4 x half>)
+declare <4 x i1> @llvm.isnan.v4bf16(<4 x bfloat>)
+declare <4 x i1> @llvm.isnan.v4f32(<4 x float>)
+declare <4 x i1> @llvm.isnan.v4f64(<4 x double>)

llvm/test/Transforms/InstCombine/fpclass.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -S -instcombine < %s | FileCheck %s
+
+define i1 @isnan_f32_noflags(float %x, float %y) {
+; CHECK-LABEL: @isnan_f32_noflags(
+; CHECK-NEXT:    [[R:%.*]] = fmul float [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT:    [[T:%.*]] = call i1 @llvm.isnan.f32(float [[R]])
+; CHECK-NEXT:    ret i1 [[T]]
+;
+  %r = fmul float %x, %y
+  %t = call i1 @llvm.isnan.f32(float %r)
+  ret i1 %t
+}
+
+define i1 @isnan_f32_ninf(float %x, float %y) {
+; CHECK-LABEL: @isnan_f32_ninf(
+; CHECK-NEXT:    [[R:%.*]] = fsub ninf float [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT:    [[T:%.*]] = call i1 @llvm.isnan.f32(float [[R]])
+; CHECK-NEXT:    ret i1 [[T]]
+;
+  %r = fsub ninf float %x, %y
+  %t = call i1 @llvm.isnan.f32(float %r)
+  ret i1 %t
+}
+
+define i1 @isnan_f32_nsz(float %x, float %y) {
+; CHECK-LABEL: @isnan_f32_nsz(
+; CHECK-NEXT:    [[R:%.*]] = fdiv nsz float [[X:%.*]], [[Y:%.*]]
+; CHECK-NEXT:    [[T:%.*]] = call i1 @llvm.isnan.f32(float [[R]])
+; CHECK-NEXT:    ret i1 [[T]]
+;
+  %r = fdiv nsz float %x, %y
+  %t = call i1 @llvm.isnan.f32(float %r)
+  ret i1 %t
+}
+
+define i1 @isnan_f32(float %x, float %y) {
+; CHECK-LABEL: @isnan_f32(
+; CHECK-NEXT:    ret i1 false
+;
+  %r = fadd nnan float %x, %y
+  %t = call i1 @llvm.isnan.f32(float %r)
+  ret i1 %t
+}
+
+define <1 x i1> @isnan_v1f32(<1 x float> %x, <1 x float> %y) {
+; CHECK-LABEL: @isnan_v1f32(
+; CHECK-NEXT:    ret <1 x i1> zeroinitializer
+;
+  %r = fadd nnan <1 x  float> %x, %y
+  %t = call <1 x i1> @llvm.isnan.v1f32(<1 x float> %r)
+  ret <1 x i1> %t
+}
+
+define <2 x i1> @isnan_v2f32(<2 x float> %x, <2 x float> %y) {
+; CHECK-LABEL: @isnan_v2f32(
+; CHECK-NEXT:    ret <2 x i1> zeroinitializer
+;
+  %r = fadd nnan <2 x  float> %x, %y
+  %t = call <2 x i1> @llvm.isnan.v2f32(<2 x float> %r)
+  ret <2 x i1> %t
+}
+
+declare i1 @llvm.isnan.f32(float %r)
+declare <1 x i1> @llvm.isnan.v1f32(<1 x float> %r)
+declare <2 x i1> @llvm.isnan.v2f32(<2 x float> %r)

llvm/test/Transforms/InstSimplify/ConstProp/fpclassify.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt < %s -instsimplify -S | FileCheck %s
+
+define i1 @isnan_01() {
+; CHECK-LABEL: @isnan_01(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    ret i1 true
+;
+entry:
+  %0 = tail call i1 @llvm.isnan.f32(float 0x7FF8000000000000)
+  ret i1 %0
+}
+
+define i1 @isnan_02() {
+; CHECK-LABEL: @isnan_02(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    ret i1 false
+;
+entry:
+  %0 = tail call i1 @llvm.isnan.f32(float 0x7FF0000000000000)
+  ret i1 %0
+}
+
+define <4 x i1> @isnan_03() {
+; CHECK-LABEL: @isnan_03(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    ret <4 x i1> <i1 true, i1 false, i1 false, i1 true>
+;
+entry:
+  %0 = tail call <4 x i1> @llvm.isnan.v4f32(<4 x float><float 0x7FF8000000000000, float 0x7FF0000000000000, float 1.0, float 0xFFF8000000000000>)
+  ret <4 x i1> %0
+}
+
+declare i1 @llvm.isnan.f32(float)
+declare <4 x i1> @llvm.isnan.v4f32(<4 x float>)