Require 'contract' fast-math flag for FMA generation

[andrew.w.kaylor]

Summary

Currently, the target-independent backend codegen will allow the generation of FMA instructions if *either* the 'contract' fast-math flag is set OR TargetOption::AllowFPOpFusion is set to FPOpFusion::Fast OR the TargetOption::UnsafeFPMath flag is set. This allows fp contraction to be controlled by a means other than the IR and prevents the generation of IR (by a front end) that would enable fusion in some functions and disable it in others.

Note: This change would render the clang -ffp-contract=fast-honor-pragma option obsolete. It also makes the llc -fp-contract option non-functional. These options will be removed in a later patch.

Also note: There are 17 additional lit tests that fail with this change. I updated tests for the X86 and AMDGPU backends (to have one I was familiar with and another I wasn't). It's tedious work, so I didn't want to update all the tests without getting feedback on this direction. Obviously, I'd fix all the tests before committing this patch. There may be a change needed to the front end for CUDA and HIP support before this patch is committed, but I'd like to keep that separate.

I'll send an RFC to llvm-dev to draw more attention to this proposed change.

[arsenm]

I would rephrase the description as removing the global flag for contraction

llvm/test/CodeGen/AMDGPU/fdot2.ll

37

This is somewhat surprising but I guess it works

[tra]

There may be a change needed to the front end for CUDA and HIP support before this patch is committed, but I'd like to keep that separate.

Do I understand it correctly is that the proposal is to explicitly annotate each instruction, instead of relying on LLVM to infer the FP contraction from the global flag. We should still be able to use -ffp-contract=fast in clang, only now it would affect IR generation directly.

In D112760#3097718, @arsenm wrote:

I would rephrase the description as removing the global flag for contraction

Maybe, we should be even more specific -- replace global flag for contractions with explicit instruction attributes.

clang/test/CodeGenCUDA/fp-contract.cu

3–7

We do need to have a way to preserve current behavior for CUDA compilation. There are many existing users that implicitly assume it.

[andrew.w.kaylor]

In D112760#3097718, @arsenm wrote:

I would rephrase the description as removing the global flag for contraction

This change also removes the behavior of the function attribute "unsafe=fp-math" enabling fp-contraction. When I started, I didn't realize that the handling of the global and the handling of the function attributes were so closely related. I guess this needs some more discussion. I'd like to see the function attributes phased out where the same semantics can be expressed by fast-math flags, but as discussed on the llvm-dev email thread this may require updates to keep from breaking out-of-tree front ends.

clang/test/CodeGenCUDA/fp-contract.cu

3–7

When you say you want to preserve "the current behavior" do you mean using "fast" as the default "fp-contract" setting, or also ignoring fp-contract-related pragmas when fp-contract=fast is used?

I certainly understand wanting fp-contract=fast to be the default behavior, but I'm puzzled by the difference in behavior that was introduced between HIP and CUDA, wherein HIP respects the pragmas but CUDA doesn't.

[arsenm]

In D112760#3101240, @andrew.w.kaylor wrote:

In D112760#3097718, @arsenm wrote:

I would rephrase the description as removing the global flag for contraction

This change also removes the behavior of the function attribute "unsafe=fp-math" enabling fp-contraction.

This attribute has never been handled consistently in the backend. We have attributes approximately corresponding to each of the individual fast math flags, so the mirror of the IR controls would be to just set all of those attributes. I've never been sure how unsafe-fp-math fits in here, other than just a convenience to avoid setting all the others. I would also like to get rid of it, since you should be checking the individual properties anyway.

[tra]

clang/test/CodeGenCUDA/fp-contract.cu

3–7

The idea is not to disturb the status quo. Major CUDA users are sort of used to clang being reasonably close to what NVCC does by default. What that is, exactly is not always clear. The current state of affairs has been working well enough. Changing how FP gets compiled will likely trigger a noticeable number of test failures due to both numerical differences and performance regressions. Former we have somewhat decent coverage for in tensorflow. Performance regressions would be harder to spot.

I can test the patch on our tensorflow tests and see how it fares.

If there are nontrivial failures, we will need to consider how to phase in the changes w/o causing unnecessary trouble for the users and/or give then an escape hatch option to keep things working until they can fix their code or tests.

puzzled by the difference in behavior that was introduced between HIP and CUDA

The details on HIP's need for fp-honor-pragma is in https://github.com/llvm/llvm-project/commit/cb08558caa3bad69213b08e6361586491232c745

For CUDA things were still working well enough with -ffp-contract=fast, so there was no need to change things.

[andrew.w.kaylor]

clang/test/CodeGenCUDA/fp-contract.cu

3–7

What I'd like to understand is whether CUDA requires ignoring the pragma when fp-contract=fast is set or if it just needs to use fp-contract=fast by default and doesn't mind that the pragma is ignored. I understand why HIP would want to honor the pragma, and I'd like that to be the normal behavior of fp-contract=fast for all targets.

I see that CUDA does respect "#pragma clang fp contract(off)" as a way to disable contraction if the global setting is "fp-contract=on" (https://godbolt.org/z/4d7En36En), so I don't understand why we wouldn't want the pragma to also work with "fp-contract=fast".

Also, Zahira Ammarguellat is working on a patch to align the clang behavior and documentation (https://reviews.llvm.org/D107994). We're trying not to break the CUDA behavior in the process. Could you take a look at that patch and provide feedback? Thanks!

[tra]

clang/test/CodeGenCUDA/fp-contract.cu

3–7

What I'd like to understand is whether CUDA requires ignoring the pragma when fp-contract=fast is set

I don't think so, but I'm not sure. I think ignoring the pragmas was the unfortunate side-effect of backend with 'Fast' fp fuse option does not respect contract flag, mentioned in D90174. Whether someone happens to rely on it is hard to tell.

or if it just needs to use fp-contract=fast by default and doesn't mind that the pragma is ignored.

I think this is roughly the case. We used fp-contract=fast because it matched what we get from nvcc, which does ignore clang pragmas: https://godbolt.org/z/fGW33fo4v

If that behavior changes, it *may* be visible to CUDA users. On the other hand, it should not be widespread and, arguably, clang pragmas that may come from clang headers or the standard library should be respected.

I see that CUDA does respect "#pragma clang fp contract(off)" as a way to disable contraction if the global setting is "fp-contract=on" (https://godbolt.org/z/4d7En36En), so I don't understand why we wouldn't want the pragma to also work with "fp-contract=fast".

We want GPU-side code to behave the same as the host-side code. For GPUs we rely on some math functions to be provided by the standard library and others are provided by CUDA headers and libdevice bitcode. I suspect that may be one of the reasons for CUDA compilation to ignore clang pragmas, because they would not be present in the GPU-side variants.

AFAICT, it's not that we want fp-contract=fast to ignore pragmas, but that it just conveniently happened to enable fma everywhere.
We could use a separate option to disable fp contract pragmas for that and let fp-contract=fast honor the pragmas.

[andrew.w.kaylor]

clang/test/CodeGenCUDA/fp-contract.cu

3–7

It's not surprising that nvcc would ignore the clang-specific pragma. Also not surprising, if I use "#pragma STDC FP_CONTRACT OFF" instead it doesn't respect that either (though in that case I get a warning saying, "unrecognized STDC pragma").

It seems very likely that CUDA developers just won't use these pragmas. If for whatever reason they do, I think it makes sense to have the pragma behave the same as it would for other targets. I'd also be OK with having clang issue a diagnostic and always disregard the pragma when compiling CUDA code (though I don't think that's the best choice). The current situation where the pragma works with fp-contract=on but not with fp-contract=fast just seems like a bug to me. Basically what I'm saying is that when compiling CUDA code, we should either not allow the pragma or it should work.

One of the things I'm trying to accomplish with this patch is to fix the target-independent backend to have it generate instructions that implement the semantics of the IR produced by the front end. So if the IR says no contract, there would be no global option that would cause the backend to do otherwise. The upshot of this would be that the pragma could be used to disable contraction, even if fp-contract=fast were used as the base setting for the compilation unit.

I feel very strongly that this is the way the backend should work relative to the IR. It sounds like we can have the front end continue working as it does for all targets, including HIP and CUDA (which would be my preference). This would effectively eliminate the need for the fp-contract=fast-honor-pragmas setting, since all targets would always honor the pragmas for fp-contract=fast.

Are there other CUDA stakeholders that I should reach out to for feedback on this?

[andrew.w.kaylor]

In D112760#3101260, @arsenm wrote:

This attribute has never been handled consistently in the backend. We have attributes approximately corresponding to each of the individual fast math flags, so the mirror of the IR controls would be to just set all of those attributes. I've never been sure how unsafe-fp-math fits in here, other than just a convenience to avoid setting all the others. I would also like to get rid of it, since you should be checking the individual properties anyway.

I'm glad to hear that. It looked like you (or someone else) had put a good bit of work into making various combinations of these options behave consistently with the AMDGPU backend, and I wasn't sure if you needed the global settings for some use case.

Can you add your feedback on my RFC thread (https://lists.llvm.org/pipermail/llvm-dev/2021-October/153460.html)? I think we're approaching consensus there, but your input would be very valuable.

[tra]

clang/test/CodeGenCUDA/fp-contract.cu

3–7

I agree with your argument in principle.

I also think that we do need a special case to preserve this CUDA quirk. I'm fine with making the standard options behave the way you describe.
But we do need to give CUDA users an escape hatch option to ignore pragmas if they run into issues.

Clang is in an unfortunate position that we want host and GPU code to behave identically, but parts of the GPU-side implementation is provided by NVIDIA and we can't change it. The only practical option I see is to have a way to preserve the current behavior when we have to.

Without the escape hatch option, the only recourse we'll have is to unroll the patch.
FP compilation changes are known to bring surprises, so the question is "what we're going to do about the issues", not whether we'll see such issues. IMO an escape hatch combined with incremental follow-up improvements is a better strategy compared to multiple patch/revert cycles.

Are there other CUDA stakeholders that I should reach out to for feedback on this?

For CUDA front-end, it's probably myself and @yaxunl as HIP shares most of the front-end functionality. OpenMP folks' (@jdoerfert ?) work overlaps some of the areas (they use NVPTX back-end). They probably would have more input on numerics, but they likely have different constraints as OpenMP doesn't need to match NVCC.

[arsenm]

Please rebase

clang/test/CodeGenCUDA/fp-contract.cu

3–7

I don't think it's worth worrying about people using clang pragmas and then having them not work in nvcc. No escape hatch required. You used a clang pragma, you expect clang's behavior. The whole point of pragma is for implementation defined features

clang/test/CodeGenCUDA/fp-contract.cu

 // REQUIRES: x86-registered-target, nvptx-registered-target, amdgpu-registered-target
 
+// FIXME: This test fails. The comment below describes broken behavior.
+//        The front end should generate IR for the semantics it expects and
+//        backends should respect the IR. Backends should never "disregard"
+//        elements of the IR.
+
 // By default CUDA uses -ffp-contract=fast, HIP uses -ffp-contract=fast-honor-pragmas.
 // we should fuse multiply/add into fma instruction.
 // In IR, fmul/fadd instructions with contract flag are emitted.
 // In backend
 //    nvptx -  assumes fast fp fuse option, which fuses
 //             mult/add insts disregarding contract flag and
 //             llvm.fmuladd intrinsics.
 //    amdgcn - assumes standard fp fuse option, which only

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

ConstantFoldBITCASTofBUILD_VECTOR(SDNode *BV, EVT DstEltVT) {
   return DAG.getBuildVector(VT, DL, Ops);
 }
 
 // Returns true if floating point contraction is allowed on the FMUL-SDValue
 // `N`
 static bool isContractableFMUL(const TargetOptions &Options, SDValue N) {
   assert(N.getOpcode() == ISD::FMUL);
 
-  return Options.AllowFPOpFusion == FPOpFusion::Fast || Options.UnsafeFPMath ||
-         N->getFlags().hasAllowContract();
+  return N->getFlags().hasAllowContract();
 }
 
 // Return true if `N` can assume no infinities involved in it's computation.
 static bool hasNoInfs(const TargetOptions &Options, SDValue N) {
   return Options.NoInfsFPMath || N.getNode()->getFlags().hasNoInfs();
 }
 
 /// Try to perform FMA combining on a given FADD node.
bool HasFMA =
       (!LegalOperations || TLI.isOperationLegalOrCustom(ISD::FMA, VT));
 
   // No valid opcode, do not combine.
   if (!HasFMAD && !HasFMA)
     return SDValue();
 
   bool CanReassociate =
       Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
-  bool AllowFusionGlobally = (Options.AllowFPOpFusion == FPOpFusion::Fast ||
-                              Options.UnsafeFPMath || HasFMAD);
   // If the addition is not contractable, do not combine.
-  if (!AllowFusionGlobally && !N->getFlags().hasAllowContract())
+  if (!HasFMAD && !N->getFlags().hasAllowContract())
     return SDValue();
 
   if (TLI.generateFMAsInMachineCombiner(VT, OptLevel))
     return SDValue();
 
   // Always prefer FMAD to FMA for precision.
   unsigned PreferredFusedOpcode = HasFMAD ? ISD::FMAD : ISD::FMA;
   bool Aggressive = TLI.enableAggressiveFMAFusion(VT);
 
   auto isFusedOp = [&](SDValue N) {
     unsigned Opcode = N.getOpcode();
     return Opcode == ISD::FMA || Opcode == ISD::FMAD;
   };
 
   // Is the node an FMUL and contractable either due to global flags or
   // SDNodeFlags.
-  auto isContractableFMUL = [AllowFusionGlobally](SDValue N) {
+  auto isContractableFMUL = [HasFMAD](SDValue N) {
     if (N.getOpcode() != ISD::FMUL)
       return false;
-    return AllowFusionGlobally || N->getFlags().hasAllowContract();
+    return HasFMAD || N->getFlags().hasAllowContract();
   };
   // If we have two choices trying to fold (fadd (fmul u, v), (fmul x, y)),
   // prefer to fold the multiply with fewer uses.
   if (Aggressive && isContractableFMUL(N0) && isContractableFMUL(N1)) {
     if (N0.getNode()->use_size() > N1.getNode()->use_size())
       std::swap(N0, N1);
   }
 
bool HasFMA =
       TLI.isFMAFasterThanFMulAndFAdd(DAG.getMachineFunction(), VT) &&
       (!LegalOperations || TLI.isOperationLegalOrCustom(ISD::FMA, VT));
 
   // No valid opcode, do not combine.
   if (!HasFMAD && !HasFMA)
     return SDValue();
 
   const SDNodeFlags Flags = N->getFlags();
-  bool AllowFusionGlobally = (Options.AllowFPOpFusion == FPOpFusion::Fast ||
-                              Options.UnsafeFPMath || HasFMAD);
 
   // If the subtraction is not contractable, do not combine.
-  if (!AllowFusionGlobally && !N->getFlags().hasAllowContract())
+  if (!HasFMAD && !N->getFlags().hasAllowContract())
     return SDValue();
 
   if (TLI.generateFMAsInMachineCombiner(VT, OptLevel))
     return SDValue();
 
   // Always prefer FMAD to FMA for precision.
   unsigned PreferredFusedOpcode = HasFMAD ? ISD::FMAD : ISD::FMA;
   bool Aggressive = TLI.enableAggressiveFMAFusion(VT);
   bool NoSignedZero = Options.NoSignedZerosFPMath || Flags.hasNoSignedZeros();
 
   // Is the node an FMUL and contractable either due to global flags or
   // SDNodeFlags.
-  auto isContractableFMUL = [AllowFusionGlobally](SDValue N) {
+  auto isContractableFMUL = [HasFMAD](SDValue N) {
     if (N.getOpcode() != ISD::FMUL)
       return false;
-    return AllowFusionGlobally || N->getFlags().hasAllowContract();
+    return HasFMAD || N->getFlags().hasAllowContract();
   };
 
   // fold (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
   auto tryToFoldXYSubZ = [&](SDValue XY, SDValue Z) {
     if (isContractableFMUL(XY) && (Aggressive || XY->hasOneUse())) {
       return DAG.getNode(PreferredFusedOpcode, SL, VT, XY.getOperand(0),
                          XY.getOperand(1), DAG.getNode(ISD::FNEG, SL, VT, Z));
     }
SDValue DAGCombiner::visitFMULForFMADistributiveCombine(SDNode *N) {
   SDLoc SL(N);
 
   assert(N->getOpcode() == ISD::FMUL && "Expected FMUL Operation");
 
   const TargetOptions &Options = DAG.getTarget().Options;
 
   // The transforms below are incorrect when x == 0 and y == inf, because the
   // intermediate multiplication produces a nan.
-  SDValue FAdd = N0.getOpcode() == ISD::FADD ? N0 : N1;
-  if (!hasNoInfs(Options, FAdd))
+  SDValue FAddOrSub;
+  if (N0.getOpcode() == ISD::FADD || N0.getOpcode() == ISD::FSUB)
+    FAddOrSub = N0;
+  else
+    FAddOrSub = N1;
+
+  if (!hasNoInfs(Options, FAddOrSub))
     return SDValue();
 
   // Floating-point multiply-add without intermediate rounding.
   bool HasFMA =
       isContractableFMUL(Options, SDValue(N, 0)) &&
       TLI.isFMAFasterThanFMulAndFAdd(DAG.getMachineFunction(), VT) &&
       (!LegalOperations || TLI.isOperationLegalOrCustom(ISD::FMA, VT));
 

llvm/test/CodeGen/AMDGPU/fdot2.ll

-; RUN: llc -march=amdgcn -mcpu=gfx900 -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s  -check-prefixes=GCN,GFX900
-; RUN: llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s  -check-prefixes=GCN,GCN-DL-UNSAFE,GFX906-DL-UNSAFE
-; RUN: llc -march=amdgcn -mcpu=gfx1011 -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s  -check-prefixes=GCN,GCN-DL-UNSAFE,GFX10-DL-UNSAFE,GFX10-CONTRACT
-; RUN: llc -march=amdgcn -mcpu=gfx1012 -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s  -check-prefixes=GCN,GCN-DL-UNSAFE,GFX10-DL-UNSAFE,GFX10-CONTRACT
-; RUN: llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck %s  -check-prefixes=GCN,GFX906
-; RUN: llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math=preserve-sign -fp-contract=fast -verify-machineinstrs < %s | FileCheck %s  -check-prefixes=GCN,GFX906-CONTRACT
-; RUN: llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck %s  -check-prefixes=GCN,GFX906-DENORM-CONTRACT
+; RUN: sed -e "s,FASTMATH_FLAGS,fast,g" %s | llc -march=amdgcn -mcpu=gfx900 -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck %s  -check-prefixes=GCN,GFX900
+; RUN: sed -e "s,FASTMATH_FLAGS,fast,g" %s | llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck %s  -check-prefixes=GCN,GCN-DL-UNSAFE,GFX906-DL-UNSAFE
+; RUN: sed -e "s,FASTMATH_FLAGS,fast,g" %s | llc -march=amdgcn -mcpu=gfx1011 -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck %s  -check-prefixes=GCN,GCN-DL-UNSAFE,GFX10-DL-UNSAFE,GFX10-CONTRACT
+; RUN: sed -e "s,FASTMATH_FLAGS,fast,g" %s | llc -march=amdgcn -mcpu=gfx1012 -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck %s  -check-prefixes=GCN,GCN-DL-UNSAFE,GFX10-DL-UNSAFE,GFX10-CONTRACT
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck %s  -check-prefixes=GCN,GFX906
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math=preserve-sign -verify-machineinstrs | FileCheck %s  -check-prefixes=GCN,GFX906-CONTRACT
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math=ieee -verify-machineinstrs | FileCheck %s  -check-prefixes=GCN,GFX906-DENORM-CONTRACT
 ; (fadd (fmul S1.x, S2.x), (fadd (fmul (S1.y, S2.y), z))) -> (fdot2 S1, S2, z)
 
 ; Tests to make sure fdot2 is not generated when vector elements of dot-product expressions
 ; are not converted from f16 to f32.
 ; GCN-LABEL: {{^}}dotproduct_f16
 ; GFX900: v_fma_f16
 ; GFX900: v_fma_f16
 
entry:
   %src2.vec = load <2 x half>, <2 x half> addrspace(1)* %src2
 
   %src1.el1 = extractelement <2 x half> %src1.vec, i64 0
   %src2.el1 = extractelement <2 x half> %src2.vec, i64 0
 
   %src1.el2 = extractelement <2 x half> %src1.vec, i64 1
   %src2.el2 = extractelement <2 x half> %src2.vec, i64 1
 
-  %mul2 = fmul half %src1.el2, %src2.el2
-  %mul1 = fmul half %src1.el1, %src2.el1
+  %mul2 = fmul FASTMATH_FLAGS half %src1.el2, %src2.el2
+  %mul1 = fmul FASTMATH_FLAGS half %src1.el1, %src2.el1
   %acc = load half, half addrspace(1)* %dst, align 2
-  %acc1 = fadd half %mul2, %acc
-  %acc2 = fadd half %mul1, %acc1
+  %acc1 = fadd FASTMATH_FLAGS half %mul2, %acc
+  %acc2 = fadd FASTMATH_FLAGS half %mul1, %acc1
   store half %acc2, half addrspace(1)* %dst, align 2
   ret void
 }
 
 
 ; We only want to generate fdot2 if vector element of dot product is converted from f16 to f32
 ; and the vectors are of type <2 x half>
 ; GCN-LABEL: {{^}}dotproduct_f16_f32
entry:
   %src2.el1 = extractelement <2 x half> %src2.vec, i64 0
   %csrc2.el1 = fpext half %src2.el1 to float
 
   %src1.el2 = extractelement <2 x half> %src1.vec, i64 1
   %csrc1.el2 = fpext half %src1.el2 to float
   %src2.el2 = extractelement <2 x half> %src2.vec, i64 1
   %csrc2.el2 = fpext half %src2.el2 to float
 
-  %mul2 = fmul float %csrc1.el2, %csrc2.el2
-  %mul1 = fmul float %csrc1.el1, %csrc2.el1
+  %mul2 = fmul FASTMATH_FLAGS float %csrc1.el2, %csrc2.el2
+  %mul1 = fmul FASTMATH_FLAGS float %csrc1.el1, %csrc2.el1
   %acc = load float, float addrspace(1)* %dst, align 4
-  %acc1 = fadd float %mul2, %acc
-  %acc2 = fadd float %mul1, %acc1
+  %acc1 = fadd FASTMATH_FLAGS float %mul2, %acc
+  %acc2 = fadd FASTMATH_FLAGS float %mul1, %acc1
   store float %acc2, float addrspace(1)* %dst, align 4
   ret void
 }
 
 ; We only want to generate fdot2 if vector element of dot product is converted from f16 to f32
 ; and the vectors are of type <2 x half>
 ; GCN-LABEL: {{^}}dotproduct_diffvecorder
 ; GFX900: v_mad_mix_f32
entry:
   %src2.el1 = extractelement <2 x half> %src2.vec, i64 0
   %csrc2.el1 = fpext half %src2.el1 to float
 
   %src1.el2 = extractelement <2 x half> %src1.vec, i64 1
   %csrc1.el2 = fpext half %src1.el2 to float
   %src2.el2 = extractelement <2 x half> %src2.vec, i64 1
   %csrc2.el2 = fpext half %src2.el2 to float
 
-  %mul2 = fmul float %csrc2.el2, %csrc1.el2
-  %mul1 = fmul float %csrc1.el1, %csrc2.el1
+  %mul2 = fmul FASTMATH_FLAGS float %csrc2.el2, %csrc1.el2
+  %mul1 = fmul FASTMATH_FLAGS float %csrc1.el1, %csrc2.el1
   %acc = load float, float addrspace(1)* %dst, align 4
-  %acc1 = fadd float %mul2, %acc
-  %acc2 = fadd float %mul1, %acc1
+  %acc1 = fadd FASTMATH_FLAGS float %mul2, %acc
+  %acc2 = fadd FASTMATH_FLAGS float %mul1, %acc1
   store float %acc2, float addrspace(1)* %dst, align 4
   ret void
 }
 
 ; Tests to make sure dot product is not generated when the vectors are not of <2 x half>.
 ; GCN-LABEL: {{^}}dotproduct_v4f16
 ; GFX900: v_mad_mix_f32
 
entry:
   %src2.el1 = extractelement <4 x half> %src2.vec, i64 0
   %csrc2.el1 = fpext half %src2.el1 to float
 
   %src1.el2 = extractelement <4 x half> %src1.vec, i64 1
   %csrc1.el2 = fpext half %src1.el2 to float
   %src2.el2 = extractelement <4 x half> %src2.vec, i64 1
   %csrc2.el2 = fpext half %src2.el2 to float
 
-  %mul2 = fmul float %csrc1.el2, %csrc2.el2
-  %mul1 = fmul float %csrc1.el1, %csrc2.el1
+  %mul2 = fmul FASTMATH_FLAGS float %csrc1.el2, %csrc2.el2
+  %mul1 = fmul FASTMATH_FLAGS float %csrc1.el1, %csrc2.el1
   %acc = load float, float addrspace(1)* %dst, align 4
-  %acc1 = fadd float %mul2, %acc
-  %acc2 = fadd float %mul1, %acc1
+  %acc1 = fadd FASTMATH_FLAGS float %mul2, %acc
+  %acc2 = fadd FASTMATH_FLAGS float %mul1, %acc1
   store float %acc2, float addrspace(1)* %dst, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}NotAdotproduct
 ; GFX900: v_mad_mix_f32
 ; GFX900: v_mad_mix_f32
 
entry:
   %src2.el1 = extractelement <2 x half> %src2.vec, i64 0
   %csrc2.el1 = fpext half %src2.el1 to float
 
   %src1.el2 = extractelement <2 x half> %src1.vec, i64 1
   %csrc1.el2 = fpext half %src1.el2 to float
   %src2.el2 = extractelement <2 x half> %src2.vec, i64 1
   %csrc2.el2 = fpext half %src2.el2 to float
 
-  %mul2 = fmul float %csrc1.el2, %csrc1.el1
-  %mul1 = fmul float %csrc2.el1, %csrc2.el2
+  %mul2 = fmul FASTMATH_FLAGS float %csrc1.el2, %csrc1.el1
+  %mul1 = fmul FASTMATH_FLAGS float %csrc2.el1, %csrc2.el2
   %acc = load float, float addrspace(1)* %dst, align 4
-  %acc1 = fadd float %mul2, %acc
-  %acc2 = fadd float %mul1, %acc1
+  %acc1 = fadd FASTMATH_FLAGS float %mul2, %acc
+  %acc2 = fadd FASTMATH_FLAGS float %mul1, %acc1
   store float %acc2, float addrspace(1)* %dst, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}Diff_Idx_NotAdotproduct
 ; GFX900: v_mad_mix_f32
 ; GFX900: v_mad_mix_f32
 
entry:
   %src2.el1 = extractelement <2 x half> %src2.vec, i64 0
   %csrc2.el1 = fpext half %src2.el1 to float
 
   %src1.el2 = extractelement <2 x half> %src1.vec, i64 1
   %csrc1.el2 = fpext half %src1.el2 to float
   %src2.el2 = extractelement <2 x half> %src2.vec, i64 1
   %csrc2.el2 = fpext half %src2.el2 to float
 
-  %mul2 = fmul float %csrc1.el2, %csrc2.el1
-  %mul1 = fmul float %csrc1.el1, %csrc2.el2
+  %mul2 = fmul FASTMATH_FLAGS float %csrc1.el2, %csrc2.el1
+  %mul1 = fmul FASTMATH_FLAGS float %csrc1.el1, %csrc2.el2
   %acc = load float, float addrspace(1)* %dst, align 4
-  %acc1 = fadd float %mul2, %acc
-  %acc2 = fadd float %mul1, %acc1
+  %acc1 = fadd FASTMATH_FLAGS float %mul2, %acc
+  %acc2 = fadd FASTMATH_FLAGS float %mul1, %acc1
   store float %acc2, float addrspace(1)* %dst, align 4
   ret void
 }

llvm/test/CodeGen/AMDGPU/fma-combine.ll

-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefix=SI-NOFMA -check-prefix=SI-SAFE -check-prefix=SI -check-prefix=FUNC %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=verde -denormal-fp-math-f32=preserve-sign -verify-machineinstrs -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefix=SI-NOFMA -check-prefix=SI-SAFE -check-prefix=SI -check-prefix=FUNC %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=ieee -verify-machineinstrs -fp-contract=fast -enable-no-infs-fp-math -enable-unsafe-fp-math < %s | FileCheck -enable-var-scope -check-prefix=SI-FMA -check-prefix=SI-UNSAFE -check-prefix=SI -check-prefix=FUNC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck -enable-var-scope -check-prefix=SI-NOFMA -check-prefix=SI-SAFE -check-prefix=SI -check-prefix=FUNC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=verde -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck -enable-var-scope -check-prefix=SI-NOFMA -check-prefix=SI-SAFE -check-prefix=SI -check-prefix=FUNC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,fast,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefix=SI-FMA -check-prefix=SI-UNSAFE -check-prefix=SI -check-prefix=FUNC %s
 
 ; FIXME: Remove enable-unsafe-fp-math in RUN line and add flags to IR instrs
 
 ; Note: The SI-FMA conversions of type x * (y + 1) --> x * y + x would be
 ; beneficial even without fp32 denormals, but they do require no-infs-fp-math
 ; for correctness.
 
 declare i32 @llvm.amdgcn.workitem.id.x() #0
define amdgpu_kernel void @combine_to_fma_f64_0(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %gep.1 = getelementptr double, double addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr double, double addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
 
-  %mul = fmul double %a, %b
-  %fma = fadd double %mul, %c
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %fma = fadd FASTMATH_FLAGS double %mul, %c
   store double %fma, double addrspace(1)* %gep.out
   ret void
 }
 
 ; (fadd (fmul x, y), z) -> (fma x, y, z)
 ; FUNC-LABEL: {{^}}combine_to_fma_f64_0_2use:
 ; SI-DAG: buffer_load_dwordx2 [[A:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[B:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8 glc{{$}}
define amdgpu_kernel void @combine_to_fma_f64_0_2use(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr double, double addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr double, double addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
   %d = load volatile double, double addrspace(1)* %gep.3
 
-  %mul = fmul double %a, %b
-  %fma0 = fadd double %mul, %c
-  %fma1 = fadd double %mul, %d
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %fma0 = fadd FASTMATH_FLAGS double %mul, %c
+  %fma1 = fadd FASTMATH_FLAGS double %mul, %d
   store volatile double %fma0, double addrspace(1)* %gep.out.0
   store volatile double %fma1, double addrspace(1)* %gep.out.1
   ret void
 }
 
 ; (fadd x, (fmul y, z)) -> (fma y, z, x)
 ; FUNC-LABEL: {{^}}combine_to_fma_f64_1:
 ; SI-DAG: buffer_load_dwordx2 [[A:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[B:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[C:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:16 glc{{$}}
 ; SI: v_fma_f64 [[RESULT:v\[[0-9]+:[0-9]+\]]], [[A]], [[B]], [[C]]
 ; SI: buffer_store_dwordx2 [[RESULT]]
 define amdgpu_kernel void @combine_to_fma_f64_1(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() #0
   %gep.0 = getelementptr double, double addrspace(1)* %in, i32 %tid
   %gep.1 = getelementptr double, double addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr double, double addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
 
-  %mul = fmul double %a, %b
-  %fma = fadd double %c, %mul
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %fma = fadd FASTMATH_FLAGS double %c, %mul
   store double %fma, double addrspace(1)* %gep.out
   ret void
 }
 
 ; (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_fma_fsub_0_f64:
 ; SI-DAG: buffer_load_dwordx2 [[A:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[B:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[C:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:16 glc{{$}}
 ; SI: v_fma_f64 [[RESULT:v\[[0-9]+:[0-9]+\]]], [[A]], [[B]], -[[C]]
 ; SI: buffer_store_dwordx2 [[RESULT]]
 define amdgpu_kernel void @combine_to_fma_fsub_0_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() #0
   %gep.0 = getelementptr double, double addrspace(1)* %in, i32 %tid
   %gep.1 = getelementptr double, double addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr double, double addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
 
-  %mul = fmul double %a, %b
-  %fma = fsub double %mul, %c
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %fma = fsub FASTMATH_FLAGS double %mul, %c
   store double %fma, double addrspace(1)* %gep.out
   ret void
 }
 
 ; (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_fma_fsub_f64_0_2use:
 ; SI-DAG: buffer_load_dwordx2 [[A:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[B:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8 glc{{$}}
define amdgpu_kernel void @combine_to_fma_fsub_f64_0_2use(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr double, double addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr double, double addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
   %d = load volatile double, double addrspace(1)* %gep.3
 
-  %mul = fmul double %a, %b
-  %fma0 = fsub double %mul, %c
-  %fma1 = fsub double %mul, %d
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %fma0 = fsub FASTMATH_FLAGS double %mul, %c
+  %fma1 = fsub FASTMATH_FLAGS double %mul, %d
   store volatile double %fma0, double addrspace(1)* %gep.out.0
   store volatile double %fma1, double addrspace(1)* %gep.out.1
   ret void
 }
 
 ; (fsub x, (fmul y, z)) -> (fma (fneg y), z, x)
 ; FUNC-LABEL: {{^}}combine_to_fma_fsub_1_f64:
 ; SI-DAG: buffer_load_dwordx2 [[A:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[B:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[C:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:16 glc{{$}}
 ; SI: v_fma_f64 [[RESULT:v\[[0-9]+:[0-9]+\]]], -[[A]], [[B]], [[C]]
 ; SI: buffer_store_dwordx2 [[RESULT]]
 define amdgpu_kernel void @combine_to_fma_fsub_1_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() #0
   %gep.0 = getelementptr double, double addrspace(1)* %in, i32 %tid
   %gep.1 = getelementptr double, double addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr double, double addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
 
-  %mul = fmul double %a, %b
-  %fma = fsub double %c, %mul
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %fma = fsub FASTMATH_FLAGS double %c, %mul
   store double %fma, double addrspace(1)* %gep.out
   ret void
 }
 
 ; (fsub x, (fmul y, z)) -> (fma (fneg y), z, x)
 ; FUNC-LABEL: {{^}}combine_to_fma_fsub_1_f64_2use:
 ; SI-DAG: buffer_load_dwordx2 [[A:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[B:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8 glc{{$}}
define amdgpu_kernel void @combine_to_fma_fsub_1_f64_2use(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr double, double addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr double, double addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
   %d = load volatile double, double addrspace(1)* %gep.3
 
-  %mul = fmul double %a, %b
-  %fma0 = fsub double %c, %mul
-  %fma1 = fsub double %d, %mul
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %fma0 = fsub FASTMATH_FLAGS double %c, %mul
+  %fma1 = fsub FASTMATH_FLAGS double %d, %mul
   store volatile double %fma0, double addrspace(1)* %gep.out.0
   store volatile double %fma1, double addrspace(1)* %gep.out.1
   ret void
 }
 
 ; (fsub (fneg (fmul x, y)), z) -> (fma (fneg x), y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_fma_fsub_2_f64:
 ; SI-DAG: buffer_load_dwordx2 [[A:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[B:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8 glc{{$}}
 ; SI-DAG: buffer_load_dwordx2 [[C:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:16 glc{{$}}
 ; SI: v_fma_f64 [[RESULT:v\[[0-9]+:[0-9]+\]]], -[[A]], [[B]], -[[C]]
 ; SI: buffer_store_dwordx2 [[RESULT]]
 define amdgpu_kernel void @combine_to_fma_fsub_2_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() #0
   %gep.0 = getelementptr double, double addrspace(1)* %in, i32 %tid
   %gep.1 = getelementptr double, double addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr double, double addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
 
-  %mul = fmul double %a, %b
-  %mul.neg = fsub double -0.0, %mul
-  %fma = fsub double %mul.neg, %c
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %mul.neg = fsub FASTMATH_FLAGS double -0.0, %mul
+  %fma = fsub FASTMATH_FLAGS double %mul.neg, %c
 
   store double %fma, double addrspace(1)* %gep.out
   ret void
 }
 
 ; (fsub (fneg (fmul x, y)), z) -> (fma (fneg x), y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_fma_fsub_2_f64_2uses_neg:
 ; SI-DAG: buffer_load_dwordx2 [[A:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
define amdgpu_kernel void @combine_to_fma_fsub_2_f64_2uses_neg(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr double, double addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr double, double addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
   %d = load volatile double, double addrspace(1)* %gep.3
 
-  %mul = fmul double %a, %b
-  %mul.neg = fsub double -0.0, %mul
-  %fma0 = fsub double %mul.neg, %c
-  %fma1 = fsub double %mul.neg, %d
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %mul.neg = fsub FASTMATH_FLAGS double -0.0, %mul
+  %fma0 = fsub FASTMATH_FLAGS double %mul.neg, %c
+  %fma1 = fsub FASTMATH_FLAGS double %mul.neg, %d
 
   store volatile double %fma0, double addrspace(1)* %gep.out.0
   store volatile double %fma1, double addrspace(1)* %gep.out.1
   ret void
 }
 
 ; (fsub (fneg (fmul x, y)), z) -> (fma (fneg x), y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_fma_fsub_2_f64_2uses_mul:
define amdgpu_kernel void @combine_to_fma_fsub_2_f64_2uses_mul(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr double, double addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr double, double addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile double, double addrspace(1)* %gep.0
   %b = load volatile double, double addrspace(1)* %gep.1
   %c = load volatile double, double addrspace(1)* %gep.2
   %d = load volatile double, double addrspace(1)* %gep.3
 
-  %mul = fmul double %a, %b
-  %mul.neg = fsub double -0.0, %mul
-  %fma0 = fsub double %mul.neg, %c
-  %fma1 = fsub double %mul, %d
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %mul.neg = fsub FASTMATH_FLAGS double -0.0, %mul
+  %fma0 = fsub FASTMATH_FLAGS double %mul.neg, %c
+  %fma1 = fsub FASTMATH_FLAGS double %mul, %d
 
   store volatile double %fma0, double addrspace(1)* %gep.out.0
   store volatile double %fma1, double addrspace(1)* %gep.out.1
   ret void
 }
 
 ; fold (fsub (fma x, y, (fmul u, v)), z) -> (fma x, y (fma u, v, (fneg z)))
 
define amdgpu_kernel void @aggressive_combine_to_fma_fsub_0_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %x = load volatile double, double addrspace(1)* %gep.0
   %y = load volatile double, double addrspace(1)* %gep.1
   %z = load volatile double, double addrspace(1)* %gep.2
   %u = load volatile double, double addrspace(1)* %gep.3
   %v = load volatile double, double addrspace(1)* %gep.4
 
-  %tmp0 = fmul double %u, %v
+  %tmp0 = fmul FASTMATH_FLAGS double %u, %v
   %tmp1 = call double @llvm.fma.f64(double %x, double %y, double %tmp0) #0
-  %tmp2 = fsub double %tmp1, %z
+  %tmp2 = fsub FASTMATH_FLAGS double %tmp1, %z
 
   store double %tmp2, double addrspace(1)* %gep.out
   ret void
 }
 
 ; fold (fsub x, (fma y, z, (fmul u, v)))
 ;   -> (fma (fneg y), z, (fma (fneg u), v, x))
 
define amdgpu_kernel void @aggressive_combine_to_fma_fsub_1_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #1 {
 
   %x = load volatile double, double addrspace(1)* %gep.0
   %y = load volatile double, double addrspace(1)* %gep.1
   %z = load volatile double, double addrspace(1)* %gep.2
   %u = load volatile double, double addrspace(1)* %gep.3
   %v = load volatile double, double addrspace(1)* %gep.4
 
   ; nsz flag is needed since this combine may change sign of zero
-  %tmp0 = fmul nsz double %u, %v
-  %tmp1 = call nsz double @llvm.fma.f64(double %y, double %z, double %tmp0) #0
-  %tmp2 = fsub nsz double %x, %tmp1
+  %tmp0 = fmul FASTMATH_FLAGS nsz double %u, %v
+  %tmp1 = call FASTMATH_FLAGS nsz double @llvm.fma.f64(double %y, double %z, double %tmp0) #0
+  %tmp2 = fsub FASTMATH_FLAGS nsz double %x, %tmp1
 
   store double %tmp2, double addrspace(1)* %gep.out
   ret void
 }
 
 ;
 ; Patterns (+ fneg variants): mul(add(1.0,x),y), mul(sub(1.0,x),y), mul(sub(x,1.0),y)
 ;
 
 ; FUNC-LABEL: {{^}}test_f32_mul_add_x_one_y:
 ; SI-NOFMA: v_add_f32_e32 [[VS:v[0-9]]], 1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VS]], [[VY:v[0-9]]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, [[VX:v[0-9]]], [[VY:v[0-9]]], [[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_add_x_one_y(float addrspace(1)* %out,
                                         float addrspace(1)* %in1,
                                         float addrspace(1)* %in2) {
   %x = load volatile float, float addrspace(1)* %in1
   %y = load volatile float, float addrspace(1)* %in2
-  %a = fadd float %x, 1.0
-  %m = fmul float %a, %y
+  %a = fadd FASTMATH_FLAGS float %x, 1.0
+  %m = fmul FASTMATH_FLAGS float %a, %y
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_y_add_x_one:
 ; SI-NOFMA: v_add_f32_e32 [[VS:v[0-9]]], 1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VY:v[0-9]]], [[VS]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, [[VX:v[0-9]]], [[VY:v[0-9]]], [[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_y_add_x_one(float addrspace(1)* %out,
                                         float addrspace(1)* %in1,
                                         float addrspace(1)* %in2) {
   %x = load volatile float, float addrspace(1)* %in1
   %y = load volatile float, float addrspace(1)* %in2
-  %a = fadd float %x, 1.0
-  %m = fmul float %y, %a
+  %a = fadd FASTMATH_FLAGS float %x, 1.0
+  %m = fmul FASTMATH_FLAGS float %y, %a
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_add_x_negone_y:
 ; SI-NOFMA: v_add_f32_e32 [[VS:v[0-9]]], -1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VS]], [[VY:v[0-9]]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, [[VX:v[0-9]]], [[VY:v[0-9]]], -[[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_add_x_negone_y(float addrspace(1)* %out,
                                            float addrspace(1)* %in1,
                                            float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %a = fadd float %x, -1.0
-  %m = fmul float %a, %y
+  %a = fadd FASTMATH_FLAGS float %x, -1.0
+  %m = fmul FASTMATH_FLAGS float %a, %y
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_y_add_x_negone:
 ; SI-NOFMA: v_add_f32_e32 [[VS:v[0-9]]], -1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VY:v[0-9]]], [[VS]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, [[VX:v[0-9]]], [[VY:v[0-9]]], -[[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_y_add_x_negone(float addrspace(1)* %out,
                                            float addrspace(1)* %in1,
                                            float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %a = fadd float %x, -1.0
-  %m = fmul float %y, %a
+  %a = fadd FASTMATH_FLAGS float %x, -1.0
+  %m = fmul FASTMATH_FLAGS float %y, %a
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_sub_one_x_y:
 ; SI-NOFMA: v_sub_f32_e32 [[VS:v[0-9]]], 1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VS]], [[VY:v[0-9]]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, -[[VX:v[0-9]]], [[VY:v[0-9]]], [[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_sub_one_x_y(float addrspace(1)* %out,
                                         float addrspace(1)* %in1,
                                         float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %s = fsub float 1.0, %x
-  %m = fmul float %s, %y
+  %s = fsub FASTMATH_FLAGS float 1.0, %x
+  %m = fmul FASTMATH_FLAGS float %s, %y
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_y_sub_one_x:
 ; SI-NOFMA: v_sub_f32_e32 [[VS:v[0-9]]], 1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VY:v[0-9]]], [[VS]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, -[[VX:v[0-9]]], [[VY:v[0-9]]], [[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_y_sub_one_x(float addrspace(1)* %out,
                                         float addrspace(1)* %in1,
                                         float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %s = fsub float 1.0, %x
-  %m = fmul float %y, %s
+  %s = fsub FASTMATH_FLAGS float 1.0, %x
+  %m = fmul FASTMATH_FLAGS float %y, %s
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_sub_negone_x_y:
 ; SI-NOFMA: v_sub_f32_e32 [[VS:v[0-9]]], -1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VS]], [[VY:v[0-9]]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, -[[VX:v[0-9]]], [[VY:v[0-9]]], -[[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_sub_negone_x_y(float addrspace(1)* %out,
                                            float addrspace(1)* %in1,
                                            float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %s = fsub float -1.0, %x
-  %m = fmul float %s, %y
+  %s = fsub FASTMATH_FLAGS float -1.0, %x
+  %m = fmul FASTMATH_FLAGS float %s, %y
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_y_sub_negone_x:
 ; SI-NOFMA: v_sub_f32_e32 [[VS:v[0-9]]], -1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VY:v[0-9]]], [[VS]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, -[[VX:v[0-9]]], [[VY:v[0-9]]], -[[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_y_sub_negone_x(float addrspace(1)* %out,
                                          float addrspace(1)* %in1,
                                          float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %s = fsub float -1.0, %x
-  %m = fmul float %y, %s
+  %s = fsub FASTMATH_FLAGS float -1.0, %x
+  %m = fmul FASTMATH_FLAGS float %y, %s
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_sub_x_one_y:
 ; SI-NOFMA: v_add_f32_e32 [[VS:v[0-9]]], -1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VS]], [[VY:v[0-9]]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, [[VX:v[0-9]]], [[VY:v[0-9]]], -[[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_sub_x_one_y(float addrspace(1)* %out,
                                         float addrspace(1)* %in1,
                                         float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %s = fsub float %x, 1.0
-  %m = fmul float %s, %y
+  %s = fsub FASTMATH_FLAGS float %x, 1.0
+  %m = fmul FASTMATH_FLAGS float %s, %y
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_y_sub_x_one:
 ; SI-NOFMA: v_add_f32_e32 [[VS:v[0-9]]], -1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VY:v[0-9]]], [[VS]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, [[VX:v[0-9]]], [[VY:v[0-9]]], -[[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_y_sub_x_one(float addrspace(1)* %out,
                                       float addrspace(1)* %in1,
                                       float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %s = fsub float %x, 1.0
-  %m = fmul float %y, %s
+  %s = fsub FASTMATH_FLAGS float %x, 1.0
+  %m = fmul FASTMATH_FLAGS float %y, %s
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_sub_x_negone_y:
 ; SI-NOFMA: v_add_f32_e32 [[VS:v[0-9]]], 1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VS]], [[VY:v[0-9]]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, [[VX:v[0-9]]], [[VY:v[0-9]]], [[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_sub_x_negone_y(float addrspace(1)* %out,
                                          float addrspace(1)* %in1,
                                          float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %s = fsub float %x, -1.0
-  %m = fmul float %s, %y
+  %s = fsub FASTMATH_FLAGS float %x, -1.0
+  %m = fmul FASTMATH_FLAGS float %s, %y
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f32_mul_y_sub_x_negone:
 ; SI-NOFMA: v_add_f32_e32 [[VS:v[0-9]]], 1.0, [[VX:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 {{v[0-9]}}, [[VY:v[0-9]]], [[VS]]
 ;
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, [[VX:v[0-9]]], [[VY:v[0-9]]], [[VY:v[0-9]]]
 define amdgpu_kernel void @test_f32_mul_y_sub_x_negone(float addrspace(1)* %out,
                                          float addrspace(1)* %in1,
                                          float addrspace(1)* %in2) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
-  %s = fsub float %x, -1.0
-  %m = fmul float %y, %s
+  %s = fsub FASTMATH_FLAGS float %x, -1.0
+  %m = fmul FASTMATH_FLAGS float %y, %s
   store float %m, float addrspace(1)* %out
   ret void
 }
 
 ;
 ; Interpolation Patterns: add(mul(x,t),mul(sub(1.0,t),y))
 ;
 
 ; FUNC-LABEL: {{^}}test_f32_interp:
 ; SI-NOFMA: v_sub_f32_e32 [[VT1:v[0-9]]], 1.0, [[VT:v[0-9]]]
 ; SI-NOFMA: v_mul_f32_e32 [[VTY:v[0-9]]], [[VY:v[0-9]]], [[VT1]]
 ; SI-NOFMA: v_mac_f32_e32 [[VTY]], [[VX:v[0-9]]], [[VT]]
 ;
 ; SI-FMA: v_fma_f32 [[VR:v[0-9]]], -[[VT:v[0-9]]], [[VY:v[0-9]]], [[VY]]
 ; SI-FMA: v_fma_f32 {{v[0-9]}}, [[VX:v[0-9]]], [[VT]], [[VR]]
 define amdgpu_kernel void @test_f32_interp(float addrspace(1)* %out,
                              float addrspace(1)* %in1,
                              float addrspace(1)* %in2,
                              float addrspace(1)* %in3) {
   %x = load float, float addrspace(1)* %in1
   %y = load float, float addrspace(1)* %in2
   %t = load float, float addrspace(1)* %in3
-  %t1 = fsub float 1.0, %t
-  %tx = fmul float %x, %t
-  %ty = fmul float %y, %t1
-  %r = fadd float %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS float 1.0, %t
+  %tx = fmul FASTMATH_FLAGS float %x, %t
+  %ty = fmul FASTMATH_FLAGS float %y, %t1
+  %r = fadd FASTMATH_FLAGS float %tx, %ty
   store float %r, float addrspace(1)* %out
   ret void
 }
 
 ; FUNC-LABEL: {{^}}test_f64_interp:
 ; SI-NOFMA: v_add_f64 [[VT1:v\[[0-9]+:[0-9]+\]]], -[[VT:v\[[0-9]+:[0-9]+\]]], 1.0
 ; SI-NOFMA: v_mul_f64 [[VTY:v\[[0-9]+:[0-9]+\]]], [[VY:v\[[0-9]+:[0-9]+\]]], [[VT1]]
 ; SI-NOFMA: v_fma_f64 v{{\[[0-9]+:[0-9]+\]}}, [[VX:v\[[0-9]+:[0-9]+\]]], [[VT]], [[VTY]]
 ;
 ; SI-FMA: v_fma_f64 [[VR:v\[[0-9]+:[0-9]+\]]], -[[VT:v\[[0-9]+:[0-9]+\]]], [[VY:v\[[0-9]+:[0-9]+\]]], [[VY]]
 ; SI-FMA: v_fma_f64 v{{\[[0-9]+:[0-9]+\]}}, [[VX:v\[[0-9]+:[0-9]+\]]], [[VT]], [[VR]]
 define amdgpu_kernel void @test_f64_interp(double addrspace(1)* %out,
                              double addrspace(1)* %in1,
                              double addrspace(1)* %in2,
                              double addrspace(1)* %in3) {
   %x = load double, double addrspace(1)* %in1
   %y = load double, double addrspace(1)* %in2
   %t = load double, double addrspace(1)* %in3
-  %t1 = fsub double 1.0, %t
-  %tx = fmul double %x, %t
-  %ty = fmul double %y, %t1
-  %r = fadd double %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS double 1.0, %t
+  %tx = fmul FASTMATH_FLAGS double %x, %t
+  %ty = fmul FASTMATH_FLAGS double %y, %t1
+  %r = fadd FASTMATH_FLAGS double %tx, %ty
   store double %r, double addrspace(1)* %out
   ret void
 }
 
 ; Make sure negative constant cancels out fneg
 ; SI-LABEL: {{^}}fma_neg_2.0_neg_a_b_f32:
 ; SI: {{buffer|flat|global}}_load_dword [[A:v[0-9]+]]
 ; SI: {{buffer|flat|global}}_load_dword [[B:v[0-9]+]]
 ; SI-NOT: [[A]]
 ; SI-NOT: [[B]]
 ; SI: v_fma_f32 v{{[0-9]+}}, [[A]], 2.0, [[B]]
 define amdgpu_kernel void @fma_neg_2.0_neg_a_b_f32(float addrspace(1)* %out, float addrspace(1)* %in) #0 {
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %r1 = load volatile float, float addrspace(1)* %gep.0
   %r2 = load volatile float, float addrspace(1)* %gep.1
 
-  %r1.fneg = fneg float %r1
+  %r1.fneg = fneg FASTMATH_FLAGS float %r1
 
   %r3 = tail call float @llvm.fma.f32(float -2.0, float %r1.fneg, float %r2)
   store float %r3, float addrspace(1)* %gep.out
   ret void
 }
 
 ; SI-LABEL: {{^}}fma_2.0_neg_a_b_f32:
 ; SI: {{buffer|flat|global}}_load_dword [[A:v[0-9]+]]
 ; SI: {{buffer|flat|global}}_load_dword [[B:v[0-9]+]]
 ; SI-NOT: [[A]]
 ; SI-NOT: [[B]]
 ; SI: v_fma_f32 v{{[0-9]+}}, [[A]], -2.0, [[B]]
 define amdgpu_kernel void @fma_2.0_neg_a_b_f32(float addrspace(1)* %out, float addrspace(1)* %in) #0 {
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %r1 = load volatile float, float addrspace(1)* %gep.0
   %r2 = load volatile float, float addrspace(1)* %gep.1
 
-  %r1.fneg = fneg float %r1
+  %r1.fneg = fneg FASTMATH_FLAGS float %r1
 
   %r3 = tail call float @llvm.fma.f32(float 2.0, float %r1.fneg, float %r2)
   store float %r3, float addrspace(1)* %gep.out
   ret void
 }
 
 ; SI-LABEL: {{^}}fma_neg_b_c_v4f32:
 ; SI: v_fma_f32 v{{[0-9]+}}, v{{[0-9]+}}, -v{{[0-9]+}}, -v{{[0-9]+}}

llvm/test/CodeGen/AMDGPU/fmul-2-combine-multi-use.ll

 ; XUN: llc -mtriple=amdgcn-amd-amdhsa -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,SI %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=tonga -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,SIVI,VI-DENORM %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=tonga -denormal-fp-math=preserve-sign -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,SIVI,VI-FLUSH %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1010 -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10,GFX8_10,GFX10-DENORM %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1010 -denormal-fp-math=preserve-sign -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10,GFX8_10,GFX10-FLUSH %s
 
 ; Make sure (fmul (fadd x, x), c) -> (fmul x, (fmul 2.0, c)) doesn't
 ; make add an instruction if the fadd has more than one use.
 
-declare half @llvm.fabs.f16(half) #1
-declare float @llvm.fabs.f32(float) #1
+declare half @llvm.fabs.f16(half) #0
+declare float @llvm.fabs.f32(float) #0
 
 ; GCN-LABEL: {{^}}multiple_fadd_use_test_f32:
 ; SI: v_max_legacy_f32_e64 [[A16:v[0-9]+]],
 ; SI: v_add_f32_e32 [[A17:v[0-9]+]], [[A16]], [[A16]]
 ; SI: v_mul_f32_e32 [[A18:v[0-9]+]], [[A17]], [[A17]]
 ; SI: v_mad_f32 [[A20:v[0-9]+]], -[[A18]], [[A17]], 1.0
 ; SI: buffer_store_dword [[A20]]
 
 ; GFX8_10: v_add_f32_e64 v{{[0-9]+}}, s{{[0-9]+}}, -1.0
 ; GFX8_10: v_add_f32_e64 v{{[0-9]+}}, s{{[0-9]+}}, -1.0
 ; GFX8_10: v_cmp_gt_f32_e64 {{vcc|vcc_lo}}, |v{{[0-9]+}}|, |v{{[0-9]+}}|
 ; GFX8_10: v_cndmask_b32_e32
 ; GFX8_10: v_add_f32_e64 v{{[0-9]+}}, |v{{[0-9]+}}|, |v{{[0-9]+}}|
 ; GFX8_10: v_mul_f32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
 ; VI:      v_mad_f32 v{{[0-9]+}}, -v{{[0-9]+}}, v{{[0-9]+}}, 1.0
 ; GFX10:   v_fma_f32 v{{[0-9]+}}, -v{{[0-9]+}}, v{{[0-9]+}}, 1.0
-define amdgpu_kernel void @multiple_fadd_use_test_f32(float addrspace(1)* %out, float %x, float %y, float %z) #0 {
-  %a11 = fadd float %y, -1.0
+define amdgpu_kernel void @multiple_fadd_use_test_f32(float addrspace(1)* %out, float %x, float %y, float %z) {
+  %a11 = fadd fast float %y, -1.0
   %a12 = call float @llvm.fabs.f32(float %a11)
-  %a13 = fadd float %x, -1.0
+  %a13 = fadd fast float %x, -1.0
   %a14 = call float @llvm.fabs.f32(float %a13)
   %a15 = fcmp ogt float %a12, %a14
   %a16 = select i1 %a15, float %a12, float %a14
-  %a17 = fmul float %a16, 2.0
-  %a18 = fmul float %a17, %a17
-  %a19 = fmul float %a18, %a17
-  %a20 = fsub float 1.0, %a19
+  %a17 = fmul fast float %a16, 2.0
+  %a18 = fmul fast float %a17, %a17
+  %a19 = fmul fast float %a18, %a17
+  %a20 = fsub fast float 1.0, %a19
   store float %a20, float addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}multiple_use_fadd_fmac_f32:
 ; GCN-DAG:   v_add_f32_e64 [[MUL2:v[0-9]+]], [[X:s[0-9]+]], s{{[0-9]+}}
 ; SIVI-DAG:  v_mac_f32_e64 [[MAD:v[0-9]+]], [[X]], 2.0
 ; GFX10-DAG: v_fma_f32 [[MAD:v[0-9]+]], [[X]], 2.0, s{{[0-9]+}}
 ; GCN-DAG:   buffer_store_dword [[MUL2]]
 ; GCN-DAG:   buffer_store_dword [[MAD]]
 ; GCN:       s_endpgm
-define amdgpu_kernel void @multiple_use_fadd_fmac_f32(float addrspace(1)* %out, float %x, [8 x i32], float %y) #0 {
+define amdgpu_kernel void @multiple_use_fadd_fmac_f32(float addrspace(1)* %out, float %x, [8 x i32], float %y) {
   %out.gep.1 = getelementptr float, float addrspace(1)* %out, i32 1
   %mul2 = fmul fast float %x, 2.0
   %mad = fadd fast float %mul2, %y
   store volatile float %mul2, float addrspace(1)* %out
   store volatile float %mad, float addrspace(1)* %out.gep.1
   ret void
 }
 
 ; GCN-LABEL: {{^}}multiple_use_fadd_fmad_f32:
 ; GCN-DAG:   v_add_f32_e64 [[MUL2:v[0-9]+]], |[[X:s[0-9]+]]|, |s{{[0-9]+}}|
 ; SIVI-DAG:  v_mad_f32 [[MAD:v[0-9]+]], |[[X]]|, 2.0, v{{[0-9]+}}
 ; GFX10-DAG: v_fma_f32 [[MAD:v[0-9]+]], |[[X]]|, 2.0, s{{[0-9]+}}
 ; GCN-DAG:   buffer_store_dword [[MUL2]]
 ; GCN-DAG:   buffer_store_dword [[MAD]]
 ; GCN:       s_endpgm
-define amdgpu_kernel void @multiple_use_fadd_fmad_f32(float addrspace(1)* %out, float %x, float %y) #0 {
+define amdgpu_kernel void @multiple_use_fadd_fmad_f32(float addrspace(1)* %out, float %x, float %y) {
   %out.gep.1 = getelementptr float, float addrspace(1)* %out, i32 1
   %x.abs = call float @llvm.fabs.f32(float %x)
   %mul2 = fmul fast float %x.abs, 2.0
   %mad = fadd fast float %mul2, %y
   store volatile float %mul2, float addrspace(1)* %out
   store volatile float %mad, float addrspace(1)* %out.gep.1
   ret void
 }
 
 ; GCN-LABEL: {{^}}multiple_use_fadd_multi_fmad_f32:
 ; SIVI:  v_mad_f32 {{v[0-9]+}}, |[[X:s[0-9]+]]|, 2.0, v{{[0-9]+}}
 ; SIVI:  v_mad_f32 {{v[0-9]+}}, |[[X]]|, 2.0, v{{[0-9]+}}
 ; GFX10: v_fma_f32 {{v[0-9]+}}, |[[X:s[0-9]+]]|, 2.0, {{s[0-9]+}}
 ; GFX10: v_fma_f32 {{v[0-9]+}}, |[[X]]|, 2.0, {{s[0-9]+}}
-define amdgpu_kernel void @multiple_use_fadd_multi_fmad_f32(float addrspace(1)* %out, float %x, float %y, float %z) #0 {
+define amdgpu_kernel void @multiple_use_fadd_multi_fmad_f32(float addrspace(1)* %out, float %x, float %y, float %z) {
   %out.gep.1 = getelementptr float, float addrspace(1)* %out, i32 1
   %x.abs = call float @llvm.fabs.f32(float %x)
   %mul2 = fmul fast float %x.abs, 2.0
   %mad0 = fadd fast float %mul2, %y
   %mad1 = fadd fast float %mul2, %z
   store volatile float %mad0, float addrspace(1)* %out
   store volatile float %mad1, float addrspace(1)* %out.gep.1
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmul_x2_xn2_f32:
 ; GCN: v_mul_f32_e64 [[TMP0:v[0-9]+]], [[X:s[0-9]+]], -4.0
 ; GCN: v_mul_f32_e32 [[RESULT:v[0-9]+]], [[X]], [[TMP0]]
 ; GCN: buffer_store_dword [[RESULT]]
-define amdgpu_kernel void @fmul_x2_xn2_f32(float addrspace(1)* %out, float %x, float %y) #0 {
+define amdgpu_kernel void @fmul_x2_xn2_f32(float addrspace(1)* %out, float %x, float %y) {
   %out.gep.1 = getelementptr float, float addrspace(1)* %out, i32 1
   %mul2 = fmul fast float %x, 2.0
   %muln2 = fmul fast float %x, -2.0
   %mul = fmul fast float %mul2, %muln2
   store volatile float %mul, float addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmul_x2_xn3_f32:
 ; SIVI:  v_mov_b32_e32 [[K:v[0-9]+]], 0xc0c00000
 ; SIVI:  v_mul_f32_e32 [[TMP0:v[0-9]+]], [[X:s[0-9]+]], [[K]]
 ; GFX10: v_mul_f32_e64 [[TMP0:v[0-9]+]], 0xc0c00000, [[X:s[0-9]+]]
 ; GCN:   v_mul_f32_e32 [[RESULT:v[0-9]+]], [[X]], [[TMP0]]
 ; GCN:   buffer_store_dword [[RESULT]]
-define amdgpu_kernel void @fmul_x2_xn3_f32(float addrspace(1)* %out, float %x, float %y) #0 {
+define amdgpu_kernel void @fmul_x2_xn3_f32(float addrspace(1)* %out, float %x, float %y) {
   %out.gep.1 = getelementptr float, float addrspace(1)* %out, i32 1
   %mul2 = fmul fast float %x, 2.0
   %muln2 = fmul fast float %x, -3.0
   %mul = fmul fast float %mul2, %muln2
   store volatile float %mul, float addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}multiple_fadd_use_test_f16:
 ; GFX8_10:      v_add_f16_e64 v{{[0-9]+}}, s{{[0-9]+}}, -1.0
 ; GFX8_10:      v_add_f16_e64 v{{[0-9]+}}, s{{[0-9]+}}, -1.0
 ; GFX8_10:      v_cmp_gt_f16_e64 {{vcc|vcc_lo}}, |v{{[0-9]+}}|, |v{{[0-9]+}}|
 ; GFX8_10:      v_cndmask_b32_e32
 ; GFX8_10:      v_add_f16_e64 v{{[0-9]+}}, |v{{[0-9]+}}|, |v{{[0-9]+}}|
 ; GFX8_10:      v_mul_f16_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
 ; VI-FLUSH:     v_mad_f16 v{{[0-9]+}}, -v{{[0-9]+}}, v{{[0-9]+}}, 1.0
 ; VI-DENORM:    v_fma_f16 v{{[0-9]+}}, -v{{[0-9]+}}, v{{[0-9]+}}, 1.0
 ; GFX10-DENORM: v_fma_f16 v{{[0-9]+}}, -v{{[0-9]+}}, v{{[0-9]+}}, 1.0
 ; GFX10-FLUSH:  v_sub_f16_e32 v{{[0-9]+}}, 1.0, v{{[0-9]+}}
-define amdgpu_kernel void @multiple_fadd_use_test_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg, i16 zeroext %z.arg) #0 {
+define amdgpu_kernel void @multiple_fadd_use_test_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg, i16 zeroext %z.arg) {
   %x = bitcast i16 %x.arg to half
   %y = bitcast i16 %y.arg to half
   %z = bitcast i16 %z.arg to half
-  %a11 = fadd half %y, -1.0
+  %a11 = fadd fast half %y, -1.0
   %a12 = call half @llvm.fabs.f16(half %a11)
-  %a13 = fadd half %x, -1.0
+  %a13 = fadd fast half %x, -1.0
   %a14 = call half @llvm.fabs.f16(half %a13)
   %a15 = fcmp ogt half %a12, %a14
   %a16 = select i1 %a15, half %a12, half %a14
-  %a17 = fmul half %a16, 2.0
-  %a18 = fmul half %a17, %a17
-  %a19 = fmul half %a18, %a17
-  %a20 = fsub half 1.0, %a19
+  %a17 = fmul fast half %a16, 2.0
+  %a18 = fmul fast half %a17, %a17
+  %a19 = fmul fast half %a18, %a17
+  %a20 = fsub fast half 1.0, %a19
   store half %a20, half addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}multiple_use_fadd_fmac_f16:
 ; GCN-DAG: v_add_f16_e64 [[MUL2:v[0-9]+]], [[X:s[0-9]+]], s{{[0-9]+}}
 
 ; VI-FLUSH-DAG:     v_mac_f16_e64 [[MAD:v[0-9]+]], [[X]], 2.0
 ; VI-DENORM-DAG:    v_fma_f16 [[MAD:v[0-9]+]], [[X]], 2.0, v{{[0-9]+}}
 ; GFX10-FLUSH-DAG:  v_add_f16_e32 [[MAD:v[0-9]+]], s{{[0-9]+}}, [[MUL2]]
 ; GFX10-DENORM-DAG: v_fma_f16 [[MAD:v[0-9]+]], [[X]], 2.0, s{{[0-9]+}}
 
 ; GCN-DAG: buffer_store_short [[MUL2]]
 ; GCN-DAG: buffer_store_short [[MAD]]
 ; GCN: s_endpgm
-define amdgpu_kernel void @multiple_use_fadd_fmac_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg) #0 {
+define amdgpu_kernel void @multiple_use_fadd_fmac_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg) {
   %x = bitcast i16 %x.arg to half
   %y = bitcast i16 %y.arg to half
   %out.gep.1 = getelementptr half, half addrspace(1)* %out, i32 1
   %mul2 = fmul fast half %x, 2.0
   %mad = fadd fast half %mul2, %y
   store volatile half %mul2, half addrspace(1)* %out
   store volatile half %mad, half addrspace(1)* %out.gep.1
   ret void
 }
 
 ; GCN-LABEL: {{^}}multiple_use_fadd_fmad_f16:
 ; GCN-DAG: v_add_f16_e64 [[MUL2:v[0-9]+]], |[[X:s[0-9]+]]|, |s{{[0-9]+}}|
 
 ; VI-FLUSH-DAG:     v_mad_f16 [[MAD:v[0-9]+]], |[[X]]|, 2.0, v{{[0-9]+}}
 ; VI-DENORM-DAG:    v_fma_f16 [[MAD:v[0-9]+]], |[[X]]|, 2.0, v{{[0-9]+}}
 ; GFX10-FLUSH-DAG:  v_add_f16_e32 [[MAD:v[0-9]+]], s{{[0-9]+}}, [[MUL2]]
 ; GFX10-DENORM-DAG: v_fma_f16 [[MAD:v[0-9]+]], |[[X]]|, 2.0, s{{[0-9]+}}
 
 ; GCN-DAG: buffer_store_short [[MUL2]]
 ; GCN-DAG: buffer_store_short [[MAD]]
 ; GCN: s_endpgm
-define amdgpu_kernel void @multiple_use_fadd_fmad_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg) #0 {
+define amdgpu_kernel void @multiple_use_fadd_fmad_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg) {
   %x = bitcast i16 %x.arg to half
   %y = bitcast i16 %y.arg to half
   %out.gep.1 = getelementptr half, half addrspace(1)* %out, i32 1
   %x.abs = call half @llvm.fabs.f16(half %x)
   %mul2 = fmul fast half %x.abs, 2.0
   %mad = fadd fast half %mul2, %y
   store volatile half %mul2, half addrspace(1)* %out
   store volatile half %mad, half addrspace(1)* %out.gep.1
   ret void
 }
 
 ; GCN-LABEL: {{^}}multiple_use_fadd_multi_fmad_f16:
 ; VI-FLUSH: v_mad_f16 {{v[0-9]+}}, |[[X:s[0-9]+]]|, 2.0, v{{[0-9]+}}
 ; VI-FLUSH: v_mad_f16 {{v[0-9]+}}, |[[X]]|, 2.0, v{{[0-9]+}}
 
 ; VI-DENORM: v_fma_f16 {{v[0-9]+}}, |[[X:s[0-9]+]]|, 2.0, v{{[0-9]+}}
 ; VI-DENORM: v_fma_f16 {{v[0-9]+}}, |[[X]]|, 2.0, v{{[0-9]+}}
 
 ; GFX10-FLUSH:  v_add_f16_e64 [[MUL2:v[0-9]+]], |[[X:s[0-9]+]]|, |{{s[0-9]+}}|
 ; GFX10-FLUSH:  v_add_f16_e32 {{v[0-9]+}}, {{s[0-9]+}}, [[MUL2]]
 ; GFX10-FLUSH:  v_add_f16_e32 {{v[0-9]+}}, {{s[0-9]+}}, [[MUL2]]
 ; GFX10-DENORM: v_fma_f16 {{v[0-9]+}}, |[[X:s[0-9]+]]|, 2.0, s{{[0-9]+}}
 ; GFX10-DENORM: v_fma_f16 {{v[0-9]+}}, |[[X]]|, 2.0, s{{[0-9]+}}
 
-define amdgpu_kernel void @multiple_use_fadd_multi_fmad_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg, i16 zeroext %z.arg) #0 {
+define amdgpu_kernel void @multiple_use_fadd_multi_fmad_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg, i16 zeroext %z.arg) {
   %x = bitcast i16 %x.arg to half
   %y = bitcast i16 %y.arg to half
   %z = bitcast i16 %z.arg to half
   %out.gep.1 = getelementptr half, half addrspace(1)* %out, i32 1
   %x.abs = call half @llvm.fabs.f16(half %x)
   %mul2 = fmul fast half %x.abs, 2.0
   %mad0 = fadd fast half %mul2, %y
   %mad1 = fadd fast half %mul2, %z
   store volatile half %mad0, half addrspace(1)* %out
   store volatile half %mad1, half addrspace(1)* %out.gep.1
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmul_x2_xn2_f16:
 ; GCN: v_mul_f16_e64 [[TMP0:v[0-9]+]], [[X:s[0-9]+]], -4.0
 ; GCN: v_mul_f16_e32 [[RESULT:v[0-9]+]], [[X]], [[TMP0]]
 ; GCN: buffer_store_short [[RESULT]]
-define amdgpu_kernel void @fmul_x2_xn2_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg) #0 {
+define amdgpu_kernel void @fmul_x2_xn2_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg) {
   %x = bitcast i16 %x.arg to half
   %y = bitcast i16 %y.arg to half
   %out.gep.1 = getelementptr half, half addrspace(1)* %out, i32 1
   %mul2 = fmul fast half %x, 2.0
   %muln2 = fmul fast half %x, -2.0
   %mul = fmul fast half %mul2, %muln2
   store volatile half %mul, half addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmul_x2_xn3_f16:
 ; SIVI:  v_mov_b32_e32 [[K:v[0-9]+]], 0xc600
 ; SIVI:  v_mul_f16_e32 [[TMP0:v[0-9]+]], [[X:s[0-9]+]], [[K]]
 ; GFX10: v_mul_f16_e64 [[TMP0:v[0-9]+]], 0xc600, [[X:s[0-9]+]]
 ; GCN:   v_mul_f16_e32 [[RESULT:v[0-9]+]], [[X]], [[TMP0]]
 ; GCN:   buffer_store_short [[RESULT]]
-define amdgpu_kernel void @fmul_x2_xn3_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg) #0 {
+define amdgpu_kernel void @fmul_x2_xn3_f16(half addrspace(1)* %out, i16 zeroext %x.arg, i16 zeroext %y.arg) {
   %x = bitcast i16 %x.arg to half
   %y = bitcast i16 %y.arg to half
   %out.gep.1 = getelementptr half, half addrspace(1)* %out, i32 1
   %mul2 = fmul fast half %x, 2.0
   %muln2 = fmul fast half %x, -3.0
   %mul = fmul fast half %mul2, %muln2
   store volatile half %mul, half addrspace(1)* %out
   ret void
 }
 
-attributes #0 = { nounwind "unsafe-fp-math"="true" }
-attributes #1 = { nounwind readnone }
+attributes #0 = { nounwind readnone }

llvm/test/CodeGen/AMDGPU/fmuladd.f16.ll

-; RUN: llc -march=amdgcn -mcpu=fiji -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI-FLUSH,VI %s
-; RUN: llc -march=amdgcn -mcpu=fiji -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI-FLUSH,VI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=fiji -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefixes=GCN,VI-FLUSH,VI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=fiji -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefixes=GCN,VI-FLUSH,VI %s
 
-; RUN: llc -march=amdgcn -mcpu=fiji -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM,GCN-DENORM-STRICT,VI-DENORM,VI %s
-; RUN: llc -march=amdgcn -mcpu=fiji -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM,GCN-DENORM-CONTRACT,VI-DENORM-CONTRACT,VI-DENORM,VI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=fiji -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM,GCN-DENORM-STRICT,VI-DENORM,VI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=fiji -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM,GCN-DENORM-CONTRACT,VI-DENORM-CONTRACT,VI-DENORM,VI %s
 
-; RUN: llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10-FLUSH,GFX10 %s
-; RUN: llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10-FLUSH,GFX10 %s
-; RUN: llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM,GCN-DENORM-STRICT,GFX10-DENORM-STRICT,GFX10-DENORM,GFX10 %s
-; RUN: llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM,GCN-DENORM-CONTRACT,GFX10-DENORM-CONTRACT,GFX10-DENORM,GFX10 %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10-FLUSH,GFX10 %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10-FLUSH,GFX10 %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM,GCN-DENORM-STRICT,GFX10-DENORM-STRICT,GFX10-DENORM,GFX10 %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM,GCN-DENORM-CONTRACT,GFX10-DENORM-CONTRACT,GFX10-DENORM,GFX10 %s
 
 declare i32 @llvm.amdgcn.workitem.id.x() #1
 declare half @llvm.fmuladd.f16(half, half, half) #1
 declare half @llvm.fabs.f16(half) #1
 
 ; GCN-LABEL: {{^}}fmuladd_f16:
 ; VI-FLUSH: v_mac_f16_e32 {{v[0-9]+, v[0-9]+, v[0-9]+}}
 

 ; GFX10-FLUSH:  v_add_f16_e32
 ; GFX10-DENORM-CONTRACT: v_fmac_f16_e32 {{v[0-9]+, v[0-9]+, v[0-9]+}}
 
 define amdgpu_kernel void @fmul_fadd_f16(half addrspace(1)* %out, half addrspace(1)* %in1,
                          half addrspace(1)* %in2, half addrspace(1)* %in3) #0 {
   %r0 = load half, half addrspace(1)* %in1
   %r1 = load half, half addrspace(1)* %in2
   %r2 = load half, half addrspace(1)* %in3
-  %mul = fmul half %r0, %r1
-  %add = fadd half %mul, %r2
+  %mul = fmul FASTMATH_FLAGS half %r0, %r1
+  %add = fadd FASTMATH_FLAGS half %mul, %r2
   store half %add, half addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmul_fadd_contract_f16:
 ; VI-FLUSH: v_mac_f16_e32 {{v[0-9]+, v[0-9]+, v[0-9]+}}
 
 ; VI-DENORM: v_fma_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}}
define amdgpu_kernel void @fadd_a_a_b_f16(half addrspace(1)* %out,
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %gep.0 = getelementptr half, half addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr half, half addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr half, half addrspace(1)* %out, i32 %tid
 
   %r0 = load volatile half, half addrspace(1)* %gep.0
   %r1 = load volatile half, half addrspace(1)* %gep.1
 
-  %add.0 = fadd half %r0, %r0
-  %add.1 = fadd half %add.0, %r1
+  %add.0 = fadd FASTMATH_FLAGS half %r0, %r0
+  %add.1 = fadd FASTMATH_FLAGS half %add.0, %r1
   store half %add.1, half addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fadd_b_a_a_f16:
 ; GCN: {{buffer|flat|global}}_load_ushort [[R1:v[0-9]+]],
 ; GCN: {{buffer|flat|global}}_load_ushort [[R2:v[0-9]+]],
 ; VI-FLUSH: v_mac_f16_e32 [[R2]], 2.0, [[R1]]
define amdgpu_kernel void @fadd_b_a_a_f16(half addrspace(1)* %out,
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %gep.0 = getelementptr half, half addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr half, half addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr half, half addrspace(1)* %out, i32 %tid
 
   %r0 = load volatile half, half addrspace(1)* %gep.0
   %r1 = load volatile half, half addrspace(1)* %gep.1
 
-  %add.0 = fadd half %r0, %r0
-  %add.1 = fadd half %r1, %add.0
+  %add.0 = fadd FASTMATH_FLAGS half %r0, %r0
+  %add.1 = fadd FASTMATH_FLAGS half %r1, %add.0
   store half %add.1, half addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmuladd_neg_2.0_a_b_f16
 ; GCN: {{buffer|flat|global}}_load_ushort [[R1:v[0-9]+]],
 ; GCN: {{buffer|flat|global}}_load_ushort [[R2:v[0-9]+]],
 ; VI-FLUSH:     v_mac_f16_e32 [[R2]], -2.0, [[R1]]
define amdgpu_kernel void @mad_sub_f16(half addrspace(1)* noalias nocapture %out, half addrspace(1)* noalias nocapture readonly %ptr) #1 {
   %add1 = add i64 %tid.ext, 1
   %gep1 = getelementptr half, half addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr half, half addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr half, half addrspace(1)* %out, i64 %tid.ext
   %a = load volatile half, half addrspace(1)* %gep0, align 2
   %b = load volatile half, half addrspace(1)* %gep1, align 2
   %c = load volatile half, half addrspace(1)* %gep2, align 2
-  %mul = fmul half %a, %b
-  %sub = fsub half %mul, %c
+  %mul = fmul FASTMATH_FLAGS half %a, %b
+  %sub = fsub FASTMATH_FLAGS half %mul, %c
   store half %sub, half addrspace(1)* %outgep, align 2
   ret void
 }
 
 ; GCN-LABEL: {{^}}mad_sub_inv_f16:
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGC:v[0-9]+]]
define amdgpu_kernel void @mad_sub_inv_f16(half addrspace(1)* noalias nocapture %out, half addrspace(1)* noalias nocapture readonly %ptr) #1 {
   %add1 = add i64 %tid.ext, 1
   %gep1 = getelementptr half, half addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr half, half addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr half, half addrspace(1)* %out, i64 %tid.ext
   %a = load volatile half, half addrspace(1)* %gep0, align 2
   %b = load volatile half, half addrspace(1)* %gep1, align 2
   %c = load volatile half, half addrspace(1)* %gep2, align 2
-  %mul = fmul half %a, %b
-  %sub = fsub half %c, %mul
+  %mul = fmul FASTMATH_FLAGS half %a, %b
+  %sub = fsub FASTMATH_FLAGS half %c, %mul
   store half %sub, half addrspace(1)* %outgep, align 2
   ret void
 }
 
 ; GCN-LABEL: {{^}}mad_sub_fabs_f16:
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGC:v[0-9]+]]
define amdgpu_kernel void @mad_sub_fabs_f16(half addrspace(1)* noalias nocapture %out, half addrspace(1)* noalias nocapture readonly %ptr) #1 {
   %gep1 = getelementptr half, half addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr half, half addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr half, half addrspace(1)* %out, i64 %tid.ext
   %a = load volatile half, half addrspace(1)* %gep0, align 2
   %b = load volatile half, half addrspace(1)* %gep1, align 2
   %c = load volatile half, half addrspace(1)* %gep2, align 2
   %c.abs = call half @llvm.fabs.f16(half %c) #0
-  %mul = fmul half %a, %b
-  %sub = fsub half %mul, %c.abs
+  %mul = fmul FASTMATH_FLAGS half %a, %b
+  %sub = fsub FASTMATH_FLAGS half %mul, %c.abs
   store half %sub, half addrspace(1)* %outgep, align 2
   ret void
 }
 
 ; GCN-LABEL: {{^}}mad_sub_fabs_inv_f16:
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGC:v[0-9]+]]
define amdgpu_kernel void @mad_sub_fabs_inv_f16(half addrspace(1)* noalias nocapture %out, half addrspace(1)* noalias nocapture readonly %ptr) #1 {
   %gep1 = getelementptr half, half addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr half, half addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr half, half addrspace(1)* %out, i64 %tid.ext
   %a = load volatile half, half addrspace(1)* %gep0, align 2
   %b = load volatile half, half addrspace(1)* %gep1, align 2
   %c = load volatile half, half addrspace(1)* %gep2, align 2
   %c.abs = call half @llvm.fabs.f16(half %c) #0
-  %mul = fmul half %a, %b
-  %sub = fsub half %c.abs, %mul
+  %mul = fmul FASTMATH_FLAGS half %a, %b
+  %sub = fsub FASTMATH_FLAGS half %c.abs, %mul
   store half %sub, half addrspace(1)* %outgep, align 2
   ret void
 }
 
 ; GCN-LABEL: {{^}}neg_neg_mad_f16:
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGC:v[0-9]+]]
define amdgpu_kernel void @neg_neg_mad_f16(half addrspace(1)* noalias nocapture %out, half addrspace(1)* noalias nocapture readonly %ptr) #1 {
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr half, half addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr half, half addrspace(1)* %out, i64 %tid.ext
   %a = load volatile half, half addrspace(1)* %gep0, align 2
   %b = load volatile half, half addrspace(1)* %gep1, align 2
   %c = load volatile half, half addrspace(1)* %gep2, align 2
   %nega = fneg half %a
   %negb = fneg half %b
-  %mul = fmul half %nega, %negb
-  %sub = fadd half %mul, %c
+  %mul = fmul FASTMATH_FLAGS half %nega, %negb
+  %sub = fadd FASTMATH_FLAGS half %mul, %c
   store half %sub, half addrspace(1)* %outgep, align 2
   ret void
 }
 
 ; GCN-LABEL: {{^}}mad_fabs_sub_f16:
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_ushort [[REGC:v[0-9]+]]
define amdgpu_kernel void @mad_fabs_sub_f16(half addrspace(1)* noalias nocapture %out, half addrspace(1)* noalias nocapture readonly %ptr) #1 {
   %gep1 = getelementptr half, half addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr half, half addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr half, half addrspace(1)* %out, i64 %tid.ext
   %a = load volatile half, half addrspace(1)* %gep0, align 2
   %b = load volatile half, half addrspace(1)* %gep1, align 2
   %c = load volatile half, half addrspace(1)* %gep2, align 2
   %b.abs = call half @llvm.fabs.f16(half %b) #0
-  %mul = fmul half %a, %b.abs
-  %sub = fsub half %mul, %c
+  %mul = fmul FASTMATH_FLAGS half %a, %b.abs
+  %sub = fsub FASTMATH_FLAGS half %mul, %c
   store half %sub, half addrspace(1)* %outgep, align 2
   ret void
 }
 
 ; GCN-LABEL: {{^}}fsub_c_fadd_a_a_f16:
 ; GCN: {{buffer|flat|global}}_load_ushort [[R1:v[0-9]+]],
 ; GCN: {{buffer|flat|global}}_load_ushort [[R2:v[0-9]+]],
 ; VI-FLUSH: v_mac_f16_e32 [[R2]], -2.0, [[R1]]
define amdgpu_kernel void @fsub_c_fadd_a_a_f16(half addrspace(1)* %out, half addrspace(1)* %in) {
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %gep.0 = getelementptr half, half addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr half, half addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr half, half addrspace(1)* %out, i32 %tid
 
   %r1 = load volatile half, half addrspace(1)* %gep.0
   %r2 = load volatile half, half addrspace(1)* %gep.1
 
-  %add = fadd half %r1, %r1
-  %r3 = fsub half %r2, %add
+  %add = fadd FASTMATH_FLAGS half %r1, %r1
+  %r3 = fsub FASTMATH_FLAGS half %r2, %add
 
   store half %r3, half addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fsub_fadd_a_a_c_f16:
 ; GCN: {{buffer|flat|global}}_load_ushort [[R1:v[0-9]+]],
 ; GCN: {{buffer|flat|global}}_load_ushort [[R2:v[0-9]+]],
define amdgpu_kernel void @fsub_fadd_a_a_c_f16(half addrspace(1)* %out, half addrspace(1)* %in) {
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %gep.0 = getelementptr half, half addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr half, half addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr half, half addrspace(1)* %out, i32 %tid
 
   %r1 = load volatile half, half addrspace(1)* %gep.0
   %r2 = load volatile half, half addrspace(1)* %gep.1
 
-  %add = fadd half %r1, %r1
-  %r3 = fsub half %add, %r2
+  %add = fadd FASTMATH_FLAGS half %r1, %r1
+  %r3 = fsub FASTMATH_FLAGS half %add, %r2
 
   store half %r3, half addrspace(1)* %gep.out
   ret void
 }
 
 attributes #0 = { nounwind }
 attributes #1 = { nounwind readnone }

llvm/test/CodeGen/AMDGPU/fmuladd.f32.ll

-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -mattr=+fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD,SI %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=ieee -mattr=+fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,SI-DENORM,GCN-DENORM-FASTFMA,SI %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=preserve-sign -mattr=-fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD,SI-FLUSH,SI %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=ieee -mattr=-fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,SI-DENORM,GCN-DENORM-SLOWFMA,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -mattr=+fast-fmaf | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=ieee -mattr=+fast-fmaf | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,SI-DENORM,GCN-DENORM-FASTFMA,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=preserve-sign -mattr=-fast-fmaf | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD,SI-FLUSH,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=ieee -mattr=-fast-fmaf | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,SI-DENORM,GCN-DENORM-SLOWFMA,SI %s
 
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -mattr=+fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD,SI-FLUSH,SI %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=ieee -mattr=+fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,SI-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-CONTRACT,SI %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=preserve-sign -mattr=-fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD,SI-FLUSH,SI %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=ieee -mattr=-fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,SI-DENORM,GCN-DENORM-SLOWFMA,GCN-DENORM-SLOWFMA-CONTRACT,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -mattr=+fast-fmaf | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD,SI-FLUSH,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=ieee -mattr=+fast-fmaf | FileCheck -enable-var-scope -check-prefixes=GCN,SI-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-CONTRACT,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=preserve-sign -mattr=-fast-fmaf | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD,SI-FLUSH,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=ieee -mattr=-fast-fmaf | FileCheck -enable-var-scope -check-prefixes=GCN,SI-DENORM,GCN-DENORM-SLOWFMA,GCN-DENORM-SLOWFMA-CONTRACT,SI %s
 
 
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx900 -denormal-fp-math-f32=preserve-sign -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx900 -denormal-fp-math-f32=ieee -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM-FASTFMA %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx900 -denormal-fp-math-f32=preserve-sign | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-MAD %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx900 -denormal-fp-math-f32=ieee | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM-FASTFMA %s
 
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx906 -denormal-fp-math-f32=preserve-sign -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-FMAC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx906 -denormal-fp-math-f32=preserve-sign | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-FMAC %s
 
 ; FIXME: Should probably test this, but sometimes selecting fmac is painful to match.
-; XUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx906 -denormal-fp-math-f32=ieee -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM-FASTFMA %s
+; XUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx906 -denormal-fp-math-f32=ieee | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM-FASTFMA %s
 
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx1030 -denormal-fp-math-f32=preserve-sign -mattr=+mad-mac-f32-insts -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-FMAC %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx1030 -denormal-fp-math-f32=ieee -mattr=+mad-mac-f32-insts -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx1030 -denormal-fp-math-f32=preserve-sign -mattr=+mad-mac-f32-insts | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-FMAC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx1030 -denormal-fp-math-f32=ieee -mattr=+mad-mac-f32-insts | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT %s
 
 ; Test all permutations of: fp32 denormals, fast fp contract, fp contract enabled for fmuladd, fmaf fast/slow.
 
 target triple = "amdgcn--"
 
 
 declare i32 @llvm.amdgcn.workitem.id.x() #1
 declare float @llvm.fmuladd.f32(float, float, float) #1

 
 ; GCN-DENORM-STRICT: v_mul_f32_e32
 ; GCN-DENORM-STRICT: v_add_f32_e32
 define amdgpu_kernel void @fmul_fadd_f32(float addrspace(1)* %out, float addrspace(1)* %in1,
                            float addrspace(1)* %in2, float addrspace(1)* %in3) #0 {
   %r0 = load volatile float, float addrspace(1)* %in1
   %r1 = load volatile float, float addrspace(1)* %in2
   %r2 = load volatile float, float addrspace(1)* %in3
-  %mul = fmul float %r0, %r1
-  %add = fadd float %mul, %r2
+  %mul = fmul FASTMATH_FLAGS float %r0, %r1
+  %add = fadd FASTMATH_FLAGS float %mul, %r2
   store float %add, float addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmul_fadd_contract_f32:
 ; GCN-FLUSH-FMAC: v_fmac_f32_e32
 
 ; GCN-DENORM-SLOWFMA-CONTRACT: v_mul_f32_e32
define amdgpu_kernel void @fadd_a_a_b_f32(float addrspace(1)* %out,
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %r0 = load volatile float, float addrspace(1)* %gep.0
   %r1 = load volatile float, float addrspace(1)* %gep.1
 
-  %add.0 = fadd float %r0, %r0
-  %add.1 = fadd float %add.0, %r1
+  %add.0 = fadd FASTMATH_FLAGS float %r0, %r0
+  %add.1 = fadd FASTMATH_FLAGS float %add.0, %r1
   store float %add.1, float addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fadd_b_a_a_f32:
 ; GCN: {{buffer|flat|global}}_load_dword [[R1:v[0-9]+]],
 ; GCN: {{buffer|flat|global}}_load_dword [[R2:v[0-9]+]],
 
define amdgpu_kernel void @fadd_b_a_a_f32(float addrspace(1)* %out,
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %r0 = load volatile float, float addrspace(1)* %gep.0
   %r1 = load volatile float, float addrspace(1)* %gep.1
 
-  %add.0 = fadd float %r0, %r0
-  %add.1 = fadd float %r1, %add.0
+  %add.0 = fadd FASTMATH_FLAGS float %r0, %r0
+  %add.1 = fadd FASTMATH_FLAGS float %r1, %add.0
   store float %add.1, float addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmuladd_neg_2.0_a_b_f32
 ; GCN: {{buffer|flat|global}}_load_dword [[R1:v[0-9]+]],
 ; GCN: {{buffer|flat|global}}_load_dword [[R2:v[0-9]+]],
 ; GCN-FLUSH-MAD: v_mac_f32_e32 [[R2]], -2.0, [[R1]]
define amdgpu_kernel void @mad_sub_f32(float addrspace(1)* noalias nocapture %out, float addrspace(1)* noalias nocapture readonly %ptr) #0 {
   %add1 = add i64 %tid.ext, 1
   %gep1 = getelementptr float, float addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr float, float addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr float, float addrspace(1)* %out, i64 %tid.ext
   %a = load volatile float, float addrspace(1)* %gep0, align 4
   %b = load volatile float, float addrspace(1)* %gep1, align 4
   %c = load volatile float, float addrspace(1)* %gep2, align 4
-  %mul = fmul float %a, %b
-  %sub = fsub float %mul, %c
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %sub = fsub FASTMATH_FLAGS float %mul, %c
   store float %sub, float addrspace(1)* %outgep, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}mad_sub_inv_f32:
 ; GCN: {{buffer|flat|global}}_load_dword [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGC:v[0-9]+]]
define amdgpu_kernel void @mad_sub_inv_f32(float addrspace(1)* noalias nocapture %out, float addrspace(1)* noalias nocapture readonly %ptr) #0 {
   %add1 = add i64 %tid.ext, 1
   %gep1 = getelementptr float, float addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr float, float addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr float, float addrspace(1)* %out, i64 %tid.ext
   %a = load volatile float, float addrspace(1)* %gep0, align 4
   %b = load volatile float, float addrspace(1)* %gep1, align 4
   %c = load volatile float, float addrspace(1)* %gep2, align 4
-  %mul = fmul float %a, %b
-  %sub = fsub float %c, %mul
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %sub = fsub FASTMATH_FLAGS float %c, %mul
   store float %sub, float addrspace(1)* %outgep, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}mad_sub_fabs_f32:
 ; GCN: {{buffer|flat|global}}_load_dword [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGC:v[0-9]+]]
define amdgpu_kernel void @mad_sub_fabs_f32(float addrspace(1)* noalias nocapture %out, float addrspace(1)* noalias nocapture readonly %ptr) #0 {
   %gep1 = getelementptr float, float addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr float, float addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr float, float addrspace(1)* %out, i64 %tid.ext
   %a = load volatile float, float addrspace(1)* %gep0, align 4
   %b = load volatile float, float addrspace(1)* %gep1, align 4
   %c = load volatile float, float addrspace(1)* %gep2, align 4
   %c.abs = call float @llvm.fabs.f32(float %c) #0
-  %mul = fmul float %a, %b
-  %sub = fsub float %mul, %c.abs
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %sub = fsub FASTMATH_FLAGS float %mul, %c.abs
   store float %sub, float addrspace(1)* %outgep, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}mad_sub_fabs_inv_f32:
 ; GCN: {{buffer|flat|global}}_load_dword [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGC:v[0-9]+]]
define amdgpu_kernel void @mad_sub_fabs_inv_f32(float addrspace(1)* noalias nocapture %out, float addrspace(1)* noalias nocapture readonly %ptr) #0 {
   %gep1 = getelementptr float, float addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr float, float addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr float, float addrspace(1)* %out, i64 %tid.ext
   %a = load volatile float, float addrspace(1)* %gep0, align 4
   %b = load volatile float, float addrspace(1)* %gep1, align 4
   %c = load volatile float, float addrspace(1)* %gep2, align 4
   %c.abs = call float @llvm.fabs.f32(float %c) #0
-  %mul = fmul float %a, %b
-  %sub = fsub float %c.abs, %mul
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %sub = fsub FASTMATH_FLAGS float %c.abs, %mul
   store float %sub, float addrspace(1)* %outgep, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}neg_neg_mad_f32:
 ; GCN: {{buffer|flat|global}}_load_dword [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGC:v[0-9]+]]
define amdgpu_kernel void @neg_neg_mad_f32(float addrspace(1)* noalias nocapture %out, float addrspace(1)* noalias nocapture readonly %ptr) #0 {
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr float, float addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr float, float addrspace(1)* %out, i64 %tid.ext
   %a = load volatile float, float addrspace(1)* %gep0, align 4
   %b = load volatile float, float addrspace(1)* %gep1, align 4
   %c = load volatile float, float addrspace(1)* %gep2, align 4
   %nega = fneg float %a
   %negb = fneg float %b
-  %mul = fmul float %nega, %negb
-  %sub = fadd float %mul, %c
+  %mul = fmul FASTMATH_FLAGS float %nega, %negb
+  %sub = fadd FASTMATH_FLAGS float %mul, %c
   store float %sub, float addrspace(1)* %outgep, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}mad_fabs_sub_f32:
 ; GCN: {{buffer|flat|global}}_load_dword [[REGA:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGB:v[0-9]+]]
 ; GCN: {{buffer|flat|global}}_load_dword [[REGC:v[0-9]+]]
define amdgpu_kernel void @mad_fabs_sub_f32(float addrspace(1)* noalias nocapture %out, float addrspace(1)* noalias nocapture readonly %ptr) #0 {
   %gep1 = getelementptr float, float addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr float, float addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr float, float addrspace(1)* %out, i64 %tid.ext
   %a = load volatile float, float addrspace(1)* %gep0, align 4
   %b = load volatile float, float addrspace(1)* %gep1, align 4
   %c = load volatile float, float addrspace(1)* %gep2, align 4
   %b.abs = call float @llvm.fabs.f32(float %b) #0
-  %mul = fmul float %a, %b.abs
-  %sub = fsub float %mul, %c
+  %mul = fmul FASTMATH_FLAGS float %a, %b.abs
+  %sub = fsub FASTMATH_FLAGS float %mul, %c
   store float %sub, float addrspace(1)* %outgep, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}fsub_c_fadd_a_a_f32:
 ; GCN: {{buffer|flat|global}}_load_dword [[R1:v[0-9]+]],
 ; GCN: {{buffer|flat|global}}_load_dword [[R2:v[0-9]+]],
 ; GCN-FLUSH: v_mac_f32_e32 [[R2]], -2.0, [[R1]]
define amdgpu_kernel void @fsub_c_fadd_a_a_f32(float addrspace(1)* %out, float addrspace(1)* %in) #0 {
   %tid = call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %r1 = load volatile float, float addrspace(1)* %gep.0
   %r2 = load volatile float, float addrspace(1)* %gep.1
 
-  %add = fadd float %r1, %r1
-  %r3 = fsub float %r2, %add
+  %add = fadd FASTMATH_FLAGS float %r1, %r1
+  %r3 = fsub FASTMATH_FLAGS float %r2, %add
 
   store float %r3, float addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fsub_fadd_a_a_c_f32:
 ; GCN: {{buffer|flat|global}}_load_dword [[R1:v[0-9]+]],
 ; GCN: {{buffer|flat|global}}_load_dword [[R2:v[0-9]+]],
define amdgpu_kernel void @fsub_fadd_a_a_c_f32(float addrspace(1)* %out, float addrspace(1)* %in) #0 {
   %tid = call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %r1 = load volatile float, float addrspace(1)* %gep.0
   %r2 = load volatile float, float addrspace(1)* %gep.1
 
-  %add = fadd float %r1, %r1
-  %r3 = fsub float %add, %r2
+  %add = fadd FASTMATH_FLAGS float %r1, %r1
+  %r3 = fsub FASTMATH_FLAGS float %add, %r2
 
   store float %r3, float addrspace(1)* %gep.out
   ret void
 }
 
 attributes #0 = { nounwind }
 attributes #1 = { nounwind readnone }

llvm/test/CodeGen/AMDGPU/fmuladd.f64.ll

-; RUN:  llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=tahiti -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,SI %s
-; RUN:  llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=verde  -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,SI %s
-; RUN:  llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=tahiti -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,SI %s
-; RUN:  llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=verde  -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,SI %s
-; RUN:  llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=tonga -mattr=-flat-for-global  -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,VI %s
-; RUN:  llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=tonga -mattr=-flat-for-global  -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,VI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=tahiti -verify-machineinstrs | FileCheck -check-prefixes=GCN,GCN-STRICT,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=verde  -verify-machineinstrs | FileCheck -check-prefixes=GCN,GCN-STRICT,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=tahiti -verify-machineinstrs | FileCheck -check-prefixes=GCN,GCN-CONTRACT,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=verde  -verify-machineinstrs | FileCheck -check-prefixes=GCN,GCN-CONTRACT,SI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=tonga -mattr=-flat-for-global  -verify-machineinstrs | FileCheck -check-prefixes=GCN,GCN-STRICT,VI %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false  -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -verify-machineinstrs | FileCheck -check-prefixes=GCN,GCN-CONTRACT,VI %s
 
 ; GCN-LABEL: {{^}}fmuladd_f64:
 ; GCN: v_fma_f64 {{v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\]}}
 define amdgpu_kernel void @fmuladd_f64(double addrspace(1)* %out, double addrspace(1)* %in1,
                          double addrspace(1)* %in2, double addrspace(1)* %in3) #0 {
   %r0 = load double, double addrspace(1)* %in1
   %r1 = load double, double addrspace(1)* %in2
   %r2 = load double, double addrspace(1)* %in3
   %r3 = tail call double @llvm.fmuladd.f64(double %r0, double %r1, double %r2)
   store double %r3, double addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmul_fadd_f64:
 ; GCN-CONTRACT: v_fma_f64 {{v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\]}}
 
 ; GCN-STRICT: v_mul_f64 {{v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\]}}
 ; GCN-STRICT: v_add_f64 {{v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\]}}
 define amdgpu_kernel void @fmul_fadd_f64(double addrspace(1)* %out, double addrspace(1)* %in1,
                            double addrspace(1)* %in2, double addrspace(1)* %in3) #0 {
   %r0 = load double, double addrspace(1)* %in1
   %r1 = load double, double addrspace(1)* %in2
   %r2 = load double, double addrspace(1)* %in3
-  %tmp = fmul double %r0, %r1
-  %r3 = fadd double %tmp, %r2
+  %tmp = fmul FASTMATH_FLAGS double %r0, %r1
+  %r3 = fadd FASTMATH_FLAGS double %tmp, %r2
   store double %r3, double addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmul_fadd_contract_f64:
 ; GCN: v_fma_f64 {{v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\], v\[[0-9]+:[0-9]+\]}}
 
 define amdgpu_kernel void @fmul_fadd_contract_f64(double addrspace(1)* %out, double addrspace(1)* %in1,
define amdgpu_kernel void @fadd_a_a_b_f64(double addrspace(1)* %out,
   %tid = call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %gep.0 = getelementptr double, double addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr double, double addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %r0 = load volatile double, double addrspace(1)* %gep.0
   %r1 = load volatile double, double addrspace(1)* %gep.1
 
-  %add.0 = fadd double %r0, %r0
-  %add.1 = fadd double %add.0, %r1
+  %add.0 = fadd FASTMATH_FLAGS double %r0, %r0
+  %add.1 = fadd FASTMATH_FLAGS double %add.0, %r1
   store double %add.1, double addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fadd_b_a_a_f64:
 ; GCN: {{buffer|flat}}_load_dwordx2 [[R1:v\[[0-9]+:[0-9]+\]]],
 ; GCN: {{buffer|flat}}_load_dwordx2 [[R2:v\[[0-9]+:[0-9]+\]]],
 
define amdgpu_kernel void @fadd_b_a_a_f64(double addrspace(1)* %out,
   %tid = call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %gep.0 = getelementptr double, double addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr double, double addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %r0 = load volatile double, double addrspace(1)* %gep.0
   %r1 = load volatile double, double addrspace(1)* %gep.1
 
-  %add.0 = fadd double %r0, %r0
-  %add.1 = fadd double %r1, %add.0
+  %add.0 = fadd FASTMATH_FLAGS double %r0, %r0
+  %add.1 = fadd FASTMATH_FLAGS double %r1, %add.0
   store double %add.1, double addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}mad_sub_f64:
 ; GCN-STRICT: v_mul_f64 v{{\[[0-9]+:[0-9]+\]}}, v{{\[[0-9]+:[0-9]+\]}}, v{{\[[0-9]+:[0-9]+\]}}
 ; GCN-STRICT: v_add_f64 v{{\[[0-9]+:[0-9]+\]}}, v{{\[[0-9]+:[0-9]+\]}}, -v{{\[[0-9]+:[0-9]+\]}}
 
 ; GCN-CONTRACT: v_fma_f64 v{{\[[0-9]+:[0-9]+\]}}, v{{\[[0-9]+:[0-9]+\]}}, v{{\[[0-9]+:[0-9]+\]}}, -v{{\[[0-9]+:[0-9]+\]}}
 define amdgpu_kernel void @mad_sub_f64(double addrspace(1)* noalias nocapture %out, double addrspace(1)* noalias nocapture readonly %ptr) #1 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() #0
   %tid.ext = sext i32 %tid to i64
   %gep0 = getelementptr double, double addrspace(1)* %ptr, i64 %tid.ext
   %add1 = add i64 %tid.ext, 1
   %gep1 = getelementptr double, double addrspace(1)* %ptr, i64 %add1
   %add2 = add i64 %tid.ext, 2
   %gep2 = getelementptr double, double addrspace(1)* %ptr, i64 %add2
   %outgep = getelementptr double, double addrspace(1)* %out, i64 %tid.ext
   %a = load volatile double, double addrspace(1)* %gep0, align 8
   %b = load volatile double, double addrspace(1)* %gep1, align 8
   %c = load volatile double, double addrspace(1)* %gep2, align 8
-  %mul = fmul double %a, %b
-  %sub = fsub double %mul, %c
+  %mul = fmul FASTMATH_FLAGS double %a, %b
+  %sub = fsub FASTMATH_FLAGS double %mul, %c
   store double %sub, double addrspace(1)* %outgep, align 8
   ret void
 }
 
 ; GCN-LABEL: {{^}}fadd_a_a_b_f64_fast_add0:
 ; GCN-STRICT: v_add_f64
 ; GCN-STRICT: v_add_f64
 
 ; GCN-CONTRACT: v_fma_f64
 define amdgpu_kernel void @fadd_a_a_b_f64_fast_add0(double addrspace(1)* %out,
                                       double addrspace(1)* %in1,
                                       double addrspace(1)* %in2) #0 {
   %tid = call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %gep.0 = getelementptr double, double addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr double, double addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %r0 = load volatile double, double addrspace(1)* %gep.0
   %r1 = load volatile double, double addrspace(1)* %gep.1
 
   %add.0 = fadd fast double %r0, %r0
-  %add.1 = fadd double %add.0, %r1
+  %add.1 = fadd FASTMATH_FLAGS double %add.0, %r1
   store double %add.1, double addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fadd_a_a_b_f64_fast_add1:
 ; GCN-STRICT: v_add_f64
 ; GCN-STRICT: v_add_f64
 
 ; GCN-CONTRACT: v_fma_f64
 define amdgpu_kernel void @fadd_a_a_b_f64_fast_add1(double addrspace(1)* %out,
                                       double addrspace(1)* %in1,
                                       double addrspace(1)* %in2) #0 {
   %tid = call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %gep.0 = getelementptr double, double addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr double, double addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr double, double addrspace(1)* %out, i32 %tid
 
   %r0 = load volatile double, double addrspace(1)* %gep.0
   %r1 = load volatile double, double addrspace(1)* %gep.1
 
-  %add.0 = fadd double %r0, %r0
+  %add.0 = fadd FASTMATH_FLAGS double %r0, %r0
   %add.1 = fadd fast double %add.0, %r1
   store double %add.1, double addrspace(1)* %gep.out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fadd_a_a_b_f64_fast:
 ; GCN: v_fma_f64
 define amdgpu_kernel void @fadd_a_a_b_f64_fast(double addrspace(1)* %out,

llvm/test/CodeGen/AMDGPU/fmuladd.v2f16.ll

-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX9-FLUSH %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX9-FLUSH %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX9-FLUSH %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX9-FLUSH %s
-
-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX9-DENORM-STRICT,GFX9-DENORM %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX9-DENORM-STRICT,GFX9-DENORM %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX9-DENORM-CONTRACT,GFX9-DENORM %s
-; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX9-DENORM-CONTRACT,GFX9-DENORM %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX9-FLUSH %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX9-FLUSH %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX9-FLUSH %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX9-FLUSH %s
+
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX9-DENORM-STRICT,GFX9-DENORM %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX9-DENORM-STRICT,GFX9-DENORM %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX9-DENORM-CONTRACT,GFX9-DENORM %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX9-DENORM-CONTRACT,GFX9-DENORM %s
 
 declare i32 @llvm.amdgcn.workitem.id.x() #1
 declare <2 x half> @llvm.fmuladd.v2f16(<2 x half>, <2 x half>, <2 x half>) #1
 declare <2 x half> @llvm.fabs.v2f16(<2 x half>) #1
 
 ; GCN-LABEL: {{^}}fmuladd_v2f16:
 ; GFX9-FLUSH: v_pk_mul_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}}
 ; GFX9-FLUSH: v_pk_add_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}}

 ; GFX9-DENORM-STRICT: v_pk_add_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}}
 
 ; GFX9-DENORM-CONTRACT: v_pk_fma_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}}
 define amdgpu_kernel void @fmul_fadd_v2f16(<2 x half> addrspace(1)* %out, <2 x half> addrspace(1)* %in1,
                          <2 x half> addrspace(1)* %in2, <2 x half> addrspace(1)* %in3) #0 {
   %r0 = load <2 x half>, <2 x half> addrspace(1)* %in1
   %r1 = load <2 x half>, <2 x half> addrspace(1)* %in2
   %r2 = load <2 x half>, <2 x half> addrspace(1)* %in3
-  %r3 = fmul <2 x half> %r0, %r1
-  %r4 = fadd <2 x half> %r3, %r2
+  %r3 = fmul FASTMATH_FLAGS <2 x half> %r0, %r1
+  %r4 = fadd FASTMATH_FLAGS <2 x half> %r3, %r2
   store <2 x half> %r4, <2 x half> addrspace(1)* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}fmul_fadd_contract_v2f16:
 ; GFX9-FLUSH: v_pk_mul_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}}
 ; GFX9-FLUSH: v_pk_add_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}}
 
define amdgpu_kernel void @fadd_a_a_b_v2f16(<2 x half> addrspace(1)* %out,
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %gep.0 = getelementptr <2 x half>, <2 x half> addrspace(1)* %out, i32 %tid
   %gep.1 = getelementptr <2 x half>, <2 x half> addrspace(1)* %gep.0, i32 1
   %gep.out = getelementptr <2 x half>, <2 x half> addrspace(1)* %out, i32 %tid
 
   %r0 = load volatile <2 x half>, <2 x half> addrspace(1)* %gep.0
   %r1 = load volatile <2 x half>, <2 x half> addrspace(1)* %gep.1
 
-  %add.0 = fadd <2 x half> %r0, %r0
-  %add.1 = fadd <2 x half> %add.0, %r1
+  %add.0 = fadd FASTMATH_FLAGS <2 x half> %r0, %r0
+  %add.1 = fadd FASTMATH_FLAGS <2 x half> %add.0, %r1
   store <2 x half> %add.1, <2 x half> addrspace(1)* %gep.out
   ret void
 }
 
 attributes #0 = { nounwind }
 attributes #1 = { nounwind readnone }

llvm/test/CodeGen/AMDGPU/mad-combine.ll

 ; Make sure we still form mad even when unsafe math or fp-contract is allowed instead of fma.
 
-; RUN: llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD  -check-prefix=SI-STD-SAFE -check-prefix=FUNC %s
-; RUN: llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-SAFE -check-prefix=FUNC %s
-; RUN: llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs -enable-unsafe-fp-math < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-UNSAFE -check-prefix=FUNC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD  -check-prefix=SI-STD-SAFE -check-prefix=FUNC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-SAFE -check-prefix=FUNC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,reassoc contract,g" %s | llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-UNSAFE -check-prefix=FUNC %s
 
 ; FIXME: Remove enable-unsafe-fp-math in RUN line and add flags to IR instrs
 
 ; Make sure we don't form mad with denormals
-; RUN: llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-DENORM -check-prefix=SI-DENORM-FASTFMAF -check-prefix=FUNC %s
-; RUN: llc -march=amdgcn -mcpu=verde -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-DENORM -check-prefix=SI-DENORM-SLOWFMAF -check-prefix=FUNC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-DENORM -check-prefix=SI-DENORM-FASTFMAF -check-prefix=FUNC %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=verde -denormal-fp-math-f32=ieee -verify-machineinstrs | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-DENORM -check-prefix=SI-DENORM-SLOWFMAF -check-prefix=FUNC %s
 
 declare i32 @llvm.amdgcn.workitem.id.x() #0
 declare float @llvm.fabs.f32(float) #0
 declare float @llvm.fma.f32(float, float, float) #0
 declare float @llvm.fmuladd.f32(float, float, float) #0
 
 ; (fadd (fmul x, y), z) -> (fma x, y, z)
 ; FUNC-LABEL: {{^}}combine_to_mad_f32_0:
define amdgpu_kernel void @combine_to_mad_f32_0(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr float, float addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
 
-  %mul = fmul float %a, %b
-  %fma = fadd float %mul, %c
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %fma = fadd FASTMATH_FLAGS float %mul, %c
   store float %fma, float addrspace(1)* %gep.out
   ret void
 }
 
 ; (fadd (fmul x, y), z) -> (fma x, y, z)
 ; FUNC-LABEL: {{^}}combine_to_mad_f32_0_2use:
 ; SI-DAG: buffer_load_dword [[A:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dword [[B:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 glc{{$}}
define amdgpu_kernel void @combine_to_mad_f32_0_2use(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr float, float addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
   %d = load volatile float, float addrspace(1)* %gep.3
 
-  %mul = fmul float %a, %b
-  %fma0 = fadd float %mul, %c
-  %fma1 = fadd float %mul, %d
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %fma0 = fadd FASTMATH_FLAGS float %mul, %c
+  %fma1 = fadd FASTMATH_FLAGS float %mul, %d
 
   store volatile float %fma0, float addrspace(1)* %gep.out.0
   store volatile float %fma1, float addrspace(1)* %gep.out.1
   ret void
 }
 
 ; (fadd x, (fmul y, z)) -> (fma y, z, x)
 ; FUNC-LABEL: {{^}}combine_to_mad_f32_1:
define amdgpu_kernel void @combine_to_mad_f32_1(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr float, float addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
 
-  %mul = fmul float %a, %b
-  %fma = fadd float %c, %mul
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %fma = fadd FASTMATH_FLAGS float %c, %mul
   store float %fma, float addrspace(1)* %gep.out
   ret void
 }
 
 ; (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_mad_fsub_0_f32:
 ; SI-DAG: buffer_load_dword [[A:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dword [[B:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 glc{{$}}
define amdgpu_kernel void @combine_to_mad_fsub_0_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr float, float addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
 
-  %mul = fmul float %a, %b
-  %fma = fsub float %mul, %c
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %fma = fsub FASTMATH_FLAGS float %mul, %c
   store float %fma, float addrspace(1)* %gep.out
   ret void
 }
 
 ; (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_mad_fsub_0_f32_2use:
 ; SI-DAG: buffer_load_dword [[A:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dword [[B:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 glc{{$}}
define amdgpu_kernel void @combine_to_mad_fsub_0_f32_2use(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr float, float addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
   %d = load volatile float, float addrspace(1)* %gep.3
 
-  %mul = fmul float %a, %b
-  %fma0 = fsub float %mul, %c
-  %fma1 = fsub float %mul, %d
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %fma0 = fsub FASTMATH_FLAGS float %mul, %c
+  %fma1 = fsub FASTMATH_FLAGS float %mul, %d
   store volatile float %fma0, float addrspace(1)* %gep.out.0
   store volatile float %fma1, float addrspace(1)* %gep.out.1
   ret void
 }
 
 ; (fsub x, (fmul y, z)) -> (fma (fneg y), z, x)
 ; FUNC-LABEL: {{^}}combine_to_mad_fsub_1_f32:
 ; SI-DAG: buffer_load_dword [[A:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
define amdgpu_kernel void @combine_to_mad_fsub_1_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr float, float addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
 
-  %mul = fmul float %a, %b
-  %fma = fsub float %c, %mul
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %fma = fsub FASTMATH_FLAGS float %c, %mul
   store float %fma, float addrspace(1)* %gep.out
   ret void
 }
 
 ; (fsub x, (fmul y, z)) -> (fma (fneg y), z, x)
 ; FUNC-LABEL: {{^}}combine_to_mad_fsub_1_f32_2use:
 ; SI-DAG: buffer_load_dword [[A:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
 ; SI-DAG: buffer_load_dword [[B:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 glc{{$}}
define amdgpu_kernel void @combine_to_mad_fsub_1_f32_2use(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr float, float addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
   %d = load volatile float, float addrspace(1)* %gep.3
 
-  %mul = fmul float %a, %b
-  %fma0 = fsub float %c, %mul
-  %fma1 = fsub float %d, %mul
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %fma0 = fsub FASTMATH_FLAGS float %c, %mul
+  %fma1 = fsub FASTMATH_FLAGS float %d, %mul
   store volatile float %fma0, float addrspace(1)* %gep.out.0
   store volatile float %fma1, float addrspace(1)* %gep.out.1
   ret void
 }
 
 ; (fsub (fneg (fmul x, y)), z) -> (fma (fneg x), y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_mad_fsub_2_f32:
 ; SI-DAG: buffer_load_dword [[A:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
define amdgpu_kernel void @combine_to_mad_fsub_2_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
   %gep.2 = getelementptr float, float addrspace(1)* %gep.0, i32 2
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
 
-  %mul = fmul float %a, %b
-  %mul.neg = fneg float %mul
-  %fma = fsub float %mul.neg, %c
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %mul.neg = fneg FASTMATH_FLAGS float %mul
+  %fma = fsub FASTMATH_FLAGS float %mul.neg, %c
 
   store float %fma, float addrspace(1)* %gep.out
   ret void
 }
 
 ; (fsub (fneg (fmul x, y)), z) -> (fma (fneg x), y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_mad_fsub_2_f32_2uses_neg:
 ; SI-DAG: buffer_load_dword [[A:v[0-9]+]], v{{\[[0-9]+:[0-9]+\]}}, s{{\[[0-9]+:[0-9]+\]}}, 0 addr64 glc{{$}}
define amdgpu_kernel void @combine_to_mad_fsub_2_f32_2uses_neg(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr float, float addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
   %d = load volatile float, float addrspace(1)* %gep.3
 
-  %mul = fmul float %a, %b
-  %mul.neg = fneg float %mul
-  %fma0 = fsub float %mul.neg, %c
-  %fma1 = fsub float %mul.neg, %d
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %mul.neg = fneg FASTMATH_FLAGS float %mul
+  %fma0 = fsub FASTMATH_FLAGS float %mul.neg, %c
+  %fma1 = fsub FASTMATH_FLAGS float %mul.neg, %d
 
   store volatile float %fma0, float addrspace(1)* %gep.out.0
   store volatile float %fma1, float addrspace(1)* %gep.out.1
   ret void
 }
 
 ; (fsub (fneg (fmul x, y)), z) -> (fma (fneg x), y, (fneg z))
 ; FUNC-LABEL: {{^}}combine_to_mad_fsub_2_f32_2uses_mul:
define amdgpu_kernel void @combine_to_mad_fsub_2_f32_2uses_mul(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.out.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %gep.out.1 = getelementptr float, float addrspace(1)* %gep.out.0, i32 1
 
   %a = load volatile float, float addrspace(1)* %gep.0
   %b = load volatile float, float addrspace(1)* %gep.1
   %c = load volatile float, float addrspace(1)* %gep.2
   %d = load volatile float, float addrspace(1)* %gep.3
 
-  %mul = fmul float %a, %b
-  %mul.neg = fneg float %mul
-  %fma0 = fsub float %mul.neg, %c
-  %fma1 = fsub float %mul, %d
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %mul.neg = fneg FASTMATH_FLAGS float %mul
+  %fma0 = fsub FASTMATH_FLAGS float %mul.neg, %c
+  %fma1 = fsub FASTMATH_FLAGS float %mul, %d
 
   store volatile float %fma0, float addrspace(1)* %gep.out.0
   store volatile float %fma1, float addrspace(1)* %gep.out.1
   ret void
 }
 
 ; fold (fsub (fma x, y, (fmul u, v)), z) -> (fma x, y (fma u, v, (fneg z)))
 
define amdgpu_kernel void @aggressive_combine_to_mad_fsub_0_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %x = load volatile float, float addrspace(1)* %gep.0
   %y = load volatile float, float addrspace(1)* %gep.1
   %z = load volatile float, float addrspace(1)* %gep.2
   %u = load volatile float, float addrspace(1)* %gep.3
   %v = load volatile float, float addrspace(1)* %gep.4
 
-  %tmp0 = fmul float %u, %v
-  %tmp1 = call float @llvm.fma.f32(float %x, float %y, float %tmp0) #0
-  %tmp2 = fsub float %tmp1, %z
+  %tmp0 = fmul FASTMATH_FLAGS float %u, %v
+  %tmp1 = call FASTMATH_FLAGS float @llvm.fma.f32(float %x, float %y, float %tmp0) #0
+  %tmp2 = fsub FASTMATH_FLAGS float %tmp1, %z
 
   store float %tmp2, float addrspace(1)* %gep.out
   ret void
 }
 
 ; fold (fsub x, (fma y, z, (fmul u, v)))
 ;   -> (fma (fneg y), z, (fma (fneg u), v, x))
 
define amdgpu_kernel void @aggressive_combine_to_mad_fsub_1_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %x = load volatile float, float addrspace(1)* %gep.0
   %y = load volatile float, float addrspace(1)* %gep.1
   %z = load volatile float, float addrspace(1)* %gep.2
   %u = load volatile float, float addrspace(1)* %gep.3
   %v = load volatile float, float addrspace(1)* %gep.4
 
-  %tmp0 = fmul float %u, %v
-  %tmp1 = call float @llvm.fma.f32(float %y, float %z, float %tmp0) #0
-  %tmp2 = fsub float %x, %tmp1
+  %tmp0 = fmul FASTMATH_FLAGS float %u, %v
+  %tmp1 = call FASTMATH_FLAGS float @llvm.fma.f32(float %y, float %z, float %tmp0) #0
+  %tmp2 = fsub FASTMATH_FLAGS float %x, %tmp1
 
   store float %tmp2, float addrspace(1)* %gep.out
   ret void
 }
 
 ; fold (fsub (fma x, y, (fmul u, v)), z) -> (fma x, y (fma u, v, (fneg z)))
 
 ; FUNC-LABEL: {{^}}aggressive_combine_to_mad_fsub_2_f32:
define amdgpu_kernel void @aggressive_combine_to_mad_fsub_2_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
   %gep.out = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %x = load volatile float, float addrspace(1)* %gep.0
   %y = load volatile float, float addrspace(1)* %gep.1
   %z = load volatile float, float addrspace(1)* %gep.2
   %u = load volatile float, float addrspace(1)* %gep.3
   %v = load volatile float, float addrspace(1)* %gep.4
 
-  %tmp0 = fmul float %u, %v
-  %tmp1 = call float @llvm.fmuladd.f32(float %x, float %y, float %tmp0) #0
-  %tmp2 = fsub float %tmp1, %z
+  %tmp0 = fmul FASTMATH_FLAGS float %u, %v
+  %tmp1 = call FASTMATH_FLAGS float @llvm.fmuladd.f32(float %x, float %y, float %tmp0) #0
+  %tmp2 = fsub FASTMATH_FLAGS float %tmp1, %z
 
   store float %tmp2, float addrspace(1)* %gep.out
   ret void
 }
 
 ; fold (fsub x, (fmuladd y, z, (fmul u, v)))
 ;   -> (fmuladd (fneg y), z, (fmuladd (fneg u), v, x))
 
define amdgpu_kernel void @aggressive_combine_to_mad_fsub_3_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #1 {
 
   %x = load volatile float, float addrspace(1)* %gep.0
   %y = load volatile float, float addrspace(1)* %gep.1
   %z = load volatile float, float addrspace(1)* %gep.2
   %u = load volatile float, float addrspace(1)* %gep.3
   %v = load volatile float, float addrspace(1)* %gep.4
 
   ; nsz flag is needed since this combine may change sign of zero
-  %tmp0 = fmul nsz float %u, %v
-  %tmp1 = call nsz float @llvm.fmuladd.f32(float %y, float %z, float %tmp0) #0
-  %tmp2 = fsub nsz float %x, %tmp1
+  %tmp0 = fmul FASTMATH_FLAGS nsz float %u, %v
+  %tmp1 = call FASTMATH_FLAGS nsz float @llvm.fmuladd.f32(float %y, float %z, float %tmp0) #0
+  %tmp2 = fsub FASTMATH_FLAGS nsz float %x, %tmp1
 
   store float %tmp2, float addrspace(1)* %gep.out
   ret void
 }
 
 attributes #0 = { nounwind readnone }
 attributes #1 = { nounwind }

llvm/test/CodeGen/AMDGPU/madak.ll

-; RUN: llc -march=amdgcn -mcpu=tahiti -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX6,GFX6_8_9,MAD %s
-; RUN: llc -march=amdgcn -mcpu=tonga -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX8,GFX6_8_9,GFX8_9,GFX8_9_10,MAD %s
-; RUN: llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX9,GFX6_8_9,GFX8_9,GFX8_9_10,MAD %s
-; RUN: llc -march=amdgcn -mcpu=gfx1010 -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX10,GFX8_9_10,GFX10-MAD %s
-; RUN: llc -march=amdgcn -mcpu=gfx1010 -verify-machineinstrs -fp-contract=fast < %s | FileCheck -check-prefixes=GCN,GFX10,GFX8_9_10,FMA %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=tahiti -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX6,GFX6_8_9,MAD %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=tonga -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX8,GFX6_8_9,GFX8_9,GFX8_9_10,MAD %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX9,GFX6_8_9,GFX8_9,GFX8_9_10,MAD %s
+; RUN: sed -e "s,FASTMATH_FLAGS,,g" %s | llc -march=amdgcn -mcpu=gfx1010 -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX10,GFX8_9_10,GFX10-MAD %s
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -march=amdgcn -mcpu=gfx1010 -verify-machineinstrs | FileCheck -check-prefixes=GCN,GFX10,GFX8_9_10,FMA %s
 
 declare i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
 declare float @llvm.fabs.f32(float) nounwind readnone
 
 ; GCN-LABEL: {{^}}madak_f32:
 ; GFX6:   buffer_load_dword [[VA:v[0-9]+]]
 ; GFX6:   buffer_load_dword [[VB:v[0-9]+]]
 ; GFX8: {{flat|global}}_load_dword [[VA:v[0-9]+]]
define amdgpu_kernel void @madak_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) #0 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
   %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
   %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load float, float addrspace(1)* %in.a.gep, align 4
   %b = load float, float addrspace(1)* %in.b.gep, align 4
 
-  %mul = fmul float %a, %b
-  %madak = fadd float %mul, 10.0
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %madak = fadd FASTMATH_FLAGS float %mul, 10.0
   store float %madak, float addrspace(1)* %out.gep, align 4
   ret void
 }
 
 ; Make sure this is only folded with one use. This is a code size
 ; optimization and if we fold the immediate multiple times, we'll undo
 ; it.
 
define amdgpu_kernel void @madak_2_use_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
 
   %out.gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
   %out.gep.1 = getelementptr float, float addrspace(1)* %in.gep.0, i32 1
 
   %a = load volatile float, float addrspace(1)* %in.gep.0, align 4
   %b = load volatile float, float addrspace(1)* %in.gep.1, align 4
   %c = load volatile float, float addrspace(1)* %in.gep.2, align 4
 
-  %mul0 = fmul float %a, %b
-  %mul1 = fmul float %a, %c
-  %madak0 = fadd float %mul0, 10.0
-  %madak1 = fadd float %mul1, 10.0
+  %mul0 = fmul FASTMATH_FLAGS float %a, %b
+  %mul1 = fmul FASTMATH_FLAGS float %a, %c
+  %madak0 = fadd FASTMATH_FLAGS float %mul0, 10.0
+  %madak1 = fadd FASTMATH_FLAGS float %mul1, 10.0
 
   store volatile float %madak0, float addrspace(1)* %out.gep.0, align 4
   store volatile float %madak1, float addrspace(1)* %out.gep.1, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}madak_m_inline_imm_f32:
 ; GCN: {{buffer|flat|global}}_load_dword [[VA:v[0-9]+]]
 ; MAD: v_madak_f32 {{v[0-9]+}}, 4.0, [[VA]], 0x41200000
 ; GFX10-MAD: v_madak_f32 {{v[0-9]+}}, 4.0, [[VA]], 0x41200000
 ; FMA: v_fmaak_f32 {{v[0-9]+}}, 4.0, [[VA]], 0x41200000
 define amdgpu_kernel void @madak_m_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a) #0 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
   %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load float, float addrspace(1)* %in.a.gep, align 4
 
-  %mul = fmul float 4.0, %a
-  %madak = fadd float %mul, 10.0
+  %mul = fmul FASTMATH_FLAGS float 4.0, %a
+  %madak = fadd FASTMATH_FLAGS float %mul, 10.0
   store float %madak, float addrspace(1)* %out.gep, align 4
   ret void
 }
 
 ; Make sure nothing weird happens with a value that is also allowed as
 ; an inline immediate.
 
 ; GCN-LABEL: {{^}}madak_inline_imm_f32:
define amdgpu_kernel void @madak_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) #0 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
   %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
   %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load float, float addrspace(1)* %in.a.gep, align 4
   %b = load float, float addrspace(1)* %in.b.gep, align 4
 
-  %mul = fmul float %a, %b
-  %madak = fadd float %mul, 4.0
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %madak = fadd FASTMATH_FLAGS float %mul, 4.0
   store float %madak, float addrspace(1)* %out.gep, align 4
   ret void
 }
 
 ; We can't use an SGPR when forming madak
 ; GCN-LABEL: {{^}}s_v_madak_f32:
 ; GCN-DAG:      s_load_dword [[SB:s[0-9]+]]
 ; GFX6_8_9-DAG: v_mov_b32_e32 [[VK:v[0-9]+]], 0x41200000
 ; GCN-DAG:      {{buffer|flat|global}}_load_dword{{(_addtid)?}} [[VA:v[0-9]+]]
 ; GCN-NOT:      v_madak_f32
 ; GFX6_8_9:     v_mac_f32_e32 [[VK]], [[SB]], [[VA]]
 ; GFX10-MAD:    v_mad_f32 v{{[0-9]+}}, [[VA]], [[SB]], 0x41200000
 ; FMA:          v_fma_f32 v{{[0-9]+}}, [[VA]], [[SB]], 0x41200000
 define amdgpu_kernel void @s_v_madak_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float %b) #0 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
   %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load float, float addrspace(1)* %in.a.gep, align 4
 
-  %mul = fmul float %a, %b
-  %madak = fadd float %mul, 10.0
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %madak = fadd FASTMATH_FLAGS float %mul, 10.0
   store float %madak, float addrspace(1)* %out.gep, align 4
   ret void
 }
 
 ; GCN-LABEL: @v_s_madak_f32
 ; GCN-DAG:       s_load_dword [[SB:s[0-9]+]]
 ; GFX6_8_9-DAG:  v_mov_b32_e32 [[VK:v[0-9]+]], 0x41200000
 ; GCN-DAG:       {{buffer|flat|global}}_load_dword{{(_addtid)?}} [[VA:v[0-9]+]]
 ; GFX6_8_9-NOT:  v_madak_f32
 ; GFX6_8_9:      v_mac_f32_e32 [[VK]], [[SB]], [[VA]]
 ; GFX10-MAD:     v_madak_f32 v{{[0-9]+}}, [[SB]], [[VA]], 0x41200000
 ; FMA:           v_fmaak_f32 v{{[0-9]+}}, [[SB]], [[VA]], 0x41200000
 define amdgpu_kernel void @v_s_madak_f32(float addrspace(1)* noalias %out, float %a, float addrspace(1)* noalias %in.b) #0 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
   %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %b = load float, float addrspace(1)* %in.b.gep, align 4
 
-  %mul = fmul float %a, %b
-  %madak = fadd float %mul, 10.0
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %madak = fadd FASTMATH_FLAGS float %mul, 10.0
   store float %madak, float addrspace(1)* %out.gep, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}s_s_madak_f32:
 ; GCN-NOT: v_madak_f32
 ; GFX8_9:  v_mac_f32_e32 {{v[0-9]+}}, {{s[0-9]+}}, {{v[0-9]+}}
 ; GFX10-MAD: v_mac_f32_e64 {{v[0-9]+}}, {{s[0-9]+}}, {{s[0-9]+}}
 ; FMA:       v_fmac_f32_e64 {{v[0-9]+}}, {{s[0-9]+}}, {{s[0-9]+}}
 define amdgpu_kernel void @s_s_madak_f32(float addrspace(1)* %out, float %a, float %b) #0 {
-  %mul = fmul float %a, %b
-  %madak = fadd float %mul, 10.0
+  %mul = fmul FASTMATH_FLAGS float %a, %b
+  %madak = fadd FASTMATH_FLAGS float %mul, 10.0
   store float %madak, float addrspace(1)* %out, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}no_madak_src0_modifier_f32:
 ; GFX6:      buffer_load_dword [[VA:v[0-9]+]]
 ; GFX6:      buffer_load_dword [[VB:v[0-9]+]]
 ; GFX8_9_10: {{flat|global}}_load_dword [[VB:v[0-9]+]]
 ; GFX8_9_10: {{flat|global}}_load_dword [[VA:v[0-9]+]]
 ; GFX6_8_9:  v_mad_f32 {{v[0-9]+}}, |{{v[0-9]+}}|, {{v[0-9]+}}, {{[sv][0-9]+}}
 ; GFX10-MAD: v_mad_f32 {{v[0-9]+}}, |{{v[0-9]+}}|, {{v[0-9]+}}, 0x41200000
 ; FMA:       v_fma_f32 {{v[0-9]+}}, |{{v[0-9]+}}|, {{v[0-9]+}}, 0x41200000
 ; GCN:       s_endpgm
 define amdgpu_kernel void @no_madak_src0_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) #0 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
   %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
   %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load float, float addrspace(1)* %in.a.gep, align 4
   %b = load float, float addrspace(1)* %in.b.gep, align 4
 
   %a.fabs = call float @llvm.fabs.f32(float %a) nounwind readnone
 
-  %mul = fmul float %a.fabs, %b
-  %madak = fadd float %mul, 10.0
+  %mul = fmul FASTMATH_FLAGS float %a.fabs, %b
+  %madak = fadd FASTMATH_FLAGS float %mul, 10.0
   store float %madak, float addrspace(1)* %out.gep, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}no_madak_src1_modifier_f32:
 ; GFX6:      buffer_load_dword [[VA:v[0-9]+]]
 ; GFX6:      buffer_load_dword [[VB:v[0-9]+]]
 ; GFX8_9_10: {{flat|global}}_load_dword [[VB:v[0-9]+]]
 ; GFX8_9_10: {{flat|global}}_load_dword [[VA:v[0-9]+]]
 ; GFX6_8_9:  v_mad_f32 {{v[0-9]+}}, {{v[0-9]+}}, |{{v[0-9]+}}|, {{[sv][0-9]+}}
 ; GFX10-MAD: v_mad_f32 {{v[0-9]+}}, {{v[0-9]+}}, |{{v[0-9]+}}|, 0x41200000
 ; FMA:       v_fma_f32 {{v[0-9]+}}, {{v[0-9]+}}, |{{v[0-9]+}}|, 0x41200000
 ; GCN:       s_endpgm
 define amdgpu_kernel void @no_madak_src1_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) #0 {
   %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
   %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
   %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
   %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
 
   %a = load float, float addrspace(1)* %in.a.gep, align 4
   %b = load float, float addrspace(1)* %in.b.gep, align 4
 
   %b.fabs = call float @llvm.fabs.f32(float %b) nounwind readnone
 
-  %mul = fmul float %a, %b.fabs
-  %madak = fadd float %mul, 10.0
+  %mul = fmul FASTMATH_FLAGS float %a, %b.fabs
+  %madak = fadd FASTMATH_FLAGS float %mul, 10.0
   store float %madak, float addrspace(1)* %out.gep, align 4
   ret void
 }
 
 ; SIFoldOperands should not fold the SGPR copy into the instruction before GFX10
 ; because the implicit immediate already uses the constant bus.
 ; On GFX10+ we can use two scalar operands.
 ; GCN-LABEL: {{^}}madak_constant_bus_violation:
bb:
   br i1 %tmp, label %bb3, label %bb4
 
 bb3:
   store volatile float 0.0, float addrspace(1)* undef
   br label %bb4
 
 bb4:
   %vgpr = load volatile float, float addrspace(1)* undef
-  %tmp0 = fmul float %sgpr0, 0.5
-  %tmp1 = fadd float %tmp0, 42.0
-  %tmp2 = fmul float %tmp1, %vgpr
+  %tmp0 = fmul FASTMATH_FLAGS float %sgpr0, 0.5
+  %tmp1 = fadd FASTMATH_FLAGS float %tmp0, 42.0
+  %tmp2 = fmul FASTMATH_FLAGS float %tmp1, %vgpr
   store volatile float %tmp2, float addrspace(1)* undef, align 4
   ret void
 }
 
 attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/X86/avx512-fma.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=x86_64-apple-darwin -mattr=+avx512f -fp-contract=fast | FileCheck %s --check-prefix=ALL --check-prefix=KNL
-; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=skx -fp-contract=fast | FileCheck %s --check-prefix=ALL --check-prefix=SKX
+; RUN: llc < %s -mtriple=x86_64-apple-darwin -mattr=+avx512f | FileCheck %s --check-prefix=ALL --check-prefix=KNL
+; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=skx | FileCheck %s --check-prefix=ALL --check-prefix=SKX
 
 define <16 x float> @test_x86_fmadd_ps_z(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) {
 ; ALL-LABEL: test_x86_fmadd_ps_z:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfmadd213ps {{.*#+}} zmm0 = (zmm1 * zmm0) + zmm2
 ; ALL-NEXT:    retq
-  %x = fmul <16 x float> %a0, %a1
-  %res = fadd <16 x float> %x, %a2
+  %x = fmul contract <16 x float> %a0, %a1
+  %res = fadd contract <16 x float> %x, %a2
   ret <16 x float> %res
 }
 
 define <16 x float> @test_x86_fmsub_ps_z(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) {
 ; ALL-LABEL: test_x86_fmsub_ps_z:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfmsub213ps {{.*#+}} zmm0 = (zmm1 * zmm0) - zmm2
 ; ALL-NEXT:    retq
-  %x = fmul <16 x float> %a0, %a1
-  %res = fsub <16 x float> %x, %a2
+  %x = fmul contract <16 x float> %a0, %a1
+  %res = fsub contract <16 x float> %x, %a2
   ret <16 x float> %res
 }
 
 define <16 x float> @test_x86_fnmadd_ps_z(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) {
 ; ALL-LABEL: test_x86_fnmadd_ps_z:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfnmadd213ps {{.*#+}} zmm0 = -(zmm1 * zmm0) + zmm2
 ; ALL-NEXT:    retq
-  %x = fmul <16 x float> %a0, %a1
-  %res = fsub <16 x float> %a2, %x
+  %x = fmul contract <16 x float> %a0, %a1
+  %res = fsub contract <16 x float> %a2, %x
   ret <16 x float> %res
 }
 
 define <16 x float> @test_x86_fnmsub_ps_z(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) {
 ; ALL-LABEL: test_x86_fnmsub_ps_z:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfnmsub213ps {{.*#+}} zmm0 = -(zmm1 * zmm0) - zmm2
 ; ALL-NEXT:    retq
-  %x = fmul <16 x float> %a0, %a1
-  %y = fsub <16 x float> <float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00,
+  %x = fmul contract <16 x float> %a0, %a1
+  %y = fsub contract <16 x float> <float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00,
                           float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00,
                           float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00,
                           float -0.000000e+00>, %x
-  %res = fsub <16 x float> %y, %a2
+  %res = fsub contract <16 x float> %y, %a2
   ret <16 x float> %res
 }
 
 define <8 x double> @test_x86_fmadd_pd_z(<8 x double> %a0, <8 x double> %a1, <8 x double> %a2) {
 ; ALL-LABEL: test_x86_fmadd_pd_z:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfmadd213pd {{.*#+}} zmm0 = (zmm1 * zmm0) + zmm2
 ; ALL-NEXT:    retq
-  %x = fmul <8 x double> %a0, %a1
-  %res = fadd <8 x double> %x, %a2
+  %x = fmul contract <8 x double> %a0, %a1
+  %res = fadd contract <8 x double> %x, %a2
   ret <8 x double> %res
 }
 
 define <8 x double> @test_x86_fmsub_pd_z(<8 x double> %a0, <8 x double> %a1, <8 x double> %a2) {
 ; ALL-LABEL: test_x86_fmsub_pd_z:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfmsub213pd {{.*#+}} zmm0 = (zmm1 * zmm0) - zmm2
 ; ALL-NEXT:    retq
-  %x = fmul <8 x double> %a0, %a1
-  %res = fsub <8 x double> %x, %a2
+  %x = fmul contract <8 x double> %a0, %a1
+  %res = fsub contract <8 x double> %x, %a2
   ret <8 x double> %res
 }
 
 define double @test_x86_fmsub_213(double %a0, double %a1, double %a2) {
 ; ALL-LABEL: test_x86_fmsub_213:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfmsub213sd {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; ALL-NEXT:    retq
-  %x = fmul double %a0, %a1
-  %res = fsub double %x, %a2
+  %x = fmul contract double %a0, %a1
+  %res = fsub contract double %x, %a2
   ret double %res
 }
 
 define double @test_x86_fmsub_213_m(double %a0, double %a1, double * %a2_ptr) {
 ; ALL-LABEL: test_x86_fmsub_213_m:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfmsub213sd {{.*#+}} xmm0 = (xmm1 * xmm0) - mem
 ; ALL-NEXT:    retq
   %a2 = load double , double *%a2_ptr
-  %x = fmul double %a0, %a1
-  %res = fsub double %x, %a2
+  %x = fmul contract double %a0, %a1
+  %res = fsub contract double %x, %a2
   ret double %res
 }
 
 define double @test_x86_fmsub_231_m(double %a0, double %a1, double * %a2_ptr) {
 ; ALL-LABEL: test_x86_fmsub_231_m:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfmsub132sd {{.*#+}} xmm0 = (xmm0 * mem) - xmm1
 ; ALL-NEXT:    retq
   %a2 = load double , double *%a2_ptr
-  %x = fmul double %a0, %a2
-  %res = fsub double %x, %a1
+  %x = fmul contract double %a0, %a2
+  %res = fsub contract double %x, %a1
   ret double %res
 }
 
 define <16 x float> @test231_br(<16 x float> %a1, <16 x float> %a2) nounwind {
 ; ALL-LABEL: test231_br:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfmadd132ps {{.*#+}} zmm0 = (zmm0 * mem) + zmm1
 ; ALL-NEXT:    retq
-  %b1 = fmul <16 x float> %a1, <float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000>
-  %b2 = fadd <16 x float> %b1, %a2
+  %b1 = fmul contract <16 x float> %a1, <float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000>
+  %b2 = fadd contract <16 x float> %b1, %a2
   ret <16 x float> %b2
 }
 
 define <16 x float> @test213_br(<16 x float> %a1, <16 x float> %a2) nounwind {
 ; ALL-LABEL: test213_br:
 ; ALL:       ## %bb.0:
 ; ALL-NEXT:    vfmadd213ps {{.*#+}} zmm0 = (zmm1 * zmm0) + mem
 ; ALL-NEXT:    retq
-  %b1 = fmul <16 x float> %a1, %a2
-  %b2 = fadd <16 x float> %b1, <float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000>
+  %b1 = fmul contract <16 x float> %a1, %a2
+  %b2 = fadd contract <16 x float> %b1, <float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000, float 0x3FB99999A0000000>
   ret <16 x float> %b2
 }
 
 ;mask (a*c+b , a)
 define <16 x float> @test_x86_fmadd132_ps(<16 x float> %a0, <16 x float> %a1, <16 x float> *%a2_ptrt, <16 x i1> %mask) {
 ; KNL-LABEL: test_x86_fmadd132_ps:
 ; KNL:       ## %bb.0:
 ; KNL-NEXT:    vpmovsxbd %xmm2, %zmm2
 ; KNL-NEXT:    vpslld $31, %zmm2, %zmm2
 ; KNL-NEXT:    vptestmd %zmm2, %zmm2, %k1
 ; KNL-NEXT:    vfmadd132ps {{.*#+}} zmm0 {%k1} = (zmm0 * mem) + zmm1
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: test_x86_fmadd132_ps:
 ; SKX:       ## %bb.0:
 ; SKX-NEXT:    vpsllw $7, %xmm2, %xmm2
 ; SKX-NEXT:    vpmovb2m %xmm2, %k1
 ; SKX-NEXT:    vfmadd132ps {{.*#+}} zmm0 {%k1} = (zmm0 * mem) + zmm1
 ; SKX-NEXT:    retq
   %a2   = load <16 x float>,<16 x float> *%a2_ptrt,align 1
-  %x = fmul <16 x float> %a0, %a2
-  %y = fadd <16 x float> %x, %a1
+  %x = fmul contract <16 x float> %a0, %a2
+  %y = fadd contract <16 x float> %x, %a1
   %res = select <16 x i1> %mask, <16 x float> %y, <16 x float> %a0
   ret <16 x float> %res
 }
 
 ;mask (a*c+b , b)
 define <16 x float> @test_x86_fmadd231_ps(<16 x float> %a0, <16 x float> %a1, <16 x float> *%a2_ptrt, <16 x i1> %mask) {
 ; KNL-LABEL: test_x86_fmadd231_ps:
 ; KNL:       ## %bb.0:
 ; KNL-NEXT:    vpmovsxbd %xmm2, %zmm2
 ; KNL-NEXT:    vpslld $31, %zmm2, %zmm2
 ; KNL-NEXT:    vptestmd %zmm2, %zmm2, %k1
 ; KNL-NEXT:    vfmadd231ps {{.*#+}} zmm1 {%k1} = (zmm0 * mem) + zmm1
 ; KNL-NEXT:    vmovaps %zmm1, %zmm0
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: test_x86_fmadd231_ps:
 ; SKX:       ## %bb.0:
 ; SKX-NEXT:    vpsllw $7, %xmm2, %xmm2
 ; SKX-NEXT:    vpmovb2m %xmm2, %k1
 ; SKX-NEXT:    vfmadd231ps {{.*#+}} zmm1 {%k1} = (zmm0 * mem) + zmm1
 ; SKX-NEXT:    vmovaps %zmm1, %zmm0
 ; SKX-NEXT:    retq
   %a2   = load <16 x float>,<16 x float> *%a2_ptrt,align 1
-  %x = fmul <16 x float> %a0, %a2
-  %y = fadd <16 x float> %x, %a1
+  %x = fmul contract <16 x float> %a0, %a2
+  %y = fadd contract <16 x float> %x, %a1
   %res = select <16 x i1> %mask, <16 x float> %y, <16 x float> %a1
   ret <16 x float> %res
 }
 
 ;mask (b*a+c , b)
 define <16 x float> @test_x86_fmadd213_ps(<16 x float> %a0, <16 x float> %a1, <16 x float> *%a2_ptrt, <16 x i1> %mask) {
 ; KNL-LABEL: test_x86_fmadd213_ps:
 ; KNL:       ## %bb.0:
 ; KNL-NEXT:    vpmovsxbd %xmm2, %zmm2
 ; KNL-NEXT:    vpslld $31, %zmm2, %zmm2
 ; KNL-NEXT:    vptestmd %zmm2, %zmm2, %k1
 ; KNL-NEXT:    vfmadd213ps {{.*#+}} zmm1 {%k1} = (zmm0 * zmm1) + mem
 ; KNL-NEXT:    vmovaps %zmm1, %zmm0
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: test_x86_fmadd213_ps:
 ; SKX:       ## %bb.0:
 ; SKX-NEXT:    vpsllw $7, %xmm2, %xmm2
 ; SKX-NEXT:    vpmovb2m %xmm2, %k1
 ; SKX-NEXT:    vfmadd213ps {{.*#+}} zmm1 {%k1} = (zmm0 * zmm1) + mem
 ; SKX-NEXT:    vmovaps %zmm1, %zmm0
 ; SKX-NEXT:    retq
   %a2   = load <16 x float>,<16 x float> *%a2_ptrt,align 1
-  %x = fmul <16 x float> %a1, %a0
-  %y = fadd <16 x float> %x, %a2
+  %x = fmul contract <16 x float> %a1, %a0
+  %y = fadd contract <16 x float> %x, %a2
   %res = select <16 x i1> %mask, <16 x float> %y, <16 x float> %a1
   ret <16 x float> %res
 }
 

llvm/test/CodeGen/X86/fma-do-not-commute.ll

-; RUN: llc -fp-contract=fast -mattr=+fma -disable-cgp < %s -o - | FileCheck %s
+; RUN: llc -mattr=+fma -disable-cgp < %s -o - | FileCheck %s
 ; Check that the 2nd and 3rd arguments of fmaXXX231 reg1, reg2, mem3 are not commuted.
 ; <rdar://problem/16800495> 
 target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-apple-macosx"
 
 ; CHECK-LABEL: test1:
 ; %arg lives in xmm0 and it shouldn't be redefined until it is used in the FMA.
 ; CHECK-NOT: {{.*}}, %xmm0
 ; %addr lives in rdi.
 ; %addr2 lives in rsi.
 ; CHECK: vmovss (%rdi), [[ADDR:%xmm[0-9]+]]
 ; The assembly syntax is in the reverse order.
 ; CHECK: vfmadd231ss (%rsi), [[ADDR]], %xmm0
 define void @test1(float* %addr, float* %addr2, float %arg) {
 entry:
   br label %loop
 
 loop:
   %sum0 = phi float [ %fma, %loop ], [ %arg, %entry ]
   %addrVal = load float, float* %addr, align 4
   %addr2Val = load float, float* %addr2, align 4
-  %fmul = fmul float %addrVal, %addr2Val
-  %fma = fadd float %sum0, %fmul
+  %fmul = fmul contract float %addrVal, %addr2Val
+  %fma = fadd contract float %sum0, %fmul
   br i1 true, label %exit, label %loop
 
 exit:
   store float %fma, float* %addr, align 4
   ret void
 }

llvm/test/CodeGen/X86/fma_patterns.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma -fp-contract=fast | FileCheck %s --check-prefixes=FMA,FMA-INFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4,+fma -fp-contract=fast | FileCheck %s --check-prefixes=FMA4,FMA4-INFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4 -fp-contract=fast | FileCheck %s --check-prefixes=FMA4,FMA4-INFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,+avx512vl -fp-contract=fast | FileCheck %s --check-prefixes=AVX512,AVX512-INFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma -fp-contract=fast -enable-no-infs-fp-math | FileCheck %s --check-prefixes=FMA,FMA-NOINFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4,+fma -fp-contract=fast -enable-no-infs-fp-math | FileCheck %s --check-prefixes=FMA4,FMA4-NOINFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4 -fp-contract=fast -enable-no-infs-fp-math | FileCheck %s --check-prefixes=FMA4,FMA4-NOINFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,+avx512vl -fp-contract=fast -enable-no-infs-fp-math | FileCheck %s --check-prefixes=AVX512,AVX512-NOINFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma | FileCheck %s --check-prefixes=FMA,FMA-INFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4,+fma | FileCheck %s --check-prefixes=FMA4,FMA4-INFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4 | FileCheck %s --check-prefixes=FMA4,FMA4-INFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,+avx512vl | FileCheck %s --check-prefixes=AVX512,AVX512-INFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract ninf,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma | FileCheck %s --check-prefixes=FMA,FMA-NOINFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract ninf,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4,+fma | FileCheck %s --check-prefixes=FMA4,FMA4-NOINFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract ninf,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4 | FileCheck %s --check-prefixes=FMA4,FMA4-NOINFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract ninf,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,+avx512vl | FileCheck %s --check-prefixes=AVX512,AVX512-NOINFS
 
 ;
 ; Pattern: (fadd (fmul x, y), z) -> (fmadd x,y,z)
 ;
 
 define float @test_f32_fmadd(float %a0, float %a1, float %a2) {
 ; FMA-LABEL: test_f32_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213ss {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_f32_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddss {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_f32_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213ss {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul float %a0, %a1
-  %res = fadd float %x, %a2
+  %x = fmul FASTMATH_FLAGS float %a0, %a1
+  %res = fadd FASTMATH_FLAGS float %x, %a2
   ret float %res
 }
 
 define <4 x float> @test_4f32_fmadd(<4 x float> %a0, <4 x float> %a1, <4 x float> %a2) {
 ; FMA-LABEL: test_4f32_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213ps {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_4f32_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddps {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_4f32_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213ps {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <4 x float> %a0, %a1
-  %res = fadd <4 x float> %x, %a2
+  %x = fmul FASTMATH_FLAGS <4 x float> %a0, %a1
+  %res = fadd FASTMATH_FLAGS <4 x float> %x, %a2
   ret <4 x float> %res
 }
 
 define <8 x float> @test_8f32_fmadd(<8 x float> %a0, <8 x float> %a1, <8 x float> %a2) {
 ; FMA-LABEL: test_8f32_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213ps {{.*#+}} ymm0 = (ymm1 * ymm0) + ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_8f32_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddps {{.*#+}} ymm0 = (ymm0 * ymm1) + ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_8f32_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213ps {{.*#+}} ymm0 = (ymm1 * ymm0) + ymm2
 ; AVX512-NEXT:    retq
-  %x = fmul <8 x float> %a0, %a1
-  %res = fadd <8 x float> %x, %a2
+  %x = fmul FASTMATH_FLAGS <8 x float> %a0, %a1
+  %res = fadd FASTMATH_FLAGS <8 x float> %x, %a2
   ret <8 x float> %res
 }
 
 define double @test_f64_fmadd(double %a0, double %a1, double %a2) {
 ; FMA-LABEL: test_f64_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213sd {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_f64_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddsd {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_f64_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213sd {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul double %a0, %a1
-  %res = fadd double %x, %a2
+  %x = fmul FASTMATH_FLAGS double %a0, %a1
+  %res = fadd FASTMATH_FLAGS double %x, %a2
   ret double %res
 }
 
 define <2 x double> @test_2f64_fmadd(<2 x double> %a0, <2 x double> %a1, <2 x double> %a2) {
 ; FMA-LABEL: test_2f64_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213pd {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_2f64_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddpd {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_2f64_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213pd {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <2 x double> %a0, %a1
-  %res = fadd <2 x double> %x, %a2
+  %x = fmul FASTMATH_FLAGS <2 x double> %a0, %a1
+  %res = fadd FASTMATH_FLAGS <2 x double> %x, %a2
   ret <2 x double> %res
 }
 
 define <4 x double> @test_4f64_fmadd(<4 x double> %a0, <4 x double> %a1, <4 x double> %a2) {
 ; FMA-LABEL: test_4f64_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213pd {{.*#+}} ymm0 = (ymm1 * ymm0) + ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_4f64_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddpd {{.*#+}} ymm0 = (ymm0 * ymm1) + ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_4f64_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213pd {{.*#+}} ymm0 = (ymm1 * ymm0) + ymm2
 ; AVX512-NEXT:    retq
-  %x = fmul <4 x double> %a0, %a1
-  %res = fadd <4 x double> %x, %a2
+  %x = fmul FASTMATH_FLAGS <4 x double> %a0, %a1
+  %res = fadd FASTMATH_FLAGS <4 x double> %x, %a2
   ret <4 x double> %res
 }
 
 ;
 ; Pattern: (fsub (fmul x, y), z) -> (fmsub x, y, z)
 ;
 
 define float @test_f32_fmsub(float %a0, float %a1, float %a2) {
 ; FMA-LABEL: test_f32_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213ss {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_f32_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubss {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_f32_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213ss {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul float %a0, %a1
-  %res = fsub float %x, %a2
+  %x = fmul FASTMATH_FLAGS float %a0, %a1
+  %res = fsub FASTMATH_FLAGS float %x, %a2
   ret float %res
 }
 
 define <4 x float> @test_4f32_fmsub(<4 x float> %a0, <4 x float> %a1, <4 x float> %a2) {
 ; FMA-LABEL: test_4f32_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_4f32_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubps {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_4f32_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <4 x float> %a0, %a1
-  %res = fsub <4 x float> %x, %a2
+  %x = fmul FASTMATH_FLAGS <4 x float> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <4 x float> %x, %a2
   ret <4 x float> %res
 }
 
 define <8 x float> @test_8f32_fmsub(<8 x float> %a0, <8 x float> %a1, <8 x float> %a2) {
 ; FMA-LABEL: test_8f32_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213ps {{.*#+}} ymm0 = (ymm1 * ymm0) - ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_8f32_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubps {{.*#+}} ymm0 = (ymm0 * ymm1) - ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_8f32_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213ps {{.*#+}} ymm0 = (ymm1 * ymm0) - ymm2
 ; AVX512-NEXT:    retq
-  %x = fmul <8 x float> %a0, %a1
-  %res = fsub <8 x float> %x, %a2
+  %x = fmul FASTMATH_FLAGS <8 x float> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <8 x float> %x, %a2
   ret <8 x float> %res
 }
 
 define double @test_f64_fmsub(double %a0, double %a1, double %a2) {
 ; FMA-LABEL: test_f64_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213sd {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_f64_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubsd {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_f64_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213sd {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul double %a0, %a1
-  %res = fsub double %x, %a2
+  %x = fmul FASTMATH_FLAGS double %a0, %a1
+  %res = fsub FASTMATH_FLAGS double %x, %a2
   ret double %res
 }
 
 define <2 x double> @test_2f64_fmsub(<2 x double> %a0, <2 x double> %a1, <2 x double> %a2) {
 ; FMA-LABEL: test_2f64_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213pd {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_2f64_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubpd {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_2f64_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213pd {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <2 x double> %a0, %a1
-  %res = fsub <2 x double> %x, %a2
+  %x = fmul FASTMATH_FLAGS <2 x double> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <2 x double> %x, %a2
   ret <2 x double> %res
 }
 
 define <4 x double> @test_4f64_fmsub(<4 x double> %a0, <4 x double> %a1, <4 x double> %a2) {
 ; FMA-LABEL: test_4f64_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213pd {{.*#+}} ymm0 = (ymm1 * ymm0) - ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_4f64_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubpd {{.*#+}} ymm0 = (ymm0 * ymm1) - ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_4f64_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213pd {{.*#+}} ymm0 = (ymm1 * ymm0) - ymm2
 ; AVX512-NEXT:    retq
-  %x = fmul <4 x double> %a0, %a1
-  %res = fsub <4 x double> %x, %a2
+  %x = fmul FASTMATH_FLAGS <4 x double> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <4 x double> %x, %a2
   ret <4 x double> %res
 }
 
 ;
 ; Pattern: (fsub z, (fmul x, y)) -> (fnmadd x, y, z)
 ;
 
 define float @test_f32_fnmadd(float %a0, float %a1, float %a2) {
 ; FMA-LABEL: test_f32_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213ss {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_f32_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddss {{.*#+}} xmm0 = -(xmm0 * xmm1) + xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_f32_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213ss {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul float %a0, %a1
-  %res = fsub float %a2, %x
+  %x = fmul FASTMATH_FLAGS float %a0, %a1
+  %res = fsub FASTMATH_FLAGS float %a2, %x
   ret float %res
 }
 
 define <4 x float> @test_4f32_fnmadd(<4 x float> %a0, <4 x float> %a1, <4 x float> %a2) {
 ; FMA-LABEL: test_4f32_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_4f32_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddps {{.*#+}} xmm0 = -(xmm0 * xmm1) + xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_4f32_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <4 x float> %a0, %a1
-  %res = fsub <4 x float> %a2, %x
+  %x = fmul FASTMATH_FLAGS <4 x float> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <4 x float> %a2, %x
   ret <4 x float> %res
 }
 
 define <8 x float> @test_8f32_fnmadd(<8 x float> %a0, <8 x float> %a1, <8 x float> %a2) {
 ; FMA-LABEL: test_8f32_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213ps {{.*#+}} ymm0 = -(ymm1 * ymm0) + ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_8f32_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddps {{.*#+}} ymm0 = -(ymm0 * ymm1) + ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_8f32_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213ps {{.*#+}} ymm0 = -(ymm1 * ymm0) + ymm2
 ; AVX512-NEXT:    retq
-  %x = fmul <8 x float> %a0, %a1
-  %res = fsub <8 x float> %a2, %x
+  %x = fmul FASTMATH_FLAGS <8 x float> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <8 x float> %a2, %x
   ret <8 x float> %res
 }
 
 define double @test_f64_fnmadd(double %a0, double %a1, double %a2) {
 ; FMA-LABEL: test_f64_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213sd {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_f64_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddsd {{.*#+}} xmm0 = -(xmm0 * xmm1) + xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_f64_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213sd {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul double %a0, %a1
-  %res = fsub double %a2, %x
+  %x = fmul FASTMATH_FLAGS double %a0, %a1
+  %res = fsub FASTMATH_FLAGS double %a2, %x
   ret double %res
 }
 
 define <2 x double> @test_2f64_fnmadd(<2 x double> %a0, <2 x double> %a1, <2 x double> %a2) {
 ; FMA-LABEL: test_2f64_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213pd {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_2f64_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddpd {{.*#+}} xmm0 = -(xmm0 * xmm1) + xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_2f64_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213pd {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <2 x double> %a0, %a1
-  %res = fsub <2 x double> %a2, %x
+  %x = fmul FASTMATH_FLAGS <2 x double> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <2 x double> %a2, %x
   ret <2 x double> %res
 }
 
 define <4 x double> @test_4f64_fnmadd(<4 x double> %a0, <4 x double> %a1, <4 x double> %a2) {
 ; FMA-LABEL: test_4f64_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213pd {{.*#+}} ymm0 = -(ymm1 * ymm0) + ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_4f64_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddpd {{.*#+}} ymm0 = -(ymm0 * ymm1) + ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_4f64_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213pd {{.*#+}} ymm0 = -(ymm1 * ymm0) + ymm2
 ; AVX512-NEXT:    retq
-  %x = fmul <4 x double> %a0, %a1
-  %res = fsub <4 x double> %a2, %x
+  %x = fmul FASTMATH_FLAGS <4 x double> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <4 x double> %a2, %x
   ret <4 x double> %res
 }
 
 ;
 ; Pattern: (fsub (fneg (fmul x, y)), z) -> (fnmsub x, y, z)
 ;
 
 define float @test_f32_fnmsub(float %a0, float %a1, float %a2) {
 ; FMA-LABEL: test_f32_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213ss {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_f32_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubss {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_f32_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213ss {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul float %a0, %a1
-  %y = fsub float -0.000000e+00, %x
-  %res = fsub float %y, %a2
+  %x = fmul FASTMATH_FLAGS float %a0, %a1
+  %y = fsub FASTMATH_FLAGS float -0.000000e+00, %x
+  %res = fsub FASTMATH_FLAGS float %y, %a2
   ret float %res
 }
 
 define <4 x float> @test_4f32_fnmsub(<4 x float> %a0, <4 x float> %a1, <4 x float> %a2) {
 ; FMA-LABEL: test_4f32_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_4f32_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubps {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_4f32_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <4 x float> %a0, %a1
-  %y = fsub <4 x float> <float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00>, %x
-  %res = fsub <4 x float> %y, %a2
+  %x = fmul FASTMATH_FLAGS <4 x float> %a0, %a1
+  %y = fsub FASTMATH_FLAGS <4 x float> <float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00>, %x
+  %res = fsub FASTMATH_FLAGS <4 x float> %y, %a2
   ret <4 x float> %res
 }
 
 define <8 x float> @test_8f32_fnmsub(<8 x float> %a0, <8 x float> %a1, <8 x float> %a2) {
 ; FMA-LABEL: test_8f32_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213ps {{.*#+}} ymm0 = -(ymm1 * ymm0) - ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_8f32_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubps {{.*#+}} ymm0 = -(ymm0 * ymm1) - ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_8f32_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213ps {{.*#+}} ymm0 = -(ymm1 * ymm0) - ymm2
 ; AVX512-NEXT:    retq
-  %x = fmul <8 x float> %a0, %a1
-  %y = fsub <8 x float> <float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00>, %x
-  %res = fsub <8 x float> %y, %a2
+  %x = fmul FASTMATH_FLAGS <8 x float> %a0, %a1
+  %y = fsub FASTMATH_FLAGS <8 x float> <float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00>, %x
+  %res = fsub FASTMATH_FLAGS <8 x float> %y, %a2
   ret <8 x float> %res
 }
 
 define double @test_f64_fnmsub(double %a0, double %a1, double %a2) {
 ; FMA-LABEL: test_f64_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213sd {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_f64_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubsd {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_f64_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213sd {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul double %a0, %a1
-  %y = fsub double -0.000000e+00, %x
-  %res = fsub double %y, %a2
+  %x = fmul FASTMATH_FLAGS double %a0, %a1
+  %y = fsub FASTMATH_FLAGS double -0.000000e+00, %x
+  %res = fsub FASTMATH_FLAGS double %y, %a2
   ret double %res
 }
 
 define <2 x double> @test_2f64_fnmsub(<2 x double> %a0, <2 x double> %a1, <2 x double> %a2) {
 ; FMA-LABEL: test_2f64_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213pd {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_2f64_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubpd {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_2f64_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213pd {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <2 x double> %a0, %a1
-  %y = fsub <2 x double> <double -0.000000e+00, double -0.000000e+00>, %x
-  %res = fsub <2 x double> %y, %a2
+  %x = fmul FASTMATH_FLAGS <2 x double> %a0, %a1
+  %y = fsub FASTMATH_FLAGS <2 x double> <double -0.000000e+00, double -0.000000e+00>, %x
+  %res = fsub FASTMATH_FLAGS <2 x double> %y, %a2
   ret <2 x double> %res
 }
 
 define <4 x double> @test_4f64_fnmsub(<4 x double> %a0, <4 x double> %a1, <4 x double> %a2) {
 ; FMA-LABEL: test_4f64_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213pd {{.*#+}} ymm0 = -(ymm1 * ymm0) - ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_4f64_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubpd {{.*#+}} ymm0 = -(ymm0 * ymm1) - ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_4f64_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213pd {{.*#+}} ymm0 = -(ymm1 * ymm0) - ymm2
 ; AVX512-NEXT:    retq
-  %x = fmul <4 x double> %a0, %a1
-  %y = fsub <4 x double> <double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00>, %x
-  %res = fsub <4 x double> %y, %a2
+  %x = fmul FASTMATH_FLAGS <4 x double> %a0, %a1
+  %y = fsub FASTMATH_FLAGS <4 x double> <double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00>, %x
+  %res = fsub FASTMATH_FLAGS <4 x double> %y, %a2
   ret <4 x double> %res
 }
 
 ;
 ; Load Folding Patterns
 ;
 
 define <4 x float> @test_4f32_fmadd_load(<4 x float>* %a0, <4 x float> %a1, <4 x float> %a2) {
 ; FMA-LABEL: test_4f32_fmadd_load:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd132ps {{.*#+}} xmm0 = (xmm0 * mem) + xmm1
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_4f32_fmadd_load:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddps {{.*#+}} xmm0 = (xmm0 * mem) + xmm1
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_4f32_fmadd_load:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd132ps {{.*#+}} xmm0 = (xmm0 * mem) + xmm1
 ; AVX512-NEXT:    retq
   %x = load <4 x float>, <4 x float>* %a0
-  %y = fmul <4 x float> %x, %a1
-  %res = fadd <4 x float> %y, %a2
+  %y = fmul FASTMATH_FLAGS <4 x float> %x, %a1
+  %res = fadd FASTMATH_FLAGS <4 x float> %y, %a2
   ret <4 x float> %res
 }
 
 define <2 x double> @test_2f64_fmsub_load(<2 x double>* %a0, <2 x double> %a1, <2 x double> %a2) {
 ; FMA-LABEL: test_2f64_fmsub_load:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub132pd {{.*#+}} xmm0 = (xmm0 * mem) - xmm1
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_2f64_fmsub_load:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubpd {{.*#+}} xmm0 = (xmm0 * mem) - xmm1
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_2f64_fmsub_load:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub132pd {{.*#+}} xmm0 = (xmm0 * mem) - xmm1
 ; AVX512-NEXT:    retq
   %x = load <2 x double>, <2 x double>* %a0
-  %y = fmul <2 x double> %x, %a1
-  %res = fsub <2 x double> %y, %a2
+  %y = fmul FASTMATH_FLAGS <2 x double> %x, %a1
+  %res = fsub FASTMATH_FLAGS <2 x double> %y, %a2
   ret <2 x double> %res
 }
 
 ;
 ; Patterns (+ fneg variants): mul(add(1.0,x),y), mul(sub(1.0,x),y), mul(sub(x,1.0),y)
 ;
 
 define <4 x float> @test_v4f32_mul_add_x_one_y(<4 x float> %x, <4 x float> %y) {

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_add_x_one_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213ps {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
-  %m = fmul <4 x float> %a, %y
+  %a = fadd FASTMATH_FLAGS <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %a, %y
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_add_x_one(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_add_x_one:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm0, %xmm1, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_add_x_one:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213ps {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
-  %m = fmul <4 x float> %y, %a
+  %a = fadd FASTMATH_FLAGS <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %a
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_add_x_one_undefs(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_add_x_one_undefs:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm0, %xmm1, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_add_x_one_undefs:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213ps {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <4 x float> %x, <float 1.0, float undef, float 1.0, float undef>
-  %m = fmul <4 x float> %y, %a
+  %a = fadd FASTMATH_FLAGS <4 x float> %x, <float 1.0, float undef, float 1.0, float undef>
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %a
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_add_x_negone_y(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_add_x_negone_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm1, %xmm0, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_add_x_negone_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <4 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0>
-  %m = fmul <4 x float> %a, %y
+  %a = fadd FASTMATH_FLAGS <4 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %a, %y
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_add_x_negone(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_add_x_negone:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm0, %xmm1, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_add_x_negone:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <4 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0>
-  %m = fmul <4 x float> %y, %a
+  %a = fadd FASTMATH_FLAGS <4 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %a
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_add_x_negone_undefs(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_add_x_negone_undefs:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm0, %xmm1, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_add_x_negone_undefs:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <4 x float> %x, <float undef, float -1.0, float undef, float -1.0>
-  %m = fmul <4 x float> %y, %a
+  %a = fadd FASTMATH_FLAGS <4 x float> %x, <float undef, float -1.0, float undef, float -1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %a
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_sub_one_x_y(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_sub_one_x_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} xmm2 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vsubps %xmm0, %xmm2, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfnmaddps {{.*#+}} xmm0 = -(xmm0 * xmm1) + xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_sub_one_x_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmadd213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> <float 1.0, float 1.0, float 1.0, float 1.0>, %x
-  %m = fmul <4 x float> %s, %y
+  %s = fsub FASTMATH_FLAGS <4 x float> <float 1.0, float 1.0, float 1.0, float 1.0>, %x
+  %m = fmul FASTMATH_FLAGS <4 x float> %s, %y
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_sub_one_x(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_sub_one_x:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} xmm2 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vsubps %xmm0, %xmm2, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfnmaddps {{.*#+}} xmm0 = -(xmm0 * xmm1) + xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_sub_one_x:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmadd213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> <float 1.0, float 1.0, float 1.0, float 1.0>, %x
-  %m = fmul <4 x float> %y, %s
+  %s = fsub FASTMATH_FLAGS <4 x float> <float 1.0, float 1.0, float 1.0, float 1.0>, %x
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %s
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_sub_one_x_undefs(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_sub_one_x_undefs:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} xmm2 = <1.0E+0,u,1.0E+0,1.0E+0>
 ; FMA-INFS-NEXT:    vsubps %xmm0, %xmm2, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfnmaddps {{.*#+}} xmm0 = -(xmm0 * xmm1) + xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_sub_one_x_undefs:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmadd213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) + xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> <float 1.0, float undef, float 1.0, float 1.0>, %x
-  %m = fmul <4 x float> %y, %s
+  %s = fsub FASTMATH_FLAGS <4 x float> <float 1.0, float undef, float 1.0, float 1.0>, %x
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %s
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_sub_negone_x_y(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_sub_negone_x_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} xmm2 = [-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0]
 ; FMA-INFS-NEXT:    vsubps %xmm0, %xmm2, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfnmsubps {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_sub_negone_x_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmsub213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> <float -1.0, float -1.0, float -1.0, float -1.0>, %x
-  %m = fmul <4 x float> %s, %y
+  %s = fsub FASTMATH_FLAGS <4 x float> <float -1.0, float -1.0, float -1.0, float -1.0>, %x
+  %m = fmul FASTMATH_FLAGS <4 x float> %s, %y
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_sub_negone_x(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_sub_negone_x:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} xmm2 = [-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0]
 ; FMA-INFS-NEXT:    vsubps %xmm0, %xmm2, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfnmsubps {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_sub_negone_x:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmsub213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> <float -1.0, float -1.0, float -1.0, float -1.0>, %x
-  %m = fmul <4 x float> %y, %s
+  %s = fsub FASTMATH_FLAGS <4 x float> <float -1.0, float -1.0, float -1.0, float -1.0>, %x
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %s
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_sub_negone_x_undefs(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_sub_negone_x_undefs:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} xmm2 = <-1.0E+0,-1.0E+0,u,-1.0E+0>
 ; FMA-INFS-NEXT:    vsubps %xmm0, %xmm2, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfnmsubps {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_sub_negone_x_undefs:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmsub213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> <float -1.0, float -1.0, float undef, float -1.0>, %x
-  %m = fmul <4 x float> %y, %s
+  %s = fsub FASTMATH_FLAGS <4 x float> <float -1.0, float -1.0, float undef, float -1.0>, %x
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %s
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_sub_x_one_y(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_sub_x_one_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm1, %xmm0, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_sub_x_one_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
-  %m = fmul <4 x float> %s, %y
+  %s = fsub FASTMATH_FLAGS <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %s, %y
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_sub_x_one(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_sub_x_one:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm0, %xmm1, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_sub_x_one:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
-  %m = fmul <4 x float> %y, %s
+  %s = fsub FASTMATH_FLAGS <4 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %s
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_sub_x_one_undefs(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_sub_x_one_undefs:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm0, %xmm1, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_sub_x_one_undefs:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> %x, <float 1.0, float 1.0, float 1.0, float undef>
-  %m = fmul <4 x float> %y, %s
+  %s = fsub FASTMATH_FLAGS <4 x float> %x, <float 1.0, float 1.0, float 1.0, float undef>
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %s
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_sub_x_negone_y(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_sub_x_negone_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm1, %xmm0, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_sub_x_negone_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213ps {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0>
-  %m = fmul <4 x float> %s, %y
+  %s = fsub FASTMATH_FLAGS <4 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %s, %y
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_sub_x_negone(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_sub_x_negone:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm0, %xmm1, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_sub_x_negone:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213ps {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0>
-  %m = fmul <4 x float> %y, %s
+  %s = fsub FASTMATH_FLAGS <4 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %s
   ret <4 x float> %m
 }
 
 define <4 x float> @test_v4f32_mul_y_sub_x_negone_undefs(<4 x float> %x, <4 x float> %y) {
 ; FMA-INFS-LABEL: test_v4f32_mul_y_sub_x_negone_undefs:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vaddps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-INFS-NEXT:    vmulps %xmm0, %xmm1, %xmm0

 ; FMA4-NOINFS:       # %bb.0:
 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_mul_y_sub_x_negone_undefs:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213ps {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <4 x float> %x, <float undef, float -1.0, float -1.0, float -1.0>
-  %m = fmul <4 x float> %y, %s
+  %s = fsub FASTMATH_FLAGS <4 x float> %x, <float undef, float -1.0, float -1.0, float -1.0>
+  %m = fmul FASTMATH_FLAGS <4 x float> %y, %s
   ret <4 x float> %m
 }
 
 ;
 ; Interpolation Patterns: add(mul(x,t),mul(sub(1.0,t),y))
 ;
 
 define float @test_f32_interp(float %x, float %y, float %t) {

 ; FMA4-NOINFS-NEXT:    vfmsubss {{.*#+}} xmm0 = (xmm0 * xmm2) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_f32_interp:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ss {{.*#+}} xmm1 = (xmm2 * xmm1) - xmm1
 ; AVX512-NOINFS-NEXT:    vfmsub213ss {{.*#+}} xmm0 = (xmm2 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %t1 = fsub nsz float 1.0, %t
-  %tx = fmul nsz float %x, %t
-  %ty = fmul nsz float %y, %t1
-  %r = fadd nsz float %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS nsz float 1.0, %t
+  %tx = fmul FASTMATH_FLAGS nsz float %x, %t
+  %ty = fmul FASTMATH_FLAGS nsz float %y, %t1
+  %r = fadd FASTMATH_FLAGS nsz float %tx, %ty
   ret float %r
 }
 
 define <4 x float> @test_v4f32_interp(<4 x float> %x, <4 x float> %y, <4 x float> %t) {
 ; FMA-INFS-LABEL: test_v4f32_interp:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} xmm3 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vsubps %xmm2, %xmm3, %xmm3

 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} xmm0 = (xmm0 * xmm2) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f32_interp:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} xmm1 = (xmm2 * xmm1) - xmm1
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm2 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %t1 = fsub nsz <4 x float> <float 1.0, float 1.0, float 1.0, float 1.0>, %t
-  %tx = fmul nsz <4 x float> %x, %t
-  %ty = fmul nsz <4 x float> %y, %t1
-  %r = fadd nsz <4 x float> %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS nsz <4 x float> <float 1.0, float 1.0, float 1.0, float 1.0>, %t
+  %tx = fmul FASTMATH_FLAGS nsz <4 x float> %x, %t
+  %ty = fmul FASTMATH_FLAGS nsz <4 x float> %y, %t1
+  %r = fadd FASTMATH_FLAGS nsz <4 x float> %tx, %ty
   ret <4 x float> %r
 }
 
 define <8 x float> @test_v8f32_interp(<8 x float> %x, <8 x float> %y, <8 x float> %t) {
 ; FMA-INFS-LABEL: test_v8f32_interp:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} ymm3 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vsubps %ymm2, %ymm3, %ymm3

 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} ymm0 = (ymm0 * ymm2) - ymm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v8f32_interp:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} ymm1 = (ymm2 * ymm1) - ymm1
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} ymm0 = (ymm2 * ymm0) - ymm1
 ; AVX512-NOINFS-NEXT:    retq
-  %t1 = fsub nsz <8 x float> <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>, %t
-  %tx = fmul nsz <8 x float> %x, %t
-  %ty = fmul nsz <8 x float> %y, %t1
-  %r = fadd nsz <8 x float> %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS nsz <8 x float> <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>, %t
+  %tx = fmul FASTMATH_FLAGS nsz <8 x float> %x, %t
+  %ty = fmul FASTMATH_FLAGS nsz <8 x float> %y, %t1
+  %r = fadd FASTMATH_FLAGS nsz <8 x float> %tx, %ty
   ret <8 x float> %r
 }
 
 define double @test_f64_interp(double %x, double %y, double %t) {
 ; FMA-INFS-LABEL: test_f64_interp:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovsd {{.*#+}} xmm3 = mem[0],zero
 ; FMA-INFS-NEXT:    vsubsd %xmm2, %xmm3, %xmm3

 ; FMA4-NOINFS-NEXT:    vfmsubsd {{.*#+}} xmm0 = (xmm0 * xmm2) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_f64_interp:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213sd {{.*#+}} xmm1 = (xmm2 * xmm1) - xmm1
 ; AVX512-NOINFS-NEXT:    vfmsub213sd {{.*#+}} xmm0 = (xmm2 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %t1 = fsub nsz double 1.0, %t
-  %tx = fmul nsz double %x, %t
-  %ty = fmul nsz double %y, %t1
-  %r = fadd nsz double %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS nsz double 1.0, %t
+  %tx = fmul FASTMATH_FLAGS nsz double %x, %t
+  %ty = fmul FASTMATH_FLAGS nsz double %y, %t1
+  %r = fadd FASTMATH_FLAGS nsz double %tx, %ty
   ret double %r
 }
 
 define <2 x double> @test_v2f64_interp(<2 x double> %x, <2 x double> %y, <2 x double> %t) {
 ; FMA-INFS-LABEL: test_v2f64_interp:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovapd {{.*#+}} xmm3 = [1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vsubpd %xmm2, %xmm3, %xmm3

 ; FMA4-NOINFS-NEXT:    vfmsubpd {{.*#+}} xmm0 = (xmm0 * xmm2) - xmm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v2f64_interp:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213pd {{.*#+}} xmm1 = (xmm2 * xmm1) - xmm1
 ; AVX512-NOINFS-NEXT:    vfmsub213pd {{.*#+}} xmm0 = (xmm2 * xmm0) - xmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %t1 = fsub nsz <2 x double> <double 1.0, double 1.0>, %t
-  %tx = fmul nsz <2 x double> %x, %t
-  %ty = fmul nsz <2 x double> %y, %t1
-  %r = fadd nsz <2 x double> %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS nsz <2 x double> <double 1.0, double 1.0>, %t
+  %tx = fmul FASTMATH_FLAGS nsz <2 x double> %x, %t
+  %ty = fmul FASTMATH_FLAGS nsz <2 x double> %y, %t1
+  %r = fadd FASTMATH_FLAGS nsz <2 x double> %tx, %ty
   ret <2 x double> %r
 }
 
 define <4 x double> @test_v4f64_interp(<4 x double> %x, <4 x double> %y, <4 x double> %t) {
 ; FMA-INFS-LABEL: test_v4f64_interp:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovapd {{.*#+}} ymm3 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vsubpd %ymm2, %ymm3, %ymm3

 ; FMA4-NOINFS-NEXT:    vfmsubpd {{.*#+}} ymm0 = (ymm0 * ymm2) - ymm1
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v4f64_interp:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213pd {{.*#+}} ymm1 = (ymm2 * ymm1) - ymm1
 ; AVX512-NOINFS-NEXT:    vfmsub213pd {{.*#+}} ymm0 = (ymm2 * ymm0) - ymm1
 ; AVX512-NOINFS-NEXT:    retq
-  %t1 = fsub nsz <4 x double> <double 1.0, double 1.0, double 1.0, double 1.0>, %t
-  %tx = fmul nsz <4 x double> %x, %t
-  %ty = fmul nsz <4 x double> %y, %t1
-  %r = fadd nsz <4 x double> %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS nsz <4 x double> <double 1.0, double 1.0, double 1.0, double 1.0>, %t
+  %tx = fmul FASTMATH_FLAGS nsz <4 x double> %x, %t
+  %ty = fmul FASTMATH_FLAGS nsz <4 x double> %y, %t1
+  %r = fadd FASTMATH_FLAGS nsz <4 x double> %tx, %ty
   ret <4 x double> %r
 }
 
 ;
 ; Pattern: (fneg (fma x, y, z)) -> (fma x, -y, -z)
 ;
 
 define <4 x float> @test_v4f32_fneg_fmadd(<4 x float> %a0, <4 x float> %a1, <4 x float> %a2) #0 {
 ; FMA-LABEL: test_v4f32_fneg_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v4f32_fneg_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubps {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v4f32_fneg_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %mul = fmul nsz <4 x float> %a0, %a1
-  %add = fadd nsz <4 x float> %mul, %a2
-  %neg = fsub nsz <4 x float> <float -0.0, float -0.0, float -0.0, float -0.0>, %add
+  %mul = fmul FASTMATH_FLAGS nsz <4 x float> %a0, %a1
+  %add = fadd FASTMATH_FLAGS nsz <4 x float> %mul, %a2
+  %neg = fsub FASTMATH_FLAGS nsz <4 x float> <float -0.0, float -0.0, float -0.0, float -0.0>, %add
   ret <4 x float> %neg
 }
 
 define <4 x double> @test_v4f64_fneg_fmsub(<4 x double> %a0, <4 x double> %a1, <4 x double> %a2) #0 {
 ; FMA-LABEL: test_v4f64_fneg_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213pd {{.*#+}} ymm0 = -(ymm1 * ymm0) + ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v4f64_fneg_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddpd {{.*#+}} ymm0 = -(ymm0 * ymm1) + ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v4f64_fneg_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213pd {{.*#+}} ymm0 = -(ymm1 * ymm0) + ymm2
 ; AVX512-NEXT:    retq
-  %mul = fmul nsz <4 x double> %a0, %a1
-  %sub = fsub nsz <4 x double> %mul, %a2
-  %neg = fsub nsz <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %sub
+  %mul = fmul FASTMATH_FLAGS nsz <4 x double> %a0, %a1
+  %sub = fsub FASTMATH_FLAGS nsz <4 x double> %mul, %a2
+  %neg = fsub FASTMATH_FLAGS nsz <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %sub
   ret <4 x double> %neg
 }
 
 define <4 x float> @test_v4f32_fneg_fnmadd(<4 x float> %a0, <4 x float> %a1, <4 x float> %a2) #0 {
 ; FMA-LABEL: test_v4f32_fneg_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v4f32_fneg_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubps {{.*#+}} xmm0 = (xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v4f32_fneg_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213ps {{.*#+}} xmm0 = (xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %mul = fmul nsz <4 x float> %a0, %a1
-  %neg0 = fsub nsz <4 x float> <float -0.0, float -0.0, float -0.0, float -0.0>, %mul
-  %add = fadd nsz <4 x float> %neg0, %a2
-  %neg1 = fsub nsz <4 x float> <float -0.0, float -0.0, float -0.0, float -0.0>, %add
+  %mul = fmul FASTMATH_FLAGS nsz <4 x float> %a0, %a1
+  %neg0 = fsub FASTMATH_FLAGS nsz <4 x float> <float -0.0, float -0.0, float -0.0, float -0.0>, %mul
+  %add = fadd FASTMATH_FLAGS nsz <4 x float> %neg0, %a2
+  %neg1 = fsub FASTMATH_FLAGS nsz <4 x float> <float -0.0, float -0.0, float -0.0, float -0.0>, %add
   ret <4 x float> %neg1
 }
 
 define <4 x double> @test_v4f64_fneg_fnmsub(<4 x double> %a0, <4 x double> %a1, <4 x double> %a2) #0 {
 ; FMA-LABEL: test_v4f64_fneg_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213pd {{.*#+}} ymm0 = (ymm1 * ymm0) + ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v4f64_fneg_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddpd {{.*#+}} ymm0 = (ymm0 * ymm1) + ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v4f64_fneg_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213pd {{.*#+}} ymm0 = (ymm1 * ymm0) + ymm2
 ; AVX512-NEXT:    retq
-  %mul = fmul nsz  <4 x double> %a0, %a1
-  %neg0 = fsub nsz <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %mul
-  %sub = fsub nsz <4 x double> %neg0, %a2
-  %neg1 = fsub nsz <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %sub
+  %mul = fmul FASTMATH_FLAGS nsz  <4 x double> %a0, %a1
+  %neg0 = fsub FASTMATH_FLAGS nsz <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %mul
+  %sub = fsub FASTMATH_FLAGS nsz <4 x double> %neg0, %a2
+  %neg1 = fsub FASTMATH_FLAGS nsz <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %sub
   ret <4 x double> %neg1
 }
 
 ;
 ; Pattern: (fma x, c1, (fmul x, c2)) -> (fmul x, c1+c2)
 ;
 
 define <4 x float> @test_v4f32_fma_x_c1_fmul_x_c2(<4 x float> %x) #0 {
 ; FMA-LABEL: test_v4f32_fma_x_c1_fmul_x_c2:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vmulps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v4f32_fma_x_c1_fmul_x_c2:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vmulps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v4f32_fma_x_c1_fmul_x_c2:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vmulps {{\.?LCPI[0-9]+_[0-9]+}}(%rip){1to4}, %xmm0, %xmm0
 ; AVX512-NEXT:    retq
-  %m0 = fmul <4 x float> %x, <float 1.0, float 2.0, float 3.0, float 4.0>
-  %m1 = fmul <4 x float> %x, <float 4.0, float 3.0, float 2.0, float 1.0>
-  %a  = fadd <4 x float> %m0, %m1
+  %m0 = fmul FASTMATH_FLAGS <4 x float> %x, <float 1.0, float 2.0, float 3.0, float 4.0>
+  %m1 = fmul FASTMATH_FLAGS <4 x float> %x, <float 4.0, float 3.0, float 2.0, float 1.0>
+  %a  = fadd FASTMATH_FLAGS <4 x float> %m0, %m1
   ret <4 x float> %a
 }
 
 ;
 ; Pattern: (fma (fmul x, c1), c2, y) -> (fma x, c1*c2, y)
 ;
 
 define <4 x float> @test_v4f32_fma_fmul_x_c1_c2_y(<4 x float> %x, <4 x float> %y) #0 {
 ; FMA-LABEL: test_v4f32_fma_fmul_x_c1_c2_y:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd132ps {{.*#+}} xmm0 = (xmm0 * mem) + xmm1
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v4f32_fma_fmul_x_c1_c2_y:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddps {{.*#+}} xmm0 = (xmm0 * mem) + xmm1
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v4f32_fma_fmul_x_c1_c2_y:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd132ps {{.*#+}} xmm0 = (xmm0 * mem) + xmm1
 ; AVX512-NEXT:    retq
-  %m0 = fmul <4 x float> %x,  <float 1.0, float 2.0, float 3.0, float 4.0>
-  %m1 = fmul <4 x float> %m0, <float 4.0, float 3.0, float 2.0, float 1.0>
-  %a  = fadd <4 x float> %m1, %y
+  %m0 = fmul FASTMATH_FLAGS <4 x float> %x,  <float 1.0, float 2.0, float 3.0, float 4.0>
+  %m1 = fmul FASTMATH_FLAGS <4 x float> %m0, <float 4.0, float 3.0, float 2.0, float 1.0>
+  %a  = fadd FASTMATH_FLAGS <4 x float> %m1, %y
   ret <4 x float> %a
 }
 
 ; Pattern: (fneg (fmul x, y)) -> (fnmsub x, y, 0)
 
 define double @test_f64_fneg_fmul(double %x, double %y) #0 {
 ; FMA-LABEL: test_f64_fneg_fmul:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vxorpd %xmm2, %xmm2, %xmm2
 ; FMA-NEXT:    vfnmsub213sd {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_f64_fneg_fmul:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vxorpd %xmm2, %xmm2, %xmm2
 ; FMA4-NEXT:    vfnmsubsd {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_f64_fneg_fmul:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vxorpd %xmm2, %xmm2, %xmm2
 ; AVX512-NEXT:    vfnmsub213sd {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %m = fmul nsz double %x, %y
-  %n = fsub double -0.0, %m
+  %m = fmul FASTMATH_FLAGS nsz double %x, %y
+  %n = fsub FASTMATH_FLAGS double -0.0, %m
   ret double %n
 }
 
 define <4 x float> @test_v4f32_fneg_fmul(<4 x float> %x, <4 x float> %y) #0 {
 ; FMA-LABEL: test_v4f32_fneg_fmul:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vxorps %xmm2, %xmm2, %xmm2
 ; FMA-NEXT:    vfnmsub213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v4f32_fneg_fmul:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vxorps %xmm2, %xmm2, %xmm2
 ; FMA4-NEXT:    vfnmsubps {{.*#+}} xmm0 = -(xmm0 * xmm1) - xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v4f32_fneg_fmul:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vxorps %xmm2, %xmm2, %xmm2
 ; AVX512-NEXT:    vfnmsub213ps {{.*#+}} xmm0 = -(xmm1 * xmm0) - xmm2
 ; AVX512-NEXT:    retq
-  %m = fmul nsz <4 x float> %x, %y
-  %n = fsub <4 x float> <float -0.0, float -0.0, float -0.0, float -0.0>, %m
+  %m = fmul FASTMATH_FLAGS nsz <4 x float> %x, %y
+  %n = fsub FASTMATH_FLAGS <4 x float> <float -0.0, float -0.0, float -0.0, float -0.0>, %m
   ret <4 x float> %n
 }
 
 define <4 x double> @test_v4f64_fneg_fmul(<4 x double> %x, <4 x double> %y) #0 {
 ; FMA-LABEL: test_v4f64_fneg_fmul:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vxorpd %xmm2, %xmm2, %xmm2
 ; FMA-NEXT:    vfnmsub213pd {{.*#+}} ymm0 = -(ymm1 * ymm0) - ymm2
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v4f64_fneg_fmul:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vxorpd %xmm2, %xmm2, %xmm2
 ; FMA4-NEXT:    vfnmsubpd {{.*#+}} ymm0 = -(ymm0 * ymm1) - ymm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v4f64_fneg_fmul:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vxorpd %xmm2, %xmm2, %xmm2
 ; AVX512-NEXT:    vfnmsub213pd {{.*#+}} ymm0 = -(ymm1 * ymm0) - ymm2
 ; AVX512-NEXT:    retq
-  %m = fmul nsz <4 x double> %x, %y
-  %n = fsub <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %m
+  %m = fmul FASTMATH_FLAGS nsz <4 x double> %x, %y
+  %n = fsub FASTMATH_FLAGS <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %m
   ret <4 x double> %n
 }
 
 define <4 x double> @test_v4f64_fneg_fmul_no_nsz(<4 x double> %x, <4 x double> %y) #0 {
 ; FMA-LABEL: test_v4f64_fneg_fmul_no_nsz:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vmulpd %ymm1, %ymm0, %ymm0
 ; FMA-NEXT:    vxorpd {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %ymm0, %ymm0
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v4f64_fneg_fmul_no_nsz:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vmulpd %ymm1, %ymm0, %ymm0
 ; FMA4-NEXT:    vxorpd {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %ymm0, %ymm0
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v4f64_fneg_fmul_no_nsz:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vmulpd %ymm1, %ymm0, %ymm0
 ; AVX512-NEXT:    vxorpd {{\.?LCPI[0-9]+_[0-9]+}}(%rip){1to4}, %ymm0, %ymm0
 ; AVX512-NEXT:    retq
-  %m = fmul <4 x double> %x, %y
-  %n = fsub <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %m
+  %m = fmul FASTMATH_FLAGS <4 x double> %x, %y
+  %n = fsub FASTMATH_FLAGS <4 x double> <double -0.0, double -0.0, double -0.0, double -0.0>, %m
   ret <4 x double> %n
 }
 
 ; ((a*b) + (c*d)) + n1 --> (a*b) + ((c*d) + n1)
 
 define double @fadd_fma_fmul_1(double %a, double %b, double %c, double %d, double %n1) nounwind {
 ; FMA-LABEL: fadd_fma_fmul_1:
 ; FMA:       # %bb.0:

 ; FMA4-NEXT:    vfmaddss {{.*#+}} xmm0 = (xmm0 * xmm1) + xmm2
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: fadd_fma_fmul_fmf:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213ss {{.*#+}} xmm2 = (xmm3 * xmm2) + xmm4
 ; AVX512-NEXT:    vfmadd213ss {{.*#+}} xmm0 = (xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    retq
-  %m1 = fmul float %a, %b
-  %m2 = fmul float %c, %d
+  %m1 = fmul FASTMATH_FLAGS float %a, %b
+  %m2 = fmul FASTMATH_FLAGS float %c, %d
   %a1 = fadd contract float %m1, %m2
-  %a2 = fadd reassoc float %n0, %a1
+  %a2 = fadd FASTMATH_FLAGS reassoc float %n0, %a1
   ret float %a2
 }
 
 ; Not minimum FMF.
 
 define float @fadd_fma_fmul_2(float %a, float %b, float %c, float %d, float %n0) nounwind {
 ; FMA-LABEL: fadd_fma_fmul_2:
 ; FMA:       # %bb.0:

 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: fadd_fma_fmul_2:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vmulss %xmm3, %xmm2, %xmm2
 ; AVX512-NEXT:    vfmadd231ss {{.*#+}} xmm2 = (xmm1 * xmm0) + xmm2
 ; AVX512-NEXT:    vaddss %xmm2, %xmm4, %xmm0
 ; AVX512-NEXT:    retq
-  %m1 = fmul float %a, %b
-  %m2 = fmul float %c, %d
+  %m1 = fmul FASTMATH_FLAGS float %a, %b
+  %m2 = fmul FASTMATH_FLAGS float %c, %d
   %a1 = fadd contract float %m1, %m2
   %a2 = fadd contract float %n0, %a1
   ret float %a2
 }
 
 ; The final fadd can be folded with either 1 of the leading fmuls.
 
 define <2 x double> @fadd_fma_fmul_3(<2 x double> %x1, <2 x double> %x2, <2 x double> %x3, <2 x double> %x4, <2 x double> %x5, <2 x double> %x6, <2 x double> %x7, <2 x double> %x8) nounwind {

llvm/test/CodeGen/X86/fma_patterns_wide.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma -fp-contract=fast | FileCheck %s --check-prefix=FMA --check-prefix=FMA-INFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4,+fma -fp-contract=fast | FileCheck %s --check-prefix=FMA4 --check-prefix=FMA4-INFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4 -fp-contract=fast | FileCheck %s --check-prefix=FMA4 --check-prefix=FMA4-INFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq -fp-contract=fast | FileCheck %s --check-prefix=AVX512 --check-prefix=AVX512-INFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma -fp-contract=fast -enable-no-infs-fp-math | FileCheck %s --check-prefix=FMA --check-prefix=FMA-NOINFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4,+fma -fp-contract=fast -enable-no-infs-fp-math | FileCheck %s --check-prefix=FMA4 --check-prefix=FMA4-NOINFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4 -fp-contract=fast -enable-no-infs-fp-math | FileCheck %s --check-prefix=FMA4 --check-prefix=FMA4-NOINFS
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq -fp-contract=fast -enable-no-infs-fp-math | FileCheck %s --check-prefix=AVX512 --check-prefix=AVX512-NOINFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma | FileCheck %s --check-prefix=FMA --check-prefix=FMA-INFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4,+fma | FileCheck %s --check-prefix=FMA4 --check-prefix=FMA4-INFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4 | FileCheck %s --check-prefix=FMA4 --check-prefix=FMA4-INFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx512dq | FileCheck %s --check-prefix=AVX512 --check-prefix=AVX512-INFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract ninf,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma | FileCheck %s --check-prefix=FMA --check-prefix=FMA-NOINFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract ninf,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4,+fma | FileCheck %s --check-prefix=FMA4 --check-prefix=FMA4-NOINFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract ninf,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma4 | FileCheck %s --check-prefix=FMA4 --check-prefix=FMA4-NOINFS
+; RUN: sed -e "s,FASTMATH_FLAGS,contract ninf,g" %s | llc -mtriple=x86_64-unknown-unknown -mattr=+avx512dq | FileCheck %s --check-prefix=AVX512 --check-prefix=AVX512-NOINFS
 
 ;
 ; Pattern: (fadd (fmul x, y), z) -> (fmadd x,y,z)
 ;
 
 define <16 x float> @test_16f32_fmadd(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) {
 ; FMA-LABEL: test_16f32_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213ps {{.*#+}} ymm0 = (ymm2 * ymm0) + ymm4
 ; FMA-NEXT:    vfmadd213ps {{.*#+}} ymm1 = (ymm3 * ymm1) + ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_16f32_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddps {{.*#+}} ymm0 = (ymm0 * ymm2) + ymm4
 ; FMA4-NEXT:    vfmaddps {{.*#+}} ymm1 = (ymm1 * ymm3) + ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_16f32_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213ps {{.*#+}} zmm0 = (zmm1 * zmm0) + zmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <16 x float> %a0, %a1
-  %res = fadd <16 x float> %x, %a2
+  %x = fmul FASTMATH_FLAGS <16 x float> %a0, %a1
+  %res = fadd FASTMATH_FLAGS <16 x float> %x, %a2
   ret <16 x float> %res
 }
 
 define <8 x double> @test_8f64_fmadd(<8 x double> %a0, <8 x double> %a1, <8 x double> %a2) {
 ; FMA-LABEL: test_8f64_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213pd {{.*#+}} ymm0 = (ymm2 * ymm0) + ymm4
 ; FMA-NEXT:    vfmadd213pd {{.*#+}} ymm1 = (ymm3 * ymm1) + ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_8f64_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddpd {{.*#+}} ymm0 = (ymm0 * ymm2) + ymm4
 ; FMA4-NEXT:    vfmaddpd {{.*#+}} ymm1 = (ymm1 * ymm3) + ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_8f64_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213pd {{.*#+}} zmm0 = (zmm1 * zmm0) + zmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <8 x double> %a0, %a1
-  %res = fadd <8 x double> %x, %a2
+  %x = fmul FASTMATH_FLAGS <8 x double> %a0, %a1
+  %res = fadd FASTMATH_FLAGS <8 x double> %x, %a2
   ret <8 x double> %res
 }
 
 ;
 ; Pattern: (fsub (fmul x, y), z) -> (fmsub x, y, z)
 ;
 
 define <16 x float> @test_16f32_fmsub(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) {
 ; FMA-LABEL: test_16f32_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213ps {{.*#+}} ymm0 = (ymm2 * ymm0) - ymm4
 ; FMA-NEXT:    vfmsub213ps {{.*#+}} ymm1 = (ymm3 * ymm1) - ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_16f32_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubps {{.*#+}} ymm0 = (ymm0 * ymm2) - ymm4
 ; FMA4-NEXT:    vfmsubps {{.*#+}} ymm1 = (ymm1 * ymm3) - ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_16f32_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213ps {{.*#+}} zmm0 = (zmm1 * zmm0) - zmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <16 x float> %a0, %a1
-  %res = fsub <16 x float> %x, %a2
+  %x = fmul FASTMATH_FLAGS <16 x float> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <16 x float> %x, %a2
   ret <16 x float> %res
 }
 
 define <8 x double> @test_8f64_fmsub(<8 x double> %a0, <8 x double> %a1, <8 x double> %a2) {
 ; FMA-LABEL: test_8f64_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213pd {{.*#+}} ymm0 = (ymm2 * ymm0) - ymm4
 ; FMA-NEXT:    vfmsub213pd {{.*#+}} ymm1 = (ymm3 * ymm1) - ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_8f64_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubpd {{.*#+}} ymm0 = (ymm0 * ymm2) - ymm4
 ; FMA4-NEXT:    vfmsubpd {{.*#+}} ymm1 = (ymm1 * ymm3) - ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_8f64_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213pd {{.*#+}} zmm0 = (zmm1 * zmm0) - zmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <8 x double> %a0, %a1
-  %res = fsub <8 x double> %x, %a2
+  %x = fmul FASTMATH_FLAGS <8 x double> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <8 x double> %x, %a2
   ret <8 x double> %res
 }
 
 ;
 ; Pattern: (fsub z, (fmul x, y)) -> (fnmadd x, y, z)
 ;
 
 define <16 x float> @test_16f32_fnmadd(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) {
 ; FMA-LABEL: test_16f32_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213ps {{.*#+}} ymm0 = -(ymm2 * ymm0) + ymm4
 ; FMA-NEXT:    vfnmadd213ps {{.*#+}} ymm1 = -(ymm3 * ymm1) + ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_16f32_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddps {{.*#+}} ymm0 = -(ymm0 * ymm2) + ymm4
 ; FMA4-NEXT:    vfnmaddps {{.*#+}} ymm1 = -(ymm1 * ymm3) + ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_16f32_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213ps {{.*#+}} zmm0 = -(zmm1 * zmm0) + zmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <16 x float> %a0, %a1
-  %res = fsub <16 x float> %a2, %x
+  %x = fmul FASTMATH_FLAGS <16 x float> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <16 x float> %a2, %x
   ret <16 x float> %res
 }
 
 define <8 x double> @test_8f64_fnmadd(<8 x double> %a0, <8 x double> %a1, <8 x double> %a2) {
 ; FMA-LABEL: test_8f64_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213pd {{.*#+}} ymm0 = -(ymm2 * ymm0) + ymm4
 ; FMA-NEXT:    vfnmadd213pd {{.*#+}} ymm1 = -(ymm3 * ymm1) + ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_8f64_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddpd {{.*#+}} ymm0 = -(ymm0 * ymm2) + ymm4
 ; FMA4-NEXT:    vfnmaddpd {{.*#+}} ymm1 = -(ymm1 * ymm3) + ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_8f64_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213pd {{.*#+}} zmm0 = -(zmm1 * zmm0) + zmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <8 x double> %a0, %a1
-  %res = fsub <8 x double> %a2, %x
+  %x = fmul FASTMATH_FLAGS <8 x double> %a0, %a1
+  %res = fsub FASTMATH_FLAGS <8 x double> %a2, %x
   ret <8 x double> %res
 }
 
 ;
 ; Pattern: (fsub (fneg (fmul x, y)), z) -> (fnmsub x, y, z)
 ;
 
 define <16 x float> @test_16f32_fnmsub(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) {
 ; FMA-LABEL: test_16f32_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213ps {{.*#+}} ymm0 = -(ymm2 * ymm0) - ymm4
 ; FMA-NEXT:    vfnmsub213ps {{.*#+}} ymm1 = -(ymm3 * ymm1) - ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_16f32_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubps {{.*#+}} ymm0 = -(ymm0 * ymm2) - ymm4
 ; FMA4-NEXT:    vfnmsubps {{.*#+}} ymm1 = -(ymm1 * ymm3) - ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_16f32_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213ps {{.*#+}} zmm0 = -(zmm1 * zmm0) - zmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <16 x float> %a0, %a1
-  %y = fsub <16 x float> <float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00>, %x
-  %res = fsub <16 x float> %y, %a2
+  %x = fmul FASTMATH_FLAGS <16 x float> %a0, %a1
+  %y = fsub FASTMATH_FLAGS <16 x float> <float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00, float -0.000000e+00>, %x
+  %res = fsub FASTMATH_FLAGS <16 x float> %y, %a2
   ret <16 x float> %res
 }
 
 define <8 x double> @test_8f64_fnmsub(<8 x double> %a0, <8 x double> %a1, <8 x double> %a2) {
 ; FMA-LABEL: test_8f64_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213pd {{.*#+}} ymm0 = -(ymm2 * ymm0) - ymm4
 ; FMA-NEXT:    vfnmsub213pd {{.*#+}} ymm1 = -(ymm3 * ymm1) - ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_8f64_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubpd {{.*#+}} ymm0 = -(ymm0 * ymm2) - ymm4
 ; FMA4-NEXT:    vfnmsubpd {{.*#+}} ymm1 = -(ymm1 * ymm3) - ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_8f64_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213pd {{.*#+}} zmm0 = -(zmm1 * zmm0) - zmm2
 ; AVX512-NEXT:    retq
-  %x = fmul <8 x double> %a0, %a1
-  %y = fsub <8 x double> <double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00>, %x
-  %res = fsub <8 x double> %y, %a2
+  %x = fmul FASTMATH_FLAGS <8 x double> %a0, %a1
+  %y = fsub FASTMATH_FLAGS <8 x double> <double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00, double -0.000000e+00>, %x
+  %res = fsub FASTMATH_FLAGS <8 x double> %y, %a2
   ret <8 x double> %res
 }
 
 ;
 ; Load Folding Patterns
 ;
 
 define <16 x float> @test_16f32_fmadd_load(<16 x float>* %a0, <16 x float> %a1, <16 x float> %a2) {

 ; FMA4-NEXT:    vfmaddps {{.*#+}} ymm1 = (ymm1 * mem) + ymm3
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_16f32_fmadd_load:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd132ps {{.*#+}} zmm0 = (zmm0 * mem) + zmm1
 ; AVX512-NEXT:    retq
   %x = load <16 x float>, <16 x float>* %a0
-  %y = fmul <16 x float> %x, %a1
-  %res = fadd <16 x float> %y, %a2
+  %y = fmul FASTMATH_FLAGS <16 x float> %x, %a1
+  %res = fadd FASTMATH_FLAGS <16 x float> %y, %a2
   ret <16 x float> %res
 }
 
 define <8 x double> @test_8f64_fmsub_load(<8 x double>* %a0, <8 x double> %a1, <8 x double> %a2) {
 ; FMA-LABEL: test_8f64_fmsub_load:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub132pd {{.*#+}} ymm0 = (ymm0 * mem) - ymm2
 ; FMA-NEXT:    vfmsub132pd {{.*#+}} ymm1 = (ymm1 * mem) - ymm3
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_8f64_fmsub_load:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubpd {{.*#+}} ymm0 = (ymm0 * mem) - ymm2
 ; FMA4-NEXT:    vfmsubpd {{.*#+}} ymm1 = (ymm1 * mem) - ymm3
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_8f64_fmsub_load:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub132pd {{.*#+}} zmm0 = (zmm0 * mem) - zmm1
 ; AVX512-NEXT:    retq
   %x = load <8 x double>, <8 x double>* %a0
-  %y = fmul <8 x double> %x, %a1
-  %res = fsub <8 x double> %y, %a2
+  %y = fmul FASTMATH_FLAGS <8 x double> %x, %a1
+  %res = fsub FASTMATH_FLAGS <8 x double> %y, %a2
   ret <8 x double> %res
 }
 
 ;
 ; Patterns (+ fneg variants): mul(add(1.0,x),y), mul(sub(1.0,x),y), mul(sub(x,1.0),y)
 ;
 
 define <16 x float> @test_v16f32_mul_add_x_one_y(<16 x float> %x, <16 x float> %y) {

 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} ymm0 = (ymm0 * ymm2) + ymm2
 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} ymm1 = (ymm1 * ymm3) + ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v16f32_mul_add_x_one_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213ps {{.*#+}} zmm0 = (zmm1 * zmm0) + zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <16 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>
-  %m = fmul <16 x float> %a, %y
+  %a = fadd FASTMATH_FLAGS <16 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>
+  %m = fmul FASTMATH_FLAGS <16 x float> %a, %y
   ret <16 x float> %m
 }
 
 define <8 x double> @test_v8f64_mul_y_add_x_one(<8 x double> %x, <8 x double> %y) {
 ; FMA-INFS-LABEL: test_v8f64_mul_y_add_x_one:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovapd {{.*#+}} ymm4 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vaddpd %ymm4, %ymm1, %ymm1

 ; FMA4-NOINFS-NEXT:    vfmaddpd {{.*#+}} ymm0 = (ymm0 * ymm2) + ymm2
 ; FMA4-NOINFS-NEXT:    vfmaddpd {{.*#+}} ymm1 = (ymm1 * ymm3) + ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v8f64_mul_y_add_x_one:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213pd {{.*#+}} zmm0 = (zmm1 * zmm0) + zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <8 x double> %x, <double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0>
-  %m = fmul <8 x double> %y, %a
+  %a = fadd FASTMATH_FLAGS <8 x double> %x, <double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0>
+  %m = fmul FASTMATH_FLAGS <8 x double> %y, %a
   ret <8 x double> %m
 }
 
 define <16 x float> @test_v16f32_mul_add_x_negone_y(<16 x float> %x, <16 x float> %y) {
 ; FMA-INFS-LABEL: test_v16f32_mul_add_x_negone_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} ymm4 = [-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0]
 ; FMA-INFS-NEXT:    vaddps %ymm4, %ymm1, %ymm1

 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} ymm0 = (ymm0 * ymm2) - ymm2
 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} ymm1 = (ymm1 * ymm3) - ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v16f32_mul_add_x_negone_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} zmm0 = (zmm1 * zmm0) - zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <16 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0>
-  %m = fmul <16 x float> %a, %y
+  %a = fadd FASTMATH_FLAGS <16 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0>
+  %m = fmul FASTMATH_FLAGS <16 x float> %a, %y
   ret <16 x float> %m
 }
 
 define <8 x double> @test_v8f64_mul_y_add_x_negone(<8 x double> %x, <8 x double> %y) {
 ; FMA-INFS-LABEL: test_v8f64_mul_y_add_x_negone:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovapd {{.*#+}} ymm4 = [-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0]
 ; FMA-INFS-NEXT:    vaddpd %ymm4, %ymm1, %ymm1

 ; FMA4-NOINFS-NEXT:    vfmsubpd {{.*#+}} ymm0 = (ymm0 * ymm2) - ymm2
 ; FMA4-NOINFS-NEXT:    vfmsubpd {{.*#+}} ymm1 = (ymm1 * ymm3) - ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v8f64_mul_y_add_x_negone:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213pd {{.*#+}} zmm0 = (zmm1 * zmm0) - zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %a = fadd <8 x double> %x, <double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0>
-  %m = fmul <8 x double> %y, %a
+  %a = fadd FASTMATH_FLAGS <8 x double> %x, <double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0>
+  %m = fmul FASTMATH_FLAGS <8 x double> %y, %a
   ret <8 x double> %m
 }
 
 define <16 x float> @test_v16f32_mul_sub_one_x_y(<16 x float> %x, <16 x float> %y) {
 ; FMA-INFS-LABEL: test_v16f32_mul_sub_one_x_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} ymm4 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vsubps %ymm1, %ymm4, %ymm1

 ; FMA4-NOINFS-NEXT:    vfnmaddps {{.*#+}} ymm0 = -(ymm0 * ymm2) + ymm2
 ; FMA4-NOINFS-NEXT:    vfnmaddps {{.*#+}} ymm1 = -(ymm1 * ymm3) + ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v16f32_mul_sub_one_x_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmadd213ps {{.*#+}} zmm0 = -(zmm1 * zmm0) + zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <16 x float> <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>, %x
-  %m = fmul <16 x float> %s, %y
+  %s = fsub FASTMATH_FLAGS <16 x float> <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>, %x
+  %m = fmul FASTMATH_FLAGS <16 x float> %s, %y
   ret <16 x float> %m
 }
 
 define <8 x double> @test_v8f64_mul_y_sub_one_x(<8 x double> %x, <8 x double> %y) {
 ; FMA-INFS-LABEL: test_v8f64_mul_y_sub_one_x:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovapd {{.*#+}} ymm4 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vsubpd %ymm1, %ymm4, %ymm1

 ; FMA4-NOINFS-NEXT:    vfnmaddpd {{.*#+}} ymm0 = -(ymm0 * ymm2) + ymm2
 ; FMA4-NOINFS-NEXT:    vfnmaddpd {{.*#+}} ymm1 = -(ymm1 * ymm3) + ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v8f64_mul_y_sub_one_x:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmadd213pd {{.*#+}} zmm0 = -(zmm1 * zmm0) + zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <8 x double> <double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0>, %x
-  %m = fmul <8 x double> %y, %s
+  %s = fsub FASTMATH_FLAGS <8 x double> <double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0>, %x
+  %m = fmul FASTMATH_FLAGS <8 x double> %y, %s
   ret <8 x double> %m
 }
 
 define <16 x float> @test_v16f32_mul_sub_negone_x_y(<16 x float> %x, <16 x float> %y) {
 ; FMA-INFS-LABEL: test_v16f32_mul_sub_negone_x_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} ymm4 = [-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0]
 ; FMA-INFS-NEXT:    vsubps %ymm1, %ymm4, %ymm1

 ; FMA4-NOINFS-NEXT:    vfnmsubps {{.*#+}} ymm0 = -(ymm0 * ymm2) - ymm2
 ; FMA4-NOINFS-NEXT:    vfnmsubps {{.*#+}} ymm1 = -(ymm1 * ymm3) - ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v16f32_mul_sub_negone_x_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmsub213ps {{.*#+}} zmm0 = -(zmm1 * zmm0) - zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <16 x float> <float -1.0, float -1.0, float -1.0, float -1.0,float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0>, %x
-  %m = fmul <16 x float> %s, %y
+  %s = fsub FASTMATH_FLAGS <16 x float> <float -1.0, float -1.0, float -1.0, float -1.0,float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0>, %x
+  %m = fmul FASTMATH_FLAGS <16 x float> %s, %y
   ret <16 x float> %m
 }
 
 define <8 x double> @test_v8f64_mul_y_sub_negone_x(<8 x double> %x, <8 x double> %y) {
 ; FMA-INFS-LABEL: test_v8f64_mul_y_sub_negone_x:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovapd {{.*#+}} ymm4 = [-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0]
 ; FMA-INFS-NEXT:    vsubpd %ymm1, %ymm4, %ymm1

 ; FMA4-NOINFS-NEXT:    vfnmsubpd {{.*#+}} ymm0 = -(ymm0 * ymm2) - ymm2
 ; FMA4-NOINFS-NEXT:    vfnmsubpd {{.*#+}} ymm1 = -(ymm1 * ymm3) - ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v8f64_mul_y_sub_negone_x:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfnmsub213pd {{.*#+}} zmm0 = -(zmm1 * zmm0) - zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <8 x double> <double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0>, %x
-  %m = fmul <8 x double> %y, %s
+  %s = fsub FASTMATH_FLAGS <8 x double> <double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0>, %x
+  %m = fmul FASTMATH_FLAGS <8 x double> %y, %s
   ret <8 x double> %m
 }
 
 define <16 x float> @test_v16f32_mul_sub_x_one_y(<16 x float> %x, <16 x float> %y) {
 ; FMA-INFS-LABEL: test_v16f32_mul_sub_x_one_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} ymm4 = [-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0]
 ; FMA-INFS-NEXT:    vaddps %ymm4, %ymm1, %ymm1

 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} ymm0 = (ymm0 * ymm2) - ymm2
 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} ymm1 = (ymm1 * ymm3) - ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v16f32_mul_sub_x_one_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} zmm0 = (zmm1 * zmm0) - zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <16 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>
-  %m = fmul <16 x float> %s, %y
+  %s = fsub FASTMATH_FLAGS <16 x float> %x, <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>
+  %m = fmul FASTMATH_FLAGS <16 x float> %s, %y
   ret <16 x float> %m
 }
 
 define <8 x double> @test_v8f64_mul_y_sub_x_one(<8 x double> %x, <8 x double> %y) {
 ; FMA-INFS-LABEL: test_v8f64_mul_y_sub_x_one:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovapd {{.*#+}} ymm4 = [-1.0E+0,-1.0E+0,-1.0E+0,-1.0E+0]
 ; FMA-INFS-NEXT:    vaddpd %ymm4, %ymm1, %ymm1

 ; FMA4-NOINFS-NEXT:    vfmsubpd {{.*#+}} ymm0 = (ymm0 * ymm2) - ymm2
 ; FMA4-NOINFS-NEXT:    vfmsubpd {{.*#+}} ymm1 = (ymm1 * ymm3) - ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v8f64_mul_y_sub_x_one:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213pd {{.*#+}} zmm0 = (zmm1 * zmm0) - zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <8 x double> %x, <double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0>
-  %m = fmul <8 x double> %y, %s
+  %s = fsub FASTMATH_FLAGS <8 x double> %x, <double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0>
+  %m = fmul FASTMATH_FLAGS <8 x double> %y, %s
   ret <8 x double> %m
 }
 
 define <16 x float> @test_v16f32_mul_sub_x_negone_y(<16 x float> %x, <16 x float> %y) {
 ; FMA-INFS-LABEL: test_v16f32_mul_sub_x_negone_y:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovaps {{.*#+}} ymm4 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vaddps %ymm4, %ymm1, %ymm1

 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} ymm0 = (ymm0 * ymm2) + ymm2
 ; FMA4-NOINFS-NEXT:    vfmaddps {{.*#+}} ymm1 = (ymm1 * ymm3) + ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v16f32_mul_sub_x_negone_y:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213ps {{.*#+}} zmm0 = (zmm1 * zmm0) + zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <16 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0>
-  %m = fmul <16 x float> %s, %y
+  %s = fsub FASTMATH_FLAGS <16 x float> %x, <float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0, float -1.0>
+  %m = fmul FASTMATH_FLAGS <16 x float> %s, %y
   ret <16 x float> %m
 }
 
 define <8 x double> @test_v8f64_mul_y_sub_x_negone(<8 x double> %x, <8 x double> %y) {
 ; FMA-INFS-LABEL: test_v8f64_mul_y_sub_x_negone:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovapd {{.*#+}} ymm4 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vaddpd %ymm4, %ymm1, %ymm1

 ; FMA4-NOINFS-NEXT:    vfmaddpd {{.*#+}} ymm0 = (ymm0 * ymm2) + ymm2
 ; FMA4-NOINFS-NEXT:    vfmaddpd {{.*#+}} ymm1 = (ymm1 * ymm3) + ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v8f64_mul_y_sub_x_negone:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmadd213pd {{.*#+}} zmm0 = (zmm1 * zmm0) + zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %s = fsub <8 x double> %x, <double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0>
-  %m = fmul <8 x double> %y, %s
+  %s = fsub FASTMATH_FLAGS <8 x double> %x, <double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0, double -1.0>
+  %m = fmul FASTMATH_FLAGS <8 x double> %y, %s
   ret <8 x double> %m
 }
 
 ;
 ; Interpolation Patterns: add(mul(x,t),mul(sub(1.0,t),y))
 ;
 
 define <16 x float> @test_v16f32_interp(<16 x float> %x, <16 x float> %y, <16 x float> %t) {

 ; FMA4-NOINFS-NEXT:    vfmsubps {{.*#+}} ymm1 = (ymm1 * ymm5) - ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v16f32_interp:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} zmm1 = (zmm2 * zmm1) - zmm1
 ; AVX512-NOINFS-NEXT:    vfmsub213ps {{.*#+}} zmm0 = (zmm2 * zmm0) - zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %t1 = fsub nsz <16 x float> <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>, %t
-  %tx = fmul nsz <16 x float> %x, %t
-  %ty = fmul nsz <16 x float> %y, %t1
-  %r = fadd nsz <16 x float> %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS nsz <16 x float> <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>, %t
+  %tx = fmul FASTMATH_FLAGS nsz <16 x float> %x, %t
+  %ty = fmul FASTMATH_FLAGS nsz <16 x float> %y, %t1
+  %r = fadd FASTMATH_FLAGS nsz <16 x float> %tx, %ty
   ret <16 x float> %r
 }
 
 define <8 x double> @test_v8f64_interp(<8 x double> %x, <8 x double> %y, <8 x double> %t) {
 ; FMA-INFS-LABEL: test_v8f64_interp:
 ; FMA-INFS:       # %bb.0:
 ; FMA-INFS-NEXT:    vmovapd {{.*#+}} ymm6 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0]
 ; FMA-INFS-NEXT:    vsubpd %ymm4, %ymm6, %ymm7

 ; FMA4-NOINFS-NEXT:    vfmsubpd {{.*#+}} ymm1 = (ymm1 * ymm5) - ymm3
 ; FMA4-NOINFS-NEXT:    retq
 ;
 ; AVX512-NOINFS-LABEL: test_v8f64_interp:
 ; AVX512-NOINFS:       # %bb.0:
 ; AVX512-NOINFS-NEXT:    vfmsub213pd {{.*#+}} zmm1 = (zmm2 * zmm1) - zmm1
 ; AVX512-NOINFS-NEXT:    vfmsub213pd {{.*#+}} zmm0 = (zmm2 * zmm0) - zmm1
 ; AVX512-NOINFS-NEXT:    retq
-  %t1 = fsub nsz <8 x double> <double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0>, %t
-  %tx = fmul nsz <8 x double> %x, %t
-  %ty = fmul nsz <8 x double> %y, %t1
-  %r = fadd nsz <8 x double> %tx, %ty
+  %t1 = fsub FASTMATH_FLAGS nsz <8 x double> <double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0, double 1.0>, %t
+  %tx = fmul FASTMATH_FLAGS nsz <8 x double> %x, %t
+  %ty = fmul FASTMATH_FLAGS nsz <8 x double> %y, %t1
+  %r = fadd FASTMATH_FLAGS nsz <8 x double> %tx, %ty
   ret <8 x double> %r
 }
 
 ;
 ; Pattern: (fneg (fma x, y, z)) -> (fma x, -y, -z)
 ;
 
 define <16 x float> @test_v16f32_fneg_fmadd(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) #0 {
 ; FMA-LABEL: test_v16f32_fneg_fmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmsub213ps {{.*#+}} ymm0 = -(ymm2 * ymm0) - ymm4
 ; FMA-NEXT:    vfnmsub213ps {{.*#+}} ymm1 = -(ymm3 * ymm1) - ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v16f32_fneg_fmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmsubps {{.*#+}} ymm0 = -(ymm0 * ymm2) - ymm4
 ; FMA4-NEXT:    vfnmsubps {{.*#+}} ymm1 = -(ymm1 * ymm3) - ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v16f32_fneg_fmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmsub213ps {{.*#+}} zmm0 = -(zmm1 * zmm0) - zmm2
 ; AVX512-NEXT:    retq
-  %mul = fmul nsz <16 x float> %a0, %a1
-  %add = fadd nsz <16 x float> %mul, %a2
-  %neg = fsub nsz <16 x float> <float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0>, %add
+  %mul = fmul FASTMATH_FLAGS nsz <16 x float> %a0, %a1
+  %add = fadd FASTMATH_FLAGS nsz <16 x float> %mul, %a2
+  %neg = fsub FASTMATH_FLAGS nsz <16 x float> <float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0>, %add
   ret <16 x float> %neg
 }
 
 define <8 x double> @test_v8f64_fneg_fmsub(<8 x double> %a0, <8 x double> %a1, <8 x double> %a2) #0 {
 ; FMA-LABEL: test_v8f64_fneg_fmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfnmadd213pd {{.*#+}} ymm0 = -(ymm2 * ymm0) + ymm4
 ; FMA-NEXT:    vfnmadd213pd {{.*#+}} ymm1 = -(ymm3 * ymm1) + ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v8f64_fneg_fmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfnmaddpd {{.*#+}} ymm0 = -(ymm0 * ymm2) + ymm4
 ; FMA4-NEXT:    vfnmaddpd {{.*#+}} ymm1 = -(ymm1 * ymm3) + ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v8f64_fneg_fmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfnmadd213pd {{.*#+}} zmm0 = -(zmm1 * zmm0) + zmm2
 ; AVX512-NEXT:    retq
-  %mul = fmul nsz <8 x double> %a0, %a1
-  %sub = fsub nsz <8 x double> %mul, %a2
-  %neg = fsub nsz <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %sub
+  %mul = fmul FASTMATH_FLAGS nsz <8 x double> %a0, %a1
+  %sub = fsub FASTMATH_FLAGS nsz <8 x double> %mul, %a2
+  %neg = fsub FASTMATH_FLAGS nsz <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %sub
   ret <8 x double> %neg
 }
 
 define <16 x float> @test_v16f32_fneg_fnmadd(<16 x float> %a0, <16 x float> %a1, <16 x float> %a2) #0 {
 ; FMA-LABEL: test_v16f32_fneg_fnmadd:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmsub213ps {{.*#+}} ymm0 = (ymm2 * ymm0) - ymm4
 ; FMA-NEXT:    vfmsub213ps {{.*#+}} ymm1 = (ymm3 * ymm1) - ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v16f32_fneg_fnmadd:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmsubps {{.*#+}} ymm0 = (ymm0 * ymm2) - ymm4
 ; FMA4-NEXT:    vfmsubps {{.*#+}} ymm1 = (ymm1 * ymm3) - ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v16f32_fneg_fnmadd:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmsub213ps {{.*#+}} zmm0 = (zmm1 * zmm0) - zmm2
 ; AVX512-NEXT:    retq
-  %mul = fmul nsz <16 x float> %a0, %a1
-  %neg0 = fsub nsz <16 x float> <float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0>, %mul
-  %add = fadd nsz <16 x float> %neg0, %a2
-  %neg1 = fsub nsz <16 x float> <float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0>, %add
+  %mul = fmul FASTMATH_FLAGS nsz <16 x float> %a0, %a1
+  %neg0 = fsub FASTMATH_FLAGS nsz <16 x float> <float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0>, %mul
+  %add = fadd FASTMATH_FLAGS nsz <16 x float> %neg0, %a2
+  %neg1 = fsub FASTMATH_FLAGS nsz <16 x float> <float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0>, %add
   ret <16 x float> %neg1
 }
 
 define <8 x double> @test_v8f64_fneg_fnmsub(<8 x double> %a0, <8 x double> %a1, <8 x double> %a2) #0 {
 ; FMA-LABEL: test_v8f64_fneg_fnmsub:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd213pd {{.*#+}} ymm0 = (ymm2 * ymm0) + ymm4
 ; FMA-NEXT:    vfmadd213pd {{.*#+}} ymm1 = (ymm3 * ymm1) + ymm5
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v8f64_fneg_fnmsub:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddpd {{.*#+}} ymm0 = (ymm0 * ymm2) + ymm4
 ; FMA4-NEXT:    vfmaddpd {{.*#+}} ymm1 = (ymm1 * ymm3) + ymm5
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v8f64_fneg_fnmsub:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd213pd {{.*#+}} zmm0 = (zmm1 * zmm0) + zmm2
 ; AVX512-NEXT:    retq
-  %mul = fmul nsz <8 x double> %a0, %a1
-  %neg0 = fsub nsz <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %mul
-  %sub = fsub nsz <8 x double> %neg0, %a2
-  %neg1 = fsub nsz <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %sub
+  %mul = fmul FASTMATH_FLAGS nsz <8 x double> %a0, %a1
+  %neg0 = fsub FASTMATH_FLAGS nsz <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %mul
+  %sub = fsub FASTMATH_FLAGS nsz <8 x double> %neg0, %a2
+  %neg1 = fsub FASTMATH_FLAGS nsz <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %sub
   ret <8 x double> %neg1
 }
 
 ;
 ; Pattern: (fma x, c1, (fmul x, c2)) -> (fmul x, c1+c2)
 ;
 
 define <16 x float> @test_v16f32_fma_x_c1_fmul_x_c2(<16 x float> %x) #0 {
 ; FMA-LABEL: test_v16f32_fma_x_c1_fmul_x_c2:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vmulps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %ymm0, %ymm0
 ; FMA-NEXT:    vmulps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %ymm1, %ymm1
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v16f32_fma_x_c1_fmul_x_c2:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vmulps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %ymm0, %ymm0
 ; FMA4-NEXT:    vmulps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %ymm1, %ymm1
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v16f32_fma_x_c1_fmul_x_c2:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vmulps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm0, %zmm0
 ; AVX512-NEXT:    retq
-  %m0 = fmul <16 x float> %x, <float 17.0, float 16.0, float 15.0, float 14.0, float 13.0, float 12.0, float 11.0, float 10.0, float 9.0, float 8.0, float 7.0, float 6.0, float 5.0, float 4.0, float 3.0, float 2.0>
-  %m1 = fmul <16 x float> %x, <float 16.0, float 15.0, float 14.0, float 13.0, float 12.0, float 11.0, float 10.0, float 9.0, float 8.0, float 7.0, float 6.0, float 5.0, float 4.0, float 3.0, float 2.0, float 1.0>
-  %a  = fadd <16 x float> %m0, %m1
+  %m0 = fmul FASTMATH_FLAGS <16 x float> %x, <float 17.0, float 16.0, float 15.0, float 14.0, float 13.0, float 12.0, float 11.0, float 10.0, float 9.0, float 8.0, float 7.0, float 6.0, float 5.0, float 4.0, float 3.0, float 2.0>
+  %m1 = fmul FASTMATH_FLAGS <16 x float> %x, <float 16.0, float 15.0, float 14.0, float 13.0, float 12.0, float 11.0, float 10.0, float 9.0, float 8.0, float 7.0, float 6.0, float 5.0, float 4.0, float 3.0, float 2.0, float 1.0>
+  %a  = fadd FASTMATH_FLAGS <16 x float> %m0, %m1
   ret <16 x float> %a
 }
 
 ;
 ; Pattern: (fma (fmul x, c1), c2, y) -> (fma x, c1*c2, y)
 ;
 
 define <16 x float> @test_v16f32_fma_fmul_x_c1_c2_y(<16 x float> %x, <16 x float> %y) #0 {
 ; FMA-LABEL: test_v16f32_fma_fmul_x_c1_c2_y:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vfmadd132ps {{.*#+}} ymm0 = (ymm0 * mem) + ymm2
 ; FMA-NEXT:    vfmadd132ps {{.*#+}} ymm1 = (ymm1 * mem) + ymm3
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v16f32_fma_fmul_x_c1_c2_y:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vfmaddps {{.*#+}} ymm0 = (ymm0 * mem) + ymm2
 ; FMA4-NEXT:    vfmaddps {{.*#+}} ymm1 = (ymm1 * mem) + ymm3
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v16f32_fma_fmul_x_c1_c2_y:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vfmadd132ps {{.*#+}} zmm0 = (zmm0 * mem) + zmm1
 ; AVX512-NEXT:    retq
-  %m0 = fmul <16 x float> %x,  <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0, float 9.0, float 10.0, float 11.0, float 12.0, float 13.0, float 14.0, float 15.0, float 16.0>
-  %m1 = fmul <16 x float> %m0, <float 16.0, float 15.0, float 14.0, float 13.0, float 12.0, float 11.0, float 10.0, float 9.0, float 8.0, float 7.0, float 6.0, float 5.0, float 4.0, float 3.0, float 2.0, float 1.0>
-  %a  = fadd <16 x float> %m1, %y
+  %m0 = fmul FASTMATH_FLAGS <16 x float> %x,  <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0, float 9.0, float 10.0, float 11.0, float 12.0, float 13.0, float 14.0, float 15.0, float 16.0>
+  %m1 = fmul FASTMATH_FLAGS <16 x float> %m0, <float 16.0, float 15.0, float 14.0, float 13.0, float 12.0, float 11.0, float 10.0, float 9.0, float 8.0, float 7.0, float 6.0, float 5.0, float 4.0, float 3.0, float 2.0, float 1.0>
+  %a  = fadd FASTMATH_FLAGS <16 x float> %m1, %y
   ret <16 x float> %a
 }
 
 ; Pattern: (fneg (fmul x, y)) -> (fnmsub x, y, 0)
 
 define <16 x float> @test_v16f32_fneg_fmul(<16 x float> %x, <16 x float> %y) #0 {
 ; FMA-LABEL: test_v16f32_fneg_fmul:
 ; FMA:       # %bb.0:

 ; FMA4-NEXT:    vfnmsubps {{.*#+}} ymm1 = -(ymm1 * ymm3) - ymm4
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v16f32_fneg_fmul:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vxorps %xmm2, %xmm2, %xmm2
 ; AVX512-NEXT:    vfnmsub213ps {{.*#+}} zmm0 = -(zmm1 * zmm0) - zmm2
 ; AVX512-NEXT:    retq
-  %m = fmul nsz <16 x float> %x, %y
-  %n = fsub <16 x float> <float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0>, %m
+  %m = fmul FASTMATH_FLAGS nsz <16 x float> %x, %y
+  %n = fsub FASTMATH_FLAGS <16 x float> <float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0, float -0.0>, %m
   ret <16 x float> %n
 }
 
 define <8 x double> @test_v8f64_fneg_fmul(<8 x double> %x, <8 x double> %y) #0 {
 ; FMA-LABEL: test_v8f64_fneg_fmul:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vxorpd %xmm4, %xmm4, %xmm4
 ; FMA-NEXT:    vfnmsub213pd {{.*#+}} ymm0 = -(ymm2 * ymm0) - ymm4
 ; FMA-NEXT:    vfnmsub213pd {{.*#+}} ymm1 = -(ymm3 * ymm1) - ymm4
 ; FMA-NEXT:    retq
 ;
 ; FMA4-LABEL: test_v8f64_fneg_fmul:
 ; FMA4:       # %bb.0:
 ; FMA4-NEXT:    vxorpd %xmm4, %xmm4, %xmm4
 ; FMA4-NEXT:    vfnmsubpd {{.*#+}} ymm0 = -(ymm0 * ymm2) - ymm4
 ; FMA4-NEXT:    vfnmsubpd {{.*#+}} ymm1 = -(ymm1 * ymm3) - ymm4
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v8f64_fneg_fmul:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vxorpd %xmm2, %xmm2, %xmm2
 ; AVX512-NEXT:    vfnmsub213pd {{.*#+}} zmm0 = -(zmm1 * zmm0) - zmm2
 ; AVX512-NEXT:    retq
-  %m = fmul nsz <8 x double> %x, %y
-  %n = fsub <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %m
+  %m = fmul FASTMATH_FLAGS nsz <8 x double> %x, %y
+  %n = fsub FASTMATH_FLAGS <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %m
   ret <8 x double> %n
 }
 
 define <8 x double> @test_v8f64_fneg_fmul_no_nsz(<8 x double> %x, <8 x double> %y) #0 {
 ; FMA-LABEL: test_v8f64_fneg_fmul_no_nsz:
 ; FMA:       # %bb.0:
 ; FMA-NEXT:    vmulpd %ymm3, %ymm1, %ymm1
 ; FMA-NEXT:    vmulpd %ymm2, %ymm0, %ymm0

 ; FMA4-NEXT:    vxorpd %ymm2, %ymm1, %ymm1
 ; FMA4-NEXT:    retq
 ;
 ; AVX512-LABEL: test_v8f64_fneg_fmul_no_nsz:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vmulpd %zmm1, %zmm0, %zmm0
 ; AVX512-NEXT:    vxorpd {{\.?LCPI[0-9]+_[0-9]+}}(%rip){1to8}, %zmm0, %zmm0
 ; AVX512-NEXT:    retq
-  %m = fmul <8 x double> %x, %y
-  %n = fsub <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %m
+  %m = fmul FASTMATH_FLAGS <8 x double> %x, %y
+  %n = fsub FASTMATH_FLAGS <8 x double> <double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0, double -0.0>, %m
   ret <8 x double> %n
 }
 
 attributes #0 = { "unsafe-fp-math"="true" }

llvm/test/CodeGen/X86/sqrt-fastmath-mir.ll

 
 define float @sqrt_ieee_ninf(float %f) #0 {
   ; CHECK-LABEL: name: sqrt_ieee_ninf
   ; CHECK: bb.0 (%ir-block.0):
   ; CHECK:   liveins: $xmm0
   ; CHECK:   [[COPY:%[0-9]+]]:fr32 = COPY $xmm0
   ; CHECK:   [[DEF:%[0-9]+]]:fr32 = IMPLICIT_DEF
   ; CHECK:   [[VRSQRTSSr:%[0-9]+]]:fr32 = VRSQRTSSr killed [[DEF]], [[COPY]]
-  ; CHECK:   %3:fr32 = ninf afn nofpexcept VMULSSrr [[COPY]], [[VRSQRTSSr]], implicit $mxcsr
+  ; CHECK:   %3:fr32 = ninf contract afn nofpexcept VMULSSrr [[COPY]], [[VRSQRTSSr]], implicit $mxcsr
   ; CHECK:   [[VMOVSSrm_alt:%[0-9]+]]:fr32 = VMOVSSrm_alt $rip, 1, $noreg, %const.0, $noreg :: (load (s32) from constant-pool)
-  ; CHECK:   %5:fr32 = ninf afn nofpexcept VFMADD213SSr [[VRSQRTSSr]], killed %3, [[VMOVSSrm_alt]], implicit $mxcsr
+  ; CHECK:   %5:fr32 = ninf contract afn nofpexcept VFMADD213SSr [[VRSQRTSSr]], killed %3, [[VMOVSSrm_alt]], implicit $mxcsr
   ; CHECK:   [[VMOVSSrm_alt1:%[0-9]+]]:fr32 = VMOVSSrm_alt $rip, 1, $noreg, %const.1, $noreg :: (load (s32) from constant-pool)
-  ; CHECK:   %7:fr32 = ninf afn nofpexcept VMULSSrr [[VRSQRTSSr]], [[VMOVSSrm_alt1]], implicit $mxcsr
-  ; CHECK:   %8:fr32 = ninf afn nofpexcept VMULSSrr killed %7, killed %5, implicit $mxcsr
-  ; CHECK:   %9:fr32 = ninf afn nofpexcept VMULSSrr [[COPY]], %8, implicit $mxcsr
-  ; CHECK:   %10:fr32 = ninf afn nofpexcept VFMADD213SSr %8, %9, [[VMOVSSrm_alt]], implicit $mxcsr
-  ; CHECK:   %11:fr32 = ninf afn nofpexcept VMULSSrr %9, [[VMOVSSrm_alt1]], implicit $mxcsr
-  ; CHECK:   %12:fr32 = ninf afn nofpexcept VMULSSrr killed %11, killed %10, implicit $mxcsr
+  ; CHECK:   %7:fr32 = ninf contract afn nofpexcept VMULSSrr [[VRSQRTSSr]], [[VMOVSSrm_alt1]], implicit $mxcsr
+  ; CHECK:   %8:fr32 = ninf contract afn nofpexcept VMULSSrr killed %7, killed %5, implicit $mxcsr
+  ; CHECK:   %9:fr32 = ninf contract afn nofpexcept VMULSSrr [[COPY]], %8, implicit $mxcsr
+  ; CHECK:   %10:fr32 = ninf contract afn nofpexcept VFMADD213SSr %8, %9, [[VMOVSSrm_alt]], implicit $mxcsr
+  ; CHECK:   %11:fr32 = ninf contract afn nofpexcept VMULSSrr %9, [[VMOVSSrm_alt1]], implicit $mxcsr
+  ; CHECK:   %12:fr32 = ninf contract afn nofpexcept VMULSSrr killed %11, killed %10, implicit $mxcsr
   ; CHECK:   [[COPY1:%[0-9]+]]:vr128 = COPY %12
   ; CHECK:   [[COPY2:%[0-9]+]]:vr128 = COPY [[COPY]]
   ; CHECK:   [[VPBROADCASTDrm:%[0-9]+]]:vr128 = VPBROADCASTDrm $rip, 1, $noreg, %const.2, $noreg :: (load (s32) from constant-pool)
   ; CHECK:   [[VPANDrr:%[0-9]+]]:vr128 = VPANDrr killed [[COPY2]], killed [[VPBROADCASTDrm]]
   ; CHECK:   [[COPY3:%[0-9]+]]:fr32 = COPY [[VPANDrr]]
   ; CHECK:   %18:fr32 = nofpexcept VCMPSSrm killed [[COPY3]], $rip, 1, $noreg, %const.3, $noreg, 1, implicit $mxcsr :: (load (s32) from constant-pool)
   ; CHECK:   [[COPY4:%[0-9]+]]:vr128 = COPY %18
   ; CHECK:   [[VPANDNrr:%[0-9]+]]:vr128 = VPANDNrr killed [[COPY4]], killed [[COPY1]]
   ; CHECK:   [[COPY5:%[0-9]+]]:fr32 = COPY [[VPANDNrr]]
   ; CHECK:   $xmm0 = COPY [[COPY5]]
   ; CHECK:   RET 0, $xmm0
-  %call = tail call ninf afn float @llvm.sqrt.f32(float %f)
+  %call = tail call ninf contract afn float @llvm.sqrt.f32(float %f)
   ret float %call
 }
 
 define float @sqrt_daz(float %f) #1 {
   ; CHECK-LABEL: name: sqrt_daz
   ; CHECK: bb.0 (%ir-block.0):
   ; CHECK:   liveins: $xmm0
   ; CHECK:   [[COPY:%[0-9]+]]:fr32 = COPY $xmm0
   ; CHECK:   [[DEF:%[0-9]+]]:fr32 = IMPLICIT_DEF
   ; CHECK:   %1:fr32 = nofpexcept VSQRTSSr killed [[DEF]], [[COPY]], implicit $mxcsr
   ; CHECK:   $xmm0 = COPY %1
   ; CHECK:   RET 0, $xmm0
   %call = tail call float @llvm.sqrt.f32(float %f)
   ret float %call
 }
 
 define float @sqrt_daz_ninf(float %f) #1 {
   ; CHECK-LABEL: name: sqrt_daz_ninf
   ; CHECK: bb.0 (%ir-block.0):
   ; CHECK:   liveins: $xmm0
   ; CHECK:   [[COPY:%[0-9]+]]:fr32 = COPY $xmm0
   ; CHECK:   [[DEF:%[0-9]+]]:fr32 = IMPLICIT_DEF
   ; CHECK:   [[VRSQRTSSr:%[0-9]+]]:fr32 = VRSQRTSSr killed [[DEF]], [[COPY]]
-  ; CHECK:   %3:fr32 = ninf afn nofpexcept VMULSSrr [[COPY]], [[VRSQRTSSr]], implicit $mxcsr
+  ; CHECK:   %3:fr32 = ninf contract afn nofpexcept VMULSSrr [[COPY]], [[VRSQRTSSr]], implicit $mxcsr
   ; CHECK:   [[VMOVSSrm_alt:%[0-9]+]]:fr32 = VMOVSSrm_alt $rip, 1, $noreg, %const.0, $noreg :: (load (s32) from constant-pool)
-  ; CHECK:   %5:fr32 = ninf afn nofpexcept VFMADD213SSr [[VRSQRTSSr]], killed %3, [[VMOVSSrm_alt]], implicit $mxcsr
+  ; CHECK:   %5:fr32 = ninf contract afn nofpexcept VFMADD213SSr [[VRSQRTSSr]], killed %3, [[VMOVSSrm_alt]], implicit $mxcsr
   ; CHECK:   [[VMOVSSrm_alt1:%[0-9]+]]:fr32 = VMOVSSrm_alt $rip, 1, $noreg, %const.1, $noreg :: (load (s32) from constant-pool)
-  ; CHECK:   %7:fr32 = ninf afn nofpexcept VMULSSrr [[VRSQRTSSr]], [[VMOVSSrm_alt1]], implicit $mxcsr
-  ; CHECK:   %8:fr32 = ninf afn nofpexcept VMULSSrr killed %7, killed %5, implicit $mxcsr
-  ; CHECK:   %9:fr32 = ninf afn nofpexcept VMULSSrr [[COPY]], %8, implicit $mxcsr
-  ; CHECK:   %10:fr32 = ninf afn nofpexcept VFMADD213SSr %8, %9, [[VMOVSSrm_alt]], implicit $mxcsr
-  ; CHECK:   %11:fr32 = ninf afn nofpexcept VMULSSrr %9, [[VMOVSSrm_alt1]], implicit $mxcsr
-  ; CHECK:   %12:fr32 = ninf afn nofpexcept VMULSSrr killed %11, killed %10, implicit $mxcsr
+  ; CHECK:   %7:fr32 = ninf contract afn nofpexcept VMULSSrr [[VRSQRTSSr]], [[VMOVSSrm_alt1]], implicit $mxcsr
+  ; CHECK:   %8:fr32 = ninf contract afn nofpexcept VMULSSrr killed %7, killed %5, implicit $mxcsr
+  ; CHECK:   %9:fr32 = ninf contract afn nofpexcept VMULSSrr [[COPY]], %8, implicit $mxcsr
+  ; CHECK:   %10:fr32 = ninf contract afn nofpexcept VFMADD213SSr %8, %9, [[VMOVSSrm_alt]], implicit $mxcsr
+  ; CHECK:   %11:fr32 = ninf contract afn nofpexcept VMULSSrr %9, [[VMOVSSrm_alt1]], implicit $mxcsr
+  ; CHECK:   %12:fr32 = ninf contract afn nofpexcept VMULSSrr killed %11, killed %10, implicit $mxcsr
   ; CHECK:   [[COPY1:%[0-9]+]]:vr128 = COPY %12
   ; CHECK:   [[FsFLD0SS:%[0-9]+]]:fr32 = FsFLD0SS
   ; CHECK:   %15:fr32 = nofpexcept VCMPSSrr [[COPY]], killed [[FsFLD0SS]], 0, implicit $mxcsr
   ; CHECK:   [[COPY2:%[0-9]+]]:vr128 = COPY %15
   ; CHECK:   [[VPANDNrr:%[0-9]+]]:vr128 = VPANDNrr killed [[COPY2]], killed [[COPY1]]
   ; CHECK:   [[COPY3:%[0-9]+]]:fr32 = COPY [[VPANDNrr]]
   ; CHECK:   $xmm0 = COPY [[COPY3]]
   ; CHECK:   RET 0, $xmm0
-  %call = tail call ninf afn float @llvm.sqrt.f32(float %f)
+  %call = tail call ninf contract afn float @llvm.sqrt.f32(float %f)
   ret float %call
 }
 
 define float @rsqrt_ieee(float %f) #0 {
   ; CHECK-LABEL: name: rsqrt_ieee
   ; CHECK: bb.0 (%ir-block.0):
   ; CHECK:   liveins: $xmm0
   ; CHECK:   [[COPY:%[0-9]+]]:fr32 = COPY $xmm0

llvm/test/CodeGen/X86/sqrt-fastmath.ll

 ; AVX512-NEXT:    vmulss %xmm1, %xmm2, %xmm1
 ; AVX512-NEXT:    vbroadcastss {{.*#+}} xmm2 = [NaN,NaN,NaN,NaN]
 ; AVX512-NEXT:    vandps %xmm2, %xmm0, %xmm0
 ; AVX512-NEXT:    vcmpltss {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %k1
 ; AVX512-NEXT:    vxorps %xmm0, %xmm0, %xmm0
 ; AVX512-NEXT:    vmovss %xmm0, %xmm1, %xmm1 {%k1}
 ; AVX512-NEXT:    vmovaps %xmm1, %xmm0
 ; AVX512-NEXT:    retq
-  %call = tail call ninf afn float @__sqrtf_finite(float %f) #2
+  %call = tail call contract ninf afn float @__sqrtf_finite(float %f) #2
   ret float %call
 }
 
 define float @finite_f32_estimate_daz(float %f) #4 {
 ; SSE-LABEL: finite_f32_estimate_daz:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    sqrtss %xmm0, %xmm0
 ; SSE-NEXT:    retq

 ; AVX512-NEXT:    vfmadd213ss {{.*#+}} xmm1 = (xmm2 * xmm1) + mem
 ; AVX512-NEXT:    vmulss {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm2, %xmm2
 ; AVX512-NEXT:    vmulss %xmm1, %xmm2, %xmm1
 ; AVX512-NEXT:    vxorps %xmm2, %xmm2, %xmm2
 ; AVX512-NEXT:    vcmpeqss %xmm2, %xmm0, %k1
 ; AVX512-NEXT:    vmovss %xmm2, %xmm1, %xmm1 {%k1}
 ; AVX512-NEXT:    vmovaps %xmm1, %xmm0
 ; AVX512-NEXT:    retq
-  %call = tail call ninf afn float @__sqrtf_finite(float %f) #2
+  %call = tail call contract ninf afn float @__sqrtf_finite(float %f) #2
   ret float %call
 }
 
 define x86_fp80 @finite_f80_no_estimate(x86_fp80 %ld) #0 {
 ; CHECK-LABEL: finite_f80_no_estimate:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fldt {{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    fsqrt

 ; AVX512-NEXT:    vmulss %xmm1, %xmm2, %xmm1
 ; AVX512-NEXT:    vbroadcastss {{.*#+}} xmm2 = [NaN,NaN,NaN,NaN]
 ; AVX512-NEXT:    vandps %xmm2, %xmm0, %xmm0
 ; AVX512-NEXT:    vcmpltss {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %k1
 ; AVX512-NEXT:    vxorps %xmm0, %xmm0, %xmm0
 ; AVX512-NEXT:    vmovss %xmm0, %xmm1, %xmm1 {%k1}
 ; AVX512-NEXT:    vmovaps %xmm1, %xmm0
 ; AVX512-NEXT:    retq
-  %call = tail call ninf afn float @__sqrtf_finite(float %x) #2
+  %call = tail call contract ninf afn float @__sqrtf_finite(float %x) #2
   ret float %call
 }
 
 define <4 x float> @sqrt_v4f32_check_denorms(<4 x float> %x) #3 {
 ; SSE-LABEL: sqrt_v4f32_check_denorms:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    sqrtps %xmm0, %xmm0
 ; SSE-NEXT:    retq

 ; AVX512-NEXT:    vmulps %xmm1, %xmm2, %xmm1
 ; AVX512-NEXT:    vmulps %xmm3, %xmm1, %xmm1
 ; AVX512-NEXT:    vbroadcastss {{.*#+}} xmm2 = [NaN,NaN,NaN,NaN]
 ; AVX512-NEXT:    vandps %xmm2, %xmm0, %xmm0
 ; AVX512-NEXT:    vbroadcastss {{.*#+}} xmm2 = [1.17549435E-38,1.17549435E-38,1.17549435E-38,1.17549435E-38]
 ; AVX512-NEXT:    vcmpleps %xmm0, %xmm2, %xmm0
 ; AVX512-NEXT:    vandps %xmm1, %xmm0, %xmm0
 ; AVX512-NEXT:    retq
-  %call = tail call ninf afn <4 x float> @llvm.sqrt.v4f32(<4 x float> %x) #2
+  %call = tail call contract ninf afn <4 x float> @llvm.sqrt.v4f32(<4 x float> %x) #2
   ret <4 x float> %call
 }
 
 define float @f32_no_estimate(float %x) #0 {
 ; SSE-LABEL: f32_no_estimate:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    sqrtss %xmm0, %xmm1
 ; SSE-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero