[RISCV] Set dependency on floating point CSRs, 1/3

[sepavloff]

Summary

There are dependencies between floating point instructions that were
missed from the target description. They involve special registers that
keep exception flags and rounding mode. Most FP instructions can set
accrued exception flags, so they are implicit definitions for 'fflags'.
Instructions that use dynamic rounding mode depend on the content of
'frm', they represent implicit uses of this register. These dependencies
impose restrictions on the ordering of FP instructions and must be
provided to the compiler.

In general there can be 4 variants of an instruction depending on
whether it depends on frm and whether the changes of fflags should
be ignored. The two most important of them are:

• defines 'fflags', depends on 'frm';
• does not depend on frm, changes of fflags should be ignored.

The first one is the general case, so it must be supported in any case.
The second corresponds to default FP environment, which is the most
widespread case. Other two might be beneficial in some cases, but are
expected to be rarer.

This change defines these two variants for relevant FP instructions. It
is split into several parts to facilitate review. This part implements
the dependency for instructions with three input registers.

[craig.topper]

I still don't understand why the existence of static rounding modes in the ISA requires that we have to use them for the default environment. X86 doesn't have static rounding mode prior to AVX512 so uses dynamic in the default mode.

Do any other targets create 3 variants of instructions like this? X86 definitely doesn't. What makes RISC-V special that it needs to go this extreme?

[sepavloff]

In D94163#2482528, @craig.topper wrote:

I still don't understand why the existence of static rounding modes in the ISA requires that we have to use them for the default environment. X86 doesn't have static rounding mode prior to AVX512 so uses dynamic in the default mode.

It is more convenient. Instructions with static rounding mode do not depend on frm so they may be scheduled more freely. Besides function with static only FP instructions may be safely called from non-default FP environment. Targets without static rounding mode don't have such possibility.

Do any other targets create 3 variants of instructions like this? X86 definitely doesn't. What makes RISC-V special that it needs to go this extreme?

It must be a target that supports both static and dynamic rounding mode. Are there other targets in officially llvm repository that support both of them?

[craig.topper]

In D94163#2489020, @sepavloff wrote:

In D94163#2482528, @craig.topper wrote:

I still don't understand why the existence of static rounding modes in the ISA requires that we have to use them for the default environment. X86 doesn't have static rounding mode prior to AVX512 so uses dynamic in the default mode.

It is more convenient. Instructions with static rounding mode do not depend on frm so they may be scheduled more freely. Besides function with static only FP instructions may be safely called from non-default FP environment. Targets without static rounding mode don't have such possibility.

If there’s no write to frm then there shouldn’t be a scheduling issue. Can you demonstrate such an issue on a target without static rounding mode?

For the exception side I thought we already solved the scheduling issue with the mayRaiseFPException bit that is treated differently than using a register def of the fflags register.

Do any other targets create 3 variants of instructions like this? X86 definitely doesn't. What makes RISC-V special that it needs to go this extreme?

It must be a target that supports both static and dynamic rounding mode. Are there other targets in officially llvm repository that support both of them?

[sepavloff]

In D94163#2489101, @craig.topper wrote:

In D94163#2489020, @sepavloff wrote:

In D94163#2482528, @craig.topper wrote:

I still don't understand why the existence of static rounding modes in the ISA requires that we have to use them for the default environment. X86 doesn't have static rounding mode prior to AVX512 so uses dynamic in the default mode.

It is more convenient. Instructions with static rounding mode do not depend on frm so they may be scheduled more freely. Besides function with static only FP instructions may be safely called from non-default FP environment. Targets without static rounding mode don't have such possibility.

If there’s no write to frm then there shouldn’t be a scheduling issue.

Sure. Such issue rises when there is write to frm. Consider the following pseudo code:

float a = ...
for (int i = ...) {
  fesetround(FE_TOWARDZERO); // csrw frm, 1
  ...
  x[i] += floor(a); // fcvt ..., rdn

floor(a) is a loop invariant and could be hoisted off the loop. It is possible as fcvt uses static rounding. However if fcvt uses dynamic rounding, it depends on frm, which is changed above, so it cannot be moved out of the loop.

[craig.topper]

In D94163#2535646, @sepavloff wrote:

In D94163#2489101, @craig.topper wrote:

In D94163#2489020, @sepavloff wrote:

In D94163#2482528, @craig.topper wrote:

I still don't understand why the existence of static rounding modes in the ISA requires that we have to use them for the default environment. X86 doesn't have static rounding mode prior to AVX512 so uses dynamic in the default mode.

It is more convenient. Instructions with static rounding mode do not depend on frm so they may be scheduled more freely. Besides function with static only FP instructions may be safely called from non-default FP environment. Targets without static rounding mode don't have such possibility.

If there’s no write to frm then there shouldn’t be a scheduling issue.

Sure. Such issue rises when there is write to frm. Consider the following pseudo code:

float a = ...
for (int i = ...) {
  fesetround(FE_TOWARDZERO); // csrw frm, 1
  ...
  x[i] += floor(a); // fcvt ..., rdn

floor(a) is a loop invariant and could be hoisted off the loop. It is possible as fcvt uses static rounding. However if fcvt uses dynamic rounding, it depends on frm, which is changed above, so it cannot be moved out of the loop.

Why wouldn't that have been hoisted out of the loop by IR LICM? Machine LICM is primarily intended to move stack reloads and constant pool loads. It only runs on the outermost loop with a preheader.

[sepavloff]

In D94163#2535653, @craig.topper wrote:

In D94163#2535646, @sepavloff wrote:

In D94163#2489101, @craig.topper wrote:

In D94163#2489020, @sepavloff wrote:

In D94163#2482528, @craig.topper wrote:

I still don't understand why the existence of static rounding modes in the ISA requires that we have to use them for the default environment. X86 doesn't have static rounding mode prior to AVX512 so uses dynamic in the default mode.

It is more convenient. Instructions with static rounding mode do not depend on frm so they may be scheduled more freely. Besides function with static only FP instructions may be safely called from non-default FP environment. Targets without static rounding mode don't have such possibility.

If there’s no write to frm then there shouldn’t be a scheduling issue.

Sure. Such issue rises when there is write to frm. Consider the following pseudo code:

float a = ...
for (int i = ...) {
  fesetround(FE_TOWARDZERO); // csrw frm, 1
  ...
  x[i] += floor(a); // fcvt ..., rdn

floor(a) is a loop invariant and could be hoisted off the loop. It is possible as fcvt uses static rounding. However if fcvt uses dynamic rounding, it depends on frm, which is changed above, so it cannot be moved out of the loop.

Why wouldn't that have been hoisted out of the loop by IR LICM? Machine LICM is primarily intended to move stack reloads and constant pool loads. It only runs on the outermost loop with a preheader.

This is another example:

%1:fpr32 = …
%2:fpr32 = …
...
csrw frm, rdn
…
%3:fpr32 = FADD_S killed %1:fpr32, killed %2:fpr32, rne

In this code scheduler can move the instruction FADD_S upward, live ranges of %1 and %2 becomes shorter and register pressure decreases. If FADD instruction implicitly depends on frm, the scheduler cannot move FADD_S above csrw, so such optimization is not possible.

[sepavloff]

Reduced number of instruction variants from 3 to 2 (generic and default)

[asb]

We discussed this briefly in the RISC-V call as I noted this patchset has been sat open for some time. One thing that might be helpful is whether you could say a little bit more about the goal for this patchset. Once this lands, what's the next step? Is there some relevant RFC, or equivalent changes being made to other in-tree architectures?

We're also still unclear about the advantage of changing codegen to default to a static rounding mode (which might be a surprising change, as all software compiled to date on both GCC and LLVM has used used the dynamic rounding mode by default).

[sepavloff]

In D94163#2603738, @asb wrote:

We discussed this briefly in the RISC-V call as I noted this patchset has been sat open for some time. One thing that might be helpful is whether you could say a little bit more about the goal for this patchset.

Most floating point instructions set accrued exception bits in fflags register. If an instruction is specified with dynamic rounding mode, it also depends on the content of frm register. So in the following code:

csrwi  frm, a1
fadd.d ft2, ft2, ft3

changing the order of instruction is not allowed, because fadd.d depends on the content of frm, which is changed by the previous instruction. Similarly, the code:

fadd.d ft2, ft2, ft3
csrrs t0, fflags, zero

does not allow to change the order of the instructions, as crsrs reads content of fflags, which is set by the first instruction.

Now nothing prevents the compiler from changing the order of instructions in these examples. Existing instruction definitions are unable to express these dependencies. It does not allow to write programs that use non-default floating point environment, for example, dynamic rounding mode.

The goal of this patchset is to establish means to express such dependencies.

Once this lands, what's the next step?

This is the first and necessary step toward full-fledged implementation of floating point support. In particular it would allow progress in D91242 and D90854. Actually this work was undertaken because the lack of dependencies prevents implementation of llvm.set.rounding (https://reviews.llvm.org/D91242#2400476). The next steps, of course, include support of constrained intrinsics for RSCV.

Is there some relevant RFC,

If RFC can facilitate review of this patchset, I will prepare it.

or equivalent changes being made to other in-tree architectures?

Targets that support full-fledged floating point operations add implicit uses and definitions of FP state and control register(s). For example:
• X86: https://github.com/llvm/llvm-project/blob/bc172e532a89754d47fef1306064a26a4dc0a76b/llvm/lib/Target/X86/X86InstrFPStack.td#L728 (see let Defs = [FPSW] and let Uses = [FPCW]),
• PowerPC: https://github.com/llvm/llvm-project/blob/e7361c8eccb7663146096622549dc03240414157/llvm/lib/Target/PowerPC/PPCInstrInfo.td#L3169 (see Uses = [RM]),
• SystemZ: https://github.com/llvm/llvm-project/blob/9e28b89827a3be4ab602b40c263839665af06b4a/llvm/lib/Target/SystemZ/SystemZInstrFP.td#L434 (see let Uses = [FPC])

We're also still unclear about the advantage of changing codegen to default to a static rounding mode (which might be a surprising change, as all software compiled to date on both GCC and LLVM has used used the dynamic rounding mode by default).

Instructions in assembler without explicit rounding mode specification get dynamic rounding mode as now. Lowering of FP operations like fadd uses static rounding mode RNE, because these operations assume default floating point environment (https://llvm.org/docs/LangRef.html#floating-point-environment). Using static rounding mode has some advantages over assuming frm to have particular value. The code that requires default rounding mode does not require setting rfm in a program where some pieces uses non-default rounding mode. Such code works as designed even if it is called from a region where other rounding mode is set. Such implementation simplifies implementation of things like #pragma STDC FENV_ROUND and make programs more robust.

[jrtc27]

In D94163#2605594, @sepavloff wrote:

In D94163#2603738, @asb wrote:

We discussed this briefly in the RISC-V call as I noted this patchset has been sat open for some time. One thing that might be helpful is whether you could say a little bit more about the goal for this patchset.

Most floating point instructions set accrued exception bits in fflags register. If an instruction is specified with dynamic rounding mode, it also depends on the content of frm register. So in the following code:

csrwi  frm, a1
fadd.d ft2, ft2, ft3

changing the order of instruction is not allowed, because fadd.d depends on the content of frm, which is changed by the previous instruction. Similarly, the code:

fadd.d ft2, ft2, ft3
csrrs t0, fflags, zero

does not allow to change the order of the instructions, as crsrs reads content of fflags, which is set by the first instruction.

Now nothing prevents the compiler from changing the order of instructions in these examples. Existing instruction definitions are unable to express these dependencies. It does not allow to write programs that use non-default floating point environment, for example, dynamic rounding mode.

The goal of this patchset is to establish means to express such dependencies.

Once this lands, what's the next step?

This is the first and necessary step toward full-fledged implementation of floating point support. In particular it would allow progress in D91242 and D90854. Actually this work was undertaken because the lack of dependencies prevents implementation of llvm.set.rounding (https://reviews.llvm.org/D91242#2400476). The next steps, of course, include support of constrained intrinsics for RSCV.

Is there some relevant RFC,

If RFC can facilitate review of this patchset, I will prepare it.

or equivalent changes being made to other in-tree architectures?

Targets that support full-fledged floating point operations add implicit uses and definitions of FP state and control register(s). For example:
• X86: https://github.com/llvm/llvm-project/blob/bc172e532a89754d47fef1306064a26a4dc0a76b/llvm/lib/Target/X86/X86InstrFPStack.td#L728 (see let Defs = [FPSW] and let Uses = [FPCW]),
• PowerPC: https://github.com/llvm/llvm-project/blob/e7361c8eccb7663146096622549dc03240414157/llvm/lib/Target/PowerPC/PPCInstrInfo.td#L3169 (see Uses = [RM]),
• SystemZ: https://github.com/llvm/llvm-project/blob/9e28b89827a3be4ab602b40c263839665af06b4a/llvm/lib/Target/SystemZ/SystemZInstrFP.td#L434 (see let Uses = [FPC])

We're also still unclear about the advantage of changing codegen to default to a static rounding mode (which might be a surprising change, as all software compiled to date on both GCC and LLVM has used used the dynamic rounding mode by default).

Instructions in assembler without explicit rounding mode specification get dynamic rounding mode as now. Lowering of FP operations like fadd uses static rounding mode RNE, because these operations assume default floating point environment (https://llvm.org/docs/LangRef.html#floating-point-environment). Using static rounding mode has some advantages over assuming frm to have particular value. The code that requires default rounding mode does not require setting rfm in a program where some pieces uses non-default rounding mode. Such code works as designed even if it is called from a region where other rounding mode is set. Such implementation simplifies implementation of things like #pragma STDC FENV_ROUND and make programs more robust.

That's going to break huge piles of C/C++ code that sets the (dynamic) rounding mode and expects it to have an effect on subsequent computations. I do not think that is a good idea.

[sepavloff]

In D94163#2606346, @jrtc27 wrote:

In D94163#2605594, @sepavloff wrote:

In D94163#2603738, @asb wrote:

We're also still unclear about the advantage of changing codegen to default to a static rounding mode (which might be a surprising change, as all software compiled to date on both GCC and LLVM has used used the dynamic rounding mode by default).

Instructions in assembler without explicit rounding mode specification get dynamic rounding mode as now. Lowering of FP operations like fadd uses static rounding mode RNE, because these operations assume default floating point environment (https://llvm.org/docs/LangRef.html#floating-point-environment). Using static rounding mode has some advantages over assuming frm to have particular value. The code that requires default rounding mode does not require setting rfm in a program where some pieces uses non-default rounding mode. Such code works as designed even if it is called from a region where other rounding mode is set. Such implementation simplifies implementation of things like #pragma STDC FENV_ROUND and make programs more robust.

That's going to break huge piles of C/C++ code that sets the (dynamic) rounding mode and expects it to have an effect on subsequent computations. I do not think that is a good idea.

You are right, it might be dangerous. While RISC-V does not support constrained intrinsics, it would be safer to use instructions with dynamic rounding mode.

[sepavloff]

Updated patch for alternative CSR solution

llvm/lib/Target/RISCV/RISCVInstrInfoD.td

 
 def RISCVBuildPairF64 : SDNode<"RISCVISD::BuildPairF64", SDT_RISCVBuildPairF64>;
 def RISCVSplitF64     : SDNode<"RISCVISD::SplitF64", SDT_RISCVSplitF64>;
 
 //===----------------------------------------------------------------------===//
 // Instruction Class Templates
 //===----------------------------------------------------------------------===//
 
-let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
-class FPFMAD_rrr_frm<RISCVOpcode opcode, string opcodestr>
-    : RVInstR4<0b01, opcode, (outs FPR64:$rd),
-               (ins FPR64:$rs1, FPR64:$rs2, FPR64:$rs3, frmarg:$funct3),
-                opcodestr, "$rd, $rs1, $rs2, $rs3, $funct3">;
-
-class FPFMADDynFrmAlias<FPFMAD_rrr_frm Inst, string OpcodeStr>
-    : InstAlias<OpcodeStr#" $rd, $rs1, $rs2, $rs3",
-                (Inst FPR64:$rd, FPR64:$rs1, FPR64:$rs2, FPR64:$rs3, 0b111)>;
+multiclass FPFMAD_mc<RISCVOpcode opcode, string OpcodeStr> {
+  def _gen : FPFMA_gen<opcode, OpcodeStr, 0b01, FPR64>;
+  def _def : FPFMA_def<opcode, OpcodeStr, 0b01, FPR64>;
+  def      : InstAlias<OpcodeStr#" $rd, $rs1, $rs2, $rs3",
+                       (!cast<Instruction>(NAME # "_gen")
+                        FPR64:$rd, FPR64:$rs1, FPR64:$rs2, FPR64:$rs3, 0b111)>;
+}
 
 let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
 class FPALUD_rr<bits<7> funct7, bits<3> funct3, string opcodestr>
     : RVInstR<funct7, funct3, OPC_OP_FP, (outs FPR64:$rd),
               (ins FPR64:$rs1, FPR64:$rs2), opcodestr, "$rd, $rs1, $rs2">;
 
 let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
 class FPALUD_rr_frm<bits<7> funct7, string opcodestr>

 // reflecting the order these fields are specified in the instruction
 // encoding.
 let hasSideEffects = 0, mayLoad = 0, mayStore = 1 in
 def FSD : RVInstS<0b011, OPC_STORE_FP, (outs),
                   (ins FPR64:$rs2, GPR:$rs1, simm12:$imm12),
                    "fsd", "$rs2, ${imm12}(${rs1})">,
           Sched<[WriteFST64, ReadStoreData, ReadFMemBase]>;
 
-def FMADD_D  : FPFMAD_rrr_frm<OPC_MADD, "fmadd.d">,
+defm FMADD_D  : FPFMAD_mc<OPC_MADD, "fmadd.d">,
                 Sched<[WriteFMulAdd64, ReadFMulAdd64, ReadFMulAdd64, ReadFMulAdd64]>;
-def          : FPFMADDynFrmAlias<FMADD_D, "fmadd.d">;
-def FMSUB_D  : FPFMAD_rrr_frm<OPC_MSUB, "fmsub.d">,
+defm FMSUB_D  : FPFMAD_mc<OPC_MSUB, "fmsub.d">,
                 Sched<[WriteFMulSub64, ReadFMulSub64, ReadFMulSub64, ReadFMulSub64]>;
-def          : FPFMADDynFrmAlias<FMSUB_D, "fmsub.d">;
-def FNMSUB_D : FPFMAD_rrr_frm<OPC_NMSUB, "fnmsub.d">,
+defm FNMSUB_D : FPFMAD_mc<OPC_NMSUB, "fnmsub.d">,
                 Sched<[WriteFMulSub64, ReadFMulSub64, ReadFMulSub64, ReadFMulSub64]>;
-def          : FPFMADDynFrmAlias<FNMSUB_D, "fnmsub.d">;
-def FNMADD_D : FPFMAD_rrr_frm<OPC_NMADD, "fnmadd.d">,
+defm FNMADD_D : FPFMAD_mc<OPC_NMADD, "fnmadd.d">,
                 Sched<[WriteFMulAdd64, ReadFMulAdd64, ReadFMulAdd64, ReadFMulAdd64]>;
-def          : FPFMADDynFrmAlias<FNMADD_D, "fnmadd.d">;
 
 def FADD_D : FPALUD_rr_frm<0b0000001, "fadd.d">,
              Sched<[WriteFALU64, ReadFALU64, ReadFALU64]>;
 def        : FPALUDDynFrmAlias<FADD_D, "fadd.d">;
 def FSUB_D : FPALUD_rr_frm<0b0000101, "fsub.d">,
              Sched<[WriteFALU64, ReadFALU64, ReadFALU64]>;
 def        : FPALUDDynFrmAlias<FSUB_D, "fsub.d">;
 def FMUL_D : FPALUD_rr_frm<0b0001001, "fmul.d">,

 def : PatFpr64Fpr64<fcopysign, FSGNJ_D>;
 def : Pat<(fcopysign FPR64:$rs1, (fneg FPR64:$rs2)), (FSGNJN_D $rs1, $rs2)>;
 def : Pat<(fcopysign FPR64:$rs1, FPR32:$rs2), (FSGNJ_D $rs1, (FCVT_D_S $rs2))>;
 def : Pat<(fcopysign FPR32:$rs1, FPR64:$rs2), (FSGNJ_S $rs1, (FCVT_S_D $rs2,
                                                               0b111))>;
 
 // fmadd: rs1 * rs2 + rs3
 def : Pat<(fma FPR64:$rs1, FPR64:$rs2, FPR64:$rs3),
-          (FMADD_D $rs1, $rs2, $rs3, 0b111)>;
+          (FMADD_D_def $rs1, $rs2, $rs3)>;
 
 // fmsub: rs1 * rs2 - rs3
 def : Pat<(fma FPR64:$rs1, FPR64:$rs2, (fneg FPR64:$rs3)),
-          (FMSUB_D FPR64:$rs1, FPR64:$rs2, FPR64:$rs3, 0b111)>;
+          (FMSUB_D_def FPR64:$rs1, FPR64:$rs2, FPR64:$rs3)>;
 
 // fnmsub: -rs1 * rs2 + rs3
 def : Pat<(fma (fneg FPR64:$rs1), FPR64:$rs2, FPR64:$rs3),
-          (FNMSUB_D FPR64:$rs1, FPR64:$rs2, FPR64:$rs3, 0b111)>;
+          (FNMSUB_D_def FPR64:$rs1, FPR64:$rs2, FPR64:$rs3)>;
 
 // fnmadd: -rs1 * rs2 - rs3
 def : Pat<(fma (fneg FPR64:$rs1), FPR64:$rs2, (fneg FPR64:$rs3)),
-          (FNMADD_D FPR64:$rs1, FPR64:$rs2, FPR64:$rs3, 0b111)>;
+          (FNMADD_D_def FPR64:$rs1, FPR64:$rs2, FPR64:$rs3)>;
 
 // The RISC-V 2.2 user-level ISA spec defines fmin and fmax as returning the
 // canonical NaN when giving a signaling NaN. This doesn't match the LLVM
 // behaviour (see https://bugs.llvm.org/show_bug.cgi?id=27363). However, the
 // draft 2.3 ISA spec changes the definition of fmin and fmax in a way that
 // matches LLVM's fminnum and fmaxnum
 // <https://github.com/riscv/riscv-isa-manual/commit/cd20cee7efd9bac7c5aa127ec3b451749d2b3cce>.
 def : PatFpr64Fpr64<fminnum, FMIN_D>;

llvm/lib/Target/RISCV/RISCVInstrInfoF.td

def frmarg : Operand<XLenVT> {
   let PrintMethod = "printFRMArg";
   let DecoderMethod = "decodeFRMArg";
 }
 
 //===----------------------------------------------------------------------===//
 // Instruction class templates
 //===----------------------------------------------------------------------===//
 
-let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
-class FPFMAS_rrr_frm<RISCVOpcode opcode, string opcodestr>
-    : RVInstR4<0b00, opcode, (outs FPR32:$rd),
-               (ins FPR32:$rs1, FPR32:$rs2, FPR32:$rs3, frmarg:$funct3),
-                opcodestr, "$rd, $rs1, $rs2, $rs3, $funct3">;
-
-class FPFMASDynFrmAlias<FPFMAS_rrr_frm Inst, string OpcodeStr>
-    : InstAlias<OpcodeStr#" $rd, $rs1, $rs2, $rs3",
-                (Inst FPR32:$rd, FPR32:$rs1, FPR32:$rs2, FPR32:$rs3, 0b111)>;
+let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in {
+  class FPFMA_gen<RISCVOpcode opcode, string opcodestr,
+                  bits<2> kind, RegisterClass rty>
+      : RVInstR4<kind, opcode, (outs rty:$rd),
+                 (ins rty:$rs1, rty:$rs2, rty:$rs3, frmarg:$funct3),
+                  opcodestr, "$rd, $rs1, $rs2, $rs3, $funct3"> {
+    let Defs = [FFLAGSReg];
+    let Uses = [FRMReg];
+  }
+
+  class FPFMA_def<RISCVOpcode opcode, string opcodestr,
+                  bits<2> kind, RegisterClass rty>
+      : RVInstR4<kind, opcode, (outs rty:$rd),
+                 (ins rty:$rs1, rty:$rs2, rty:$rs3),
+                  opcodestr, "$rd, $rs1, $rs2, $rs3, rne"> {
+    let funct3 = 0b000;
+  }
+}
+
+multiclass FPFMAS_mc<RISCVOpcode opcode, string OpcodeStr> {
+  def _gen : FPFMA_gen<opcode, OpcodeStr, 0b00, FPR32>;
+  def _def : FPFMA_def<opcode, OpcodeStr, 0b00, FPR32>;
+  def      : InstAlias<OpcodeStr#" $rd, $rs1, $rs2, $rs3",
+                       (!cast<Instruction>(NAME # "_gen")
+                        FPR32:$rd, FPR32:$rs1, FPR32:$rs2, FPR32:$rs3, 0b111)>;
+}
 
-let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
 class FPALUS_rr<bits<7> funct7, bits<3> funct3, string opcodestr>
     : RVInstR<funct7, funct3, OPC_OP_FP, (outs FPR32:$rd),
               (ins FPR32:$rs1, FPR32:$rs2), opcodestr, "$rd, $rs1, $rs2">;
 
 let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
 class FPALUS_rr_frm<bits<7> funct7, string opcodestr>
     : RVInstRFrm<funct7, OPC_OP_FP, (outs FPR32:$rd),
                  (ins FPR32:$rs1, FPR32:$rs2, frmarg:$funct3), opcodestr,

 // reflecting the order these fields are specified in the instruction
 // encoding.
 let hasSideEffects = 0, mayLoad = 0, mayStore = 1 in
 def FSW : RVInstS<0b010, OPC_STORE_FP, (outs),
                   (ins FPR32:$rs2, GPR:$rs1, simm12:$imm12),
                    "fsw", "$rs2, ${imm12}(${rs1})">,
           Sched<[WriteFST32, ReadStoreData, ReadFMemBase]>;
 
-def FMADD_S  : FPFMAS_rrr_frm<OPC_MADD, "fmadd.s">,
+defm FMADD_S  : FPFMAS_mc<OPC_MADD, "fmadd.s">,
                 Sched<[WriteFMulAdd32, ReadFMulAdd32, ReadFMulAdd32, ReadFMulAdd32]>;
-def          : FPFMASDynFrmAlias<FMADD_S, "fmadd.s">;
-def FMSUB_S  : FPFMAS_rrr_frm<OPC_MSUB, "fmsub.s">,
+defm FMSUB_S  : FPFMAS_mc<OPC_MSUB, "fmsub.s">,
                 Sched<[WriteFMulSub32, ReadFMulSub32, ReadFMulSub32, ReadFMulSub32]>;
-def          : FPFMASDynFrmAlias<FMSUB_S, "fmsub.s">;
-def FNMSUB_S : FPFMAS_rrr_frm<OPC_NMSUB, "fnmsub.s">,
+defm FNMSUB_S : FPFMAS_mc<OPC_NMSUB, "fnmsub.s">,
                 Sched<[WriteFMulSub32, ReadFMulSub32, ReadFMulSub32, ReadFMulSub32]>;
-def          : FPFMASDynFrmAlias<FNMSUB_S, "fnmsub.s">;
-def FNMADD_S : FPFMAS_rrr_frm<OPC_NMADD, "fnmadd.s">,
+defm FNMADD_S : FPFMAS_mc<OPC_NMADD, "fnmadd.s">,
                 Sched<[WriteFMulAdd32, ReadFMulAdd32, ReadFMulAdd32, ReadFMulAdd32]>;
-def          : FPFMASDynFrmAlias<FNMADD_S, "fnmadd.s">;
 
 def FADD_S : FPALUS_rr_frm<0b0000000, "fadd.s">,
              Sched<[WriteFALU32, ReadFALU32, ReadFALU32]>;
 def        : FPALUSDynFrmAlias<FADD_S, "fadd.s">;
 def FSUB_S : FPALUS_rr_frm<0b0000100, "fsub.s">,
              Sched<[WriteFALU32, ReadFALU32, ReadFALU32]>;
 def        : FPALUSDynFrmAlias<FSUB_S, "fsub.s">;
 def FMUL_S : FPALUS_rr_frm<0b0001000, "fmul.s">,

 def : Pat<(fneg FPR32:$rs1), (FSGNJN_S $rs1, $rs1)>;
 def : Pat<(fabs FPR32:$rs1), (FSGNJX_S $rs1, $rs1)>;
 
 def : PatFpr32Fpr32<fcopysign, FSGNJ_S>;
 def : Pat<(fcopysign FPR32:$rs1, (fneg FPR32:$rs2)), (FSGNJN_S $rs1, $rs2)>;
 
 // fmadd: rs1 * rs2 + rs3
 def : Pat<(fma FPR32:$rs1, FPR32:$rs2, FPR32:$rs3),
-          (FMADD_S $rs1, $rs2, $rs3, 0b111)>;
+          (FMADD_S_def $rs1, $rs2, $rs3)>;
 
 // fmsub: rs1 * rs2 - rs3
 def : Pat<(fma FPR32:$rs1, FPR32:$rs2, (fneg FPR32:$rs3)),
-          (FMSUB_S FPR32:$rs1, FPR32:$rs2, FPR32:$rs3, 0b111)>;
+          (FMSUB_S_def FPR32:$rs1, FPR32:$rs2, FPR32:$rs3)>;
 
 // fnmsub: -rs1 * rs2 + rs3
 def : Pat<(fma (fneg FPR32:$rs1), FPR32:$rs2, FPR32:$rs3),
-          (FNMSUB_S FPR32:$rs1, FPR32:$rs2, FPR32:$rs3, 0b111)>;
+          (FNMSUB_S_def FPR32:$rs1, FPR32:$rs2, FPR32:$rs3)>;
 
 // fnmadd: -rs1 * rs2 - rs3
 def : Pat<(fma (fneg FPR32:$rs1), FPR32:$rs2, (fneg FPR32:$rs3)),
-          (FNMADD_S FPR32:$rs1, FPR32:$rs2, FPR32:$rs3, 0b111)>;
+          (FNMADD_S_def FPR32:$rs1, FPR32:$rs2, FPR32:$rs3)>;
 
 // The RISC-V 2.2 user-level ISA spec defines fmin and fmax as returning the
 // canonical NaN when given a signaling NaN. This doesn't match the LLVM
 // behaviour (see https://bugs.llvm.org/show_bug.cgi?id=27363). However, the
 // draft 2.3 ISA spec changes the definition of fmin and fmax in a way that
 // matches LLVM's fminnum and fmaxnum
 // <https://github.com/riscv/riscv-isa-manual/commit/cd20cee7efd9bac7c5aa127ec3b451749d2b3cce>.
 def : PatFpr32Fpr32<fminnum, FMIN_S>;

llvm/lib/Target/RISCV/RISCVInstrInfoZfh.td

def riscv_fmv_h_x
     : SDNode<"RISCVISD::FMV_H_X", SDT_RISCVFMV_H_X>;
 def riscv_fmv_x_anyexth
     : SDNode<"RISCVISD::FMV_X_ANYEXTH", SDT_RISCVFMV_X_ANYEXTH>;
 
 //===----------------------------------------------------------------------===//
 // Instruction class templates
 //===----------------------------------------------------------------------===//
 
-let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
-class FPFMAH_rrr_frm<RISCVOpcode opcode, string opcodestr>
-    : RVInstR4<0b10, opcode, (outs FPR16:$rd),
-               (ins FPR16:$rs1, FPR16:$rs2, FPR16:$rs3, frmarg:$funct3),
-                opcodestr, "$rd, $rs1, $rs2, $rs3, $funct3">;
-
-class FPFMAHDynFrmAlias<FPFMAH_rrr_frm Inst, string OpcodeStr>
-    : InstAlias<OpcodeStr#" $rd, $rs1, $rs2, $rs3",
-                (Inst FPR16:$rd, FPR16:$rs1, FPR16:$rs2, FPR16:$rs3, 0b111)>;
+multiclass FPFMAH_mc<RISCVOpcode opcode, string OpcodeStr> {
+  def _gen : FPFMA_gen<opcode, OpcodeStr, 0b10, FPR16>;
+  def _def : FPFMA_def<opcode, OpcodeStr, 0b10, FPR16>;
+  def      : InstAlias<OpcodeStr#" $rd, $rs1, $rs2, $rs3",
+                       (!cast<Instruction>(NAME # "_gen")
+                        FPR16:$rd, FPR16:$rs1, FPR16:$rs2, FPR16:$rs3, 0b111)>;
+}
 
 let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
 class FPALUH_rr<bits<7> funct7, bits<3> funct3, string opcodestr>
     : RVInstR<funct7, funct3, OPC_OP_FP, (outs FPR16:$rd),
               (ins FPR16:$rs1, FPR16:$rs2), opcodestr, "$rd, $rs1, $rs2">;
 
 let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
 class FPALUH_rr_frm<bits<7> funct7, string opcodestr>

 // reflecting the order these fields are specified in the instruction
 // encoding.
 let hasSideEffects = 0, mayLoad = 0, mayStore = 1 in
 def FSH : RVInstS<0b001, OPC_STORE_FP, (outs),
                   (ins FPR16:$rs2, GPR:$rs1, simm12:$imm12),
                    "fsh", "$rs2, ${imm12}(${rs1})">,
           Sched<[]>;
 
-def FMADD_H  : FPFMAH_rrr_frm<OPC_MADD, "fmadd.h">,
+defm FMADD_H  : FPFMAH_mc<OPC_MADD, "fmadd.h">,
                 Sched<[]>;
-def          : FPFMAHDynFrmAlias<FMADD_H, "fmadd.h">;
-def FMSUB_H  : FPFMAH_rrr_frm<OPC_MSUB, "fmsub.h">,
+defm FMSUB_H  : FPFMAH_mc<OPC_MSUB, "fmsub.h">,
                 Sched<[]>;
-def          : FPFMAHDynFrmAlias<FMSUB_H, "fmsub.h">;
-def FNMSUB_H : FPFMAH_rrr_frm<OPC_NMSUB, "fnmsub.h">,
+defm FNMSUB_H : FPFMAH_mc<OPC_NMSUB, "fnmsub.h">,
                 Sched<[]>;
-def          : FPFMAHDynFrmAlias<FNMSUB_H, "fnmsub.h">;
-def FNMADD_H : FPFMAH_rrr_frm<OPC_NMADD, "fnmadd.h">,
+defm FNMADD_H : FPFMAH_mc<OPC_NMADD, "fnmadd.h">,
                 Sched<[]>;
-def          : FPFMAHDynFrmAlias<FNMADD_H, "fnmadd.h">;
 
 def FADD_H : FPALUH_rr_frm<0b0000010, "fadd.h">,
              Sched<[]>;
 def        : FPALUHDynFrmAlias<FADD_H, "fadd.h">;
 def FSUB_H : FPALUH_rr_frm<0b0000110, "fsub.h">,
              Sched<[]>;
 def        : FPALUHDynFrmAlias<FSUB_H, "fsub.h">;
 def FMUL_H : FPALUH_rr_frm<0b0001010, "fmul.h">,
def : Pat<(fcopysign FPR16:$rs1, FPR32:$rs2),
           (FSGNJ_H $rs1, (FCVT_H_S $rs2, 0b111))>;
 def : Pat<(fcopysign FPR16:$rs1, FPR64:$rs2),
           (FSGNJ_H $rs1, (FCVT_H_D $rs2, 0b111))>;
 def : Pat<(fcopysign FPR32:$rs1, FPR16:$rs2), (FSGNJ_S $rs1, (FCVT_S_H $rs2))>;
 def : Pat<(fcopysign FPR64:$rs1, FPR16:$rs2), (FSGNJ_D $rs1, (FCVT_D_H $rs2))>;
 
 // fmadd: rs1 * rs2 + rs3
 def : Pat<(fma FPR16:$rs1, FPR16:$rs2, FPR16:$rs3),
-          (FMADD_H $rs1, $rs2, $rs3, 0b111)>;
+          (FMADD_H_def $rs1, $rs2, $rs3)>;
 
 // fmsub: rs1 * rs2 - rs3
 def : Pat<(fma FPR16:$rs1, FPR16:$rs2, (fneg FPR16:$rs3)),
-          (FMSUB_H FPR16:$rs1, FPR16:$rs2, FPR16:$rs3, 0b111)>;
+          (FMSUB_H_def FPR16:$rs1, FPR16:$rs2, FPR16:$rs3)>;
 
 // fnmsub: -rs1 * rs2 + rs3
 def : Pat<(fma (fneg FPR16:$rs1), FPR16:$rs2, FPR16:$rs3),
-          (FNMSUB_H FPR16:$rs1, FPR16:$rs2, FPR16:$rs3, 0b111)>;
+          (FNMSUB_H_def FPR16:$rs1, FPR16:$rs2, FPR16:$rs3)>;
 
 // fnmadd: -rs1 * rs2 - rs3
 def : Pat<(fma (fneg FPR16:$rs1), FPR16:$rs2, (fneg FPR16:$rs3)),
-          (FNMADD_H FPR16:$rs1, FPR16:$rs2, FPR16:$rs3, 0b111)>;
+          (FNMADD_H_def FPR16:$rs1, FPR16:$rs2, FPR16:$rs3)>;
 
 def : PatFpr16Fpr16<fminnum, FMIN_H>;
 def : PatFpr16Fpr16<fmaxnum, FMAX_H>;
 
 /// Setcc
 
 def : PatFpr16Fpr16<seteq, FEQ_H>;
 def : PatFpr16Fpr16<setoeq, FEQ_H>;

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

if (TFI->hasBP(MF))
     markSuperRegs(Reserved, RISCVABI::getBPReg()); // bp
 
   // V registers for code generation. We handle them manually.
   markSuperRegs(Reserved, RISCV::VL);
   markSuperRegs(Reserved, RISCV::VTYPE);
   markSuperRegs(Reserved, RISCV::VXSAT);
   markSuperRegs(Reserved, RISCV::VXRM);
 
+  // Floating point environment registers.
+  markSuperRegs(Reserved, RISCV::FRMReg);
+  markSuperRegs(Reserved, RISCV::FFLAGSReg);
+  markSuperRegs(Reserved, RISCV::FCSRReg);
+
   assert(checkAllSuperRegsMarked(Reserved));
   return Reserved;
 }
 
 bool RISCVRegisterInfo::isAsmClobberable(const MachineFunction &MF,
                                          MCRegister PhysReg) const {
   return !MF.getSubtarget<RISCVSubtarget>().isRegisterReservedByUser(PhysReg);
 }

llvm/lib/Target/RISCV/RISCVRegisterInfo.td

foreach nf = NFList<m>.L in {
     def "VRN" # nf # "M" # m: VReg<[untyped],
                                (add !cast<RegisterTuples>("VN" # nf # "M" # m # "V0"), !cast<RegisterTuples>("VN" # nf # "M" # m # "NoV0")),
                                     !mul(nf, m)>;
     def "VRN" # nf # "M" # m # "NoV0": VReg<[untyped],
                                (add !cast<RegisterTuples>("VN" # nf # "M" # m # "NoV0")),
                                     !mul(nf, m)>;
   }
 }
+
+// Special registers
+def FFLAGSReg : RISCVReg<1, "fflags">;
+def FRMReg    : RISCVReg<2, "frm">;
+def FCSRReg   : RISCVReg<3, "fcsr">;

llvm/test/CodeGen/RISCV/double-arith.ll

 ; RV32IFD-NEXT:    sw a5, 12(sp)
 ; RV32IFD-NEXT:    fld ft0, 8(sp)
 ; RV32IFD-NEXT:    sw a2, 8(sp)
 ; RV32IFD-NEXT:    sw a3, 12(sp)
 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a0, 8(sp)
 ; RV32IFD-NEXT:    sw a1, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
-; RV32IFD-NEXT:    fmadd.d ft0, ft2, ft1, ft0
+; RV32IFD-NEXT:    fmadd.d ft0, ft2, ft1, ft0, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fmadd_d:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a2
 ; RV64IFD-NEXT:    fmv.d.x ft1, a1
 ; RV64IFD-NEXT:    fmv.d.x ft2, a0
-; RV64IFD-NEXT:    fmadd.d ft0, ft2, ft1, ft0
+; RV64IFD-NEXT:    fmadd.d ft0, ft2, ft1, ft0, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %1 = call double @llvm.fma.f64(double %a, double %b, double %c)
   ret double %1
 }
 
 define double @fmsub_d(double %a, double %b, double %c) nounwind {
 ; RV32IFD-LABEL: fmsub_d:
 ; RV32IFD:       # %bb.0:
 ; RV32IFD-NEXT:    addi sp, sp, -16
 ; RV32IFD-NEXT:    sw a2, 8(sp)
 ; RV32IFD-NEXT:    sw a3, 12(sp)
 ; RV32IFD-NEXT:    fld ft0, 8(sp)
 ; RV32IFD-NEXT:    sw a0, 8(sp)
 ; RV32IFD-NEXT:    sw a1, 12(sp)
 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a4, 8(sp)
 ; RV32IFD-NEXT:    sw a5, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
 ; RV32IFD-NEXT:    fcvt.d.w ft3, zero
 ; RV32IFD-NEXT:    fadd.d ft2, ft2, ft3
-; RV32IFD-NEXT:    fmsub.d ft0, ft1, ft0, ft2
+; RV32IFD-NEXT:    fmsub.d ft0, ft1, ft0, ft2, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fmsub_d:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a1
 ; RV64IFD-NEXT:    fmv.d.x ft1, a0
 ; RV64IFD-NEXT:    fmv.d.x ft2, a2
 ; RV64IFD-NEXT:    fmv.d.x ft3, zero
 ; RV64IFD-NEXT:    fadd.d ft2, ft2, ft3
-; RV64IFD-NEXT:    fmsub.d ft0, ft1, ft0, ft2
+; RV64IFD-NEXT:    fmsub.d ft0, ft1, ft0, ft2, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %c_ = fadd double 0.0, %c ; avoid negation using xor
   %negc = fsub double -0.0, %c_
   %1 = call double @llvm.fma.f64(double %a, double %b, double %negc)
   ret double %1
 }
 
 define double @fnmadd_d(double %a, double %b, double %c) nounwind {
 ; RV32IFD-LABEL: fnmadd_d:
 ; RV32IFD:       # %bb.0:
 ; RV32IFD-NEXT:    addi sp, sp, -16
 ; RV32IFD-NEXT:    sw a2, 8(sp)
 ; RV32IFD-NEXT:    sw a3, 12(sp)
 ; RV32IFD-NEXT:    fld ft0, 8(sp)
 ; RV32IFD-NEXT:    sw a4, 8(sp)
 ; RV32IFD-NEXT:    sw a5, 12(sp)
 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a0, 8(sp)
 ; RV32IFD-NEXT:    sw a1, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
 ; RV32IFD-NEXT:    fcvt.d.w ft3, zero
 ; RV32IFD-NEXT:    fadd.d ft2, ft2, ft3
 ; RV32IFD-NEXT:    fadd.d ft1, ft1, ft3
-; RV32IFD-NEXT:    fnmadd.d ft0, ft2, ft0, ft1
+; RV32IFD-NEXT:    fnmadd.d ft0, ft2, ft0, ft1, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fnmadd_d:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a1
 ; RV64IFD-NEXT:    fmv.d.x ft1, a2
 ; RV64IFD-NEXT:    fmv.d.x ft2, a0
 ; RV64IFD-NEXT:    fmv.d.x ft3, zero
 ; RV64IFD-NEXT:    fadd.d ft2, ft2, ft3
 ; RV64IFD-NEXT:    fadd.d ft1, ft1, ft3
-; RV64IFD-NEXT:    fnmadd.d ft0, ft2, ft0, ft1
+; RV64IFD-NEXT:    fnmadd.d ft0, ft2, ft0, ft1, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %a_ = fadd double 0.0, %a
   %c_ = fadd double 0.0, %c
   %nega = fsub double -0.0, %a_
   %negc = fsub double -0.0, %c_
   %1 = call double @llvm.fma.f64(double %nega, double %b, double %negc)
   ret double %1

 ; RV32IFD-NEXT:    sw a5, 12(sp)
 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a2, 8(sp)
 ; RV32IFD-NEXT:    sw a3, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
 ; RV32IFD-NEXT:    fcvt.d.w ft3, zero
 ; RV32IFD-NEXT:    fadd.d ft2, ft2, ft3
 ; RV32IFD-NEXT:    fadd.d ft1, ft1, ft3
-; RV32IFD-NEXT:    fnmadd.d ft0, ft2, ft0, ft1
+; RV32IFD-NEXT:    fnmadd.d ft0, ft2, ft0, ft1, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fnmadd_d_2:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a0
 ; RV64IFD-NEXT:    fmv.d.x ft1, a2
 ; RV64IFD-NEXT:    fmv.d.x ft2, a1
 ; RV64IFD-NEXT:    fmv.d.x ft3, zero
 ; RV64IFD-NEXT:    fadd.d ft2, ft2, ft3
 ; RV64IFD-NEXT:    fadd.d ft1, ft1, ft3
-; RV64IFD-NEXT:    fnmadd.d ft0, ft2, ft0, ft1
+; RV64IFD-NEXT:    fnmadd.d ft0, ft2, ft0, ft1, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %b_ = fadd double 0.0, %b
   %c_ = fadd double 0.0, %c
   %negb = fsub double -0.0, %b_
   %negc = fsub double -0.0, %c_
   %1 = call double @llvm.fma.f64(double %a, double %negb, double %negc)
   ret double %1

 ; RV32IFD-NEXT:    sw a2, 8(sp)
 ; RV32IFD-NEXT:    sw a3, 12(sp)
 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a0, 8(sp)
 ; RV32IFD-NEXT:    sw a1, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
 ; RV32IFD-NEXT:    fcvt.d.w ft3, zero
 ; RV32IFD-NEXT:    fadd.d ft2, ft2, ft3
-; RV32IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0
+; RV32IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fnmsub_d:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a2
 ; RV64IFD-NEXT:    fmv.d.x ft1, a1
 ; RV64IFD-NEXT:    fmv.d.x ft2, a0
 ; RV64IFD-NEXT:    fmv.d.x ft3, zero
 ; RV64IFD-NEXT:    fadd.d ft2, ft2, ft3
-; RV64IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0
+; RV64IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %a_ = fadd double 0.0, %a
   %nega = fsub double -0.0, %a_
   %1 = call double @llvm.fma.f64(double %nega, double %b, double %c)
   ret double %1
 }
 
 define double @fnmsub_d_2(double %a, double %b, double %c) nounwind {
 ; RV32IFD-LABEL: fnmsub_d_2:
 ; RV32IFD:       # %bb.0:
 ; RV32IFD-NEXT:    addi sp, sp, -16
 ; RV32IFD-NEXT:    sw a4, 8(sp)
 ; RV32IFD-NEXT:    sw a5, 12(sp)
 ; RV32IFD-NEXT:    fld ft0, 8(sp)
 ; RV32IFD-NEXT:    sw a0, 8(sp)
 ; RV32IFD-NEXT:    sw a1, 12(sp)
 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a2, 8(sp)
 ; RV32IFD-NEXT:    sw a3, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
 ; RV32IFD-NEXT:    fcvt.d.w ft3, zero
 ; RV32IFD-NEXT:    fadd.d ft2, ft2, ft3
-; RV32IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0
+; RV32IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fnmsub_d_2:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a2
 ; RV64IFD-NEXT:    fmv.d.x ft1, a0
 ; RV64IFD-NEXT:    fmv.d.x ft2, a1
 ; RV64IFD-NEXT:    fmv.d.x ft3, zero
 ; RV64IFD-NEXT:    fadd.d ft2, ft2, ft3
-; RV64IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0
+; RV64IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %b_ = fadd double 0.0, %b
   %negb = fsub double -0.0, %b_
   %1 = call double @llvm.fma.f64(double %a, double %negb, double %c)
   ret double %1
 }
 
 define double @fmadd_d_contract(double %a, double %b, double %c) nounwind {
 ; RV32IFD-LABEL: fmadd_d_contract:
 ; RV32IFD:       # %bb.0:
 ; RV32IFD-NEXT:    addi sp, sp, -16
 ; RV32IFD-NEXT:    sw a4, 8(sp)
 ; RV32IFD-NEXT:    sw a5, 12(sp)
 ; RV32IFD-NEXT:    fld ft0, 8(sp)
 ; RV32IFD-NEXT:    sw a2, 8(sp)
 ; RV32IFD-NEXT:    sw a3, 12(sp)
 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a0, 8(sp)
 ; RV32IFD-NEXT:    sw a1, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
-; RV32IFD-NEXT:    fmadd.d ft0, ft2, ft1, ft0
+; RV32IFD-NEXT:    fmadd.d ft0, ft2, ft1, ft0, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fmadd_d_contract:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a2
 ; RV64IFD-NEXT:    fmv.d.x ft1, a1
 ; RV64IFD-NEXT:    fmv.d.x ft2, a0
-; RV64IFD-NEXT:    fmadd.d ft0, ft2, ft1, ft0
+; RV64IFD-NEXT:    fmadd.d ft0, ft2, ft1, ft0, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %1 = fmul contract double %a, %b
   %2 = fadd contract double %1, %c
   ret double %2
 }
 
 define double @fmsub_d_contract(double %a, double %b, double %c) nounwind {
 ; RV32IFD-LABEL: fmsub_d_contract:
 ; RV32IFD:       # %bb.0:
 ; RV32IFD-NEXT:    addi sp, sp, -16
 ; RV32IFD-NEXT:    sw a2, 8(sp)
 ; RV32IFD-NEXT:    sw a3, 12(sp)
 ; RV32IFD-NEXT:    fld ft0, 8(sp)
 ; RV32IFD-NEXT:    sw a0, 8(sp)
 ; RV32IFD-NEXT:    sw a1, 12(sp)
 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a4, 8(sp)
 ; RV32IFD-NEXT:    sw a5, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
 ; RV32IFD-NEXT:    fcvt.d.w ft3, zero
 ; RV32IFD-NEXT:    fadd.d ft2, ft2, ft3
-; RV32IFD-NEXT:    fmsub.d ft0, ft1, ft0, ft2
+; RV32IFD-NEXT:    fmsub.d ft0, ft1, ft0, ft2, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fmsub_d_contract:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a1
 ; RV64IFD-NEXT:    fmv.d.x ft1, a0
 ; RV64IFD-NEXT:    fmv.d.x ft2, a2
 ; RV64IFD-NEXT:    fmv.d.x ft3, zero
 ; RV64IFD-NEXT:    fadd.d ft2, ft2, ft3
-; RV64IFD-NEXT:    fmsub.d ft0, ft1, ft0, ft2
+; RV64IFD-NEXT:    fmsub.d ft0, ft1, ft0, ft2, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %c_ = fadd double 0.0, %c ; avoid negation using xor
   %1 = fmul contract double %a, %b
   %2 = fsub contract double %1, %c_
   ret double %2
 }
 

 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a0, 8(sp)
 ; RV32IFD-NEXT:    sw a1, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
 ; RV32IFD-NEXT:    fcvt.d.w ft3, zero
 ; RV32IFD-NEXT:    fadd.d ft2, ft2, ft3
 ; RV32IFD-NEXT:    fadd.d ft1, ft1, ft3
 ; RV32IFD-NEXT:    fadd.d ft0, ft0, ft3
-; RV32IFD-NEXT:    fnmadd.d ft0, ft2, ft1, ft0
+; RV32IFD-NEXT:    fnmadd.d ft0, ft2, ft1, ft0, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fnmadd_d_contract:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a2
 ; RV64IFD-NEXT:    fmv.d.x ft1, a1
 ; RV64IFD-NEXT:    fmv.d.x ft2, a0
 ; RV64IFD-NEXT:    fmv.d.x ft3, zero
 ; RV64IFD-NEXT:    fadd.d ft2, ft2, ft3
 ; RV64IFD-NEXT:    fadd.d ft1, ft1, ft3
 ; RV64IFD-NEXT:    fadd.d ft0, ft0, ft3
-; RV64IFD-NEXT:    fnmadd.d ft0, ft2, ft1, ft0
+; RV64IFD-NEXT:    fnmadd.d ft0, ft2, ft1, ft0, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %a_ = fadd double 0.0, %a ; avoid negation using xor
   %b_ = fadd double 0.0, %b ; avoid negation using xor
   %c_ = fadd double 0.0, %c ; avoid negation using xor
   %1 = fmul contract double %a_, %b_
   %2 = fneg double %1
   %3 = fsub contract double %2, %c_

 ; RV32IFD-NEXT:    sw a3, 12(sp)
 ; RV32IFD-NEXT:    fld ft1, 8(sp)
 ; RV32IFD-NEXT:    sw a0, 8(sp)
 ; RV32IFD-NEXT:    sw a1, 12(sp)
 ; RV32IFD-NEXT:    fld ft2, 8(sp)
 ; RV32IFD-NEXT:    fcvt.d.w ft3, zero
 ; RV32IFD-NEXT:    fadd.d ft2, ft2, ft3
 ; RV32IFD-NEXT:    fadd.d ft1, ft1, ft3
-; RV32IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0
+; RV32IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0, rne
 ; RV32IFD-NEXT:    fsd ft0, 8(sp)
 ; RV32IFD-NEXT:    lw a0, 8(sp)
 ; RV32IFD-NEXT:    lw a1, 12(sp)
 ; RV32IFD-NEXT:    addi sp, sp, 16
 ; RV32IFD-NEXT:    ret
 ;
 ; RV64IFD-LABEL: fnmsub_d_contract:
 ; RV64IFD:       # %bb.0:
 ; RV64IFD-NEXT:    fmv.d.x ft0, a2
 ; RV64IFD-NEXT:    fmv.d.x ft1, a1
 ; RV64IFD-NEXT:    fmv.d.x ft2, a0
 ; RV64IFD-NEXT:    fmv.d.x ft3, zero
 ; RV64IFD-NEXT:    fadd.d ft2, ft2, ft3
 ; RV64IFD-NEXT:    fadd.d ft1, ft1, ft3
-; RV64IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0
+; RV64IFD-NEXT:    fnmsub.d ft0, ft2, ft1, ft0, rne
 ; RV64IFD-NEXT:    fmv.x.d a0, ft0
 ; RV64IFD-NEXT:    ret
   %a_ = fadd double 0.0, %a ; avoid negation using xor
   %b_ = fadd double 0.0, %b ; avoid negation using xor
   %1 = fmul contract double %a_, %b_
   %2 = fsub contract double %c, %1
   ret double %2
 }

llvm/test/CodeGen/RISCV/float-arith.ll

 declare float @llvm.fma.f32(float, float, float)
 
 define float @fmadd_s(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fmadd_s:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a2
 ; RV32IF-NEXT:    fmv.w.x ft1, a1
 ; RV32IF-NEXT:    fmv.w.x ft2, a0
-; RV32IF-NEXT:    fmadd.s ft0, ft2, ft1, ft0
+; RV32IF-NEXT:    fmadd.s ft0, ft2, ft1, ft0, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fmadd_s:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a2
 ; RV64IF-NEXT:    fmv.w.x ft1, a1
 ; RV64IF-NEXT:    fmv.w.x ft2, a0
-; RV64IF-NEXT:    fmadd.s ft0, ft2, ft1, ft0
+; RV64IF-NEXT:    fmadd.s ft0, ft2, ft1, ft0, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %1 = call float @llvm.fma.f32(float %a, float %b, float %c)
   ret float %1
 }
 
 define float @fmsub_s(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fmsub_s:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a1
 ; RV32IF-NEXT:    fmv.w.x ft1, a0
 ; RV32IF-NEXT:    fmv.w.x ft2, a2
 ; RV32IF-NEXT:    fmv.w.x ft3, zero
 ; RV32IF-NEXT:    fadd.s ft2, ft2, ft3
-; RV32IF-NEXT:    fmsub.s ft0, ft1, ft0, ft2
+; RV32IF-NEXT:    fmsub.s ft0, ft1, ft0, ft2, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fmsub_s:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a1
 ; RV64IF-NEXT:    fmv.w.x ft1, a0
 ; RV64IF-NEXT:    fmv.w.x ft2, a2
 ; RV64IF-NEXT:    fmv.w.x ft3, zero
 ; RV64IF-NEXT:    fadd.s ft2, ft2, ft3
-; RV64IF-NEXT:    fmsub.s ft0, ft1, ft0, ft2
+; RV64IF-NEXT:    fmsub.s ft0, ft1, ft0, ft2, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %c_ = fadd float 0.0, %c ; avoid negation using xor
   %negc = fsub float -0.0, %c_
   %1 = call float @llvm.fma.f32(float %a, float %b, float %negc)
   ret float %1
 }
 
 define float @fnmadd_s(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fnmadd_s:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a1
 ; RV32IF-NEXT:    fmv.w.x ft1, a2
 ; RV32IF-NEXT:    fmv.w.x ft2, a0
 ; RV32IF-NEXT:    fmv.w.x ft3, zero
 ; RV32IF-NEXT:    fadd.s ft2, ft2, ft3
 ; RV32IF-NEXT:    fadd.s ft1, ft1, ft3
-; RV32IF-NEXT:    fnmadd.s ft0, ft2, ft0, ft1
+; RV32IF-NEXT:    fnmadd.s ft0, ft2, ft0, ft1, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fnmadd_s:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a1
 ; RV64IF-NEXT:    fmv.w.x ft1, a2
 ; RV64IF-NEXT:    fmv.w.x ft2, a0
 ; RV64IF-NEXT:    fmv.w.x ft3, zero
 ; RV64IF-NEXT:    fadd.s ft2, ft2, ft3
 ; RV64IF-NEXT:    fadd.s ft1, ft1, ft3
-; RV64IF-NEXT:    fnmadd.s ft0, ft2, ft0, ft1
+; RV64IF-NEXT:    fnmadd.s ft0, ft2, ft0, ft1, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %a_ = fadd float 0.0, %a
   %c_ = fadd float 0.0, %c
   %nega = fsub float -0.0, %a_
   %negc = fsub float -0.0, %c_
   %1 = call float @llvm.fma.f32(float %nega, float %b, float %negc)
   ret float %1
 }
 
 define float @fnmadd_s_2(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fnmadd_s_2:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a0
 ; RV32IF-NEXT:    fmv.w.x ft1, a2
 ; RV32IF-NEXT:    fmv.w.x ft2, a1
 ; RV32IF-NEXT:    fmv.w.x ft3, zero
 ; RV32IF-NEXT:    fadd.s ft2, ft2, ft3
 ; RV32IF-NEXT:    fadd.s ft1, ft1, ft3
-; RV32IF-NEXT:    fnmadd.s ft0, ft2, ft0, ft1
+; RV32IF-NEXT:    fnmadd.s ft0, ft2, ft0, ft1, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fnmadd_s_2:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a0
 ; RV64IF-NEXT:    fmv.w.x ft1, a2
 ; RV64IF-NEXT:    fmv.w.x ft2, a1
 ; RV64IF-NEXT:    fmv.w.x ft3, zero
 ; RV64IF-NEXT:    fadd.s ft2, ft2, ft3
 ; RV64IF-NEXT:    fadd.s ft1, ft1, ft3
-; RV64IF-NEXT:    fnmadd.s ft0, ft2, ft0, ft1
+; RV64IF-NEXT:    fnmadd.s ft0, ft2, ft0, ft1, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %b_ = fadd float 0.0, %b
   %c_ = fadd float 0.0, %c
   %negb = fsub float -0.0, %b_
   %negc = fsub float -0.0, %c_
   %1 = call float @llvm.fma.f32(float %a, float %negb, float %negc)
   ret float %1
 }
 
 define float @fnmsub_s(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fnmsub_s:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a2
 ; RV32IF-NEXT:    fmv.w.x ft1, a1
 ; RV32IF-NEXT:    fmv.w.x ft2, a0
 ; RV32IF-NEXT:    fmv.w.x ft3, zero
 ; RV32IF-NEXT:    fadd.s ft2, ft2, ft3
-; RV32IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0
+; RV32IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fnmsub_s:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a2
 ; RV64IF-NEXT:    fmv.w.x ft1, a1
 ; RV64IF-NEXT:    fmv.w.x ft2, a0
 ; RV64IF-NEXT:    fmv.w.x ft3, zero
 ; RV64IF-NEXT:    fadd.s ft2, ft2, ft3
-; RV64IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0
+; RV64IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %a_ = fadd float 0.0, %a
   %nega = fsub float -0.0, %a_
   %1 = call float @llvm.fma.f32(float %nega, float %b, float %c)
   ret float %1
 }
 
 define float @fnmsub_s_2(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fnmsub_s_2:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a2
 ; RV32IF-NEXT:    fmv.w.x ft1, a0
 ; RV32IF-NEXT:    fmv.w.x ft2, a1
 ; RV32IF-NEXT:    fmv.w.x ft3, zero
 ; RV32IF-NEXT:    fadd.s ft2, ft2, ft3
-; RV32IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0
+; RV32IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fnmsub_s_2:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a2
 ; RV64IF-NEXT:    fmv.w.x ft1, a0
 ; RV64IF-NEXT:    fmv.w.x ft2, a1
 ; RV64IF-NEXT:    fmv.w.x ft3, zero
 ; RV64IF-NEXT:    fadd.s ft2, ft2, ft3
-; RV64IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0
+; RV64IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %b_ = fadd float 0.0, %b
   %negb = fsub float -0.0, %b_
   %1 = call float @llvm.fma.f32(float %a, float %negb, float %c)
   ret float %1
 }
 
 define float @fmadd_s_contract(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fmadd_s_contract:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a2
 ; RV32IF-NEXT:    fmv.w.x ft1, a1
 ; RV32IF-NEXT:    fmv.w.x ft2, a0
-; RV32IF-NEXT:    fmadd.s ft0, ft2, ft1, ft0
+; RV32IF-NEXT:    fmadd.s ft0, ft2, ft1, ft0, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fmadd_s_contract:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a2
 ; RV64IF-NEXT:    fmv.w.x ft1, a1
 ; RV64IF-NEXT:    fmv.w.x ft2, a0
-; RV64IF-NEXT:    fmadd.s ft0, ft2, ft1, ft0
+; RV64IF-NEXT:    fmadd.s ft0, ft2, ft1, ft0, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %1 = fmul contract float %a, %b
   %2 = fadd contract float %1, %c
   ret float %2
 }
 
 define float @fmsub_s_contract(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fmsub_s_contract:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a1
 ; RV32IF-NEXT:    fmv.w.x ft1, a0
 ; RV32IF-NEXT:    fmv.w.x ft2, a2
 ; RV32IF-NEXT:    fmv.w.x ft3, zero
 ; RV32IF-NEXT:    fadd.s ft2, ft2, ft3
-; RV32IF-NEXT:    fmsub.s ft0, ft1, ft0, ft2
+; RV32IF-NEXT:    fmsub.s ft0, ft1, ft0, ft2, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fmsub_s_contract:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a1
 ; RV64IF-NEXT:    fmv.w.x ft1, a0
 ; RV64IF-NEXT:    fmv.w.x ft2, a2
 ; RV64IF-NEXT:    fmv.w.x ft3, zero
 ; RV64IF-NEXT:    fadd.s ft2, ft2, ft3
-; RV64IF-NEXT:    fmsub.s ft0, ft1, ft0, ft2
+; RV64IF-NEXT:    fmsub.s ft0, ft1, ft0, ft2, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %c_ = fadd float 0.0, %c ; avoid negation using xor
   %1 = fmul contract float %a, %b
   %2 = fsub contract float %1, %c_
   ret float %2
 }
 
 define float @fnmadd_s_contract(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fnmadd_s_contract:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a2
 ; RV32IF-NEXT:    fmv.w.x ft1, a1
 ; RV32IF-NEXT:    fmv.w.x ft2, a0
 ; RV32IF-NEXT:    fmv.w.x ft3, zero
 ; RV32IF-NEXT:    fadd.s ft2, ft2, ft3
 ; RV32IF-NEXT:    fadd.s ft1, ft1, ft3
 ; RV32IF-NEXT:    fadd.s ft0, ft0, ft3
-; RV32IF-NEXT:    fnmadd.s ft0, ft2, ft1, ft0
+; RV32IF-NEXT:    fnmadd.s ft0, ft2, ft1, ft0, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fnmadd_s_contract:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a2
 ; RV64IF-NEXT:    fmv.w.x ft1, a1
 ; RV64IF-NEXT:    fmv.w.x ft2, a0
 ; RV64IF-NEXT:    fmv.w.x ft3, zero
 ; RV64IF-NEXT:    fadd.s ft2, ft2, ft3
 ; RV64IF-NEXT:    fadd.s ft1, ft1, ft3
 ; RV64IF-NEXT:    fadd.s ft0, ft0, ft3
-; RV64IF-NEXT:    fnmadd.s ft0, ft2, ft1, ft0
+; RV64IF-NEXT:    fnmadd.s ft0, ft2, ft1, ft0, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %a_ = fadd float 0.0, %a ; avoid negation using xor
   %b_ = fadd float 0.0, %b ; avoid negation using xor
   %c_ = fadd float 0.0, %c ; avoid negation using xor
   %1 = fmul contract float %a_, %b_
   %2 = fneg float %1
   %3 = fsub contract float %2, %c_
   ret float %3
 }
 
 define float @fnmsub_s_contract(float %a, float %b, float %c) nounwind {
 ; RV32IF-LABEL: fnmsub_s_contract:
 ; RV32IF:       # %bb.0:
 ; RV32IF-NEXT:    fmv.w.x ft0, a2
 ; RV32IF-NEXT:    fmv.w.x ft1, a1
 ; RV32IF-NEXT:    fmv.w.x ft2, a0
 ; RV32IF-NEXT:    fmv.w.x ft3, zero
 ; RV32IF-NEXT:    fadd.s ft2, ft2, ft3
 ; RV32IF-NEXT:    fadd.s ft1, ft1, ft3
-; RV32IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0
+; RV32IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0, rne
 ; RV32IF-NEXT:    fmv.x.w a0, ft0
 ; RV32IF-NEXT:    ret
 ;
 ; RV64IF-LABEL: fnmsub_s_contract:
 ; RV64IF:       # %bb.0:
 ; RV64IF-NEXT:    fmv.w.x ft0, a2
 ; RV64IF-NEXT:    fmv.w.x ft1, a1
 ; RV64IF-NEXT:    fmv.w.x ft2, a0
 ; RV64IF-NEXT:    fmv.w.x ft3, zero
 ; RV64IF-NEXT:    fadd.s ft2, ft2, ft3
 ; RV64IF-NEXT:    fadd.s ft1, ft1, ft3
-; RV64IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0
+; RV64IF-NEXT:    fnmsub.s ft0, ft2, ft1, ft0, rne
 ; RV64IF-NEXT:    fmv.x.w a0, ft0
 ; RV64IF-NEXT:    ret
   %a_ = fadd float 0.0, %a ; avoid negation using xor
   %b_ = fadd float 0.0, %b ; avoid negation using xor
   %1 = fmul contract float %a_, %b_
   %2 = fsub contract float %c, %1
   ret float %2
 }

llvm/test/CodeGen/RISCV/half-arith.ll

; RV64IZFH-NEXT:    ret
   ret i32 %2
 }
 
 declare half @llvm.fma.f16(half, half, half)
 
 define half @fmadd_s(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fmadd_s:
 ; RV32IZFH:       # %bb.0:
-; RV32IZFH-NEXT:    fmadd.h fa0, fa0, fa1, fa2
+; RV32IZFH-NEXT:    fmadd.h fa0, fa0, fa1, fa2, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fmadd_s:
 ; RV64IZFH:       # %bb.0:
-; RV64IZFH-NEXT:    fmadd.h fa0, fa0, fa1, fa2
+; RV64IZFH-NEXT:    fmadd.h fa0, fa0, fa1, fa2, rne
 ; RV64IZFH-NEXT:    ret
   %1 = call half @llvm.fma.f16(half %a, half %b, half %c)
   ret half %1
 }
 
 define half @fmsub_s(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fmsub_s:
 ; RV32IZFH:       # %bb.0:
 ; RV32IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV32IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV32IZFH-NEXT:    fmsub.h fa0, fa0, fa1, ft0
+; RV32IZFH-NEXT:    fmsub.h fa0, fa0, fa1, ft0, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fmsub_s:
 ; RV64IZFH:       # %bb.0:
 ; RV64IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV64IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV64IZFH-NEXT:    fmsub.h fa0, fa0, fa1, ft0
+; RV64IZFH-NEXT:    fmsub.h fa0, fa0, fa1, ft0, rne
 ; RV64IZFH-NEXT:    ret
   %c_ = fadd half 0.0, %c ; avoid negation using xor
   %negc = fsub half -0.0, %c_
   %1 = call half @llvm.fma.f16(half %a, half %b, half %negc)
   ret half %1
 }
 
 define half @fnmadd_s(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fnmadd_s:
 ; RV32IZFH:       # %bb.0:
 ; RV32IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV32IZFH-NEXT:    fadd.h ft1, fa0, ft0
 ; RV32IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV32IZFH-NEXT:    fnmadd.h fa0, ft1, fa1, ft0
+; RV32IZFH-NEXT:    fnmadd.h fa0, ft1, fa1, ft0, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fnmadd_s:
 ; RV64IZFH:       # %bb.0:
 ; RV64IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV64IZFH-NEXT:    fadd.h ft1, fa0, ft0
 ; RV64IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV64IZFH-NEXT:    fnmadd.h fa0, ft1, fa1, ft0
+; RV64IZFH-NEXT:    fnmadd.h fa0, ft1, fa1, ft0, rne
 ; RV64IZFH-NEXT:    ret
   %a_ = fadd half 0.0, %a
   %c_ = fadd half 0.0, %c
   %nega = fsub half -0.0, %a_
   %negc = fsub half -0.0, %c_
   %1 = call half @llvm.fma.f16(half %nega, half %b, half %negc)
   ret half %1
 }
 
 define half @fnmadd_s_2(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fnmadd_s_2:
 ; RV32IZFH:       # %bb.0:
 ; RV32IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV32IZFH-NEXT:    fadd.h ft1, fa1, ft0
 ; RV32IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV32IZFH-NEXT:    fnmadd.h fa0, ft1, fa0, ft0
+; RV32IZFH-NEXT:    fnmadd.h fa0, ft1, fa0, ft0, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fnmadd_s_2:
 ; RV64IZFH:       # %bb.0:
 ; RV64IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV64IZFH-NEXT:    fadd.h ft1, fa1, ft0
 ; RV64IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV64IZFH-NEXT:    fnmadd.h fa0, ft1, fa0, ft0
+; RV64IZFH-NEXT:    fnmadd.h fa0, ft1, fa0, ft0, rne
 ; RV64IZFH-NEXT:    ret
   %b_ = fadd half 0.0, %b
   %c_ = fadd half 0.0, %c
   %negb = fsub half -0.0, %b_
   %negc = fsub half -0.0, %c_
   %1 = call half @llvm.fma.f16(half %a, half %negb, half %negc)
   ret half %1
 }
 
 define half @fnmsub_s(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fnmsub_s:
 ; RV32IZFH:       # %bb.0:
 ; RV32IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV32IZFH-NEXT:    fadd.h ft0, fa0, ft0
-; RV32IZFH-NEXT:    fnmsub.h fa0, ft0, fa1, fa2
+; RV32IZFH-NEXT:    fnmsub.h fa0, ft0, fa1, fa2, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fnmsub_s:
 ; RV64IZFH:       # %bb.0:
 ; RV64IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV64IZFH-NEXT:    fadd.h ft0, fa0, ft0
-; RV64IZFH-NEXT:    fnmsub.h fa0, ft0, fa1, fa2
+; RV64IZFH-NEXT:    fnmsub.h fa0, ft0, fa1, fa2, rne
 ; RV64IZFH-NEXT:    ret
   %a_ = fadd half 0.0, %a
   %nega = fsub half -0.0, %a_
   %1 = call half @llvm.fma.f16(half %nega, half %b, half %c)
   ret half %1
 }
 
 define half @fnmsub_s_2(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fnmsub_s_2:
 ; RV32IZFH:       # %bb.0:
 ; RV32IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV32IZFH-NEXT:    fadd.h ft0, fa1, ft0
-; RV32IZFH-NEXT:    fnmsub.h fa0, ft0, fa0, fa2
+; RV32IZFH-NEXT:    fnmsub.h fa0, ft0, fa0, fa2, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fnmsub_s_2:
 ; RV64IZFH:       # %bb.0:
 ; RV64IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV64IZFH-NEXT:    fadd.h ft0, fa1, ft0
-; RV64IZFH-NEXT:    fnmsub.h fa0, ft0, fa0, fa2
+; RV64IZFH-NEXT:    fnmsub.h fa0, ft0, fa0, fa2, rne
 ; RV64IZFH-NEXT:    ret
   %b_ = fadd half 0.0, %b
   %negb = fsub half -0.0, %b_
   %1 = call half @llvm.fma.f16(half %a, half %negb, half %c)
   ret half %1
 }
 
 define half @fmadd_s_contract(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fmadd_s_contract:
 ; RV32IZFH:       # %bb.0:
-; RV32IZFH-NEXT:    fmadd.h fa0, fa0, fa1, fa2
+; RV32IZFH-NEXT:    fmadd.h fa0, fa0, fa1, fa2, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fmadd_s_contract:
 ; RV64IZFH:       # %bb.0:
-; RV64IZFH-NEXT:    fmadd.h fa0, fa0, fa1, fa2
+; RV64IZFH-NEXT:    fmadd.h fa0, fa0, fa1, fa2, rne
 ; RV64IZFH-NEXT:    ret
   %1 = fmul contract half %a, %b
   %2 = fadd contract half %1, %c
   ret half %2
 }
 
 define half @fmsub_s_contract(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fmsub_s_contract:
 ; RV32IZFH:       # %bb.0:
 ; RV32IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV32IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV32IZFH-NEXT:    fmsub.h fa0, fa0, fa1, ft0
+; RV32IZFH-NEXT:    fmsub.h fa0, fa0, fa1, ft0, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fmsub_s_contract:
 ; RV64IZFH:       # %bb.0:
 ; RV64IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV64IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV64IZFH-NEXT:    fmsub.h fa0, fa0, fa1, ft0
+; RV64IZFH-NEXT:    fmsub.h fa0, fa0, fa1, ft0, rne
 ; RV64IZFH-NEXT:    ret
   %c_ = fadd half 0.0, %c ; avoid negation using xor
   %1 = fmul contract half %a, %b
   %2 = fsub contract half %1, %c_
   ret half %2
 }
 
 define half @fnmadd_s_contract(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fnmadd_s_contract:
 ; RV32IZFH:       # %bb.0:
 ; RV32IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV32IZFH-NEXT:    fadd.h ft1, fa0, ft0
 ; RV32IZFH-NEXT:    fadd.h ft2, fa1, ft0
 ; RV32IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV32IZFH-NEXT:    fnmadd.h fa0, ft1, ft2, ft0
+; RV32IZFH-NEXT:    fnmadd.h fa0, ft1, ft2, ft0, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fnmadd_s_contract:
 ; RV64IZFH:       # %bb.0:
 ; RV64IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV64IZFH-NEXT:    fadd.h ft1, fa0, ft0
 ; RV64IZFH-NEXT:    fadd.h ft2, fa1, ft0
 ; RV64IZFH-NEXT:    fadd.h ft0, fa2, ft0
-; RV64IZFH-NEXT:    fnmadd.h fa0, ft1, ft2, ft0
+; RV64IZFH-NEXT:    fnmadd.h fa0, ft1, ft2, ft0, rne
 ; RV64IZFH-NEXT:    ret
   %a_ = fadd half 0.0, %a ; avoid negation using xor
   %b_ = fadd half 0.0, %b ; avoid negation using xor
   %c_ = fadd half 0.0, %c ; avoid negation using xor
   %1 = fmul contract half %a_, %b_
   %2 = fneg half %1
   %3 = fsub contract half %2, %c_
   ret half %3
 }
 
 define half @fnmsub_s_contract(half %a, half %b, half %c) nounwind {
 ; RV32IZFH-LABEL: fnmsub_s_contract:
 ; RV32IZFH:       # %bb.0:
 ; RV32IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV32IZFH-NEXT:    fadd.h ft1, fa0, ft0
 ; RV32IZFH-NEXT:    fadd.h ft0, fa1, ft0
-; RV32IZFH-NEXT:    fnmsub.h fa0, ft1, ft0, fa2
+; RV32IZFH-NEXT:    fnmsub.h fa0, ft1, ft0, fa2, rne
 ; RV32IZFH-NEXT:    ret
 ;
 ; RV64IZFH-LABEL: fnmsub_s_contract:
 ; RV64IZFH:       # %bb.0:
 ; RV64IZFH-NEXT:    fmv.h.x ft0, zero
 ; RV64IZFH-NEXT:    fadd.h ft1, fa0, ft0
 ; RV64IZFH-NEXT:    fadd.h ft0, fa1, ft0
-; RV64IZFH-NEXT:    fnmsub.h fa0, ft1, ft0, fa2
+; RV64IZFH-NEXT:    fnmsub.h fa0, ft1, ft0, fa2, rne
 ; RV64IZFH-NEXT:    ret
   %a_ = fadd half 0.0, %a ; avoid negation using xor
   %b_ = fadd half 0.0, %b ; avoid negation using xor
   %1 = fmul contract half %a_, %b_
   %2 = fsub contract half %c, %1
   ret half %2
 }