[MachineCombiner][RISCV] Enable MachineCombiner for RISCV

[asi-sc]

Summary

Initial implementation to match basic FP reassociation patterns.

[asi-sc]

This patch does not contain the full test to make changes more readable during the review.

Whetstone double-precision perf results for sifive-u74 (-O3 -funroll-loops -finline-functions -ffast-math -mtune=sifive-u74)
N1 +8.9%
N2 +1%

Machine combiner is off

Loop content                  Result              MFLOPS      MOPS   Seconds

N1 floating point     -1.12398255667391944       261.534              0.743
N2 floating point     -1.12187079889293351       222.128              6.123
N3 if then else        1.00000000000000000               14291.202    0.073
N4 fixed point        12.00000000000000000               289771357.091    0.000
N5 sin,cos etc.        0.49902937281518078                  20.891   40.300
N6 floating point      0.99999987890802811       170.563             32.001
N7 assignments         3.00000000000000000               28564.747    0.065
N8 exp,sqrt etc.       0.75100163018453681                  21.163   17.787

MWIPS                                           1042.205             97.092

Machine combiner is on

Loop content                  Result              MFLOPS      MOPS   Seconds

N1 floating point     -1.12398255667391900       284.892              0.692
N2 floating point     -1.12187079889295083       224.735              6.144
N3 if then else        1.00000000000000000               14281.518    0.074
N4 fixed point        12.00000000000000000               269692501.333    0.000
N5 sin,cos etc.        0.49902937281518078                  20.787   41.122
N6 floating point      0.99999987890802811       170.559             32.492
N7 assignments         3.00000000000000000               28605.105    0.066
N8 exp,sqrt etc.       0.75100163018453681                  21.534   17.748

MWIPS                                           1044.744             98.340

[craig.topper]

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

1150

Probably want to make sure that the FrmOpIdx is the same as the current number of operands. If it's not then adding to the end is wrong.

1154

I'm not sure I like this assumption that FRM physical register is the next operand. Can we just create a FRM implicit use operand instead of copying it from the old?

1211

Why not use fast math flags?

llvm/lib/Target/RISCV/RISCVTargetMachine.cpp

271

Doesn't every other target put this in addILPOpts?

[asi-sc]

Addressed review comments

[asi-sc]

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

1211

I changed current implementation to use fast math flags, but I have a question related to that.
Clang distributes fast math flags to instructions if -ffast-math is passed, whereas llc does not. So, theoretically we can have UnsafeFPMath option enabled and don't have fast math flags on instructions. What do you think, should we combine instructions in this case?

llvm/lib/Target/RISCV/RISCVTargetMachine.cpp

271

Thanks for the question, it's definitely something we should discuss. You are right, addILPOpts is the common insertion point for this pass. However, my experiments show that it's more profitable to run machine combiner after machine LICM and machine sinking which are inserted right after ILPOpts customization point. I also need to say that my local machine combiner version has many more patterns to combine, e.g. it can reassociate integer additions.
One example I remember is that on Coremark machine sinking gathers a chain of additions in the loop:

bb.291.for.body3.us.us.i66:
; predecessors: %bb.289, %bb.290 
  successors: %bb.63(0x04000000), %bb.62(0x7c000000); %bb.63(3.12%), %bb.62(96.88%)

  %76:gpr = PHI %1287:gpr, %bb.289, %1285:gpr, %bb.290 
  %1288:gpr = PHI %1210:gpr, %bb.289, %1286:gpr, %bb.290 
  %1212:gpr = ADD %1211:gpr, %72:gpr 
  %1223:gpr = ADD %1222:gpr, %1212:gpr
  %1234:gpr = ADD %1233:gpr, %1223:gpr
  %1245:gpr = ADD %1244:gpr, %1234:gpr
  %1256:gpr = ADD %1255:gpr, %1245:gpr
  %1267:gpr = ADD %1266:gpr, %1256:gpr
  %1278:gpr = ADD %1277:gpr, %1267:gpr
  %77:gpr = ADD killed %1288:gpr, killed %1278:gpr
  %78:gpr = nuw ADDI %71:gpr, 8
  BNE %3114:gpr, %78:gpr, %bb.62
  PseudoBR %bb.63

Reassociation allows using multiple ALUs for this code.

At the same time I wasn't able to find any examples against inserting machine combiner a little later than other targets do.

[asi-sc]

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

1186

I'd like to comment why I don't follow the default path here (TargetInstrInfo::getMachineCombinerPatterns -> TargetInstrInfo::isReassociationCandidate -> ...). There are two main reasons:

• Generic implementation relies on isAssociativeAndCommutative function that takes MachineInstruction. Having it, we can check the opcode, fast math flags, etc. However, instructions are checked independently, so it's not possible to test if two instructions have equal rounding modes. Adding one more interface method required only by RISCV target doesn't seem to be a good idea.
• If we decide to introduce machine combiner for RISCV, I'd like to add patterns for FSUB instruction as well, e.g. (X + A) - Y => (X - Y) + A. These FSUB patterns need approximately the same checks as the patterns from this patch but cannot follow default (associative and commutative) path. It means that this code will be required anyway.

[craig.topper]

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

1211

Here's the patch where X86 removed the UnsafeFPMath check https://reviews.llvm.org/D74851

[asi-sc]

Small code refactoring and tests update (change 'fast' flag to 'reassoc')

[asi-sc]

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

1205

I looked into how quite similar transformations are implemented in InstCombine. Usually it's enough to check that the root instruction allows reassociation. So, we can think of relaxing this condition.

One more thing to mention is that other targets in addititon to FmReassoc checks FmNsz. I don't see any need to do this for fadd and fmul cases. Please, correct me if I'm wrong.

1211

Thanks for the link. Then I think we also should use only instructions flags.

[craig.topper]

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

1165

Should we intersect the MIFlags too like X86 and PowerPC do? Not sure where AArch64 doesn't.

1205

I don't think Nsz is needed, but I'd rather us be consistent with the other 3 targets that do this.

[asi-sc]

Make flags intersection for new instructions, add Nsz flag check.

[asi-sc]

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

1165

Thanks for drawing my attention to this. I think we should. As an improvement we can think of propagating some fast-math flags from the root instruction, but not in this patch.
I added flags intersection and two test cases.

[asi-sc]

Move FRM operands creation from setSpecialOperandAttr to finalizeInsInstrs

[craig.topper]

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

1156

auto *NewMi

1221

Use Added = true. No need for |

[asi-sc]

Addressed review comments

[asi-sc]

Rebase after D135776

[craig.topper]

LGTM

[asi-sc]

Rebase to fetch precommitted tests

[asi-sc]

We have to revert it because tests were not correctly updated by 'update_mir_test_checks.py' script. Fix in D136170. I'll regenerate tests and update the review soon.

[asi-sc]

Re-generate machine-combiner-mir.ll test checks

llvm/lib/Target/RISCV/RISCVInstrInfo.h

createMIROperandComment(const MachineInstr &MI, const MachineOperand &Op,
                           unsigned OpIdx,
                           const TargetRegisterInfo *TRI) const override;
 
   void getVLENFactoredAmount(
       MachineFunction &MF, MachineBasicBlock &MBB,
       MachineBasicBlock::iterator II, const DebugLoc &DL, Register DestReg,
       int64_t Amount, MachineInstr::MIFlag Flag = MachineInstr::NoFlags) const;
 
+  bool useMachineCombiner() const override { return true; }
+
+  void setSpecialOperandAttr(MachineInstr &OldMI1, MachineInstr &OldMI2,
+                             MachineInstr &NewMI1,
+                             MachineInstr &NewMI2) const override;
+  bool
+  getMachineCombinerPatterns(MachineInstr &Root,
+                             SmallVectorImpl<MachineCombinerPattern> &Patterns,
+                             bool DoRegPressureReduce) const override;
+
+  void
+  finalizeInsInstrs(MachineInstr &Root, MachineCombinerPattern &P,
+                    SmallVectorImpl<MachineInstr *> &InsInstrs) const override;
+
 protected:
   const RISCVSubtarget &STI;
 };
 
 namespace RISCV {
 
 // Returns true if this is the sext.w pattern, addiw rd, rs1, 0.
 bool isSEXT_W(const MachineInstr &MI);
 bool isZEXT_W(const MachineInstr &MI);
 bool isZEXT_B(const MachineInstr &MI);
 
 // Returns true if the given MI is an RVV instruction opcode for which we may
 // expect to see a FrameIndex operand.
 bool isRVVSpill(const MachineInstr &MI);
 
 Optional<std::pair<unsigned, unsigned>> isRVVSpillForZvlsseg(unsigned Opcode);
 
 bool isFaultFirstLoad(const MachineInstr &MI);
 
 // Implemented in RISCVGenInstrInfo.inc
 int16_t getNamedOperandIdx(uint16_t Opcode, uint16_t NamedIndex);
 
+// Return true if both input instructions have equal rounding mode. If at least
+// one of the instructions does not have rounding mode, false will be returned.
+bool hasEqualFRM(const MachineInstr &MI1, const MachineInstr &MI2);
+
 // Special immediate for AVL operand of V pseudo instructions to indicate VLMax.
 static constexpr int64_t VLMaxSentinel = -1LL;
 } // namespace RISCV
 
 namespace RISCVVPseudosTable {
 
 struct PseudoInfo {
   uint16_t Pseudo;

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

 #include "RISCVMachineFunctionInfo.h"
 #include "RISCVSubtarget.h"
 #include "RISCVTargetMachine.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Analysis/MemoryLocation.h"
 #include "llvm/CodeGen/LiveIntervals.h"
 #include "llvm/CodeGen/LiveVariables.h"
+#include "llvm/CodeGen/MachineCombinerPattern.h"
 #include "llvm/CodeGen/MachineFunctionPass.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/RegisterScavenging.h"
 #include "llvm/MC/MCInstBuilder.h"
 #include "llvm/MC/TargetRegistry.h"
 #include "llvm/Support/ErrorHandling.h"
 
case RISCV::FSGNJ_H:
     if (MI.getOperand(1).isReg() && MI.getOperand(2).isReg() &&
         MI.getOperand(1).getReg() == MI.getOperand(2).getReg())
       return DestSourcePair{MI.getOperand(0), MI.getOperand(1)};
     break;
   }
   return None;
 }
 
+void RISCVInstrInfo::setSpecialOperandAttr(MachineInstr &OldMI1,
+                                           MachineInstr &OldMI2,
+                                           MachineInstr &NewMI1,
+                                           MachineInstr &NewMI2) const {
+  uint16_t IntersectedFlags = OldMI1.getFlags() & OldMI2.getFlags();
+  NewMI1.setFlags(IntersectedFlags);
+  NewMI2.setFlags(IntersectedFlags);
+}
+
+void RISCVInstrInfo::finalizeInsInstrs(
+    MachineInstr &Root, MachineCombinerPattern &P,
+    SmallVectorImpl<MachineInstr *> &InsInstrs) const {
+  int16_t FrmOpIdx =
+      RISCV::getNamedOperandIdx(Root.getOpcode(), RISCV::OpName::frm);
+  if (FrmOpIdx < 0) {
+    assert(all_of(InsInstrs,
+                  [](MachineInstr *MI) {
+                    return RISCV::getNamedOperandIdx(MI->getOpcode(),
+                                                     RISCV::OpName::frm) < 0;
+                  }) &&
+           "New instructions require FRM whereas the old one does not have it");
+    return;
+  }
+
+  const MachineOperand &FRM = Root.getOperand(FrmOpIdx);
+  MachineFunction &MF = *Root.getMF();
+
+  for (auto *NewMI : InsInstrs) {
+    assert(static_cast<unsigned>(RISCV::getNamedOperandIdx(
+               NewMI->getOpcode(), RISCV::OpName::frm)) ==
+               NewMI->getNumOperands() &&
+           "Instruction has unexpected number of operands");
+    MachineInstrBuilder MIB(MF, NewMI);
+    MIB.add(FRM);
+    if (FRM.getImm() == RISCVFPRndMode::DYN)
+      MIB.addUse(RISCV::FRM, RegState::Implicit);
+  }
+}
+
+static bool isFADD(unsigned Opc) {
+  switch (Opc) {
+  default:
+    return false;
+  case RISCV::FADD_H:
+  case RISCV::FADD_S:
+  case RISCV::FADD_D:
+    return true;
+  }
+}
+
+static bool isFMUL(unsigned Opc) {
+  switch (Opc) {
+  default:
+    return false;
+  case RISCV::FMUL_H:
+  case RISCV::FMUL_S:
+  case RISCV::FMUL_D:
+    return true;
+  }
+}
+
+static bool isAssociativeAndCommutativeFPOpcode(unsigned Opc) {
+  return isFADD(Opc) || isFMUL(Opc);
+}
+
+static bool canReassociate(MachineInstr &Root, MachineOperand &MO) {
+  if (!MO.isReg() || !Register::isVirtualRegister(MO.getReg()))
+    return false;
+  MachineRegisterInfo &MRI = Root.getMF()->getRegInfo();
+  MachineInstr *MI = MRI.getVRegDef(MO.getReg());
+  if (!MI || !MRI.hasOneNonDBGUse(MO.getReg()))
+    return false;
+
+  if (MI->getOpcode() != Root.getOpcode())
+    return false;
+
+  if (!Root.getFlag(MachineInstr::MIFlag::FmReassoc) ||
+      !Root.getFlag(MachineInstr::MIFlag::FmNsz) ||
+      !MI->getFlag(MachineInstr::MIFlag::FmReassoc) ||
+      !MI->getFlag(MachineInstr::MIFlag::FmNsz))
+    return false;
+
+  return RISCV::hasEqualFRM(Root, *MI);
+}
+
+static bool
+getFPReassocPatterns(MachineInstr &Root,
+                     SmallVectorImpl<MachineCombinerPattern> &Patterns) {
+  bool Added = false;
+  if (canReassociate(Root, Root.getOperand(1))) {
+    Patterns.push_back(MachineCombinerPattern::REASSOC_AX_BY);
+    Patterns.push_back(MachineCombinerPattern::REASSOC_XA_BY);
+    Added = true;
+  }
+  if (canReassociate(Root, Root.getOperand(2))) {
+    Patterns.push_back(MachineCombinerPattern::REASSOC_AX_YB);
+    Patterns.push_back(MachineCombinerPattern::REASSOC_XA_YB);
+    Added = true;
+  }
+  return Added;
+}
+
+static bool getFPPatterns(MachineInstr &Root,
+                          SmallVectorImpl<MachineCombinerPattern> &Patterns) {
+  unsigned Opc = Root.getOpcode();
+  if (isAssociativeAndCommutativeFPOpcode(Opc))
+    return getFPReassocPatterns(Root, Patterns);
+  return false;
+}
+
+bool RISCVInstrInfo::getMachineCombinerPatterns(
+    MachineInstr &Root, SmallVectorImpl<MachineCombinerPattern> &Patterns,
+    bool DoRegPressureReduce) const {
+
+  if (getFPPatterns(Root, Patterns))
+    return true;
+
+  return TargetInstrInfo::getMachineCombinerPatterns(Root, Patterns,
+                                                     DoRegPressureReduce);
+}
+
 bool RISCVInstrInfo::verifyInstruction(const MachineInstr &MI,
                                        StringRef &ErrInfo) const {
   MCInstrDesc const &Desc = MI.getDesc();
 
   for (auto &OI : enumerate(Desc.operands())) {
     unsigned OpType = OI.value().OperandType;
     if (OpType >= RISCVOp::OPERAND_FIRST_RISCV_IMM &&
         OpType <= RISCVOp::OPERAND_LAST_RISCV_IMM) {
case RISCV::PseudoVRELOAD8_M1:
     return std::make_pair(8u, 1u);
   }
 }
 
 bool RISCV::isFaultFirstLoad(const MachineInstr &MI) {
   return MI.getNumExplicitDefs() == 2 && MI.modifiesRegister(RISCV::VL) &&
          !MI.isInlineAsm();
 }
+
+bool RISCV::hasEqualFRM(const MachineInstr &MI1, const MachineInstr &MI2) {
+  int16_t MI1FrmOpIdx =
+      RISCV::getNamedOperandIdx(MI1.getOpcode(), RISCV::OpName::frm);
+  int16_t MI2FrmOpIdx =
+      RISCV::getNamedOperandIdx(MI2.getOpcode(), RISCV::OpName::frm);
+  if (MI1FrmOpIdx < 0 || MI2FrmOpIdx < 0)
+    return false;
+  MachineOperand FrmOp1 = MI1.getOperand(MI1FrmOpIdx);
+  MachineOperand FrmOp2 = MI2.getOperand(MI2FrmOpIdx);
+  return FrmOp1.getImm() == FrmOp2.getImm();
+}

llvm/lib/Target/RISCV/RISCVTargetMachine.cpp

static cl::opt<bool> EnableRedundantCopyElimination(
     cl::desc("Enable the redundant copy elimination pass"), cl::init(true),
     cl::Hidden);
 
 // FIXME: Unify control over GlobalMerge.
 static cl::opt<cl::boolOrDefault>
     EnableGlobalMerge("riscv-enable-global-merge", cl::Hidden,
                       cl::desc("Enable the global merge pass"));
 
+static cl::opt<bool>
+    EnableMachineCombiner("riscv-enable-machine-combiner",
+                          cl::desc("Enable the machine combiner pass"),
+                          cl::init(true), cl::Hidden);
+
 extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeRISCVTarget() {
   RegisterTargetMachine<RISCVTargetMachine> X(getTheRISCV32Target());
   RegisterTargetMachine<RISCVTargetMachine> Y(getTheRISCV64Target());
   auto *PR = PassRegistry::getPassRegistry();
   initializeGlobalISel(*PR);
   initializeRISCVMakeCompressibleOptPass(*PR);
   initializeRISCVGatherScatterLoweringPass(*PR);
   initializeRISCVCodeGenPreparePass(*PR);
void RISCVPassConfig::addPreEmitPass2() {
   // Schedule the expansion of AMOs at the last possible moment, avoiding the
   // possibility for other passes to break the requirements for forward
   // progress in the LR/SC block.
   addPass(createRISCVExpandAtomicPseudoPass());
 }
 
 void RISCVPassConfig::addMachineSSAOptimization() {
   TargetPassConfig::addMachineSSAOptimization();
+  if (TM->getOptLevel() == CodeGenOpt::Aggressive && EnableMachineCombiner)
+    addPass(&MachineCombinerID);
 
   if (TM->getTargetTriple().getArch() == Triple::riscv64)
     addPass(createRISCVSExtWRemovalPass());
 }
 
 void RISCVPassConfig::addPreRegAlloc() {
   addPass(createRISCVPreRAExpandPseudoPass());
   if (TM->getOptLevel() != CodeGenOpt::None)

llvm/test/CodeGen/RISCV/O3-pipeline.ll

 ; CHECK-NEXT:       MachineDominator Tree Construction
 ; CHECK-NEXT:       Machine Block Frequency Analysis
 ; CHECK-NEXT:       Machine Common Subexpression Elimination
 ; CHECK-NEXT:       MachinePostDominator Tree Construction
 ; CHECK-NEXT:       Machine Cycle Info Analysis
 ; CHECK-NEXT:       Machine code sinking
 ; CHECK-NEXT:       Peephole Optimizations
 ; CHECK-NEXT:       Remove dead machine instructions
+; CHECK-NEXT:       Machine Trace Metrics
+; CHECK-NEXT:       Lazy Machine Block Frequency Analysis
+; CHECK-NEXT:       Machine InstCombiner
 ; RV64-NEXT:        RISCV sext.w Removal
 ; CHECK-NEXT:       RISCV Pre-RA pseudo instruction expansion pass
 ; CHECK-NEXT:       RISCV Merge Base Offset
 ; CHECK-NEXT:       RISCV Insert VSETVLI pass
 ; CHECK-NEXT:       Detect Dead Lanes
 ; CHECK-NEXT:       Process Implicit Definitions
 ; CHECK-NEXT:       Remove unreachable machine basic blocks
 ; CHECK-NEXT:       Live Variable Analysis

llvm/test/CodeGen/RISCV/machine-combiner-mir.ll

+; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+; RUN: llc -mtriple=riscv64 -mattr=+d -verify-machineinstrs -mcpu=sifive-u74 \
+; RUN: -O3 -enable-unsafe-fp-math -riscv-enable-machine-combiner=true        \
+; RUN: -stop-after machine-combiner < %s | FileCheck %s
+
+define double @test_reassoc_fadd1(double %a0, double %a1, double %a2, double %a3) {
+  ; CHECK-LABEL: name: test_reassoc_fadd1
+  ; CHECK: bb.0 (%ir-block.0):
+  ; CHECK-NEXT:   liveins: $f10_d, $f11_d, $f12_d, $f13_d
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:fpr64 = COPY $f13_d
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:fpr64 = COPY $f12_d
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:fpr64 = COPY $f11_d
+  ; CHECK-NEXT:   [[COPY3:%[0-9]+]]:fpr64 = COPY $f10_d
+  ; CHECK-NEXT:   [[FADD_D:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FADD_D [[COPY3]], [[COPY2]], 7, implicit $frm
+  ; CHECK-NEXT:   [[FADD_D1:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FADD_D [[COPY1]], [[COPY]], 7, implicit $frm
+  ; CHECK-NEXT:   [[FADD_D2:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FADD_D killed [[FADD_D]], killed [[FADD_D1]], 7, implicit $frm
+  ; CHECK-NEXT:   $f10_d = COPY [[FADD_D2]]
+  ; CHECK-NEXT:   PseudoRET implicit $f10_d
+  %t0 = fadd nsz reassoc double %a0, %a1
+  %t1 = fadd nsz reassoc double %t0, %a2
+  %t2 = fadd nsz reassoc double %t1, %a3
+  ret double %t2
+}
+
+define double @test_reassoc_fmul1(double %a0, double %a1, double %a2, double %a3) {
+  ; CHECK-LABEL: name: test_reassoc_fmul1
+  ; CHECK: bb.0 (%ir-block.0):
+  ; CHECK-NEXT:   liveins: $f10_d, $f11_d, $f12_d, $f13_d
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:fpr64 = COPY $f13_d
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:fpr64 = COPY $f12_d
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:fpr64 = COPY $f11_d
+  ; CHECK-NEXT:   [[COPY3:%[0-9]+]]:fpr64 = COPY $f10_d
+  ; CHECK-NEXT:   [[FMUL_D:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FMUL_D [[COPY3]], [[COPY2]], 7, implicit $frm
+  ; CHECK-NEXT:   [[FMUL_D1:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FMUL_D [[COPY1]], [[COPY]], 7, implicit $frm
+  ; CHECK-NEXT:   [[FMUL_D2:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FMUL_D killed [[FMUL_D]], killed [[FMUL_D1]], 7, implicit $frm
+  ; CHECK-NEXT:   $f10_d = COPY [[FMUL_D2]]
+  ; CHECK-NEXT:   PseudoRET implicit $f10_d
+  %t0 = fmul nsz reassoc double %a0, %a1
+  %t1 = fmul nsz reassoc double %t0, %a2
+  %t2 = fmul nsz reassoc double %t1, %a3
+  ret double %t2
+}
+
+; Verify flags intersection
+define double @test_reassoc_flags1(double %a0, double %a1, double %a2, double %a3) {
+  ; CHECK-LABEL: name: test_reassoc_flags1
+  ; CHECK: bb.0 (%ir-block.0):
+  ; CHECK-NEXT:   liveins: $f10_d, $f11_d, $f12_d, $f13_d
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:fpr64 = COPY $f13_d
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:fpr64 = COPY $f12_d
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:fpr64 = COPY $f11_d
+  ; CHECK-NEXT:   [[COPY3:%[0-9]+]]:fpr64 = COPY $f10_d
+  ; CHECK-NEXT:   [[FADD_D:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FADD_D [[COPY3]], [[COPY2]], 7, implicit $frm
+  ; CHECK-NEXT:   [[FADD_D1:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FADD_D [[COPY1]], [[COPY]], 7, implicit $frm
+  ; CHECK-NEXT:   [[FADD_D2:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FADD_D killed [[FADD_D]], killed [[FADD_D1]], 7, implicit $frm
+  ; CHECK-NEXT:   $f10_d = COPY [[FADD_D2]]
+  ; CHECK-NEXT:   PseudoRET implicit $f10_d
+  %t0 = fadd nsz reassoc double %a0, %a1
+  %t1 = fadd contract nsz reassoc double %t0, %a2
+  %t2 = fadd nsz reassoc double %t1, %a3
+  ret double %t2
+}
+
+; Verify flags intersection
+define double @test_reassoc_flags2(double %a0, double %a1, double %a2, double %a3) {
+  ; CHECK-LABEL: name: test_reassoc_flags2
+  ; CHECK: bb.0 (%ir-block.0):
+  ; CHECK-NEXT:   liveins: $f10_d, $f11_d, $f12_d, $f13_d
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:fpr64 = COPY $f13_d
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:fpr64 = COPY $f12_d
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:fpr64 = COPY $f11_d
+  ; CHECK-NEXT:   [[COPY3:%[0-9]+]]:fpr64 = COPY $f10_d
+  ; CHECK-NEXT:   [[FADD_D:%[0-9]+]]:fpr64 = nsz reassoc nofpexcept FADD_D [[COPY3]], [[COPY2]], 7, implicit $frm
+  ; CHECK-NEXT:   [[FADD_D1:%[0-9]+]]:fpr64 = nsz contract reassoc nofpexcept FADD_D [[COPY1]], [[COPY]], 7, implicit $frm
+  ; CHECK-NEXT:   [[FADD_D2:%[0-9]+]]:fpr64 = nsz contract reassoc nofpexcept FADD_D killed [[FADD_D]], killed [[FADD_D1]], 7, implicit $frm
+  ; CHECK-NEXT:   $f10_d = COPY [[FADD_D2]]
+  ; CHECK-NEXT:   PseudoRET implicit $f10_d
+  %t0 = fadd nsz reassoc double %a0, %a1
+  %t1 = fadd contract nsz reassoc double %t0, %a2
+  %t2 = fadd contract nsz reassoc double %t1, %a3
+  ret double %t2
+}

llvm/test/CodeGen/RISCV/machine-combiner.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=riscv64 -mattr=+d -verify-machineinstrs -mcpu=sifive-u74 \
-; RUN: -O3 -enable-unsafe-fp-math  < %s | FileCheck %s --check-prefixes=CHECK
+; RUN: -O3 -enable-unsafe-fp-math -riscv-enable-machine-combiner=true < %s | \
+; RUN: FileCheck %s
 
 define double @test_reassoc_fadd1(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fadd1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fadd.d ft0, fa0, fa1
-; CHECK-NEXT:    fadd.d ft0, ft0, fa2
-; CHECK-NEXT:    fadd.d fa0, ft0, fa3
+; CHECK-NEXT:    fadd.d ft1, fa2, fa3
+; CHECK-NEXT:    fadd.d fa0, ft0, ft1
 ; CHECK-NEXT:    ret
   %t0 = fadd nsz reassoc double %a0, %a1
   %t1 = fadd nsz reassoc double %t0, %a2
   %t2 = fadd nsz reassoc double %t1, %a3
   ret double %t2
 }
 
 define double @test_reassoc_fadd2(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fadd2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fadd.d ft0, fa0, fa1
-; CHECK-NEXT:    fadd.d ft0, fa2, ft0
-; CHECK-NEXT:    fadd.d fa0, ft0, fa3
+; CHECK-NEXT:    fadd.d ft1, fa2, fa3
+; CHECK-NEXT:    fadd.d fa0, ft0, ft1
 ; CHECK-NEXT:    ret
   %t0 = fadd nsz reassoc double %a0, %a1
   %t1 = fadd nsz reassoc double %a2, %t0
   %t2 = fadd nsz reassoc double %t1, %a3
   ret double %t2
 }
 
 define double @test_reassoc_fadd3(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fadd3:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fadd.d ft0, fa0, fa1
-; CHECK-NEXT:    fadd.d ft0, ft0, fa2
-; CHECK-NEXT:    fadd.d fa0, fa3, ft0
+; CHECK-NEXT:    fadd.d ft1, fa2, fa3
+; CHECK-NEXT:    fadd.d fa0, ft0, ft1
 ; CHECK-NEXT:    ret
   %t0 = fadd nsz reassoc double %a0, %a1
   %t1 = fadd nsz reassoc double %t0, %a2
   %t2 = fadd nsz reassoc double %a3, %t1
   ret double %t2
 }
 
 define double @test_reassoc_fadd4(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fadd4:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fadd.d ft0, fa0, fa1
-; CHECK-NEXT:    fadd.d ft0, fa2, ft0
-; CHECK-NEXT:    fadd.d fa0, fa3, ft0
+; CHECK-NEXT:    fadd.d ft1, fa2, fa3
+; CHECK-NEXT:    fadd.d fa0, ft0, ft1
 ; CHECK-NEXT:    ret
   %t0 = fadd nsz reassoc double %a0, %a1
   %t1 = fadd nsz reassoc double %a2, %t0
   %t2 = fadd nsz reassoc double %a3, %t1
   ret double %t2
 }
 
 define double @test_reassoc_fmul1(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fmul1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fmul.d ft0, fa0, fa1
-; CHECK-NEXT:    fmul.d ft0, ft0, fa2
-; CHECK-NEXT:    fmul.d fa0, ft0, fa3
+; CHECK-NEXT:    fmul.d ft1, fa2, fa3
+; CHECK-NEXT:    fmul.d fa0, ft0, ft1
 ; CHECK-NEXT:    ret
   %t0 = fmul nsz reassoc double %a0, %a1
   %t1 = fmul nsz reassoc double %t0, %a2
   %t2 = fmul nsz reassoc double %t1, %a3
   ret double %t2
 }
 
 define double @test_reassoc_fmul2(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fmul2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fmul.d ft0, fa0, fa1
-; CHECK-NEXT:    fmul.d ft0, fa2, ft0
-; CHECK-NEXT:    fmul.d fa0, ft0, fa3
+; CHECK-NEXT:    fmul.d ft1, fa2, fa3
+; CHECK-NEXT:    fmul.d fa0, ft0, ft1
 ; CHECK-NEXT:    ret
   %t0 = fmul nsz reassoc double %a0, %a1
   %t1 = fmul nsz reassoc double %a2, %t0
   %t2 = fmul nsz reassoc double %t1, %a3
   ret double %t2
 }
 
 define double @test_reassoc_fmul3(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fmul3:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fmul.d ft0, fa0, fa1
-; CHECK-NEXT:    fmul.d ft0, ft0, fa2
-; CHECK-NEXT:    fmul.d fa0, fa3, ft0
+; CHECK-NEXT:    fmul.d ft1, fa2, fa3
+; CHECK-NEXT:    fmul.d fa0, ft0, ft1
 ; CHECK-NEXT:    ret
   %t0 = fmul nsz reassoc double %a0, %a1
   %t1 = fmul nsz reassoc double %t0, %a2
   %t2 = fmul nsz reassoc double %a3, %t1
   ret double %t2
 }
 
 define double @test_reassoc_fmul4(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fmul4:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fmul.d ft0, fa0, fa1
-; CHECK-NEXT:    fmul.d ft0, fa2, ft0
-; CHECK-NEXT:    fmul.d fa0, fa3, ft0
+; CHECK-NEXT:    fmul.d ft1, fa2, fa3
+; CHECK-NEXT:    fmul.d fa0, ft0, ft1
 ; CHECK-NEXT:    ret
   %t0 = fmul nsz reassoc double %a0, %a1
   %t1 = fmul nsz reassoc double %a2, %t0
   %t2 = fmul nsz reassoc double %a3, %t1
   ret double %t2
 }
 
 define double @test_reassoc_big1(double %a0, double %a1, double %a2, double %a3, double %a4, double %a5, double %a6) {
 ; CHECK-LABEL: test_reassoc_big1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fadd.d ft0, fa0, fa1
-; CHECK-NEXT:    fadd.d ft0, ft0, fa2
-; CHECK-NEXT:    fadd.d ft0, ft0, fa3
-; CHECK-NEXT:    fadd.d ft0, ft0, fa4
-; CHECK-NEXT:    fadd.d ft0, ft0, fa5
-; CHECK-NEXT:    fadd.d fa0, ft0, fa6
+; CHECK-NEXT:    fadd.d ft1, fa2, fa3
+; CHECK-NEXT:    fadd.d ft2, fa4, fa5
+; CHECK-NEXT:    fadd.d ft0, ft0, ft1
+; CHECK-NEXT:    fadd.d ft1, ft2, fa6
+; CHECK-NEXT:    fadd.d fa0, ft0, ft1
 ; CHECK-NEXT:    ret
   %t0 = fadd nsz reassoc double %a0, %a1
   %t1 = fadd nsz reassoc double %t0, %a2
   %t2 = fadd nsz reassoc double %t1, %a3
   %t3 = fadd nsz reassoc double %t2, %a4
   %t4 = fadd nsz reassoc double %t3, %a5
   %t5 = fadd nsz reassoc double %t4, %a6
   ret double %t5
 }
 
 define double @test_reassoc_big2(double %a0, double %a1, i32 %a2, double %a3, i32 %a4, double %a5) {
 ; CHECK-LABEL: test_reassoc_big2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fcvt.d.w ft0, a0
-; CHECK-NEXT:    fmul.d ft0, fa2, ft0
 ; CHECK-NEXT:    fcvt.d.w ft1, a1
-; CHECK-NEXT:    fmul.d ft0, ft1, ft0
-; CHECK-NEXT:    fadd.d ft1, fa0, fa1
-; CHECK-NEXT:    fmul.d ft0, ft0, fa1
-; CHECK-NEXT:    fadd.d ft1, fa2, ft1
-; CHECK-NEXT:    fadd.d ft2, fa2, fa1
-; CHECK-NEXT:    fmul.d ft0, ft0, fa0
+; CHECK-NEXT:    fmul.d ft0, fa2, ft0
+; CHECK-NEXT:    fmul.d ft1, ft1, fa1
+; CHECK-NEXT:    fadd.d ft2, fa0, fa1
+; CHECK-NEXT:    fadd.d ft3, fa2, fa1
+; CHECK-NEXT:    fmul.d ft0, ft0, ft1
+; CHECK-NEXT:    fadd.d ft1, fa2, ft2
+; CHECK-NEXT:    fmul.d ft2, fa0, ft3
 ; CHECK-NEXT:    fsub.d ft1, fa3, ft1
 ; CHECK-NEXT:    fmul.d ft0, ft0, ft2
 ; CHECK-NEXT:    fmul.d fa0, ft1, ft0
 ; CHECK-NEXT:    ret
   %cvt1 = sitofp i32 %a2 to double
   %cvt2 = sitofp i32 %a4 to double
   %t5 = fmul nsz reassoc double %a3, %cvt1
   %t9 = fmul nsz reassoc double %cvt2, %t5
   %t4 = fmul nsz reassoc double %t9, %a1
   %t0 = fadd nsz reassoc double %a0, %a1
   %t1 = fadd nsz reassoc double %a3, %t0
   %t3 = fadd nsz reassoc double %a3, %a1
   %t6 = fmul nsz reassoc double %t4, %a0
   %t2 = fsub nsz reassoc double %a5, %t1
   %t7 = fmul nsz reassoc double %t6, %t3
   %t8 = fmul nsz reassoc double %t2, %t7
   ret double %t8
 }
 
+; Negative test
 define double @test_reassoc_fadd_flags_1(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fadd_flags_1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fadd.d ft0, fa0, fa1
 ; CHECK-NEXT:    fadd.d ft0, ft0, fa2
 ; CHECK-NEXT:    fadd.d fa0, ft0, fa3
 ; CHECK-NEXT:    ret
   %t0 = fadd nsz reassoc double %a0, %a1
   %t1 = fadd double %t0, %a2
   %t2 = fadd nsz reassoc double %t1, %a3
   ret double %t2
 }
 
+; Negative test
 define double @test_reassoc_fadd_flags_2(double %a0, double %a1, double %a2, double %a3) {
 ; CHECK-LABEL: test_reassoc_fadd_flags_2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    fadd.d ft0, fa0, fa1
 ; CHECK-NEXT:    fadd.d ft0, ft0, fa2
 ; CHECK-NEXT:    fadd.d fa0, ft0, fa3
 ; CHECK-NEXT:    ret
   %t0 = fadd nsz reassoc double %a0, %a1
   %t1 = fadd nsz reassoc double %t0, %a2
   %t2 = fadd double %t1, %a3
   ret double %t2
 }