[AMDGPU][GlobalISel] Transform (fsub (fmul x, y), z) -> (fma x, y, -z)

[matejam]

Summary

Instead of sub and mul instructions, use v_mad, v_mac or v_fma if fma
instructions are faster and are legal for the given architecture.
Combiner for a simple case that has only one subtraction and one
multiplication instruction and transforms them into some of the fma
instructions depending on the architecture.

[arsenm]

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

4085–4086

This should probably allow vectors we can break down later too

4088

Don't see where isFMADLegal is fedined

4107–4119

I'm not sure I follow this heuristic, or what SwapPriority means

4160–4161

The types are all identical, there's no reason to query every type

4164

You can directly use the type and avoid the explicit createGenericVirtualRegister with
auto Neg = B.buildFNeg(Ty, X)

llvm/test/CodeGen/AMDGPU/GlobalISel/combine-fma-sub-mul.ll

118–119

Why did we fail to fold the modifier here?

[matejam]

Added support for vector types and some refactoring.

[matejam]

Thanks for the suggestions!

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

4088

In the other revision (the parent): D93305

4107–4119

If SwapPriority is equal to 0 that means that the first and second operands aren't both fmul instructions, if it's equal to 2 it means that both of the arguments are fmul and that the second arg has fewer uses so we pick him for folding, vice versa if it's equal to 1. I will make it more simple in the next version.

[matejam]

This comment was removed by matejam.

[matejam]

Put back the accidentally deleted combiner from the list of combiners (load_or_combine).

[matejam]

Minor changes in CombinerHelper.cpp and in the tests.

[matejam]

Use mi_match for comparing instructions instead of comparing the opcodes.

[foad]

Does this assume that all targets can do the fneg for free? Or can a target choose to fold only (fadd (fmul x, y), z), not (fsub (fmul x, y), z) ?

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

4438

Typo "refers", and they're called MI0 and MI1.

4443

Use hasMoreUses() from the previous patch.

[matejam]

In D96614#2787052, @foad wrote:

Does this assume that all targets can do the fneg for free? Or can a target choose to fold only (fadd (fmul x, y), z), not (fsub (fmul x, y), z) ?

Similar to SelectionDAG which negates an SDValue:

fold (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
DAG.getNode(PreferredFusedOpcode, SL, VT, XY.getOperand(0),
            XY.getOperand(1), DAG.getNode(ISD::FNEG, SL, VT, Z));

[matejam]

Typos and refactoring.

[matejam]

Rebased.

[matejam]

Use m_MInstr instead of m_Reg in matching patterns (mi_match).
A few minor bug fixes.
Formatting and refactoring.

[matejam]

Use applyBuildFn instead of writing my own apply.

[mbrkusanin]

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

5104–5108

Rename Src3 to NegZ so it matches the comment above.
Use B not Builder.

5117–5121

Rename Src1 to NegY so it matches the comment above.
Use B not Builder.

[matejam]

Changes in tests.

[mbrkusanin]

clang-format for CombinerHelper.cpp

llvm/test/CodeGen/AMDGPU/GlobalISel/combine-fma-sub-mul.ll

203

%a and %z should be swapped here, otherwise this is the same test as the one above.

Also combiner fails for this test for -mcpu=gfx900 --denormal-fp-math=preserve-sign.

Same for test above (test_half_sub_mul). It produces correct result only because fsub is replaced by fadd + fneg in legalizer and then is probably matched by one of other combiners that start from fadd.

680

Same here, swap %a and %z.

[matejam]

Minor changes in tests.

[mbrkusanin]

LGTM

[matejam]

Added missing comments and an FMA combiner group in table gen.

llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h

bool matchCombineFAddFMAFMulToFMadOrFMA(MachineInstr &MI,
                                           BuildFnTy &MatchInfo);
 
   // Transform (fadd (fma x, y, (fpext (fmul u, v))), z)
   //            -> (fma x, y, (fma (fpext u), (fpext v), z))
   //            -> (fmad x, y, (fma (fpext u), (fpext v), z))
   bool matchCombineFAddFpExtFMulToFMadOrFMAAggressive(MachineInstr &MI,
                                                       BuildFnTy &MatchInfo);
 
+  /// Transform (fsub (fmul x, y), z) -> (fma x, y, -z)
+  ///                                 -> (fmad x, y, -z)
+  bool matchCombineFSubFMulToFMadOrFMA(MachineInstr &MI, BuildFnTy &MatchInfo);
+
   /// Transform trunc ([asz]ext x) to x or ([asz]ext x) or (trunc x).
   bool matchCombineTruncOfExt(MachineInstr &MI,
                               std::pair<Register, unsigned> &MatchInfo);
   void applyCombineTruncOfExt(MachineInstr &MI,
                               std::pair<Register, unsigned> &MatchInfo);
 
   /// Transform trunc (shl x, K) to shl (trunc x),
   /// K => K < VT.getScalarSizeInBits().

llvm/include/llvm/Target/GlobalISel/Combine.td

 //           (fma x, y, (fma (fpext u), (fpext v), z))
 def combine_fadd_fpext_fma_fmul_to_fmad_or_fma: GICombineRule<
   (defs root:$root, build_fn_matchinfo:$info),
   (match (wip_match_opcode G_FADD):$root,
          [{ return Helper.matchCombineFAddFpExtFMulToFMadOrFMAAggressive(
                                                   *${root}, ${info}); }]),
   (apply [{ Helper.applyBuildFn(*${root}, ${info}); }])>;
 
+// Transform (fsub (fmul x, y), z) -> (fma x, y, -z)
+def combine_fsub_fmul_to_fmad_or_fma: GICombineRule<
+  (defs root:$root, build_fn_matchinfo:$info),
+  (match (wip_match_opcode G_FSUB):$root,
+         [{ return Helper.matchCombineFSubFMulToFMadOrFMA(*${root},
+                                                          ${info}); }]),
+  (apply [{ Helper.applyBuildFn(*${root}, ${info}); }])>;
+
 // Currently only the one combine above.
 def insert_vec_elt_combines : GICombineGroup<
                             [combine_insert_vec_elts_build_vector]>;
 
 def extract_vec_elt_build_vec : GICombineRule<
   (defs root:$root, register_matchinfo:$matchinfo),
   (match (wip_match_opcode G_EXTRACT_VECTOR_ELT):$root,
     [{ return Helper.matchExtractVecEltBuildVec(*${root}, ${matchinfo}); }]),
def all_combines : GICombineGroup<[trivial_combines, insert_vec_elt_combines,
     unmerge_zext_to_zext, merge_unmerge, trunc_ext_fold, trunc_shl,
     const_combines, xor_of_and_with_same_reg, ptr_add_with_zero,
     shift_immed_chain, shift_of_shifted_logic_chain, load_or_combine,
     truncstore_merge, div_rem_to_divrem, funnel_shift_combines,
     form_bitfield_extract, constant_fold, fabs_fneg_fold,
     intdiv_combines, mulh_combines, redundant_neg_operands,
     combine_fadd_fmul_to_fmad_or_fma, combine_fadd_fpext_fmul_to_fmad_or_fma,
     combine_fadd_fma_fmul_to_fmad_or_fma,
-    combine_fadd_fpext_fma_fmul_to_fmad_or_fma]>;
+    combine_fadd_fpext_fma_fmul_to_fmad_or_fma,
+    combine_fsub_fmul_to_fmad_or_fma]>;
 
 // A combine group used to for prelegalizer combiners at -O0. The combines in
 // this group have been selected based on experiments to balance code size and
 // compile time performance.
 def optnone_combines : GICombineGroup<[trivial_combines,
     ptr_add_immed_chain, combines_for_extload,
     not_cmp_fold, opt_brcond_by_inverting_cond]>;

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

 }
 
 bool CombinerHelper::matchBitfieldExtractFromShr(
     MachineInstr &MI, std::function<void(MachineIRBuilder &)> &MatchInfo) {
   const unsigned Opcode = MI.getOpcode();
   assert(Opcode == TargetOpcode::G_ASHR || Opcode == TargetOpcode::G_LSHR);
 
   const Register Dst = MI.getOperand(0).getReg();
 
   const unsigned ExtrOpcode = Opcode == TargetOpcode::G_ASHR
                                   ? TargetOpcode::G_SBFX
                                   : TargetOpcode::G_UBFX;
 
   // Check if the type we would use for the extract is legal
   LLT Ty = MRI.getType(Dst);
   LLT ExtractTy = getTargetLowering().getPreferredShiftAmountTy(Ty);
   if (!LI || !LI->isLegalOrCustom({ExtrOpcode, {Ty, ExtractTy}}))
     return false;
 
   Register ShlSrc;
   int64_t ShrAmt;
   int64_t ShlAmt;
   const unsigned Size = Ty.getScalarSizeInBits();
 
   // Try to match shr (shl x, c1), c2
   if (!mi_match(Dst, MRI,
                 m_BinOp(Opcode,
                         m_OneNonDBGUse(m_GShl(m_Reg(ShlSrc), m_ICst(ShlAmt))),
                         m_ICst(ShrAmt))))
     return false;
 
   // Make sure that the shift sizes can fit a bitfield extract
   if (ShlAmt < 0 || ShlAmt > ShrAmt || ShrAmt >= Size)
     return false;
 
   // Skip this combine if the G_SEXT_INREG combine could handle it
   if (Opcode == TargetOpcode::G_ASHR && ShlAmt == ShrAmt)
     return false;
 
   // Calculate start position and width of the extract
   const int64_t Pos = ShrAmt - ShlAmt;
   const int64_t Width = Size - ShrAmt;
 
   MatchInfo = [=](MachineIRBuilder &B) {
     auto WidthCst = B.buildConstant(ExtractTy, Width);
     auto PosCst = B.buildConstant(ExtractTy, Pos);
     B.buildInstr(ExtrOpcode, {Dst}, {ShlSrc, PosCst, WidthCst});
   };
   return true;
 }
 
bool CombinerHelper::matchBitfieldExtractFromShrAnd(
 
   // Check that ubfx can do the extraction, with no holes in the mask.
   uint64_t UMask = SMask;
   UMask |= maskTrailingOnes<uint64_t>(ShrAmt);
   UMask &= maskTrailingOnes<uint64_t>(Size);
   if (!isMask_64(UMask))
     return false;
 
   // Calculate start position and width of the extract.
   const int64_t Pos = ShrAmt;
   const int64_t Width = countTrailingOnes(UMask) - ShrAmt;
 
   // It's preferable to keep the shift, rather than form G_SBFX.
   // TODO: remove the G_AND via demanded bits analysis.
   if (Opcode == TargetOpcode::G_ASHR && Width + ShrAmt == Size)
     return false;
 
   MatchInfo = [=](MachineIRBuilder &B) {
     auto WidthCst = B.buildConstant(Ty, Width);
     auto PosCst = B.buildConstant(Ty, Pos);
     B.buildInstr(TargetOpcode::G_UBFX, {Dst}, {AndSrc, PosCst, WidthCst});
bool CombinerHelper::matchNarrowBinopFeedingAnd(
   unsigned NarrowWidth = Mask.countTrailingOnes();
   if (NarrowWidth == WideTy.getSizeInBits())
     return false;
   LLT NarrowTy = LLT::scalar(NarrowWidth);
 
   // Check if adding the zext + truncates could be harmful.
   auto &MF = *MI.getMF();
   const auto &TLI = getTargetLowering();
   LLVMContext &Ctx = MF.getFunction().getContext();
   auto &DL = MF.getDataLayout();
   if (!TLI.isTruncateFree(WideTy, NarrowTy, DL, Ctx) ||
       !TLI.isZExtFree(NarrowTy, WideTy, DL, Ctx))
     return false;
   if (!isLegalOrBeforeLegalizer({TargetOpcode::G_TRUNC, {NarrowTy, WideTy}}) ||
       !isLegalOrBeforeLegalizer({TargetOpcode::G_ZEXT, {WideTy, NarrowTy}}))
     return false;
   Register BinOpLHS = LHSInst->getOperand(1).getReg();
   Register BinOpRHS = LHSInst->getOperand(2).getReg();
   MatchInfo = [=, &MI](MachineIRBuilder &B) {
     auto NarrowLHS = Builder.buildTrunc(NarrowTy, BinOpLHS);
     auto NarrowRHS = Builder.buildTrunc(NarrowTy, BinOpRHS);
     auto NarrowBinOp =
if (isContractableFMul(*FMulMI, AllowFusionGlobally) &&
       };
       return true;
     }
   }
 
   return false;
 }
 
+bool CombinerHelper::matchCombineFSubFMulToFMadOrFMA(
+    MachineInstr &MI,
+    std::function<void(MachineIRBuilder &)> &MatchInfo) {
+  assert(MI.getOpcode() == TargetOpcode::G_FSUB);
+
+  bool AllowFusionGlobally, HasFMAD, Aggressive;
+  if (!canCombineFMadOrFMA(MI, AllowFusionGlobally, HasFMAD, Aggressive))
+    return false;
+
+  MachineInstr *LHS = MRI.getVRegDef(MI.getOperand(1).getReg());
+  MachineInstr *RHS = MRI.getVRegDef(MI.getOperand(2).getReg());
+  LLT DstTy = MRI.getType(MI.getOperand(0).getReg());
+
+  // If we have two choices trying to fold (fadd (fmul u, v), (fmul x, y)),
+  // prefer to fold the multiply with fewer uses.
+  int FirstMulHasFewerUses = true;
+  if (isContractableFMul(*LHS, AllowFusionGlobally) &&
+      isContractableFMul(*RHS, AllowFusionGlobally) &&
+      hasMoreUses(*LHS, *RHS, MRI))
+    FirstMulHasFewerUses = false;
+
+  unsigned PreferredFusedOpcode =
+      HasFMAD ? TargetOpcode::G_FMAD : TargetOpcode::G_FMA;
+
+  // fold (fsub (fmul x, y), z) -> (fma x, y, -z)
+  if (FirstMulHasFewerUses &&
+      (isContractableFMul(*LHS, AllowFusionGlobally) &&
+       (Aggressive || MRI.hasOneNonDBGUse(LHS->getOperand(0).getReg())))) {
+    MatchInfo = [=, &MI](MachineIRBuilder &B) {
+      Register NegZ = B.buildFNeg(DstTy, RHS->getOperand(0).getReg())
+                       .getReg(0);
+      B.buildInstr(PreferredFusedOpcode, {MI.getOperand(0).getReg()},
+                   {LHS->getOperand(1).getReg(), LHS->getOperand(2).getReg(),
+                    NegZ});
+    };
+
+    return true;
+  }
+  // fold (fsub x, (fmul y, z)) -> (fma -y, z, x)
+  else if ((isContractableFMul(*RHS, AllowFusionGlobally) &&
+       (Aggressive || MRI.hasOneNonDBGUse(RHS->getOperand(0).getReg())))) {
+    MatchInfo = [=, &MI](MachineIRBuilder &B) {
+      Register NegY = B.buildFNeg(DstTy, RHS->getOperand(1).getReg())
+                       .getReg(0);
+      B.buildInstr(PreferredFusedOpcode, {MI.getOperand(0).getReg()},
+                   {NegY, RHS->getOperand(2).getReg(),
+                    LHS->getOperand(0).getReg()});
+    };
+
+    return true;
+  }
+
+  return false;
+}
+
 bool CombinerHelper::tryCombine(MachineInstr &MI) {
   if (tryCombineCopy(MI))
     return true;
   if (tryCombineExtendingLoads(MI))
     return true;
   if (tryCombineIndexedLoadStore(MI))
     return true;
   return false;
 }

llvm/test/CodeGen/AMDGPU/GlobalISel/combine-fma-sub-mul.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx900 < %s | FileCheck -check-prefix=GFX9 %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx900 -fp-contract=fast < %s | FileCheck -check-prefix=GFX9-CONTRACT %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx900 --denormal-fp-math=preserve-sign < %s | FileCheck -check-prefix=GFX9-DENORM %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx1010 < %s | FileCheck -check-prefix=GFX10 %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx1010 -fp-contract=fast < %s | FileCheck -check-prefix=GFX10-CONTRACT %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx1010 --denormal-fp-math=preserve-sign < %s | FileCheck -check-prefix=GFX10-DENORM %s
+
+; fold (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
+; fold (fsub x, (fmul y, z)) -> (fma (fneg y), z, x)
+
+define float @test_f32_sub_mul(float %x, float %y, float %z) {
+; GFX9-LABEL: test_f32_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f32_e32 v0, v0, v1
+; GFX9-NEXT:    v_sub_f32_e32 v0, v0, v2
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_f32_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v0, v0, v1, -v2
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_f32_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_f32 v0, v0, v1, -v2
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_f32_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f32_e32 v0, v0, v1
+; GFX10-NEXT:    v_sub_f32_e32 v0, v0, v2
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_f32_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v0, v0, v1, -v2
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_f32_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mad_f32 v0, v0, v1, -v2
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul float %x, %y
+  %b = fsub float %a, %z
+  ret float %b
+}
+
+define float @test_f32_sub_mul_rhs(float %x, float %y, float %z) {
+; GFX9-LABEL: test_f32_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f32_e32 v0, v0, v1
+; GFX9-NEXT:    v_sub_f32_e32 v0, v2, v0
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_f32_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v0, -v0, v1, v2
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_f32_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_f32 v0, -v0, v1, v2
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_f32_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f32_e32 v0, v0, v1
+; GFX10-NEXT:    v_sub_f32_e32 v0, v2, v0
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_f32_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v0, -v0, v1, v2
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_f32_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mad_f32 v0, -v0, v1, v2
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul float %x, %y
+  %b = fsub float %z, %a
+  ret float %b
+}
+
+define half @test_half_sub_mul(half %x, half %y, half %z) {
+; GFX9-LABEL: test_half_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX9-NEXT:    v_add_f16_e64 v0, v0, -v2
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_half_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_xor_b32_e32 v2, 0x8000, v2
+; GFX9-CONTRACT-NEXT:    v_fma_f16 v0, v0, v1, v2
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_half_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_legacy_f16 v0, v0, v1, -v2
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_half_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX10-NEXT:    v_add_f16_e64 v0, v0, -v2
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_half_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_xor_b32_e32 v2, 0x8000, v2
+; GFX10-CONTRACT-NEXT:    v_fma_f16 v0, v0, v1, v2
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_half_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v0, v0, -v2
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul half %x, %y
+  %b = fsub half %a, %z
+  ret half %b
+}
+
+define half @test_half_sub_mul_rhs(half %x, half %y, half %z) {
+; GFX9-LABEL: test_half_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX9-NEXT:    v_add_f16_e64 v0, v2, -v0
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_half_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_xor_b32_e32 v0, 0x8000, v0
+; GFX9-CONTRACT-NEXT:    v_fma_f16 v0, v0, v1, v2
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_half_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX9-DENORM-NEXT:    v_add_f16_e64 v0, v2, -v0
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_half_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX10-NEXT:    v_add_f16_e64 v0, v2, -v0
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_half_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_xor_b32_e32 v0, 0x8000, v0
+; GFX10-CONTRACT-NEXT:    v_fma_f16 v0, v0, v1, v2
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_half_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v0, v2, -v0
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul half %x, %y
+  %b = fsub half %z, %a
+  ret half %b
+}
+
+define double @test_double_sub_mul(double %x, double %y, double %z) {
+; GFX9-LABEL: test_double_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX9-NEXT:    v_add_f64 v[0:1], v[0:1], -v[4:5]
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_double_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[0:1], v[0:1], v[2:3], -v[4:5]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_double_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX9-DENORM-NEXT:    v_add_f64 v[0:1], v[0:1], -v[4:5]
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_double_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX10-NEXT:    v_add_f64 v[0:1], v[0:1], -v[4:5]
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_double_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[0:1], v[0:1], v[2:3], -v[4:5]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_double_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX10-DENORM-NEXT:    v_add_f64 v[0:1], v[0:1], -v[4:5]
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul double %x, %y
+  %b = fsub double %a, %z
+  ret double %b
+}
+
+define double @test_double_sub_mul_rhs(double %x, double %y, double %z) {
+; GFX9-LABEL: test_double_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX9-NEXT:    v_add_f64 v[0:1], v[4:5], -v[0:1]
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_double_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[0:1], -v[0:1], v[2:3], v[4:5]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_double_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX9-DENORM-NEXT:    v_add_f64 v[0:1], v[4:5], -v[0:1]
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_double_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX10-NEXT:    v_add_f64 v[0:1], v[4:5], -v[0:1]
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_double_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[0:1], -v[0:1], v[2:3], v[4:5]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_double_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX10-DENORM-NEXT:    v_add_f64 v[0:1], v[4:5], -v[0:1]
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul double %x, %y
+  %b = fsub double %z, %a
+  ret double %b
+}
+
+define <4 x float> @test_v4f32_sub_mul(<4 x float> %x, <4 x float> %y, <4 x float> %z) {
+; GFX9-LABEL: test_v4f32_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f32_e32 v0, v0, v4
+; GFX9-NEXT:    v_mul_f32_e32 v1, v1, v5
+; GFX9-NEXT:    v_mul_f32_e32 v2, v2, v6
+; GFX9-NEXT:    v_mul_f32_e32 v3, v3, v7
+; GFX9-NEXT:    v_sub_f32_e32 v0, v0, v8
+; GFX9-NEXT:    v_sub_f32_e32 v1, v1, v9
+; GFX9-NEXT:    v_sub_f32_e32 v2, v2, v10
+; GFX9-NEXT:    v_sub_f32_e32 v3, v3, v11
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f32_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v0, v0, v4, -v8
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v1, v1, v5, -v9
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v2, v2, v6, -v10
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v3, v3, v7, -v11
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f32_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_f32 v0, v0, v4, -v8
+; GFX9-DENORM-NEXT:    v_mad_f32 v1, v1, v5, -v9
+; GFX9-DENORM-NEXT:    v_mad_f32 v2, v2, v6, -v10
+; GFX9-DENORM-NEXT:    v_mad_f32 v3, v3, v7, -v11
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f32_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f32_e32 v0, v0, v4
+; GFX10-NEXT:    v_mul_f32_e32 v1, v1, v5
+; GFX10-NEXT:    v_mul_f32_e32 v2, v2, v6
+; GFX10-NEXT:    v_mul_f32_e32 v3, v3, v7
+; GFX10-NEXT:    v_sub_f32_e32 v0, v0, v8
+; GFX10-NEXT:    v_sub_f32_e32 v1, v1, v9
+; GFX10-NEXT:    v_sub_f32_e32 v2, v2, v10
+; GFX10-NEXT:    v_sub_f32_e32 v3, v3, v11
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f32_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v0, v0, v4, -v8
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v1, v1, v5, -v9
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v2, v2, v6, -v10
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v3, v3, v7, -v11
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f32_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mad_f32 v0, v0, v4, -v8
+; GFX10-DENORM-NEXT:    v_mad_f32 v1, v1, v5, -v9
+; GFX10-DENORM-NEXT:    v_mad_f32 v2, v2, v6, -v10
+; GFX10-DENORM-NEXT:    v_mad_f32 v3, v3, v7, -v11
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x float> %x, %y
+  %b = fsub <4 x float> %a, %z
+  ret <4 x float> %b
+}
+
+define <4 x float> @test_v4f32_sub_mul_rhs(<4 x float> %x, <4 x float> %y, <4 x float> %z) {
+; GFX9-LABEL: test_v4f32_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f32_e32 v0, v0, v4
+; GFX9-NEXT:    v_mul_f32_e32 v1, v1, v5
+; GFX9-NEXT:    v_mul_f32_e32 v2, v2, v6
+; GFX9-NEXT:    v_mul_f32_e32 v3, v3, v7
+; GFX9-NEXT:    v_sub_f32_e32 v0, v8, v0
+; GFX9-NEXT:    v_sub_f32_e32 v1, v9, v1
+; GFX9-NEXT:    v_sub_f32_e32 v2, v10, v2
+; GFX9-NEXT:    v_sub_f32_e32 v3, v11, v3
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f32_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v0, -v0, v4, v8
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v1, -v1, v5, v9
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v2, -v2, v6, v10
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v3, -v3, v7, v11
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f32_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_f32 v0, -v0, v4, v8
+; GFX9-DENORM-NEXT:    v_mad_f32 v1, -v1, v5, v9
+; GFX9-DENORM-NEXT:    v_mad_f32 v2, -v2, v6, v10
+; GFX9-DENORM-NEXT:    v_mad_f32 v3, -v3, v7, v11
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f32_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f32_e32 v0, v0, v4
+; GFX10-NEXT:    v_mul_f32_e32 v1, v1, v5
+; GFX10-NEXT:    v_mul_f32_e32 v2, v2, v6
+; GFX10-NEXT:    v_mul_f32_e32 v3, v3, v7
+; GFX10-NEXT:    v_sub_f32_e32 v0, v8, v0
+; GFX10-NEXT:    v_sub_f32_e32 v1, v9, v1
+; GFX10-NEXT:    v_sub_f32_e32 v2, v10, v2
+; GFX10-NEXT:    v_sub_f32_e32 v3, v11, v3
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f32_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v0, -v0, v4, v8
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v1, -v1, v5, v9
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v2, -v2, v6, v10
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v3, -v3, v7, v11
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f32_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mad_f32 v0, -v0, v4, v8
+; GFX10-DENORM-NEXT:    v_mad_f32 v1, -v1, v5, v9
+; GFX10-DENORM-NEXT:    v_mad_f32 v2, -v2, v6, v10
+; GFX10-DENORM-NEXT:    v_mad_f32 v3, -v3, v7, v11
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x float> %x, %y
+  %b = fsub <4 x float> %z, %a
+  ret <4 x float> %b
+}
+
+define <4 x half> @test_v4f16_sub_mul(<4 x half> %x, <4 x half> %y, <4 x half> %z) {
+; GFX9-LABEL: test_v4f16_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX9-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX9-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX9-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX9-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX9-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX9-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f16_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_pk_fma_f16 v0, v0, v2, v4 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX9-CONTRACT-NEXT:    v_pk_fma_f16 v1, v1, v3, v5 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f16_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX9-DENORM-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX9-DENORM-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX9-DENORM-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-DENORM-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX9-DENORM-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-DENORM-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX9-DENORM-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX9-DENORM-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f16_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX10-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX10-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX10-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX10-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX10-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX10-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f16_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_pk_fma_f16 v0, v0, v2, v4 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX10-CONTRACT-NEXT:    v_pk_fma_f16 v1, v1, v3, v5 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f16_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX10-DENORM-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX10-DENORM-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX10-DENORM-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX10-DENORM-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-DENORM-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX10-DENORM-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x half> %x, %y
+  %b = fsub <4 x half> %a, %z
+  ret <4 x half> %b
+}
+
+define <4 x half> @test_v4f16_sub_mul_rhs(<4 x half> %x, <4 x half> %y, <4 x half> %z) {
+; GFX9-LABEL: test_v4f16_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX9-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX9-NEXT:    v_add_f16_e64 v2, v4, -v0
+; GFX9-NEXT:    v_add_f16_sdwa v0, v4, -v0 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-NEXT:    v_add_f16_e64 v3, v5, -v1
+; GFX9-NEXT:    v_add_f16_sdwa v1, v5, -v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX9-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX9-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f16_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_pk_fma_f16 v0, v0, v2, v4 neg_lo:[1,0,0] neg_hi:[1,0,0]
+; GFX9-CONTRACT-NEXT:    v_pk_fma_f16 v1, v1, v3, v5 neg_lo:[1,0,0] neg_hi:[1,0,0]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f16_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX9-DENORM-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX9-DENORM-NEXT:    v_add_f16_e64 v2, v4, -v0
+; GFX9-DENORM-NEXT:    v_add_f16_sdwa v0, v4, -v0 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-DENORM-NEXT:    v_add_f16_e64 v3, v5, -v1
+; GFX9-DENORM-NEXT:    v_add_f16_sdwa v1, v5, -v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-DENORM-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX9-DENORM-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX9-DENORM-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f16_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX10-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX10-NEXT:    v_add_f16_e64 v2, v4, -v0
+; GFX10-NEXT:    v_add_f16_sdwa v0, v4, -v0 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-NEXT:    v_add_f16_e64 v3, v5, -v1
+; GFX10-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX10-NEXT:    v_add_f16_sdwa v1, v5, -v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX10-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f16_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_pk_fma_f16 v0, v0, v2, v4 neg_lo:[1,0,0] neg_hi:[1,0,0]
+; GFX10-CONTRACT-NEXT:    v_pk_fma_f16 v1, v1, v3, v5 neg_lo:[1,0,0] neg_hi:[1,0,0]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f16_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX10-DENORM-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v2, v4, -v0
+; GFX10-DENORM-NEXT:    v_add_f16_sdwa v0, v4, -v0 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v3, v5, -v1
+; GFX10-DENORM-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX10-DENORM-NEXT:    v_add_f16_sdwa v1, v5, -v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-DENORM-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX10-DENORM-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x half> %x, %y
+  %b = fsub <4 x half> %z, %a
+  ret <4 x half> %b
+}
+
+define <4 x double> @test_v4f64_sub_mul(<4 x double> %x, <4 x double> %y, <4 x double> %z) {
+; GFX9-LABEL: test_v4f64_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX9-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX9-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX9-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX9-NEXT:    v_add_f64 v[0:1], v[0:1], -v[16:17]
+; GFX9-NEXT:    v_add_f64 v[2:3], v[2:3], -v[18:19]
+; GFX9-NEXT:    v_add_f64 v[4:5], v[4:5], -v[20:21]
+; GFX9-NEXT:    v_add_f64 v[6:7], v[6:7], -v[22:23]
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f64_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[0:1], v[0:1], v[8:9], -v[16:17]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[2:3], v[2:3], v[10:11], -v[18:19]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[4:5], v[4:5], v[12:13], -v[20:21]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[6:7], v[6:7], v[14:15], -v[22:23]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f64_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX9-DENORM-NEXT:    v_add_f64 v[0:1], v[0:1], -v[16:17]
+; GFX9-DENORM-NEXT:    v_add_f64 v[2:3], v[2:3], -v[18:19]
+; GFX9-DENORM-NEXT:    v_add_f64 v[4:5], v[4:5], -v[20:21]
+; GFX9-DENORM-NEXT:    v_add_f64 v[6:7], v[6:7], -v[22:23]
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f64_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX10-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX10-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX10-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX10-NEXT:    v_add_f64 v[0:1], v[0:1], -v[16:17]
+; GFX10-NEXT:    v_add_f64 v[2:3], v[2:3], -v[18:19]
+; GFX10-NEXT:    v_add_f64 v[4:5], v[4:5], -v[20:21]
+; GFX10-NEXT:    v_add_f64 v[6:7], v[6:7], -v[22:23]
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f64_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[0:1], v[0:1], v[8:9], -v[16:17]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[2:3], v[2:3], v[10:11], -v[18:19]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[4:5], v[4:5], v[12:13], -v[20:21]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[6:7], v[6:7], v[14:15], -v[22:23]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f64_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX10-DENORM-NEXT:    v_add_f64 v[0:1], v[0:1], -v[16:17]
+; GFX10-DENORM-NEXT:    v_add_f64 v[2:3], v[2:3], -v[18:19]
+; GFX10-DENORM-NEXT:    v_add_f64 v[4:5], v[4:5], -v[20:21]
+; GFX10-DENORM-NEXT:    v_add_f64 v[6:7], v[6:7], -v[22:23]
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x double> %x, %y
+  %b = fsub <4 x double> %a, %z
+  ret <4 x double> %b
+}
+
+define <4 x double> @test_v4f64_sub_mul_rhs(<4 x double> %x, <4 x double> %y, <4 x double> %z) {
+; GFX9-LABEL: test_v4f64_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX9-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX9-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX9-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX9-NEXT:    v_add_f64 v[0:1], v[16:17], -v[0:1]
+; GFX9-NEXT:    v_add_f64 v[2:3], v[18:19], -v[2:3]
+; GFX9-NEXT:    v_add_f64 v[4:5], v[20:21], -v[4:5]
+; GFX9-NEXT:    v_add_f64 v[6:7], v[22:23], -v[6:7]
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f64_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[0:1], -v[0:1], v[8:9], v[16:17]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[2:3], -v[2:3], v[10:11], v[18:19]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[4:5], -v[4:5], v[12:13], v[20:21]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[6:7], -v[6:7], v[14:15], v[22:23]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f64_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX9-DENORM-NEXT:    v_add_f64 v[0:1], v[16:17], -v[0:1]
+; GFX9-DENORM-NEXT:    v_add_f64 v[2:3], v[18:19], -v[2:3]
+; GFX9-DENORM-NEXT:    v_add_f64 v[4:5], v[20:21], -v[4:5]
+; GFX9-DENORM-NEXT:    v_add_f64 v[6:7], v[22:23], -v[6:7]
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f64_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX10-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX10-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX10-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX10-NEXT:    v_add_f64 v[0:1], v[16:17], -v[0:1]
+; GFX10-NEXT:    v_add_f64 v[2:3], v[18:19], -v[2:3]
+; GFX10-NEXT:    v_add_f64 v[4:5], v[20:21], -v[4:5]
+; GFX10-NEXT:    v_add_f64 v[6:7], v[22:23], -v[6:7]
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f64_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[0:1], -v[0:1], v[8:9], v[16:17]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[2:3], -v[2:3], v[10:11], v[18:19]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[4:5], -v[4:5], v[12:13], v[20:21]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[6:7], -v[6:7], v[14:15], v[22:23]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f64_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX10-DENORM-NEXT:    v_add_f64 v[0:1], v[16:17], -v[0:1]
+; GFX10-DENORM-NEXT:    v_add_f64 v[2:3], v[18:19], -v[2:3]
+; GFX10-DENORM-NEXT:    v_add_f64 v[4:5], v[20:21], -v[4:5]
+; GFX10-DENORM-NEXT:    v_add_f64 v[6:7], v[22:23], -v[6:7]
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x double> %x, %y
+  %b = fsub <4 x double> %z, %a
+  ret <4 x double> %b
+}