[AMDGPU][GlobalISel] Transform (fsub (fmul x, y), z) -> (fma x, y, -z)

[matejam]

Summary

Instead of sub and mul instructions, use v_mad, v_mac or v_fma if fma
instructions are faster and are legal for the given architecture.
Combiner for a simple case that has only one subtraction and one
multiplication instruction and transforms them into some of the fma
instructions depending on the architecture.

[arsenm]

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

4085–4086

This should probably allow vectors we can break down later too

4088

Don't see where isFMADLegal is fedined

4107–4119

I'm not sure I follow this heuristic, or what SwapPriority means

4160–4161

The types are all identical, there's no reason to query every type

4164

You can directly use the type and avoid the explicit createGenericVirtualRegister with
auto Neg = B.buildFNeg(Ty, X)

llvm/test/CodeGen/AMDGPU/GlobalISel/combine-fma-sub-mul.ll

118–119

Why did we fail to fold the modifier here?

[matejam]

Added support for vector types and some refactoring.

[matejam]

Thanks for the suggestions!

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

4088

In the other revision (the parent): D93305

4107–4119

If SwapPriority is equal to 0 that means that the first and second operands aren't both fmul instructions, if it's equal to 2 it means that both of the arguments are fmul and that the second arg has fewer uses so we pick him for folding, vice versa if it's equal to 1. I will make it more simple in the next version.

[matejam]

This comment was removed by matejam.

[matejam]

Put back the accidentally deleted combiner from the list of combiners (load_or_combine).

[matejam]

Minor changes in CombinerHelper.cpp and in the tests.

[matejam]

Use mi_match for comparing instructions instead of comparing the opcodes.

[foad]

Does this assume that all targets can do the fneg for free? Or can a target choose to fold only (fadd (fmul x, y), z), not (fsub (fmul x, y), z) ?

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

4438

Typo "refers", and they're called MI0 and MI1.

4443

Use hasMoreUses() from the previous patch.

[matejam]

In D96614#2787052, @foad wrote:

Does this assume that all targets can do the fneg for free? Or can a target choose to fold only (fadd (fmul x, y), z), not (fsub (fmul x, y), z) ?

Similar to SelectionDAG which negates an SDValue:

fold (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
DAG.getNode(PreferredFusedOpcode, SL, VT, XY.getOperand(0),
            XY.getOperand(1), DAG.getNode(ISD::FNEG, SL, VT, Z));

[matejam]

Typos and refactoring.

[matejam]

Rebased.

[matejam]

Use m_MInstr instead of m_Reg in matching patterns (mi_match).
A few minor bug fixes.
Formatting and refactoring.

[matejam]

Use applyBuildFn instead of writing my own apply.

[mbrkusanin]

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

4633–4637

Rename Src3 to NegZ so it matches the comment above.
Use B not Builder.

4646–4650

Rename Src1 to NegY so it matches the comment above.
Use B not Builder.

[matejam]

Changes in tests.

[mbrkusanin]

clang-format for CombinerHelper.cpp

llvm/test/CodeGen/AMDGPU/GlobalISel/combine-fma-sub-mul.ll

203

%a and %z should be swapped here, otherwise this is the same test as the one above.

Also combiner fails for this test for -mcpu=gfx900 --denormal-fp-math=preserve-sign.

Same for test above (test_half_sub_mul). It produces correct result only because fsub is replaced by fadd + fneg in legalizer and then is probably matched by one of other combiners that start from fadd.

680

Same here, swap %a and %z.

[matejam]

Minor changes in tests.

[mbrkusanin]

LGTM

[matejam]

Added missing comments and an FMA combiner group in table gen.

llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h

bool matchCombineFAddFpExtFMulToFMadOrFMAAggressive(
       MachineInstr &MI,
       std::tuple<Register, Register, Register, Register,
                  Register, unsigned> &MatchInfo);
   bool applyCombineFAddFpExtFMulToFMadOrFMAAggressive(
       MachineInstr &MI,
       std::tuple<Register, Register, Register, Register,
                  Register, unsigned> &MatchInfo);
 
+  /// Transform (fsub (fmul x, y), z) -> (fma x, y, -z)
+  ///                                 -> (fmad x, y, -z)
+  bool matchCombineFSubFMulToFMadOrFMA(
+      MachineInstr &MI,
+      std::tuple<Register, Register, Register, unsigned, bool> &MatchInfo);
+  bool applyCombineFSubFMulToFMadOrFMA(
+      MachineInstr &MI,
+      std::tuple<Register, Register, Register, unsigned, bool> &MatchInfo);
+
   /// Transform trunc ([asz]ext x) to x or ([asz]ext x) or (trunc x).
   bool matchCombineTruncOfExt(MachineInstr &MI,
                               std::pair<Register, unsigned> &MatchInfo);
   void applyCombineTruncOfExt(MachineInstr &MI,
                               std::pair<Register, unsigned> &MatchInfo);
 
   /// Transform trunc (shl x, K) to shl (trunc x),
   /// K => K < VT.getScalarSizeInBits().

llvm/include/llvm/Target/GlobalISel/Combine.td

 def combine_fadd_fpext_fma_fmul_to_fmad_or_fma: GICombineRule<
   (defs root:$root, combine_fadd_fpext_fma_fmul_to_fmad_or_fma_info:$info),
   (match (wip_match_opcode G_FADD):$root,
          [{ return Helper.matchCombineFAddFpExtFMulToFMadOrFMAAggressive(
                                                   *${root}, ${info}); }]),
   (apply [{ return Helper.applyCombineFAddFpExtFMulToFMadOrFMAAggressive(
                                                   *${root}, ${info}); }])>;
 
+// Transform (fsub (fmul x, y), z) -> (fma x, y, -z)
+def combine_fsub_fmul_to_fmad_or_fma_info :
+    GIDefMatchData<"std::tuple<Register, Register, Register, unsigned, bool>">;
+def combine_fsub_fmul_to_fmad_or_fma: GICombineRule<
+  (defs root:$root, combine_fsub_fmul_to_fmad_or_fma_info:$info),
+  (match (wip_match_opcode G_FSUB):$root,
+         [{ return Helper.matchCombineFSubFMulToFMadOrFMA(*${root},
+                                                          ${info}); }]),
+  (apply [{ return Helper.applyCombineFSubFMulToFMadOrFMA(*${root},
+                                                          ${info}); }])>;
+
 // Currently only the one combine above.
 def insert_vec_elt_combines : GICombineGroup<
                             [combine_insert_vec_elts_build_vector]>;
 
 def extract_vec_elt_build_vec : GICombineRule<
   (defs root:$root, register_matchinfo:$matchinfo),
   (match (wip_match_opcode G_EXTRACT_VECTOR_ELT):$root,
     [{ return Helper.matchExtractVecEltBuildVec(*${root}, ${matchinfo}); }]),
def all_combines : GICombineGroup<[trivial_combines, insert_vec_elt_combines,
     unmerge_merge, fabs_fabs_fold, unmerge_cst, unmerge_dead_to_trunc,
     unmerge_zext_to_zext, merge_unmerge, trunc_ext_fold, trunc_shl,
     const_combines, xor_of_and_with_same_reg, ptr_add_with_zero,
     shift_immed_chain, shift_of_shifted_logic_chain, load_or_combine,
     truncstore_merge, div_rem_to_divrem, funnel_shift_combines,
     form_bitfield_extract, constant_fold, combine_fadd_fmul_to_fmad_or_fma,
     combine_fadd_fpext_fmul_to_fmad_or_fma,
     combine_fadd_fma_fmul_to_fmad_or_fma,
-    combine_fadd_fpext_fma_fmul_to_fmad_or_fma]>;
+    combine_fadd_fpext_fma_fmul_to_fmad_or_fma,
+    combine_fsub_fmul_to_fmad_or_fma]>;
 
 // A combine group used to for prelegalizer combiners at -O0. The combines in
 // this group have been selected based on experiments to balance code size and
 // compile time performance.
 def optnone_combines : GICombineGroup<[trivial_combines,
     ptr_add_immed_chain, combines_for_extload,
     not_cmp_fold, opt_brcond_by_inverting_cond]>;

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

Type *AccessTy =
         getTypeForLLT(MRI.getType(ConvUseMI->getOperand(0).getReg()),
                       PtrAdd.getMF()->getFunction().getContext());
     const auto &TLI = *PtrAdd.getMF()->getSubtarget().getTargetLowering();
     if (!TLI.isLegalAddressingMode(PtrAdd.getMF()->getDataLayout(), AM,
                                    AccessTy, AS))
       continue;
 
     // Would x[offset1+offset2] still be a legal addressing mode?
     AM.BaseOffs = CombinedValue;
     if (!TLI.isLegalAddressingMode(PtrAdd.getMF()->getDataLayout(), AM,
                                    AccessTy, AS))
       return true;
   }
 
   return false;
 }
 
 bool CombinerHelper::matchReassocPtrAdd(
     MachineInstr &MI, std::function<void(MachineIRBuilder &)> &MatchInfo) {
   assert(MI.getOpcode() == TargetOpcode::G_PTR_ADD);
   // We're trying to match a few pointer computation patterns here for
   // re-association opportunities.
   // 1) Isolating a constant operand to be on the RHS, e.g.:
   // G_PTR_ADD(BASE, G_ADD(X, C)) -> G_PTR_ADD(G_PTR_ADD(BASE, X), C)
   //
   // 2) Folding two constants in each sub-tree as long as such folding
   // doesn't break a legal addressing mode.
   // G_PTR_ADD(G_PTR_ADD(BASE, C1), C2) -> G_PTR_ADD(BASE, C1+C2)
   Register Src1Reg = MI.getOperand(1).getReg();
   Register Src2Reg = MI.getOperand(2).getReg();
   MachineInstr *LHS = MRI.getVRegDef(Src1Reg);
   MachineInstr *RHS = MRI.getVRegDef(Src2Reg);
 
   if (LHS->getOpcode() != TargetOpcode::G_PTR_ADD) {
     // Try to match example 1).
     if (RHS->getOpcode() != TargetOpcode::G_ADD)
       return false;
     auto C2 = getConstantVRegVal(RHS->getOperand(2).getReg(), MRI);
     if (!C2)
       return false;
 
     MatchInfo = [=,&MI](MachineIRBuilder &B) {
       LLT PtrTy = MRI.getType(MI.getOperand(0).getReg());
 
       auto NewBase =
           Builder.buildPtrAdd(PtrTy, Src1Reg, RHS->getOperand(1).getReg());
       Observer.changingInstr(MI);
       MI.getOperand(1).setReg(NewBase.getReg(0));
       MI.getOperand(2).setReg(RHS->getOperand(2).getReg());
       Observer.changedInstr(MI);
     };
bool CombinerHelper::matchConstantFold(MachineInstr &MI, APInt &MatchInfo) {
   Register Op2 = MI.getOperand(2).getReg();
   auto MaybeCst = ConstantFoldBinOp(MI.getOpcode(), Op1, Op2, MRI);
   if (!MaybeCst)
     return false;
   MatchInfo = *MaybeCst;
   return true;
 }
 
 bool CombinerHelper::matchNarrowBinopFeedingAnd(
     MachineInstr &MI, std::function<void(MachineIRBuilder &)> &MatchInfo) {
   // Look for a binop feeding into an AND with a mask:
   //
   // %add = G_ADD %lhs, %rhs
   // %and = G_AND %add, 000...11111111
   //
   // Check if it's possible to perform the binop at a narrower width and zext
   // back to the original width like so:
   //
   // %narrow_lhs = G_TRUNC %lhs
   // %narrow_rhs = G_TRUNC %rhs
   // %narrow_add = G_ADD %narrow_lhs, %narrow_rhs
bool CombinerHelper::matchCombineFAddFMAFMulToFMadOrFMA(
   assert(MI.getOpcode() == TargetOpcode::G_FADD);
 
   bool AllowFusionGlobally, HasFMAD, Aggressive;
   if (!canCombineFMadOrFMA(MI, AllowFusionGlobally, HasFMAD, Aggressive, true))
     return false;
 
   MachineInstr *LHS = MRI.getVRegDef(MI.getOperand(1).getReg());
   MachineInstr *RHS = MRI.getVRegDef(MI.getOperand(2).getReg());
 
   unsigned PreferredFusedOpcode =
       HasFMAD ? TargetOpcode::G_FMAD : TargetOpcode::G_FMA;
 
   // If we have two choices trying to fold (fadd (fmul u, v), (fmul x, y)),
   // prefer to fold the multiply with fewer uses.
   if (Aggressive && isContractableFMul(*LHS, AllowFusionGlobally) &&
       isContractableFMul(*RHS, AllowFusionGlobally)) {
     if (std::distance(MRI.use_instr_nodbg_begin(LHS->getOperand(0).getReg()),
                       MRI.use_instr_nodbg_end()) >
         std::distance(MRI.use_instr_nodbg_begin(RHS->getOperand(0).getReg()),
                       MRI.use_instr_nodbg_end()))
       std::swap(LHS, RHS);
   }
if (Ty1 != DstType && Ty2 != DstType) {
     Y = Builder.buildFPExt(DstType, Y).getReg(0);
   }
 
   Builder.buildInstr(PreferredFusedOpcode, {MI.getOperand(0).getReg()},
                      {X, Y, InnerFMA});
   MI.eraseFromParent();
 
   return true;
 }
 
+bool CombinerHelper::matchCombineFSubFMulToFMadOrFMA(
+    MachineInstr &MI,
+    std::tuple<Register, Register, Register, unsigned, bool> &MatchInfo) {
+  assert(MI.getOpcode() == TargetOpcode::G_FSUB);
+
+  bool AllowFusionGlobally, HasFMAD, Aggressive;
+  if (!canCombineFMadOrFMA(MI, AllowFusionGlobally, HasFMAD, Aggressive))
+    return false;
+
+  MachineInstr *LHS = MRI.getVRegDef(MI.getOperand(1).getReg());
+  MachineInstr *RHS = MRI.getVRegDef(MI.getOperand(2).getReg());
+
+  // If we have two choices trying to fold (fadd (fmul u, v), (fmul x, y)),
+  // prefer to fold the multiply with fewer uses.
+  int FirstMulHasFewerUses = true;
+  if (isContractableFMul(*LHS, AllowFusionGlobally) &&
+      isContractableFMul(*RHS, AllowFusionGlobally) &&
+      hasMoreUses(*LHS, *RHS, MRI))
+    FirstMulHasFewerUses = false;
+
+  unsigned PreferredFusedOpcode =
+      HasFMAD ? TargetOpcode::G_FMAD : TargetOpcode::G_FMA;
+
+  // fold (fsub (fmul x, y), z) -> (fma x, y, -z)
+  if (FirstMulHasFewerUses &&
+      (isContractableFMul(*LHS, AllowFusionGlobally) &&
+       (Aggressive || MRI.hasOneNonDBGUse(LHS->getOperand(0).getReg())))) {
+    MatchInfo = {LHS->getOperand(1).getReg(), LHS->getOperand(2).getReg(),
+                 RHS->getOperand(0).getReg(), PreferredFusedOpcode, true};
+    return true;
+  }
+  // fold (fsub x, (fmul y, z)) -> (fma -y, z, x)
+  else if ((isContractableFMul(*RHS, AllowFusionGlobally) &&
+       (Aggressive || MRI.hasOneNonDBGUse(RHS->getOperand(0).getReg())))) {
+    MatchInfo = {RHS->getOperand(1).getReg(), RHS->getOperand(2).getReg(),
+                 LHS->getOperand(0).getReg(), PreferredFusedOpcode, false};
+    return true;
+  }
+
+  return false;
+}
+
+bool CombinerHelper::applyCombineFSubFMulToFMadOrFMA(
+    MachineInstr &MI,
+    std::tuple<Register, Register, Register, unsigned, bool> &MatchInfo) {
+  Register Src1, Src2, Src3;
+  unsigned PreferredFusedOpcode;
+  bool NegateLast;
+  std::tie(Src1, Src2, Src3, PreferredFusedOpcode, NegateLast) = MatchInfo;
+
+  Builder.setInstrAndDebugLoc(MI);
+
+  LLT DstTy = MRI.getType(Src1);
+  if (NegateLast)
+    Src3 = Builder.buildFNeg(DstTy, Src3).getReg(0);
+  else
+    Src1 = Builder.buildFNeg(DstTy, Src1).getReg(0);
+
+  Builder.buildInstr(PreferredFusedOpcode, {MI.getOperand(0).getReg()},
+                     {Src1, Src2, Src3});
+  MI.eraseFromParent();
+  return true;
+}
+
 bool CombinerHelper::tryCombine(MachineInstr &MI) {
   if (tryCombineCopy(MI))
     return true;
   if (tryCombineExtendingLoads(MI))
     return true;
   if (tryCombineIndexedLoadStore(MI))
     return true;
   return false;
 }

llvm/test/CodeGen/AMDGPU/GlobalISel/combine-fma-sub-mul.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx900 < %s | FileCheck -check-prefix=GFX9 %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx900 -fp-contract=fast < %s | FileCheck -check-prefix=GFX9-CONTRACT %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx900 --denormal-fp-math=preserve-sign < %s | FileCheck -check-prefix=GFX9-DENORM %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx1010 < %s | FileCheck -check-prefix=GFX10 %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx1010 -fp-contract=fast < %s | FileCheck -check-prefix=GFX10-CONTRACT %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx1010 --denormal-fp-math=preserve-sign < %s | FileCheck -check-prefix=GFX10-DENORM %s
+
+; fold (fsub (fmul x, y), z) -> (fma x, y, (fneg z))
+; fold (fsub x, (fmul y, z)) -> (fma (fneg y), z, x)
+
+define float @test_f32_sub_mul(float %x, float %y, float %z) {
+; GFX9-LABEL: test_f32_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f32_e32 v0, v0, v1
+; GFX9-NEXT:    v_sub_f32_e32 v0, v0, v2
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_f32_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v0, v0, v1, -v2
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_f32_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_f32 v0, v0, v1, -v2
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_f32_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f32_e32 v0, v0, v1
+; GFX10-NEXT:    v_sub_f32_e32 v0, v0, v2
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_f32_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v0, v0, v1, -v2
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_f32_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mad_f32 v0, v0, v1, -v2
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul float %x, %y
+  %b = fsub float %a, %z
+  ret float %b
+}
+
+define float @test_f32_sub_mul_rhs(float %x, float %y, float %z) {
+; GFX9-LABEL: test_f32_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f32_e32 v0, v0, v1
+; GFX9-NEXT:    v_sub_f32_e32 v0, v2, v0
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_f32_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v0, -v0, v1, v2
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_f32_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_f32 v0, -v0, v1, v2
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_f32_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f32_e32 v0, v0, v1
+; GFX10-NEXT:    v_sub_f32_e32 v0, v2, v0
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_f32_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v0, v1, -v0, v2
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_f32_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mad_f32 v0, -v0, v1, v2
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul float %x, %y
+  %b = fsub float %z, %a
+  ret float %b
+}
+
+define half @test_half_sub_mul(half %x, half %y, half %z) {
+; GFX9-LABEL: test_half_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX9-NEXT:    v_add_f16_e64 v0, v0, -v2
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_half_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_xor_b32_e32 v2, 0x8000, v2
+; GFX9-CONTRACT-NEXT:    v_fma_f16 v0, v0, v1, v2
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_half_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_legacy_f16 v0, v0, v1, -v2
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_half_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX10-NEXT:    v_add_f16_e64 v0, v0, -v2
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_half_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_xor_b32_e32 v2, 0x8000, v2
+; GFX10-CONTRACT-NEXT:    v_fma_f16 v0, v0, v1, v2
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_half_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v0, v0, -v2
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul half %x, %y
+  %b = fsub half %a, %z
+  ret half %b
+}
+
+define half @test_half_sub_mul_rhs(half %x, half %y, half %z) {
+; GFX9-LABEL: test_half_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX9-NEXT:    v_add_f16_e64 v0, v0, -v2
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_half_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_xor_b32_e32 v2, 0x8000, v2
+; GFX9-CONTRACT-NEXT:    v_fma_f16 v0, v0, v1, v2
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_half_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_legacy_f16 v0, v0, v1, -v2
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_half_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX10-NEXT:    v_add_f16_e64 v0, v0, -v2
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_half_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_xor_b32_e32 v2, 0x8000, v2
+; GFX10-CONTRACT-NEXT:    v_fma_f16 v0, v0, v1, v2
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_half_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f16_e32 v0, v0, v1
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v0, v0, -v2
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul half %x, %y
+  %b = fsub half %a, %z
+  ret half %b
+}
+
+define double @test_double_sub_mul(double %x, double %y, double %z) {
+; GFX9-LABEL: test_double_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX9-NEXT:    v_add_f64 v[0:1], v[0:1], -v[4:5]
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_double_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[0:1], v[0:1], v[2:3], -v[4:5]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_double_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX9-DENORM-NEXT:    v_add_f64 v[0:1], v[0:1], -v[4:5]
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_double_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX10-NEXT:    v_add_f64 v[0:1], v[0:1], -v[4:5]
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_double_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[0:1], v[0:1], v[2:3], -v[4:5]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_double_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX10-DENORM-NEXT:    v_add_f64 v[0:1], v[0:1], -v[4:5]
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul double %x, %y
+  %b = fsub double %a, %z
+  ret double %b
+}
+
+define double @test_double_sub_mul_rhs(double %x, double %y, double %z) {
+; GFX9-LABEL: test_double_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX9-NEXT:    v_add_f64 v[0:1], v[4:5], -v[0:1]
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_double_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[0:1], -v[0:1], v[2:3], v[4:5]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_double_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX9-DENORM-NEXT:    v_add_f64 v[0:1], v[4:5], -v[0:1]
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_double_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX10-NEXT:    v_add_f64 v[0:1], v[4:5], -v[0:1]
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_double_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[0:1], -v[0:1], v[2:3], v[4:5]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_double_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[2:3]
+; GFX10-DENORM-NEXT:    v_add_f64 v[0:1], v[4:5], -v[0:1]
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul double %x, %y
+  %b = fsub double %z, %a
+  ret double %b
+}
+
+define <4 x float> @test_v4f32_sub_mul(<4 x float> %x, <4 x float> %y, <4 x float> %z) {
+; GFX9-LABEL: test_v4f32_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f32_e32 v0, v0, v4
+; GFX9-NEXT:    v_mul_f32_e32 v1, v1, v5
+; GFX9-NEXT:    v_mul_f32_e32 v2, v2, v6
+; GFX9-NEXT:    v_mul_f32_e32 v3, v3, v7
+; GFX9-NEXT:    v_sub_f32_e32 v0, v0, v8
+; GFX9-NEXT:    v_sub_f32_e32 v1, v1, v9
+; GFX9-NEXT:    v_sub_f32_e32 v2, v2, v10
+; GFX9-NEXT:    v_sub_f32_e32 v3, v3, v11
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f32_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v0, v0, v4, -v8
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v1, v1, v5, -v9
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v2, v2, v6, -v10
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v3, v3, v7, -v11
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f32_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_f32 v0, v0, v4, -v8
+; GFX9-DENORM-NEXT:    v_mad_f32 v1, v1, v5, -v9
+; GFX9-DENORM-NEXT:    v_mad_f32 v2, v2, v6, -v10
+; GFX9-DENORM-NEXT:    v_mad_f32 v3, v3, v7, -v11
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f32_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f32_e32 v0, v0, v4
+; GFX10-NEXT:    v_mul_f32_e32 v1, v1, v5
+; GFX10-NEXT:    v_mul_f32_e32 v2, v2, v6
+; GFX10-NEXT:    v_mul_f32_e32 v3, v3, v7
+; GFX10-NEXT:    v_sub_f32_e32 v0, v0, v8
+; GFX10-NEXT:    v_sub_f32_e32 v1, v1, v9
+; GFX10-NEXT:    v_sub_f32_e32 v2, v2, v10
+; GFX10-NEXT:    v_sub_f32_e32 v3, v3, v11
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f32_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v0, v0, v4, -v8
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v1, v1, v5, -v9
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v2, v2, v6, -v10
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v3, v3, v7, -v11
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f32_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mad_f32 v0, v0, v4, -v8
+; GFX10-DENORM-NEXT:    v_mad_f32 v1, v1, v5, -v9
+; GFX10-DENORM-NEXT:    v_mad_f32 v2, v2, v6, -v10
+; GFX10-DENORM-NEXT:    v_mad_f32 v3, v3, v7, -v11
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x float> %x, %y
+  %b = fsub <4 x float> %a, %z
+  ret <4 x float> %b
+}
+
+define <4 x float> @test_v4f32_sub_mul_rhs(<4 x float> %x, <4 x float> %y, <4 x float> %z) {
+; GFX9-LABEL: test_v4f32_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f32_e32 v0, v0, v4
+; GFX9-NEXT:    v_mul_f32_e32 v1, v1, v5
+; GFX9-NEXT:    v_mul_f32_e32 v2, v2, v6
+; GFX9-NEXT:    v_mul_f32_e32 v3, v3, v7
+; GFX9-NEXT:    v_sub_f32_e32 v0, v8, v0
+; GFX9-NEXT:    v_sub_f32_e32 v1, v9, v1
+; GFX9-NEXT:    v_sub_f32_e32 v2, v10, v2
+; GFX9-NEXT:    v_sub_f32_e32 v3, v11, v3
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f32_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v0, -v0, v4, v8
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v1, -v1, v5, v9
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v2, -v2, v6, v10
+; GFX9-CONTRACT-NEXT:    v_fma_f32 v3, -v3, v7, v11
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f32_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mad_f32 v0, -v0, v4, v8
+; GFX9-DENORM-NEXT:    v_mad_f32 v1, -v1, v5, v9
+; GFX9-DENORM-NEXT:    v_mad_f32 v2, -v2, v6, v10
+; GFX9-DENORM-NEXT:    v_mad_f32 v3, -v3, v7, v11
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f32_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f32_e32 v0, v0, v4
+; GFX10-NEXT:    v_mul_f32_e32 v1, v1, v5
+; GFX10-NEXT:    v_mul_f32_e32 v2, v2, v6
+; GFX10-NEXT:    v_mul_f32_e32 v3, v3, v7
+; GFX10-NEXT:    v_sub_f32_e32 v0, v8, v0
+; GFX10-NEXT:    v_sub_f32_e32 v1, v9, v1
+; GFX10-NEXT:    v_sub_f32_e32 v2, v10, v2
+; GFX10-NEXT:    v_sub_f32_e32 v3, v11, v3
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f32_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v0, v4, -v0, v8
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v1, v5, -v1, v9
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v2, v6, -v2, v10
+; GFX10-CONTRACT-NEXT:    v_fma_f32 v3, v7, -v3, v11
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f32_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mad_f32 v0, -v0, v4, v8
+; GFX10-DENORM-NEXT:    v_mad_f32 v1, -v1, v5, v9
+; GFX10-DENORM-NEXT:    v_mad_f32 v2, -v2, v6, v10
+; GFX10-DENORM-NEXT:    v_mad_f32 v3, -v3, v7, v11
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x float> %x, %y
+  %b = fsub <4 x float> %z, %a
+  ret <4 x float> %b
+}
+
+define <4 x half> @test_v4f16_sub_mul(<4 x half> %x, <4 x half> %y, <4 x half> %z) {
+; GFX9-LABEL: test_v4f16_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX9-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX9-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX9-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX9-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX9-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX9-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f16_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_pk_fma_f16 v0, v0, v2, v4 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX9-CONTRACT-NEXT:    v_pk_fma_f16 v1, v1, v3, v5 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f16_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX9-DENORM-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX9-DENORM-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX9-DENORM-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-DENORM-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX9-DENORM-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX9-DENORM-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-DENORM-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX9-DENORM-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f16_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX10-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX10-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX10-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX10-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX10-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX10-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f16_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_pk_fma_f16 v0, v0, v2, v4 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX10-CONTRACT-NEXT:    v_pk_fma_f16 v1, v1, v3, v5 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f16_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX10-DENORM-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX10-DENORM-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX10-DENORM-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX10-DENORM-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-DENORM-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX10-DENORM-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x half> %x, %y
+  %b = fsub <4 x half> %a, %z
+  ret <4 x half> %b
+}
+
+define <4 x half> @test_v4f16_sub_mul_rhs(<4 x half> %x, <4 x half> %y, <4 x half> %z) {
+; GFX9-LABEL: test_v4f16_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX9-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX9-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX9-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX9-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX9-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX9-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f16_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_pk_fma_f16 v0, v0, v2, v4 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX9-CONTRACT-NEXT:    v_pk_fma_f16 v1, v1, v3, v5 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f16_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX9-DENORM-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX9-DENORM-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX9-DENORM-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-DENORM-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX9-DENORM-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX9-DENORM-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX9-DENORM-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX9-DENORM-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f16_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX10-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX10-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX10-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX10-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX10-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX10-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f16_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_pk_fma_f16 v0, v0, v2, v4 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX10-CONTRACT-NEXT:    v_pk_fma_f16 v1, v1, v3, v5 neg_lo:[0,0,1] neg_hi:[0,0,1]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f16_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_pk_mul_f16 v0, v0, v2
+; GFX10-DENORM-NEXT:    v_pk_mul_f16 v1, v1, v3
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v2, v0, -v4
+; GFX10-DENORM-NEXT:    v_add_f16_sdwa v0, v0, -v4 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-DENORM-NEXT:    v_add_f16_e64 v3, v1, -v5
+; GFX10-DENORM-NEXT:    v_mov_b32_e32 v4, 0xffff
+; GFX10-DENORM-NEXT:    v_add_f16_sdwa v1, v1, -v5 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
+; GFX10-DENORM-NEXT:    v_and_or_b32 v0, v2, v4, v0
+; GFX10-DENORM-NEXT:    v_and_or_b32 v1, v3, v4, v1
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x half> %x, %y
+  %b = fsub <4 x half> %a, %z
+  ret <4 x half> %b
+}
+
+define <4 x double> @test_v4f64_sub_mul(<4 x double> %x, <4 x double> %y, <4 x double> %z) {
+; GFX9-LABEL: test_v4f64_sub_mul:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX9-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX9-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX9-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX9-NEXT:    v_add_f64 v[0:1], v[0:1], -v[16:17]
+; GFX9-NEXT:    v_add_f64 v[2:3], v[2:3], -v[18:19]
+; GFX9-NEXT:    v_add_f64 v[4:5], v[4:5], -v[20:21]
+; GFX9-NEXT:    v_add_f64 v[6:7], v[6:7], -v[22:23]
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f64_sub_mul:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[0:1], v[0:1], v[8:9], -v[16:17]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[2:3], v[2:3], v[10:11], -v[18:19]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[4:5], v[4:5], v[12:13], -v[20:21]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[6:7], v[6:7], v[14:15], -v[22:23]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f64_sub_mul:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX9-DENORM-NEXT:    v_add_f64 v[0:1], v[0:1], -v[16:17]
+; GFX9-DENORM-NEXT:    v_add_f64 v[2:3], v[2:3], -v[18:19]
+; GFX9-DENORM-NEXT:    v_add_f64 v[4:5], v[4:5], -v[20:21]
+; GFX9-DENORM-NEXT:    v_add_f64 v[6:7], v[6:7], -v[22:23]
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f64_sub_mul:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX10-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX10-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX10-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX10-NEXT:    v_add_f64 v[0:1], v[0:1], -v[16:17]
+; GFX10-NEXT:    v_add_f64 v[2:3], v[2:3], -v[18:19]
+; GFX10-NEXT:    v_add_f64 v[4:5], v[4:5], -v[20:21]
+; GFX10-NEXT:    v_add_f64 v[6:7], v[6:7], -v[22:23]
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f64_sub_mul:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[0:1], v[0:1], v[8:9], -v[16:17]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[2:3], v[2:3], v[10:11], -v[18:19]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[4:5], v[4:5], v[12:13], -v[20:21]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[6:7], v[6:7], v[14:15], -v[22:23]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f64_sub_mul:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX10-DENORM-NEXT:    v_add_f64 v[0:1], v[0:1], -v[16:17]
+; GFX10-DENORM-NEXT:    v_add_f64 v[2:3], v[2:3], -v[18:19]
+; GFX10-DENORM-NEXT:    v_add_f64 v[4:5], v[4:5], -v[20:21]
+; GFX10-DENORM-NEXT:    v_add_f64 v[6:7], v[6:7], -v[22:23]
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x double> %x, %y
+  %b = fsub <4 x double> %a, %z
+  ret <4 x double> %b
+}
+
+define <4 x double> @test_v4f64_sub_mul_rhs(<4 x double> %x, <4 x double> %y, <4 x double> %z) {
+; GFX9-LABEL: test_v4f64_sub_mul_rhs:
+; GFX9:       ; %bb.0: ; %.entry
+; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX9-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX9-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX9-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX9-NEXT:    v_add_f64 v[0:1], v[16:17], -v[0:1]
+; GFX9-NEXT:    v_add_f64 v[2:3], v[18:19], -v[2:3]
+; GFX9-NEXT:    v_add_f64 v[4:5], v[20:21], -v[4:5]
+; GFX9-NEXT:    v_add_f64 v[6:7], v[22:23], -v[6:7]
+; GFX9-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-CONTRACT-LABEL: test_v4f64_sub_mul_rhs:
+; GFX9-CONTRACT:       ; %bb.0: ; %.entry
+; GFX9-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[0:1], -v[0:1], v[8:9], v[16:17]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[2:3], -v[2:3], v[10:11], v[18:19]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[4:5], -v[4:5], v[12:13], v[20:21]
+; GFX9-CONTRACT-NEXT:    v_fma_f64 v[6:7], -v[6:7], v[14:15], v[22:23]
+; GFX9-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX9-DENORM-LABEL: test_v4f64_sub_mul_rhs:
+; GFX9-DENORM:       ; %bb.0: ; %.entry
+; GFX9-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX9-DENORM-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX9-DENORM-NEXT:    v_add_f64 v[0:1], v[16:17], -v[0:1]
+; GFX9-DENORM-NEXT:    v_add_f64 v[2:3], v[18:19], -v[2:3]
+; GFX9-DENORM-NEXT:    v_add_f64 v[4:5], v[20:21], -v[4:5]
+; GFX9-DENORM-NEXT:    v_add_f64 v[6:7], v[22:23], -v[6:7]
+; GFX9-DENORM-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: test_v4f64_sub_mul_rhs:
+; GFX10:       ; %bb.0: ; %.entry
+; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX10-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX10-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX10-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX10-NEXT:    v_add_f64 v[0:1], v[16:17], -v[0:1]
+; GFX10-NEXT:    v_add_f64 v[2:3], v[18:19], -v[2:3]
+; GFX10-NEXT:    v_add_f64 v[4:5], v[20:21], -v[4:5]
+; GFX10-NEXT:    v_add_f64 v[6:7], v[22:23], -v[6:7]
+; GFX10-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-CONTRACT-LABEL: test_v4f64_sub_mul_rhs:
+; GFX10-CONTRACT:       ; %bb.0: ; %.entry
+; GFX10-CONTRACT-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-CONTRACT-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[0:1], -v[0:1], v[8:9], v[16:17]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[2:3], -v[2:3], v[10:11], v[18:19]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[4:5], -v[4:5], v[12:13], v[20:21]
+; GFX10-CONTRACT-NEXT:    v_fma_f64 v[6:7], -v[6:7], v[14:15], v[22:23]
+; GFX10-CONTRACT-NEXT:    s_setpc_b64 s[30:31]
+;
+; GFX10-DENORM-LABEL: test_v4f64_sub_mul_rhs:
+; GFX10-DENORM:       ; %bb.0: ; %.entry
+; GFX10-DENORM-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-DENORM-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-DENORM-NEXT:    v_mul_f64 v[0:1], v[0:1], v[8:9]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[2:3], v[2:3], v[10:11]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[4:5], v[4:5], v[12:13]
+; GFX10-DENORM-NEXT:    v_mul_f64 v[6:7], v[6:7], v[14:15]
+; GFX10-DENORM-NEXT:    v_add_f64 v[0:1], v[16:17], -v[0:1]
+; GFX10-DENORM-NEXT:    v_add_f64 v[2:3], v[18:19], -v[2:3]
+; GFX10-DENORM-NEXT:    v_add_f64 v[4:5], v[20:21], -v[4:5]
+; GFX10-DENORM-NEXT:    v_add_f64 v[6:7], v[22:23], -v[6:7]
+; GFX10-DENORM-NEXT:    s_setpc_b64 s[30:31]
+.entry:
+  %a = fmul <4 x double> %x, %y
+  %b = fsub <4 x double> %z, %a
+  ret <4 x double> %b
+}