[GISel] Add fpext/fptrunc combines

[pnappa]

Summary

Adds the following combines, plus AArch64 tests:

• (fptrunc (fpext x)) -> x, when dest and source types match.
• (fptrunc ([us]itofp x)) -> ([us]itofp x)
• (fpext C) -> C
• (fpext ([us]itofp x)) -> ([us]itofp x), when x is 8 bit.

[pnappa]

Fixed AMDGPU's fdiv.f16.ll test.

ninja check-llvm-codegen

Successfully passes locally.

[aemerson]

llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h

355

The apply functions should return void.

llvm/include/llvm/Target/GlobalISel/Combine.td

480

likewise, the apply clause doesn't need to return.

[paquette]

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

2508

It's unfortunate that there's nothing in MIPatternMatch for G_SITOFP and G_UITOFP right now, because it looks like this would work nicely with mi_match.

2533

May be worth reducing indentation here for the sake of slight readability gains.

Also there are some extra braces here which aren't needed.

[paquette]

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

2548

The apply functions here are basically identical. To reduce duplication, I think it would be better to either

a) Make a helper which handles this type of combine
b) Use applyBuildFn

[paquette]

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

1680

I think that we should have an assert that we get one of the expected opcodes here.

[pnappa]

Addressed @aemerson & @paquette's feedback.

[foad]

llvm/include/llvm/Target/GlobalISel/Combine.td

353

I didn't know we had these as combines. Do we really want this, instead of constant-folding them in CSEMIRBuilder? See D99036 and D104528 for some steps in that direction.

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

2531

Is there a precedent for using "UnsafeFPMath" to enable this particular unsafe combine?

[paquette]

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

2531

This looks really similar to the isKnownExactCastIntToFP case in InstCombinerImpl::visitFPExt.

Taking a peek in isKnownExactCastIntToFP doesn't have any mention of unsafe math. I couldn't find anything in SelectionDAG either (but maybe I missed something?)

(The checks in isKnownExactCastIntToFP seem to do a lot of work actually; I wonder if it would make sense to do something similar here? I guess that could be a follow-up patch.)

[pnappa]

llvm/include/llvm/Target/GlobalISel/Combine.td

353

I don't know the answer to this, any one else want to chime in?

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

2531

Precedent, as in have we done something similar before, enabling a different combine depending on fastmath? Or as in, does this optimisation help in practice?

Yes to both, I think, but the latter is fairly niche

[foad]

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

2531

Precedent as in, does the unsafe math flag enable this specific optimization anywhere else in the compiler, e.g. on IR or in SelectionDAG? I'm not sure it's a good idea to add new unsafe optimizations into the existing (pretty well established) unsafe math flag.

[aditya_nandakumar]

llvm/include/llvm/Target/GlobalISel/Combine.td

353

I think we've tried constant folding size changing ops (such as ZEXT) in the builder and it has the potential to cause infinite loops in the legalizer algorithm.
In general, since the builder doesn't care about the legality of ops it's folding and producing, we should be a bit careful on what transformations we introduce as part of building instructions.

[pnappa]

Addressed @foad 's concerns about fast-math. Removed the combine applying when unsafe math is enabled.

[foad]

Removed the combine applying when unsafe math is enabled.

Thanks. Please remove it from the commit message too.

llvm/include/llvm/Target/GlobalISel/Combine.td

483

Is this safe? E.g. does converting i32->f32->f16 always give the same result as i32->f16, even though the former does a kind of double rounding (from 32 bits to 24 bit mantissa to 11 bit mantissa)? Does SelectionDAG do the equivalent optimization?

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

2508

Could return early here.

2531

Could return early here.

[pnappa]

llvm/include/llvm/Target/GlobalISel/Combine.td

483

Re: correctness: https://alive2.llvm.org/ce/z/7xZjnH, https://alive2.llvm.org/ce/z/azpy9w. Yeah, it's sound.

I don't know if SDAG does the equivalent optimisation, I had a look in DAGCombiner.cpp, and can't see it. I don't have any experience with SDAG, unfortunately.

[pnappa]

Rebased to main. Sorry for the noise!

[foad]

llvm/include/llvm/Target/GlobalISel/Combine.td

483

Re: correctness: https://alive2.llvm.org/ce/z/7xZjnH, https://alive2.llvm.org/ce/z/azpy9w. Yeah, it's sound.

That's not a good enough test. You're only testing an i16 input, which is small enough to fit exactly in the 24-bit float mantissa without being rounded off.

[foad]

llvm/include/llvm/Target/GlobalISel/Combine.td

483

Here's a counterexample: https://godbolt.org/z/h14bob8jd

[foad]

llvm/include/llvm/Target/GlobalISel/Combine.td

483

And here's alive2 saying the same thing: https://alive2.llvm.org/ce/z/UEZDuD

So, I think this might be safe for the specific case of float->half (*) but I don't think it's safe in general for any two floating point types.

(*) Because all integers up to the largest value you can represent as a half (about 2^16) are exactly representable as a float.

[arsenm]

Can you try this with the new tablegen combiner support?

llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h

public:
 
   /// Transform anyext(trunc(x)) to x.
   bool matchCombineAnyExtTrunc(MachineInstr &MI, Register &Reg);
   void applyCombineAnyExtTrunc(MachineInstr &MI, Register &Reg);
 
   /// Transform zext(trunc(x)) to x.
   bool matchCombineZextTrunc(MachineInstr &MI, Register &Reg);
 
+  /// Transform fptrunc(fpext(x)) to x.
+  bool matchCombineFPTruncExt(MachineInstr &MI, Register &Reg);
+
+  /// Transform fptrunc([su]itofp(x)) to [su]itofp x.
+  bool matchCombineFPTruncIntToFloat(
+      MachineInstr &MI, std::function<void(MachineIRBuilder &)> &MatchInfo);
+
+  /// Transform fpext([su]itofp(x)) to [su]itofp x.
+  bool matchCombineFPExtIntToFloat(
+      MachineInstr &MI, std::function<void(MachineIRBuilder &)> &MatchInfo);
+
   /// Transform [asz]ext([asz]ext(x)) to [asz]ext x.
   bool matchCombineExtOfExt(MachineInstr &MI,
                             std::tuple<Register, unsigned> &MatchInfo);
   void applyCombineExtOfExt(MachineInstr &MI,
                             std::tuple<Register, unsigned> &MatchInfo);
 
   /// Transform fneg(fneg(x)) to x.
   bool matchCombineFNegOfFNeg(MachineInstr &MI, Register &Reg);

llvm/include/llvm/Target/GlobalISel/Combine.td

 def simplify_add_to_sub_matchinfo: GIDefMatchData<"std::tuple<Register, Register>">;
 def simplify_add_to_sub: GICombineRule <
   (defs root:$root, simplify_add_to_sub_matchinfo:$info),
   (match (wip_match_opcode G_ADD):$root,
     [{ return Helper.matchSimplifyAddToSub(*${root}, ${info}); }]),
   (apply [{ Helper.applySimplifyAddToSub(*${root}, ${info});}])
 >;
 
 // Fold fp_op(cst) to the constant result of the floating point operation.
 def constant_fp_op_matchinfo: GIDefMatchData<"Optional<APFloat>">;
 def constant_fp_op: GICombineRule <
   (defs root:$root, constant_fp_op_matchinfo:$info),
-  (match (wip_match_opcode G_FNEG, G_FABS, G_FPTRUNC, G_FSQRT, G_FLOG2):$root,
+  (match (wip_match_opcode G_FNEG, G_FABS, G_FPTRUNC, G_FPEXT, G_FSQRT, G_FLOG2):$root,
     [{ return Helper.matchCombineConstantFoldFpUnary(*${root}, ${info}); }]),
   (apply [{ Helper.applyCombineConstantFoldFpUnary(*${root}, ${info}); }])
 >;
 
 // Fold int2ptr(ptr2int(x)) -> x
 def p2i_to_i2p: GICombineRule<
   (defs root:$root, register_matchinfo:$info),
   (match (wip_match_opcode G_INTTOPTR):$root,

 def ext_ext_fold_matchinfo : GIDefMatchData<"std::tuple<Register, unsigned>">;
 def ext_ext_fold: GICombineRule <
   (defs root:$root, ext_ext_fold_matchinfo:$matchinfo),
   (match (wip_match_opcode G_ANYEXT, G_SEXT, G_ZEXT):$root,
          [{ return Helper.matchCombineExtOfExt(*${root}, ${matchinfo}); }]),
   (apply [{ Helper.applyCombineExtOfExt(*${root}, ${matchinfo}); }])
 >;
 
+// Fold (fptrunc (fpext x)) -> x if source type is same as destination type.
+def fptrunc_fpext_fold: GICombineRule <
+  (defs root:$root, register_matchinfo:$matchinfo),
+  (match (wip_match_opcode G_FPTRUNC):$root,
+         [{ return Helper.matchCombineFPTruncExt(*${root}, ${matchinfo}); }]),
+  (apply [{ Helper.replaceSingleDefInstWithReg(*${root}, ${matchinfo}); }])
+>;
+
+// Fold (fptrunc ([su]itofp x)) -> ([su]itofp x)
+def fptrunc_int_to_float: GICombineRule<
+  (defs root:$d, build_fn_matchinfo:$info),
+  (match (wip_match_opcode G_FPTRUNC): $d,
+  [{ return Helper.matchCombineFPTruncIntToFloat(*${d}, ${info}); }]),
+  (apply [{ Helper.applyBuildFn(*${d}, ${info}); }])
+>;
+
+// Fold (fpext ([su]itofp x)) -> ([su]itofp x), if x is representable in Dst
+// without losing precision.
+def fpext_int_to_float: GICombineRule<
+  (defs root:$d, build_fn_matchinfo:$info),
+  (match (wip_match_opcode G_FPEXT): $d,
+  [{ return Helper.matchCombineFPExtIntToFloat(*${d}, ${info}); }]),
+  (apply [{ Helper.applyBuildFn(*${d}, ${info}); }])
+>;
+
 def not_cmp_fold_matchinfo : GIDefMatchData<"SmallVector<Register, 4>">;
 def not_cmp_fold : GICombineRule<
   (defs root:$d, not_cmp_fold_matchinfo:$info),
   (match (wip_match_opcode G_XOR): $d,
   [{ return Helper.matchNotCmp(*${d}, ${info}); }]),
   (apply [{ Helper.applyNotCmp(*${d}, ${info}); }])
 >;
 
def undef_combines : GICombineGroup<[undef_to_fp_zero, undef_to_int_zero,
                                      propagate_undef_all_ops,
                                      propagate_undef_shuffle_mask,
                                      erase_undef_store]>;
 
 def identity_combines : GICombineGroup<[select_same_val, right_identity_zero,
                                         binop_same_val, binop_left_to_zero,
                                         binop_right_to_zero, p2i_to_i2p,
                                         i2p_to_p2i, anyext_trunc_fold,
-                                        fneg_fneg_fold, right_identity_one]>;
+                                        fneg_fneg_fold, right_identity_one,
+                                        fptrunc_fpext_fold]>;
 
 def const_combines : GICombineGroup<[constant_fp_op, const_ptradd_to_i2p,
                                      overlapping_and]>;
 
 def known_bits_simplifications : GICombineGroup<[
   redundant_and, redundant_sext_inreg, redundant_or, urem_pow2_to_mask,
   zext_trunc_fold, icmp_to_true_false_known_bits]>;
 
def all_combines : GICombineGroup<[trivial_combines, insert_vec_elt_combines,
     width_reduction_combines, select_combines,
     known_bits_simplifications, ext_ext_fold,
     not_cmp_fold, opt_brcond_by_inverting_cond,
     unmerge_merge, fabs_fabs_fold, unmerge_cst, unmerge_dead_to_trunc,
     unmerge_zext_to_zext, merge_unmerge, trunc_ext_fold, trunc_shl,
     const_combines, xor_of_and_with_same_reg, ptr_add_with_zero,
     shift_immed_chain, shift_of_shifted_logic_chain, load_or_combine,
     div_rem_to_divrem, funnel_shift_combines, form_bitfield_extract,
-    constant_fold]>;
+    constant_fold, fptrunc_int_to_float, fpext_int_to_float]>;
 
 // A combine group used to for prelegalizer combiners at -O0. The combines in
 // this group have been selected based on experiments to balance code size and
 // compile time performance.
 def optnone_combines : GICombineGroup<[trivial_combines,
     ptr_add_immed_chain, combines_for_extload,
     not_cmp_fold, opt_brcond_by_inverting_cond]>;

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

case TargetOpcode::G_FNEG: {
     V.changeSign();
     return V;
   }
   case TargetOpcode::G_FABS: {
     V.clearSign();
     return V;
   }
   case TargetOpcode::G_FPTRUNC:
+  case TargetOpcode::G_FPEXT:
     break;
   case TargetOpcode::G_FSQRT: {
     bool Unused;
     V.convert(APFloat::IEEEdouble(), APFloat::rmNearestTiesToEven, &Unused);
     V = APFloat(sqrt(V.convertToDouble()));
     break;
   }
   case TargetOpcode::G_FLOG2: {
     bool Unused;
     V.convert(APFloat::IEEEdouble(), APFloat::rmNearestTiesToEven, &Unused);
     V = APFloat(log2(V.convertToDouble()));
     break;
   }
   }
   // Convert `APFloat` to appropriate IEEE type depending on `DstTy`. Otherwise,
-  // `buildFConstant` will assert on size mismatch. Only `G_FPTRUNC`, `G_FSQRT`,
-  // and `G_FLOG2` reach here.
+  // `buildFConstant` will assert on size mismatch.
+  assert(Opcode == TargetOpcode::G_FPTRUNC || Opcode == TargetOpcode::G_FPEXT ||
+         Opcode == TargetOpcode::G_FSQRT || Opcode == TargetOpcode::G_FLOG2);
   bool Unused;
   V.convert(getFltSemanticForLLT(DstTy), APFloat::rmNearestTiesToEven, &Unused);
   return V;
 }
 
 bool CombinerHelper::matchCombineConstantFoldFpUnary(MachineInstr &MI,
                                                      Optional<APFloat> &Cst) {
   Register DstReg = MI.getOperand(0).getReg();
if (mi_match(SrcReg, MRI,
                m_GTrunc(m_all_of(m_Reg(Reg), m_SpecificType(DstTy))))) {
     unsigned DstSize = DstTy.getScalarSizeInBits();
     unsigned SrcSize = MRI.getType(SrcReg).getScalarSizeInBits();
     return KB->getKnownBits(Reg).countMinLeadingZeros() >= DstSize - SrcSize;
   }
   return false;
 }
 
+bool CombinerHelper::matchCombineFPTruncExt(MachineInstr &MI, Register &Reg) {
+  assert(MI.getOpcode() == TargetOpcode::G_FPTRUNC && "Expected a G_FPTRUNC");
+  Register DstReg = MI.getOperand(0).getReg();
+  Register SrcReg = MI.getOperand(1).getReg();
+  LLT DstTy = MRI.getType(DstReg);
+  return mi_match(SrcReg, MRI,
+                  m_GFPExt(m_all_of(m_Reg(Reg), m_SpecificType(DstTy))));
+}
+
+bool CombinerHelper::matchCombineFPTruncIntToFloat(
+    MachineInstr &MI, std::function<void(MachineIRBuilder &)> &MatchInfo) {
+  assert(MI.getOpcode() == TargetOpcode::G_FPTRUNC && "Expected a G_FPTRUNC");
+  Register SrcReg = MI.getOperand(1).getReg();
+  MachineInstr *SrcMI = MRI.getVRegDef(SrcReg);
+  unsigned SrcOpc = SrcMI->getOpcode();
+  if (SrcOpc != TargetOpcode::G_SITOFP && SrcOpc != TargetOpcode::G_UITOFP)
+    return false;
+  Register SrcInputReg = SrcMI->getOperand(1).getReg();
+  Register DstReg = MI.getOperand(0).getReg();
+  MatchInfo = [=](MachineIRBuilder &B) {
+    B.buildInstr(SrcOpc, {DstReg}, {SrcInputReg});
+  };
+  return true;
+}
+
+bool CombinerHelper::matchCombineFPExtIntToFloat(
+    MachineInstr &MI, std::function<void(MachineIRBuilder &)> &MatchInfo) {
+  assert(MI.getOpcode() == TargetOpcode::G_FPEXT && "Expected a G_FPEXT");
+  Register SrcReg = MI.getOperand(1).getReg();
+  MachineInstr *SrcMI = MRI.getVRegDef(SrcReg);
+  unsigned SrcOpc = SrcMI->getOpcode();
+  Register DstReg = MI.getOperand(0).getReg();
+  if (SrcOpc != TargetOpcode::G_SITOFP && SrcOpc != TargetOpcode::G_UITOFP)
+    return false;
+  // 8-bit integers are fully representable in supported floating point types.
+  Register SrcInputReg = SrcMI->getOperand(1).getReg();
+  if (MRI.getType(SrcInputReg).getSizeInBits() != 8)
+    return false;
+  MatchInfo = [=](MachineIRBuilder &B) {
+    B.buildInstr(SrcOpc, {DstReg}, {SrcInputReg});
+  };
+  return true;
+}
+
 bool CombinerHelper::matchCombineExtOfExt(
     MachineInstr &MI, std::tuple<Register, unsigned> &MatchInfo) {
   assert((MI.getOpcode() == TargetOpcode::G_ANYEXT ||
           MI.getOpcode() == TargetOpcode::G_SEXT ||
           MI.getOpcode() == TargetOpcode::G_ZEXT) &&
          "Expected a G_[ASZ]EXT");
   Register SrcReg = MI.getOperand(1).getReg();
   MachineInstr *SrcMI = MRI.getVRegDef(SrcReg);
   // Match exts with the same opcode, anyext([sz]ext) and sext(zext).
   unsigned Opc = MI.getOpcode();
   unsigned SrcOpc = SrcMI->getOpcode();
   if (Opc == SrcOpc ||
       (Opc == TargetOpcode::G_ANYEXT &&
        (SrcOpc == TargetOpcode::G_SEXT || SrcOpc == TargetOpcode::G_ZEXT)) ||
       (Opc == TargetOpcode::G_SEXT && SrcOpc == TargetOpcode::G_ZEXT)) {
     MatchInfo = std::make_tuple(SrcMI->getOperand(1).getReg(), SrcOpc);
     return true;
   }
   return false;
 }

llvm/test/CodeGen/AArch64/GlobalISel/combine-ext.mir

liveins: $s0
     ; CHECK: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $s0
     ; CHECK: [[ZEXT:%[0-9]+]]:_(<2 x s64>) = G_ZEXT [[COPY]](<2 x s16>)
     ; CHECK: $q0 = COPY [[ZEXT]](<2 x s64>)
     %0:_(<2 x s16>) = COPY $s0
     %1:_(<2 x s32>) = G_ZEXT %0(<2 x s16>)
     %2:_(<2 x s64>) = G_ZEXT %1(<2 x s32>)
     $q0 = COPY %2(<2 x s64>)
 ...
+---
+name:            test_combine_half_to_float_fpext_constant
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: test_combine_half_to_float_fpext_constant
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_FCONSTANT float -5.500000e+00
+    ; CHECK: $w0 = COPY [[C]](s32)
+    %0:_(s16) = G_FCONSTANT half 0xHC580
+    %1:_(s32) = G_FPEXT %0(s16)
+    $w0 = COPY %1(s32)
+...
+---
+name:            test_combine_half_to_double_fpext_constant
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: test_combine_half_to_double_fpext_constant
+    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_FCONSTANT double -5.500000e+00
+    ; CHECK: $x0 = COPY [[C]](s64)
+    %0:_(s16) = G_FCONSTANT half 0xHC580
+    %1:_(s64) = G_FPEXT %0(s16)
+    $x0 = COPY %1(s64)
+...
+---
+name:            test_combine_float_to_double_fpext_constant
+body:             |
+  bb.1:
+    ; CHECK-LABEL: name: test_combine_float_to_double_fpext_constant
+    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_FCONSTANT double 5.500000e+00
+    ; CHECK: $x0 = COPY [[C]](s64)
+    %0:_(s32) = G_FCONSTANT float 5.500000e+00
+    %1:_(s64) = G_FPEXT %0(s32)
+    $x0 = COPY %1(s64)
+...

llvm/test/CodeGen/AArch64/GlobalISel/combine-fpext-suitofp.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -o - -mtriple=aarch64-unknown-unknown -run-pass=aarch64-prelegalizer-combiner -verify-machineinstrs  %s | FileCheck %s
+
+# (fpext ([us]itofp x)) -> ([us]itofp x)
+# Due to loss of precision, this is only enabled for x = i8.
+---
+name:            test_combine_fpext_sitofp_8_to_16
+body:             |
+  bb.1:
+    liveins: $b0
+    ; CHECK-LABEL: name: test_combine_fpext_sitofp_8_to_16
+    ; CHECK: [[COPY:%[0-9]+]]:_(s8) = COPY $b0
+    ; CHECK: [[SITOFP:%[0-9]+]]:_(s16) = G_SITOFP [[COPY]](s8)
+    ; CHECK: $h1 = COPY [[SITOFP]](s16)
+    %0:_(s8) = COPY $b0
+    %1:_(s8) = G_SITOFP %0(s8)
+    %2:_(s16) = G_FPEXT %1(s8)
+    $h1 = COPY %2(s16)
+...
+---
+name:            test_combine_fpext_sitofp_8_to_32
+body:             |
+  bb.1:
+    liveins: $b0
+    ; CHECK-LABEL: name: test_combine_fpext_sitofp_8_to_32
+    ; CHECK: [[COPY:%[0-9]+]]:_(s8) = COPY $b0
+    ; CHECK: [[SITOFP:%[0-9]+]]:_(s32) = G_SITOFP [[COPY]](s8)
+    ; CHECK: $w1 = COPY [[SITOFP]](s32)
+    %0:_(s8) = COPY $b0
+    %1:_(s8) = G_SITOFP %0(s8)
+    %2:_(s32) = G_FPEXT %1(s8)
+    $w1 = COPY %2(s32)
+...

llvm/test/CodeGen/AArch64/GlobalISel/combine-fptrunc.mir

body:             |
   bb.1:
     ; CHECK-LABEL: name: test_combine_double_to_foat_fptrunc_constant
     ; CHECK: [[C:%[0-9]+]]:_(s32) = G_FCONSTANT float 0x4010CCCCC0000000
     ; CHECK: $w0 = COPY [[C]](s32)
     %0:_(s64) = G_FCONSTANT double 4.200000e+00
     %1:_(s32) = G_FPTRUNC %0(s64)
     $w0 = COPY %1(s32)
 ...
+# (fptrunc (fpext x)) -> x
+---
+name:            test_combine_truncate_ext_32
+body:             |
+  bb.1:
+    liveins: $w0
+    ; CHECK-LABEL: name: test_combine_truncate_ext_32
+    ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+    ; CHECK: $w1 = COPY [[COPY]](s32)
+    %0:_(s32) = COPY $w0
+    %1:_(s64) = G_FPEXT %0(s32)
+    %2:_(s32) = G_FPTRUNC %1(s64)
+    $w1 = COPY %2(s32)
+...
+---
+name:            test_combine_truncate_ext_16
+body:             |
+  bb.1:
+    liveins: $h0
+    ; CHECK-LABEL: name: test_combine_truncate_ext_16
+    ; CHECK: [[COPY:%[0-9]+]]:_(s16) = COPY $h0
+    ; CHECK: $h1 = COPY [[COPY]](s16)
+    %0:_(s16) = COPY $h0
+    %1:_(s32) = G_FPEXT %0(s16)
+    %2:_(s16) = G_FPTRUNC %1(s32)
+    $h1 = COPY %2(s16)
+...
+# Don't combine when the types mismatch.
+---
+name:            test_combine_truncate_ext_32_non_identity
+body:             |
+  bb.1:
+    liveins: $w0
+    ; CHECK-LABEL: name: test_combine_truncate_ext_32_non_identity
+    ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+    ; CHECK: [[FPEXT:%[0-9]+]]:_(s64) = G_FPEXT [[COPY]](s32)
+    ; CHECK: [[FPTRUNC:%[0-9]+]]:_(s16) = G_FPTRUNC [[FPEXT]](s64)
+    ; CHECK: $h1 = COPY [[FPTRUNC]](s16)
+    %0:_(s32) = COPY $w0
+    %1:_(s64) = G_FPEXT %0(s32)
+    %2:_(s16) = G_FPTRUNC %1(s64)
+    $h1 = COPY %2(s16)
+...
+---
+name:            test_combine_truncate_ext_16_non_identity
+body:             |
+  bb.1:
+    liveins: $h0
+    ; CHECK-LABEL: name: test_combine_truncate_ext_16_non_identity
+    ; CHECK: [[COPY:%[0-9]+]]:_(s16) = COPY $h0
+    ; CHECK: [[FPEXT:%[0-9]+]]:_(s64) = G_FPEXT [[COPY]](s16)
+    ; CHECK: [[FPTRUNC:%[0-9]+]]:_(s32) = G_FPTRUNC [[FPEXT]](s64)
+    ; CHECK: $w1 = COPY [[FPTRUNC]](s32)
+    %0:_(s16) = COPY $h0
+    %1:_(s64) = G_FPEXT %0(s16)
+    %2:_(s32) = G_FPTRUNC %1(s64)
+    $w1 = COPY %2(s32)
+...
+# (fptrunc ([us]itofp x)) -> ([us]itofp x)
+---
+name:            test_combine_truncate_sitofp_16
+body:             |
+  bb.1:
+    liveins: $h0
+    ; CHECK-LABEL: name: test_combine_truncate_sitofp_16
+    ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+    ; CHECK: [[SITOFP:%[0-9]+]]:_(s16) = G_SITOFP [[COPY]](s32)
+    ; CHECK: $h1 = COPY [[SITOFP]](s16)
+    %0:_(s32) = COPY $w0
+    %1:_(s32) = G_SITOFP %0(s32)
+    %2:_(s16) = G_FPTRUNC %1(s32)
+    $h1 = COPY %2(s16)
+...
+---
+name:            test_combine_truncate_uitofp_32
+body:             |
+  bb.1:
+    liveins: $h0
+    ; CHECK-LABEL: name: test_combine_truncate_uitofp_32
+    ; CHECK: [[COPY:%[0-9]+]]:_(s64) = COPY $x0
+    ; CHECK: [[UITOFP:%[0-9]+]]:_(s32) = G_UITOFP [[COPY]](s64)
+    ; CHECK: $w1 = COPY [[UITOFP]](s32)
+    %0:_(s64) = COPY $x0
+    %1:_(s64) = G_UITOFP %0(s64)
+    %2:_(s32) = G_FPTRUNC %1(s64)
+    $w1 = COPY %2(s32)
+...

llvm/test/CodeGen/AMDGPU/GlobalISel/fdiv.f16.ll

; GFX10-NEXT:    s_setpc_b64 s[30:31]
   %fdiv = fdiv half %a, %b, !fpmath !0
   ret half %fdiv
 }
 
 define half @v_rcp_f16(half %x) {
 ; GFX6-IEEE-LABEL: v_rcp_f16:
 ; GFX6-IEEE:       ; %bb.0:
 ; GFX6-IEEE-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v1, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, v1
-; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v3, v2
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v4, vcc, v1, v0, v1
-; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
-; GFX6-IEEE-NEXT:    v_fma_f32 v3, v5, v3, v3
-; GFX6-IEEE-NEXT:    v_mul_f32_e32 v5, v4, v3
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v2, v5, v4
-; GFX6-IEEE-NEXT:    v_fma_f32 v5, v6, v3, v5
-; GFX6-IEEE-NEXT:    v_fma_f32 v2, -v2, v5, v4
-; GFX6-IEEE-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
-; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v2, v0, v1
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v1, s[4:5], v0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v2, v1
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v3, vcc, 1.0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v4, -v1, v2, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v2, v4, v2, v2
+; GFX6-IEEE-NEXT:    v_mul_f32_e32 v4, v3, v2
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v1, v4, v3
+; GFX6-IEEE-NEXT:    v_fma_f32 v4, v5, v2, v4
+; GFX6-IEEE-NEXT:    v_fma_f32 v1, -v1, v4, v3
+; GFX6-IEEE-NEXT:    v_div_fmas_f32 v1, v1, v2, v4
+; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v1, v0, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f16_f32_e32 v0, v0
 ; GFX6-IEEE-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX6-FLUSH-LABEL: v_rcp_f16:
 ; GFX6-FLUSH:       ; %bb.0:
 ; GFX6-FLUSH-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v1, 1.0
 ; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, v1
-; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v3, v2
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v4, vcc, v1, v0, v1
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v1, s[4:5], v0, v0, 1.0
+; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v2, v1
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v3, vcc, 1.0, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
-; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v5, v3, v3
-; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v5, v4, v3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v5, v4
-; GFX6-FLUSH-NEXT:    v_fma_f32 v5, v6, v3, v5
-; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v5, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v4, -v1, v2, 1.0
+; GFX6-FLUSH-NEXT:    v_fma_f32 v2, v4, v2, v2
+; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v4, v3, v2
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v1, v4, v3
+; GFX6-FLUSH-NEXT:    v_fma_f32 v4, v5, v2, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v1, -v1, v4, v3
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
-; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
-; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v2, v0, v1
+; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v1, v1, v2, v4
+; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v1, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
 ; GFX6-FLUSH-NEXT:    v_cvt_f16_f32_e32 v0, v0
 ; GFX6-FLUSH-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX89-LABEL: v_rcp_f16:
 ; GFX89:       ; %bb.0:
 ; GFX89-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX89-NEXT:    v_cvt_f32_f16_e32 v1, v0
-; GFX89-NEXT:    v_cvt_f32_f16_e32 v2, 1.0
 ; GFX89-NEXT:    v_rcp_f32_e32 v1, v1
-; GFX89-NEXT:    v_mul_f32_e32 v1, v2, v1
+; GFX89-NEXT:    v_mul_f32_e32 v1, 1.0, v1
 ; GFX89-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX89-NEXT:    v_div_fixup_f16 v0, v1, v0, 1.0
 ; GFX89-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: v_rcp_f16:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    v_cvt_f32_f16_e32 v1, v0
-; GFX10-NEXT:    v_cvt_f32_f16_e32 v2, 1.0
 ; GFX10-NEXT:    v_rcp_f32_e32 v1, v1
-; GFX10-NEXT:    v_mul_f32_e32 v1, v2, v1
+; GFX10-NEXT:    v_mul_f32_e32 v1, 1.0, v1
 ; GFX10-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX10-NEXT:    v_div_fixup_f16 v0, v1, v0, 1.0
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]
   %fdiv = fdiv half 1.0, %x
   ret half %fdiv
 }
 
 define half @v_rcp_f16_arcp(half %x) {
 ; GFX6-IEEE-LABEL: v_rcp_f16_arcp:
 ; GFX6-IEEE:       ; %bb.0:
 ; GFX6-IEEE-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v1, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, v1
-; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v3, v2
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v4, vcc, v1, v0, v1
-; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
-; GFX6-IEEE-NEXT:    v_fma_f32 v3, v5, v3, v3
-; GFX6-IEEE-NEXT:    v_mul_f32_e32 v5, v4, v3
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v2, v5, v4
-; GFX6-IEEE-NEXT:    v_fma_f32 v5, v6, v3, v5
-; GFX6-IEEE-NEXT:    v_fma_f32 v2, -v2, v5, v4
-; GFX6-IEEE-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
-; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v2, v0, v1
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v1, s[4:5], v0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v2, v1
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v3, vcc, 1.0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v4, -v1, v2, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v2, v4, v2, v2
+; GFX6-IEEE-NEXT:    v_mul_f32_e32 v4, v3, v2
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v1, v4, v3
+; GFX6-IEEE-NEXT:    v_fma_f32 v4, v5, v2, v4
+; GFX6-IEEE-NEXT:    v_fma_f32 v1, -v1, v4, v3
+; GFX6-IEEE-NEXT:    v_div_fmas_f32 v1, v1, v2, v4
+; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v1, v0, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f16_f32_e32 v0, v0
 ; GFX6-IEEE-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX6-FLUSH-LABEL: v_rcp_f16_arcp:
 ; GFX6-FLUSH:       ; %bb.0:
 ; GFX6-FLUSH-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v1, 1.0
 ; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, v1
-; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v3, v2
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v4, vcc, v1, v0, v1
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v1, s[4:5], v0, v0, 1.0
+; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v2, v1
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v3, vcc, 1.0, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
-; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v5, v3, v3
-; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v5, v4, v3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v5, v4
-; GFX6-FLUSH-NEXT:    v_fma_f32 v5, v6, v3, v5
-; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v5, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v4, -v1, v2, 1.0
+; GFX6-FLUSH-NEXT:    v_fma_f32 v2, v4, v2, v2
+; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v4, v3, v2
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v1, v4, v3
+; GFX6-FLUSH-NEXT:    v_fma_f32 v4, v5, v2, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v1, -v1, v4, v3
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
-; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
-; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v2, v0, v1
+; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v1, v1, v2, v4
+; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v1, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
 ; GFX6-FLUSH-NEXT:    v_cvt_f16_f32_e32 v0, v0
 ; GFX6-FLUSH-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX89-LABEL: v_rcp_f16_arcp:
 ; GFX89:       ; %bb.0:
 ; GFX89-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX89-NEXT:    v_cvt_f32_f16_e32 v1, v0
-; GFX89-NEXT:    v_cvt_f32_f16_e32 v2, 1.0
 ; GFX89-NEXT:    v_rcp_f32_e32 v1, v1
-; GFX89-NEXT:    v_mul_f32_e32 v1, v2, v1
+; GFX89-NEXT:    v_mul_f32_e32 v1, 1.0, v1
 ; GFX89-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX89-NEXT:    v_div_fixup_f16 v0, v1, v0, 1.0
 ; GFX89-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: v_rcp_f16_arcp:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    v_cvt_f32_f16_e32 v1, v0
-; GFX10-NEXT:    v_cvt_f32_f16_e32 v2, 1.0
 ; GFX10-NEXT:    v_rcp_f32_e32 v1, v1
-; GFX10-NEXT:    v_mul_f32_e32 v1, v2, v1
+; GFX10-NEXT:    v_mul_f32_e32 v1, 1.0, v1
 ; GFX10-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX10-NEXT:    v_div_fixup_f16 v0, v1, v0, 1.0
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]
   %fdiv = fdiv arcp half 1.0, %x
   ret half %fdiv
 }
 
 define half @v_rcp_f16_arcp_afn(half %x) {
 ; GFX6-LABEL: v_rcp_f16_arcp_afn:
 ; GFX6:       ; %bb.0:
 ; GFX6-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX6-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-NEXT:    v_cvt_f32_f16_e32 v1, 1.0
 ; GFX6-NEXT:    v_rcp_f32_e32 v0, v0
-; GFX6-NEXT:    v_mul_f32_e32 v0, v1, v0
+; GFX6-NEXT:    v_mul_f32_e32 v0, 1.0, v0
 ; GFX6-NEXT:    v_cvt_f16_f32_e32 v0, v0
 ; GFX6-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX89-LABEL: v_rcp_f16_arcp_afn:
 ; GFX89:       ; %bb.0:
 ; GFX89-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX89-NEXT:    v_rcp_f16_e32 v0, v0
 ; GFX89-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: v_rcp_f16_arcp_afn:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    v_rcp_f16_e32 v0, v0
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]
   %fdiv = fdiv arcp afn half 1.0, %x
   ret half %fdiv
 }
 
 define half @v_rcp_f16_ulp25(half %x) {
 ; GFX6-IEEE-LABEL: v_rcp_f16_ulp25:
 ; GFX6-IEEE:       ; %bb.0:
 ; GFX6-IEEE-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v1, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, v1
-; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v3, v2
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v4, vcc, v1, v0, v1
-; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
-; GFX6-IEEE-NEXT:    v_fma_f32 v3, v5, v3, v3
-; GFX6-IEEE-NEXT:    v_mul_f32_e32 v5, v4, v3
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v2, v5, v4
-; GFX6-IEEE-NEXT:    v_fma_f32 v5, v6, v3, v5
-; GFX6-IEEE-NEXT:    v_fma_f32 v2, -v2, v5, v4
-; GFX6-IEEE-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
-; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v2, v0, v1
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v1, s[4:5], v0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v2, v1
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v3, vcc, 1.0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v4, -v1, v2, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v2, v4, v2, v2
+; GFX6-IEEE-NEXT:    v_mul_f32_e32 v4, v3, v2
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v1, v4, v3
+; GFX6-IEEE-NEXT:    v_fma_f32 v4, v5, v2, v4
+; GFX6-IEEE-NEXT:    v_fma_f32 v1, -v1, v4, v3
+; GFX6-IEEE-NEXT:    v_div_fmas_f32 v1, v1, v2, v4
+; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v1, v0, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f16_f32_e32 v0, v0
 ; GFX6-IEEE-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX6-FLUSH-LABEL: v_rcp_f16_ulp25:
 ; GFX6-FLUSH:       ; %bb.0:
 ; GFX6-FLUSH-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v1, 1.0
 ; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, v1
-; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v3, v2
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v4, vcc, v1, v0, v1
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v1, s[4:5], v0, v0, 1.0
+; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v2, v1
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v3, vcc, 1.0, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
-; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v5, v3, v3
-; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v5, v4, v3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v5, v4
-; GFX6-FLUSH-NEXT:    v_fma_f32 v5, v6, v3, v5
-; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v5, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v4, -v1, v2, 1.0
+; GFX6-FLUSH-NEXT:    v_fma_f32 v2, v4, v2, v2
+; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v4, v3, v2
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v1, v4, v3
+; GFX6-FLUSH-NEXT:    v_fma_f32 v4, v5, v2, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v1, -v1, v4, v3
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
-; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
-; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v2, v0, v1
+; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v1, v1, v2, v4
+; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v1, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
 ; GFX6-FLUSH-NEXT:    v_cvt_f16_f32_e32 v0, v0
 ; GFX6-FLUSH-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX89-LABEL: v_rcp_f16_ulp25:
 ; GFX89:       ; %bb.0:
 ; GFX89-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX89-NEXT:    v_rcp_f16_e32 v0, v0
; GFX10-NEXT:    s_setpc_b64 s[30:31]
   %fdiv = fdiv <2 x half> %a, %b, !fpmath !0
   ret <2 x half> %fdiv
 }
 
 define <2 x half> @v_rcp_v2f16(<2 x half> %x) {
 ; GFX6-IEEE-LABEL: v_rcp_v2f16:
 ; GFX6-IEEE:       ; %bb.0:
 ; GFX6-IEEE-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v2, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v0, v0
 ; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v1, v1
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v3, s[4:5], v0, v0, v2
-; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v4, v3
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v5, vcc, v2, v0, v2
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v3, v4, 1.0
-; GFX6-IEEE-NEXT:    v_fma_f32 v4, v6, v4, v4
-; GFX6-IEEE-NEXT:    v_mul_f32_e32 v6, v5, v4
-; GFX6-IEEE-NEXT:    v_fma_f32 v7, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, v7, v4, v6
-; GFX6-IEEE-NEXT:    v_fma_f32 v3, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_div_fmas_f32 v3, v3, v4, v6
-; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v3, v0, v2
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v3, s[4:5], v1, v1, v2
-; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v4, v3
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v5, vcc, v2, v1, v2
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v3, v2
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-IEEE-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-IEEE-NEXT:    v_fma_f32 v2, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v2, v0, 1.0
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v2, s[4:5], v1, v1, 1.0
+; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v3, v2
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v1, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f16_f32_e32 v0, v0
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v3, v4, 1.0
-; GFX6-IEEE-NEXT:    v_fma_f32 v4, v6, v4, v4
-; GFX6-IEEE-NEXT:    v_mul_f32_e32 v6, v5, v4
-; GFX6-IEEE-NEXT:    v_fma_f32 v7, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, v7, v4, v6
-; GFX6-IEEE-NEXT:    v_fma_f32 v3, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_div_fmas_f32 v3, v3, v4, v6
-; GFX6-IEEE-NEXT:    v_div_fixup_f32 v1, v3, v1, v2
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-IEEE-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-IEEE-NEXT:    v_fma_f32 v2, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-IEEE-NEXT:    v_div_fixup_f32 v1, v2, v1, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX6-IEEE-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX6-FLUSH-LABEL: v_rcp_v2f16:
 ; GFX6-FLUSH:       ; %bb.0:
 ; GFX6-FLUSH-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-FLUSH-NEXT:    s_movk_i32 s6, 0x3c00
-; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v2, s6
 ; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v3, s[4:5], v0, v0, v2
-; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v4, v3
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v5, vcc, v2, v0, v2
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, 1.0
+; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v3, v2
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v3, v4, 1.0
-; GFX6-FLUSH-NEXT:    v_fma_f32 v4, v6, v4, v4
-; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v6, v5, v4
-; GFX6-FLUSH-NEXT:    v_fma_f32 v7, -v3, v6, v5
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, v7, v4, v6
-; GFX6-FLUSH-NEXT:    v_fma_f32 v3, -v3, v6, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v5, v4
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
-; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v3, v3, v4, v6
-; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v4, s6
 ; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v1, v1
-; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v3, v0, v2
+; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v2, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
 ; GFX6-FLUSH-NEXT:    v_cvt_f16_f32_e32 v0, v0
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v1, v1, v4
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v1, v1, 1.0
 ; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v3, v2
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v5, vcc, v4, v1, v4
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v1, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v3, 1.0
-; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v6, v3, v3
-; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v6, v5, v3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v7, -v2, v6, v5
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, v7, v3, v6
-; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v6, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v5, v4
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
-; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v6
-; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v1, v2, v1, v4
+; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v1, v2, v1, 1.0
 ; GFX6-FLUSH-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX6-FLUSH-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX8-LABEL: v_rcp_v2f16:
 ; GFX8:       ; %bb.0:
 ; GFX8-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX8-NEXT:    v_lshrrev_b32_e32 v2, 16, v0
 ; GFX8-NEXT:    v_cvt_f32_f16_e32 v1, v0
 ; GFX8-NEXT:    v_cvt_f32_f16_e32 v3, v2
-; GFX8-NEXT:    v_cvt_f32_f16_e32 v4, 1.0
 ; GFX8-NEXT:    v_rcp_f32_e32 v1, v1
 ; GFX8-NEXT:    v_rcp_f32_e32 v3, v3
-; GFX8-NEXT:    v_mul_f32_e32 v1, v4, v1
-; GFX8-NEXT:    v_mul_f32_e32 v3, v4, v3
+; GFX8-NEXT:    v_mul_f32_e32 v1, 1.0, v1
+; GFX8-NEXT:    v_mul_f32_e32 v3, 1.0, v3
 ; GFX8-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX8-NEXT:    v_cvt_f16_f32_e32 v3, v3
 ; GFX8-NEXT:    v_div_fixup_f16 v0, v1, v0, 1.0
 ; GFX8-NEXT:    v_div_fixup_f16 v1, v3, v2, 1.0
 ; GFX8-NEXT:    v_mov_b32_e32 v2, 16
 ; GFX8-NEXT:    v_lshlrev_b32_sdwa v1, v2, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
 ; GFX8-NEXT:    v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
 ; GFX8-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-LABEL: v_rcp_v2f16:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    v_lshrrev_b32_e32 v2, 16, v0
 ; GFX9-NEXT:    v_cvt_f32_f16_e32 v1, v0
 ; GFX9-NEXT:    v_cvt_f32_f16_e32 v3, v2
-; GFX9-NEXT:    v_cvt_f32_f16_e32 v4, 1.0
 ; GFX9-NEXT:    v_rcp_f32_e32 v1, v1
 ; GFX9-NEXT:    v_rcp_f32_e32 v3, v3
-; GFX9-NEXT:    v_mul_f32_e32 v1, v4, v1
-; GFX9-NEXT:    v_mul_f32_e32 v3, v4, v3
+; GFX9-NEXT:    v_mul_f32_e32 v1, 1.0, v1
+; GFX9-NEXT:    v_mul_f32_e32 v3, 1.0, v3
 ; GFX9-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX9-NEXT:    v_cvt_f16_f32_e32 v3, v3
 ; GFX9-NEXT:    v_div_fixup_f16 v0, v1, v0, 1.0
 ; GFX9-NEXT:    v_div_fixup_f16 v1, v3, v2, 1.0
 ; GFX9-NEXT:    v_mov_b32_e32 v2, 0xffff
 ; GFX9-NEXT:    v_lshlrev_b32_e32 v1, 16, v1
 ; GFX9-NEXT:    v_and_or_b32 v0, v0, v2, v1
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: v_rcp_v2f16:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    v_lshrrev_b32_e32 v1, 16, v0
 ; GFX10-NEXT:    v_cvt_f32_f16_e32 v3, v0
-; GFX10-NEXT:    v_cvt_f32_f16_e32 v4, 1.0
 ; GFX10-NEXT:    v_cvt_f32_f16_e32 v2, v1
 ; GFX10-NEXT:    v_rcp_f32_e32 v3, v3
 ; GFX10-NEXT:    v_rcp_f32_e32 v2, v2
-; GFX10-NEXT:    v_mul_f32_e32 v3, v4, v3
-; GFX10-NEXT:    v_mul_f32_e32 v2, v4, v2
+; GFX10-NEXT:    v_mul_f32_e32 v3, 1.0, v3
+; GFX10-NEXT:    v_mul_f32_e32 v2, 1.0, v2
 ; GFX10-NEXT:    v_cvt_f16_f32_e32 v3, v3
 ; GFX10-NEXT:    v_cvt_f16_f32_e32 v2, v2
 ; GFX10-NEXT:    v_div_fixup_f16 v0, v3, v0, 1.0
 ; GFX10-NEXT:    v_div_fixup_f16 v1, v2, v1, 1.0
 ; GFX10-NEXT:    v_lshlrev_b32_e32 v1, 16, v1
 ; GFX10-NEXT:    v_and_or_b32 v0, 0xffff, v0, v1
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]
   %fdiv = fdiv <2 x half> <half 1.0, half 1.0>, %x
   ret <2 x half> %fdiv
 }
 
 define <2 x half> @v_rcp_v2f16_arcp(<2 x half> %x) {
 ; GFX6-IEEE-LABEL: v_rcp_v2f16_arcp:
 ; GFX6-IEEE:       ; %bb.0:
 ; GFX6-IEEE-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v2, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v0, v0
 ; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v1, v1
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v3, s[4:5], v0, v0, v2
-; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v4, v3
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v5, vcc, v2, v0, v2
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v3, v4, 1.0
-; GFX6-IEEE-NEXT:    v_fma_f32 v4, v6, v4, v4
-; GFX6-IEEE-NEXT:    v_mul_f32_e32 v6, v5, v4
-; GFX6-IEEE-NEXT:    v_fma_f32 v7, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, v7, v4, v6
-; GFX6-IEEE-NEXT:    v_fma_f32 v3, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_div_fmas_f32 v3, v3, v4, v6
-; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v3, v0, v2
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v3, s[4:5], v1, v1, v2
-; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v4, v3
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v5, vcc, v2, v1, v2
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v3, v2
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-IEEE-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-IEEE-NEXT:    v_fma_f32 v2, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v2, v0, 1.0
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v2, s[4:5], v1, v1, 1.0
+; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v3, v2
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v1, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f16_f32_e32 v0, v0
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v3, v4, 1.0
-; GFX6-IEEE-NEXT:    v_fma_f32 v4, v6, v4, v4
-; GFX6-IEEE-NEXT:    v_mul_f32_e32 v6, v5, v4
-; GFX6-IEEE-NEXT:    v_fma_f32 v7, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, v7, v4, v6
-; GFX6-IEEE-NEXT:    v_fma_f32 v3, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_div_fmas_f32 v3, v3, v4, v6
-; GFX6-IEEE-NEXT:    v_div_fixup_f32 v1, v3, v1, v2
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-IEEE-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-IEEE-NEXT:    v_fma_f32 v2, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-IEEE-NEXT:    v_div_fixup_f32 v1, v2, v1, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX6-IEEE-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX6-FLUSH-LABEL: v_rcp_v2f16_arcp:
 ; GFX6-FLUSH:       ; %bb.0:
 ; GFX6-FLUSH-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-FLUSH-NEXT:    s_movk_i32 s6, 0x3c00
-; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v2, s6
 ; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v3, s[4:5], v0, v0, v2
-; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v4, v3
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v5, vcc, v2, v0, v2
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, 1.0
+; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v3, v2
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v3, v4, 1.0
-; GFX6-FLUSH-NEXT:    v_fma_f32 v4, v6, v4, v4
-; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v6, v5, v4
-; GFX6-FLUSH-NEXT:    v_fma_f32 v7, -v3, v6, v5
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, v7, v4, v6
-; GFX6-FLUSH-NEXT:    v_fma_f32 v3, -v3, v6, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v5, v4
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
-; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v3, v3, v4, v6
-; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v4, s6
 ; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v1, v1
-; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v3, v0, v2
+; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v2, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
 ; GFX6-FLUSH-NEXT:    v_cvt_f16_f32_e32 v0, v0
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v1, v1, v4
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v1, v1, 1.0
 ; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v3, v2
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v5, vcc, v4, v1, v4
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v1, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v3, 1.0
-; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v6, v3, v3
-; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v6, v5, v3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v7, -v2, v6, v5
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, v7, v3, v6
-; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v6, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v5, v4
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
-; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v6
-; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v1, v2, v1, v4
+; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v1, v2, v1, 1.0
 ; GFX6-FLUSH-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX6-FLUSH-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX8-LABEL: v_rcp_v2f16_arcp:
 ; GFX8:       ; %bb.0:
 ; GFX8-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX8-NEXT:    v_lshrrev_b32_e32 v2, 16, v0
 ; GFX8-NEXT:    v_cvt_f32_f16_e32 v1, v0
 ; GFX8-NEXT:    v_cvt_f32_f16_e32 v3, v2
-; GFX8-NEXT:    v_cvt_f32_f16_e32 v4, 1.0
 ; GFX8-NEXT:    v_rcp_f32_e32 v1, v1
 ; GFX8-NEXT:    v_rcp_f32_e32 v3, v3
-; GFX8-NEXT:    v_mul_f32_e32 v1, v4, v1
-; GFX8-NEXT:    v_mul_f32_e32 v3, v4, v3
+; GFX8-NEXT:    v_mul_f32_e32 v1, 1.0, v1
+; GFX8-NEXT:    v_mul_f32_e32 v3, 1.0, v3
 ; GFX8-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX8-NEXT:    v_cvt_f16_f32_e32 v3, v3
 ; GFX8-NEXT:    v_div_fixup_f16 v0, v1, v0, 1.0
 ; GFX8-NEXT:    v_div_fixup_f16 v1, v3, v2, 1.0
 ; GFX8-NEXT:    v_mov_b32_e32 v2, 16
 ; GFX8-NEXT:    v_lshlrev_b32_sdwa v1, v2, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
 ; GFX8-NEXT:    v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
 ; GFX8-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-LABEL: v_rcp_v2f16_arcp:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    v_lshrrev_b32_e32 v2, 16, v0
 ; GFX9-NEXT:    v_cvt_f32_f16_e32 v1, v0
 ; GFX9-NEXT:    v_cvt_f32_f16_e32 v3, v2
-; GFX9-NEXT:    v_cvt_f32_f16_e32 v4, 1.0
 ; GFX9-NEXT:    v_rcp_f32_e32 v1, v1
 ; GFX9-NEXT:    v_rcp_f32_e32 v3, v3
-; GFX9-NEXT:    v_mul_f32_e32 v1, v4, v1
-; GFX9-NEXT:    v_mul_f32_e32 v3, v4, v3
+; GFX9-NEXT:    v_mul_f32_e32 v1, 1.0, v1
+; GFX9-NEXT:    v_mul_f32_e32 v3, 1.0, v3
 ; GFX9-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX9-NEXT:    v_cvt_f16_f32_e32 v3, v3
 ; GFX9-NEXT:    v_div_fixup_f16 v0, v1, v0, 1.0
 ; GFX9-NEXT:    v_div_fixup_f16 v1, v3, v2, 1.0
 ; GFX9-NEXT:    v_mov_b32_e32 v2, 0xffff
 ; GFX9-NEXT:    v_lshlrev_b32_e32 v1, 16, v1
 ; GFX9-NEXT:    v_and_or_b32 v0, v0, v2, v1
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: v_rcp_v2f16_arcp:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    v_lshrrev_b32_e32 v1, 16, v0
 ; GFX10-NEXT:    v_cvt_f32_f16_e32 v3, v0
-; GFX10-NEXT:    v_cvt_f32_f16_e32 v4, 1.0
 ; GFX10-NEXT:    v_cvt_f32_f16_e32 v2, v1
 ; GFX10-NEXT:    v_rcp_f32_e32 v3, v3
 ; GFX10-NEXT:    v_rcp_f32_e32 v2, v2
-; GFX10-NEXT:    v_mul_f32_e32 v3, v4, v3
-; GFX10-NEXT:    v_mul_f32_e32 v2, v4, v2
+; GFX10-NEXT:    v_mul_f32_e32 v3, 1.0, v3
+; GFX10-NEXT:    v_mul_f32_e32 v2, 1.0, v2
 ; GFX10-NEXT:    v_cvt_f16_f32_e32 v3, v3
 ; GFX10-NEXT:    v_cvt_f16_f32_e32 v2, v2
 ; GFX10-NEXT:    v_div_fixup_f16 v0, v3, v0, 1.0
 ; GFX10-NEXT:    v_div_fixup_f16 v1, v2, v1, 1.0
 ; GFX10-NEXT:    v_lshlrev_b32_e32 v1, 16, v1
 ; GFX10-NEXT:    v_and_or_b32 v0, 0xffff, v0, v1
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]
   %fdiv = fdiv arcp <2 x half> <half 1.0, half 1.0>, %x
   ret <2 x half> %fdiv
 }
 
 define <2 x half> @v_rcp_v2f16_arcp_afn(<2 x half> %x) {
 ; GFX6-LABEL: v_rcp_v2f16_arcp_afn:
 ; GFX6:       ; %bb.0:
 ; GFX6-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX6-NEXT:    v_cvt_f32_f16_e32 v0, v0
 ; GFX6-NEXT:    v_cvt_f32_f16_e32 v1, v1
-; GFX6-NEXT:    v_cvt_f32_f16_e32 v2, 1.0
 ; GFX6-NEXT:    v_rcp_f32_e32 v0, v0
 ; GFX6-NEXT:    v_rcp_f32_e32 v1, v1
-; GFX6-NEXT:    v_mul_f32_e32 v0, v2, v0
-; GFX6-NEXT:    v_mul_f32_e32 v1, v2, v1
+; GFX6-NEXT:    v_mul_f32_e32 v0, 1.0, v0
+; GFX6-NEXT:    v_mul_f32_e32 v1, 1.0, v1
 ; GFX6-NEXT:    v_cvt_f16_f32_e32 v0, v0
 ; GFX6-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX6-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX8-LABEL: v_rcp_v2f16_arcp_afn:
 ; GFX8:       ; %bb.0:
 ; GFX8-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX8-NEXT:    v_rcp_f16_sdwa v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
; GFX10-NEXT:    s_setpc_b64 s[30:31]
   %fdiv = fdiv arcp afn <2 x half> <half 1.0, half 1.0>, %x
   ret <2 x half> %fdiv
 }
 
 define <2 x half> @v_rcp_v2f16_ulp25(<2 x half> %x) {
 ; GFX6-IEEE-LABEL: v_rcp_v2f16_ulp25:
 ; GFX6-IEEE:       ; %bb.0:
 ; GFX6-IEEE-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v2, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v0, v0
 ; GFX6-IEEE-NEXT:    v_cvt_f32_f16_e32 v1, v1
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v3, s[4:5], v0, v0, v2
-; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v4, v3
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v5, vcc, v2, v0, v2
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v3, v4, 1.0
-; GFX6-IEEE-NEXT:    v_fma_f32 v4, v6, v4, v4
-; GFX6-IEEE-NEXT:    v_mul_f32_e32 v6, v5, v4
-; GFX6-IEEE-NEXT:    v_fma_f32 v7, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, v7, v4, v6
-; GFX6-IEEE-NEXT:    v_fma_f32 v3, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_div_fmas_f32 v3, v3, v4, v6
-; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v3, v0, v2
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v3, s[4:5], v1, v1, v2
-; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v4, v3
-; GFX6-IEEE-NEXT:    v_div_scale_f32 v5, vcc, v2, v1, v2
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v3, v2
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v0, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-IEEE-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-IEEE-NEXT:    v_fma_f32 v2, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-IEEE-NEXT:    v_div_fixup_f32 v0, v2, v0, 1.0
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v2, s[4:5], v1, v1, 1.0
+; GFX6-IEEE-NEXT:    v_rcp_f32_e32 v3, v2
+; GFX6-IEEE-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v1, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f16_f32_e32 v0, v0
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v3, v4, 1.0
-; GFX6-IEEE-NEXT:    v_fma_f32 v4, v6, v4, v4
-; GFX6-IEEE-NEXT:    v_mul_f32_e32 v6, v5, v4
-; GFX6-IEEE-NEXT:    v_fma_f32 v7, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_fma_f32 v6, v7, v4, v6
-; GFX6-IEEE-NEXT:    v_fma_f32 v3, -v3, v6, v5
-; GFX6-IEEE-NEXT:    v_div_fmas_f32 v3, v3, v4, v6
-; GFX6-IEEE-NEXT:    v_div_fixup_f32 v1, v3, v1, v2
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-IEEE-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-IEEE-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-IEEE-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-IEEE-NEXT:    v_fma_f32 v2, -v2, v5, v4
+; GFX6-IEEE-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-IEEE-NEXT:    v_div_fixup_f32 v1, v2, v1, 1.0
 ; GFX6-IEEE-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX6-IEEE-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX6-FLUSH-LABEL: v_rcp_v2f16_ulp25:
 ; GFX6-FLUSH:       ; %bb.0:
 ; GFX6-FLUSH-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX6-FLUSH-NEXT:    s_movk_i32 s6, 0x3c00
-; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v2, s6
 ; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v0, v0
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v3, s[4:5], v0, v0, v2
-; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v4, v3
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v5, vcc, v2, v0, v2
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v0, v0, 1.0
+; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v3, v2
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v3, v4, 1.0
-; GFX6-FLUSH-NEXT:    v_fma_f32 v4, v6, v4, v4
-; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v6, v5, v4
-; GFX6-FLUSH-NEXT:    v_fma_f32 v7, -v3, v6, v5
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, v7, v4, v6
-; GFX6-FLUSH-NEXT:    v_fma_f32 v3, -v3, v6, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v5, v4
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
-; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v3, v3, v4, v6
-; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v4, s6
 ; GFX6-FLUSH-NEXT:    v_cvt_f32_f16_e32 v1, v1
-; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v3, v0, v2
+; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v0, v2, v0, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
 ; GFX6-FLUSH-NEXT:    v_cvt_f16_f32_e32 v0, v0
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v1, v1, v4
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v2, s[4:5], v1, v1, 1.0
 ; GFX6-FLUSH-NEXT:    v_rcp_f32_e32 v3, v2
-; GFX6-FLUSH-NEXT:    v_div_scale_f32 v5, vcc, v4, v1, v4
+; GFX6-FLUSH-NEXT:    v_div_scale_f32 v4, vcc, 1.0, v1, 1.0
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v3, 1.0
-; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v6, v3, v3
-; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v6, v5, v3
-; GFX6-FLUSH-NEXT:    v_fma_f32 v7, -v2, v6, v5
-; GFX6-FLUSH-NEXT:    v_fma_f32 v6, v7, v3, v6
-; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v6, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, -v2, v3, 1.0
+; GFX6-FLUSH-NEXT:    v_fma_f32 v3, v5, v3, v3
+; GFX6-FLUSH-NEXT:    v_mul_f32_e32 v5, v4, v3
+; GFX6-FLUSH-NEXT:    v_fma_f32 v6, -v2, v5, v4
+; GFX6-FLUSH-NEXT:    v_fma_f32 v5, v6, v3, v5
+; GFX6-FLUSH-NEXT:    v_fma_f32 v2, -v2, v5, v4
 ; GFX6-FLUSH-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
-; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v6
-; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v1, v2, v1, v4
+; GFX6-FLUSH-NEXT:    v_div_fmas_f32 v2, v2, v3, v5
+; GFX6-FLUSH-NEXT:    v_div_fixup_f32 v1, v2, v1, 1.0
 ; GFX6-FLUSH-NEXT:    v_cvt_f16_f32_e32 v1, v1
 ; GFX6-FLUSH-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX8-LABEL: v_rcp_v2f16_ulp25:
 ; GFX8:       ; %bb.0:
 ; GFX8-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX8-NEXT:    v_rcp_f16_sdwa v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
 ; GFX8-NEXT:    v_rcp_f16_e32 v0, v0