[GISel] Add new combines for G_ADD

[mkitzan]

Summary

Patch adds new GICombineRules for G_ADD:

• G_ADD(x, G_SUB(y, x)) -> y
• G_ADD(G_SUB(y, x), x) -> y

Patch additionally adds new combine tests for AArch64 target for these new rules.

[tschuett]

Out of curiosity, is the first rule a win or canonicalisation?

[arsenm]

llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h

415

APInt to allow it to work for wide integers?

llvm/test/CodeGen/AArch64/GlobalISel/combine-add.mir

86

Add an s128 case? This currently won't work for vectors, but it would be nice to handle a few vector tests too

llvm/test/CodeGen/AMDGPU/GlobalISel/extractelement.i128.ll

23–24 ↗

(On Diff #292884)

I don't understand where this regression came from (but isn't particularly important)

54–62 ↗

(On Diff #292884)

This is a much more interesting regression

[mkitzan]

In D87936#2282772, @tschuett wrote:

Out of curiosity, is the first rule a win or canonicalisation?

Primarily canonicalization, however depending on the target there can be perf wins. By eliminating the signed constant, there's a better chance the constant can be folded into the opcode as an immediate operand rather than being loaded into a register before being used (if the target supports immediate operands, otherwise nothing lost, eh?).

Assume hypothetical target only supports 2b immediate operands to arithmetic ops and registers are 8b. Here's an example where the combine is a perf win.

mov r1, -1
add r0, r1
---
sub r0, 1

llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h

415

Make sense, will add

llvm/test/CodeGen/AArch64/GlobalISel/combine-add.mir

86

Will add an s128 and a vector test case for each of these

llvm/test/CodeGen/AMDGPU/GlobalISel/extractelement.i128.ll

54–62 ↗

(On Diff #292884)

I noticed regressions in this test file too. The combine causing them is the: G_ADD(x, y) -> G_OR(x, y) (iff x and y share no common bits). That combine rule is the cause of nearly all the AMDGPU test changes (with the exception of three or four tests).

[arsenm]

llvm/test/CodeGen/AMDGPU/GlobalISel/extractelement.i128.ll

54–62 ↗

(On Diff #292884)

So we need the equivalent utility for SelectionDAG::isBaseWithConstantOffset(

[mkitzan]

llvm/test/CodeGen/AMDGPU/GlobalISel/extractelement.i128.ll

54–62 ↗

(On Diff #292884)

Not quite sure where that fits into the implementation of the combine. The added GlobalISel/Utils.cpp:haveNoCommonBitsSet acts the same as SelectionDAG.pp:haveNoCommonBitsSet which is used to gate the DAG combine version (x + y) -> (x | y) iff x and y share no bits.

[mkitzan]

• Changed G_ADD(x, -cst) -> G_SUB(x, cst) to use APInt
• Added vector and s128 test cases (though some combines don't fire with these input types)

[arsenm]

llvm/test/CodeGen/AMDGPU/GlobalISel/extractelement.i128.ll

54–62 ↗

(On Diff #292884)

It's not for the combine itself, it's for all of the other code to deal with the side effect of it. In general this is a terrible combine because nobody expects adds to turn into ors. Without a utility function you can easily use to match adds that were turned into ors, I think it's generally worse to do this combine. I think having this utility function is a prerequisite for this

[mkitzan]

I can remove the G_ADD(x, y)->G_OR(x, y) combine if that's a blocker. Let me know if there's anything else specifically blocking the approval of this patch.

[mkitzan]

Updates:

• Rebased patch
• Removed G_ADD(x, y) -> G_OR(x, y) combine
• Exchanged my custom applyAddSubSameReg function for replaceSingleDefInstWithReg
• Add @aemerson and @foad to reviewers list

[arsenm]

I do think the add -> or has some value, but it should be separate with the helper function

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

5633–5638

Probably should use m_match here (maybe even have a negative constant matcher)

[mkitzan]

Good point about using mi_match, updated the diff.

Can work on the add -> or combine + helper function in a future patch. Better to not block committing the simpler combines.

[foad]

llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h

739

Does SelectionDAG do this? I suppose it can be useful to canonicalize, but my gut feeling is that it would be better to do the converse, i.e. canonicalize G_SUB(x, cst) -> G_ADD(x, -cst). That way you are canonicalizing on using G_ADD instead of G_SUB wherever possible, which feels more useful than using positive constants instead of negative constants wherever possible. E.g. if you're writing a pattern to match a "reg + simm" addressing mode you would only need to match G_ADD.

[mkitzan]

llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h

739

No, I can't seem to find it in DAGCombiner. Would you rather it be removed from the patch?

[arsenm]

I think addressing modes are less of a factor given that g_ptr_add is separate

[arsenm]

In D87936#3525578, @arsenm wrote:

I think addressing modes are less of a factor given that g_ptr_add is separate

I guess this breaks down as soon as you have a 2 component mode

[mkitzan]

I'll remove that combine as well. If there's more discussion and it's found to be beneficial then I can open another revision for it.

[mkitzan]

Updates:

• Removed G_ADD(x, -cst) -> G_SUB(x, cst)
• Very minor NFC addressing https://reviews.llvm.org/D125819#inline-1207969

[paquette]

I think this looks reasonable now? I basically only have one nit.

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

5639

FYI, in Register.h:

/// Wrapper class representing virtual and physical registers. Should be passed
/// by value.
class Register {
  unsigned Reg;

Since neither MaybeSub nor MaybeSameReg is being mutated, they might as well be passed by value.

[mkitzan]

Thanks for the review @paquette. I'll update the diff addressing your nit when I push the patch

llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h

public:
   /// Transform trunc (shl x, K) to shl (trunc x),
   /// K => K < VT.getScalarSizeInBits().
   bool matchCombineTruncOfShl(MachineInstr &MI,
                               std::pair<Register, Register> &MatchInfo);
   void applyCombineTruncOfShl(MachineInstr &MI,
                               std::pair<Register, Register> &MatchInfo);
 
   /// Transform G_MUL(x, -1) to G_SUB(0, x)
   void applyCombineMulByNegativeOne(MachineInstr &MI);
 
   /// Return true if any explicit use operand on \p MI is defined by a
   /// G_IMPLICIT_DEF.
   bool matchAnyExplicitUseIsUndef(MachineInstr &MI);
 
   /// Return true if all register explicit use operands on \p MI are defined by
   /// a G_IMPLICIT_DEF.
   bool matchAllExplicitUsesAreUndef(MachineInstr &MI);
public:
   bool matchCombineFSubFpExtFNegFMulToFMadOrFMA(MachineInstr &MI,
                                                 BuildFnTy &MatchInfo);
 
   /// Fold boolean selects to logical operations.
   bool matchSelectToLogical(MachineInstr &MI, BuildFnTy &MatchInfo);
 
   bool matchCombineFMinMaxNaN(MachineInstr &MI, unsigned &Info);
 
+  /// Transform G_ADD(x, -cst) to G_SUB(x, cst).
+  bool matchAddNegConstant(MachineInstr &MI, APInt &Cst);
+  bool applyAddNegConstant(MachineInstr &MI, APInt &Cst);
+
+  /// Transform G_ADD(x, G_SUB(y, x)) to y.
+  /// Transform G_ADD(G_SUB(y, x), x) to y.
+  bool matchAddSubSameReg(MachineInstr &MI, Register &Src);
+
 private:
   /// Given a non-indexed load or store instruction \p MI, find an offset that
   /// can be usefully and legally folded into it as a post-indexing operation.
   ///
   /// \returns true if a candidate is found.
   bool findPostIndexCandidate(MachineInstr &MI, Register &Addr, Register &Base,
                               Register &Offset);
 

llvm/include/llvm/Target/GlobalISel/Combine.td

 
 def combine_minmax_nan_matchinfo: GIDefMatchData<"unsigned">;
 def combine_minmax_nan: GICombineRule<
   (defs root:$root, combine_minmax_nan_matchinfo:$info),
   (match (wip_match_opcode G_FMINNUM, G_FMAXNUM, G_FMINIMUM, G_FMAXIMUM):$root,
          [{ return Helper.matchCombineFMinMaxNaN(*${root}, ${info}); }]),
   (apply [{ Helper.replaceSingleDefInstWithOperand(*${root}, ${info}); }])>;
 
+// Transform (add x, -cst) -> (sub x, cst)
+def add_neg_constant_matchinfo : GIDefMatchData<"APInt">;
+def add_neg_constant: GICombineRule<
+  (defs root:$root, add_neg_constant_matchinfo:$matchinfo),
+  (match (wip_match_opcode G_ADD):$root,
+         [{ return Helper.matchAddNegConstant(*${root}, ${matchinfo}); }]),
+  (apply [{ return Helper.applyAddNegConstant(*${root}, ${matchinfo}); }])>;
+
+// Transform (add x, (sub y, x)) -> y
+// Transform (add (sub y, x), x) -> y
+def add_sub_reg_matchinfo : GIDefMatchData<"Register">;
+def add_sub_reg: GICombineRule <
+  (defs root:$root, add_sub_reg_matchinfo:$matchinfo),
+  (match (wip_match_opcode G_ADD):$root,
+         [{ return Helper.matchAddSubSameReg(*${root}, ${matchinfo}); }]),
+  (apply [{ return Helper.replaceSingleDefInstWithReg(*${root},
+                                                      ${matchinfo}); }])>;
+
 // FIXME: These should use the custom predicate feature once it lands.
 def undef_combines : GICombineGroup<[undef_to_fp_zero, undef_to_int_zero,
                                      undef_to_negative_one,
                                      binop_left_undef_to_zero,
                                      binop_right_undef_to_undef,
                                      propagate_undef_any_op,
                                      propagate_undef_all_ops,
                                      propagate_undef_shuffle_mask,
                                      erase_undef_store,
                                      unmerge_undef]>;
 
 def identity_combines : GICombineGroup<[select_same_val, right_identity_zero,
                                         binop_same_val, binop_left_to_zero,
                                         binop_right_to_zero, p2i_to_i2p,
                                         i2p_to_p2i, anyext_trunc_fold,
-                                        fneg_fneg_fold, right_identity_one]>;
+                                        fneg_fneg_fold, right_identity_one,
+                                        add_sub_reg]>;
 
 def const_combines : GICombineGroup<[constant_fp_op, const_ptradd_to_i2p,
                                      overlapping_and, mulo_by_2, mulo_by_0,
-                                     addo_by_0, combine_minmax_nan]>;
+                                     addo_by_0, combine_minmax_nan,
+                                     add_neg_constant]>;
 
 def known_bits_simplifications : GICombineGroup<[
   redundant_and, redundant_sext_inreg, redundant_or, urem_pow2_to_mask,
   zext_trunc_fold, icmp_to_true_false_known_bits, icmp_to_lhs_known_bits]>;
 
 def width_reduction_combines : GICombineGroup<[reduce_shl_of_extend,
                                                narrow_binop_feeding_and]>;
 

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

if (!MaybeCst || !MaybeCst->getValueAPF().isNaN())
       return false;
     IdxToPropagate = PropagateNaN ? Idx : (Idx == 1 ? 2 : 1);
     return true;
   };
 
   return MatchNaN(1) || MatchNaN(2);
 }
 
+bool CombinerHelper::matchAddNegConstant(MachineInstr &MI, APInt &Cst) {
+  assert(MI.getOpcode() == TargetOpcode::G_ADD && "Expected a G_ADD");
+  Register CstSrc = MI.getOperand(2).getReg();
+  if (!mi_match(CstSrc, MRI, m_ICst(Cst)))
+    return false;
+  return Cst.isNegative();
+}
+
+bool CombinerHelper::applyAddNegConstant(MachineInstr &MI, APInt &Cst) {
+  assert(MI.getOpcode() == TargetOpcode::G_ADD && Cst.isNegative() &&
+         "Expected a G_ADD and negative constant");
+  Register Dst = MI.getOperand(0).getReg();
+  Register Reg = MI.getOperand(1).getReg();
+
+  Builder.setInstrAndDebugLoc(MI);
+  Builder.buildSub(Dst, Reg, Builder.buildConstant(MRI.getType(Reg), -Cst),
+                   MI.getFlags());
+  MI.eraseFromParent();
+  return true;
+}
+
+bool CombinerHelper::matchAddSubSameReg(MachineInstr &MI, Register &Src) {
+  assert(MI.getOpcode() == TargetOpcode::G_ADD && "Expected a G_ADD");
+  Register LHS = MI.getOperand(1).getReg();
+  Register RHS = MI.getOperand(2).getReg();
+
+  // Helper lambda to check for opportunities for
+  // A + (B - A) -> B
+  // (B - A) + A -> B
+  auto CheckFold = [&](Register &MaybeSub, Register &MaybeSameReg) {
+    Register Reg;
+    return mi_match(MaybeSub, MRI, m_GSub(m_Reg(Src), m_Reg(Reg))) &&
+           Reg == MaybeSameReg;
+  };
+  return CheckFold(LHS, RHS) || CheckFold(RHS, LHS);
+}
+
 bool CombinerHelper::tryCombine(MachineInstr &MI) {
   if (tryCombineCopy(MI))
     return true;
   if (tryCombineExtendingLoads(MI))
     return true;
   if (tryCombineIndexedLoadStore(MI))
     return true;
   return false;
 }

llvm/test/CodeGen/AArch64/GlobalISel/combine-add.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -run-pass=aarch64-prelegalizer-combiner -verify-machineinstrs -mtriple aarch64-unknown-unknown %s -o - | FileCheck %s
+
+---
+name:            add_neg_constant
+alignment:       4
+tracksRegLiveness: true
+frameInfo:
+  maxAlignment:    1
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    liveins: $w0
+    ; CHECK-LABEL: name: add_neg_constant
+    ; CHECK: liveins: $w0
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+    ; CHECK-NEXT: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 1
+    ; CHECK-NEXT: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[COPY]], [[C]]
+    ; CHECK-NEXT: $w0 = COPY [[SUB]](s32)
+    %0:_(s32) = COPY $w0
+    %1:_(s32) = G_CONSTANT i32 -1
+    %2:_(s32) = G_ADD %0, %1
+    $w0 = COPY %2
+...
+---
+name:            add_neg_constant_wide
+alignment:       4
+tracksRegLiveness: true
+frameInfo:
+  maxAlignment:    1
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    liveins: $q0
+    ; CHECK-LABEL: name: add_neg_constant_wide
+    ; CHECK: liveins: $q0
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(s128) = COPY $q0
+    ; CHECK-NEXT: [[C:%[0-9]+]]:_(s128) = G_CONSTANT i128 1
+    ; CHECK-NEXT: [[SUB:%[0-9]+]]:_(s128) = G_SUB [[COPY]], [[C]]
+    ; CHECK-NEXT: $q0 = COPY [[SUB]](s128)
+    %0:_(s128) = COPY $q0
+    %1:_(s128) = G_CONSTANT i128 -1
+    %2:_(s128) = G_ADD %0, %1
+    $q0 = COPY %2
+...
+---
+name:            add_neg_constant_vec
+alignment:       4
+tracksRegLiveness: true
+frameInfo:
+  maxAlignment:    1
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    liveins: $x0
+    ; CHECK-LABEL: name: add_neg_constant_vec
+    ; CHECK: liveins: $x0
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(<4 x s16>) = COPY $x0
+    ; CHECK-NEXT: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 -1
+    ; CHECK-NEXT: [[BUILD_VECTOR:%[0-9]+]]:_(<4 x s16>) = G_BUILD_VECTOR [[C]](s16), [[C]](s16), [[C]](s16), [[C]](s16)
+    ; CHECK-NEXT: [[ADD:%[0-9]+]]:_(<4 x s16>) = G_ADD [[COPY]], [[BUILD_VECTOR]]
+    ; CHECK-NEXT: $x0 = COPY [[ADD]](<4 x s16>)
+    %0:_(<4 x s16>) = COPY $x0
+    %1:_(s16) = G_CONSTANT i16 -1
+    %2:_(<4 x s16>) = G_BUILD_VECTOR %1, %1, %1, %1
+    %3:_(<4 x s16>) = G_ADD %0, %2
+    $x0 = COPY %3(<4 x s16>)
+...
+---
+name:            add_lhs_sub_reg
+alignment:       4
+tracksRegLiveness: true
+frameInfo:
+  maxAlignment:    1
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    liveins: $w0, $w1
+    ; CHECK-LABEL: name: add_lhs_sub_reg
+    ; CHECK: liveins: $w0, $w1
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+    ; CHECK-NEXT: $w0 = COPY [[COPY]](s32)
+    %0:_(s32) = COPY $w0
+    %1:_(s32) = COPY $w1
+    %2:_(s32) = G_SUB %0, %1
+    %3:_(s32) = G_ADD %2, %1
+    $w0 = COPY %3
+...
+---
+name:            add_lhs_sub_reg_wide
+alignment:       4
+tracksRegLiveness: true
+frameInfo:
+  maxAlignment:    1
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+    ; CHECK-LABEL: name: add_lhs_sub_reg_wide
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(s128) = COPY $q0
+    ; CHECK-NEXT: $q0 = COPY [[COPY]](s128)
+    %0:_(s128) = COPY $q0
+    %1:_(s128) = COPY $q1
+    %2:_(s128) = G_SUB %0, %1
+    %3:_(s128) = G_ADD %2, %1
+    $q0 = COPY %3
+...
+---
+name:            add_lhs_sub_reg_vec
+alignment:       4
+tracksRegLiveness: true
+frameInfo:
+  maxAlignment:    1
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    liveins: $x0, $x1
+    ; CHECK-LABEL: name: add_lhs_sub_reg_vec
+    ; CHECK: liveins: $x0, $x1
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(<4 x s16>) = COPY $x0
+    ; CHECK-NEXT: $x0 = COPY [[COPY]](<4 x s16>)
+    %0:_(<4 x s16>) = COPY $x0
+    %1:_(<4 x s16>) = COPY $x1
+    %2:_(<4 x s16>) = G_SUB %0, %1
+    %3:_(<4 x s16>) = G_ADD %2, %1
+    $x0 = COPY %3
+...
+---
+name:            add_rhs_sub_reg
+alignment:       4
+tracksRegLiveness: true
+frameInfo:
+  maxAlignment:    1
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    liveins: $w0, $w1
+    ; CHECK-LABEL: name: add_rhs_sub_reg
+    ; CHECK: liveins: $w0, $w1
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+    ; CHECK-NEXT: $w0 = COPY [[COPY]](s32)
+    %0:_(s32) = COPY $w0
+    %1:_(s32) = COPY $w1
+    %2:_(s32) = G_SUB %0, %1
+    %3:_(s32) = G_ADD %1, %2
+    $w0 = COPY %3
+...
+---
+name:            add_rhs_sub_reg_wide
+alignment:       4
+tracksRegLiveness: true
+frameInfo:
+  maxAlignment:    1
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+    ; CHECK-LABEL: name: add_rhs_sub_reg_wide
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(s128) = COPY $q0
+    ; CHECK-NEXT: $q0 = COPY [[COPY]](s128)
+    %0:_(s128) = COPY $q0
+    %1:_(s128) = COPY $q1
+    %2:_(s128) = G_SUB %0, %1
+    %3:_(s128) = G_ADD %1, %2
+    $q0 = COPY %3
+...
+---
+name:            add_rhs_sub_reg_vec
+alignment:       4
+tracksRegLiveness: true
+frameInfo:
+  maxAlignment:    1
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    liveins: $x0, $x1
+    ; CHECK-LABEL: name: add_rhs_sub_reg_vec
+    ; CHECK: liveins: $x0, $x1
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(<4 x s16>) = COPY $x0
+    ; CHECK-NEXT: $x0 = COPY [[COPY]](<4 x s16>)
+    %0:_(<4 x s16>) = COPY $x0
+    %1:_(<4 x s16>) = COPY $x1
+    %2:_(<4 x s16>) = G_SUB %0, %1
+    %3:_(<4 x s16>) = G_ADD %1, %2
+    $x0 = COPY %3
+...

llvm/test/CodeGen/AMDGPU/GlobalISel/add.v2i16.ll

 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    v_mov_b32_e32 v1, 0xffc0ffc0
 ; GFX9-NEXT:    v_pk_add_u16 v0, v0, v1
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX8-LABEL: v_add_v2i16_neg_inline_imm_splat:
 ; GFX8:       ; %bb.0:
 ; GFX8-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-NEXT:    v_mov_b32_e32 v2, 0xffffffc0
-; GFX8-NEXT:    v_add_u16_e32 v1, 0xffc0, v0
-; GFX8-NEXT:    v_add_u16_sdwa v0, v0, v2 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
+; GFX8-NEXT:    v_mov_b32_e32 v2, 64
+; GFX8-NEXT:    v_subrev_u16_e32 v1, 64, v0
+; GFX8-NEXT:    v_sub_u16_sdwa v0, v0, v2 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
 ; GFX8-NEXT:    v_or_b32_e32 v0, v1, v0
 ; GFX8-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: v_add_v2i16_neg_inline_imm_splat:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    v_pk_add_u16 v0, 0xffc0, v0 op_sel_hi:[0,1]

 ; GFX9-NEXT:    v_mov_b32_e32 v1, 0x4ffc0
 ; GFX9-NEXT:    v_pk_add_u16 v0, v0, v1
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX8-LABEL: v_add_v2i16_neg_inline_imm_lo:
 ; GFX8:       ; %bb.0:
 ; GFX8-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX8-NEXT:    v_mov_b32_e32 v2, 4
-; GFX8-NEXT:    v_add_u16_e32 v1, 0xffc0, v0
+; GFX8-NEXT:    v_subrev_u16_e32 v1, 64, v0
 ; GFX8-NEXT:    v_add_u16_sdwa v0, v0, v2 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
 ; GFX8-NEXT:    v_or_b32_e32 v0, v1, v0
 ; GFX8-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: v_add_v2i16_neg_inline_imm_lo:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0

 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    v_mov_b32_e32 v1, 0xffc00004
 ; GFX9-NEXT:    v_pk_add_u16 v0, v0, v1
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX8-LABEL: v_add_v2i16_neg_inline_imm_hi:
 ; GFX8:       ; %bb.0:
 ; GFX8-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX8-NEXT:    v_mov_b32_e32 v1, 0xffffffc0
-; GFX8-NEXT:    v_add_u16_e32 v2, 4, v0
-; GFX8-NEXT:    v_add_u16_sdwa v0, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
-; GFX8-NEXT:    v_or_b32_e32 v0, v2, v0
+; GFX8-NEXT:    v_mov_b32_e32 v2, 64
+; GFX8-NEXT:    v_add_u16_e32 v1, 4, v0
+; GFX8-NEXT:    v_sub_u16_sdwa v0, v0, v2 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
+; GFX8-NEXT:    v_or_b32_e32 v0, v1, v0
 ; GFX8-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: v_add_v2i16_neg_inline_imm_hi:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
 ; GFX10-NEXT:    v_pk_add_u16 v0, 0xffc00004, v0
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]

llvm/test/CodeGen/AMDGPU/GlobalISel/combine-urem-pow-2.mir

bb.0:
 
     ; GCN-LABEL: name: urem_s32_var_shl1
     ; GCN: liveins: $vgpr0, $vgpr1
     ; GCN-NEXT: {{  $}}
     ; GCN-NEXT: %var:_(s32) = COPY $vgpr0
     ; GCN-NEXT: %shift_amt:_(s32) = COPY $vgpr1
     ; GCN-NEXT: %one:_(s32) = G_CONSTANT i32 1
     ; GCN-NEXT: %one_bit:_(s32) = G_SHL %one, %shift_amt(s32)
-    ; GCN-NEXT: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 -1
-    ; GCN-NEXT: [[ADD:%[0-9]+]]:_(s32) = G_ADD %one_bit, [[C]]
-    ; GCN-NEXT: %rem:_(s32) = G_AND %var, [[ADD]]
+    ; GCN-NEXT: [[SUB:%[0-9]+]]:_(s32) = G_SUB %one_bit, %one
+    ; GCN-NEXT: %rem:_(s32) = G_AND %var, [[SUB]]
     ; GCN-NEXT: $vgpr0 = COPY %rem(s32)
     %var:_(s32) = COPY $vgpr0
     %shift_amt:_(s32) = COPY $vgpr1
     %one:_(s32) = G_CONSTANT i32 1
     %one_bit:_(s32) = G_SHL %one, %shift_amt
     %rem:_(s32) = G_UREM %var, %one_bit
     $vgpr0 = COPY %rem
 ...
 
 ---
 name: urem_s64_var_shl1
 tracksRegLiveness: true
 body:             |
   bb.0:
     liveins: $vgpr0_vgpr1, $vgpr2
 
     ; GCN-LABEL: name: urem_s64_var_shl1
     ; GCN: liveins: $vgpr0_vgpr1, $vgpr2
     ; GCN-NEXT: {{  $}}
     ; GCN-NEXT: %var:_(s64) = COPY $vgpr0_vgpr1
     ; GCN-NEXT: %shiftamt:_(s32) = COPY $vgpr2
     ; GCN-NEXT: %one:_(s64) = G_CONSTANT i64 1
     ; GCN-NEXT: %one_bit:_(s64) = G_SHL %one, %shiftamt(s32)
-    ; GCN-NEXT: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 -1
-    ; GCN-NEXT: [[ADD:%[0-9]+]]:_(s64) = G_ADD %one_bit, [[C]]
-    ; GCN-NEXT: %rem:_(s64) = G_AND %var, [[ADD]]
+    ; GCN-NEXT: [[SUB:%[0-9]+]]:_(s64) = G_SUB %one_bit, %one
+    ; GCN-NEXT: %rem:_(s64) = G_AND %var, [[SUB]]
     ; GCN-NEXT: $vgpr0_vgpr1 = COPY %rem(s64)
     %var:_(s64) = COPY $vgpr0_vgpr1
     %shiftamt:_(s32) = COPY $vgpr2
     %one:_(s64) = G_CONSTANT i64 1
     %one_bit:_(s64) = G_SHL %one, %shiftamt
     %rem:_(s64) = G_UREM %var, %one_bit
     $vgpr0_vgpr1 = COPY %rem
 ...

llvm/test/CodeGen/AMDGPU/GlobalISel/extractelement.ll

entry:
   ret double %ext
 }
 
 define amdgpu_ps double @dyn_extract_v8f64_s_s_offsetm1(<8 x double> inreg %vec, i32 inreg %sel) {
 ; GCN-LABEL: dyn_extract_v8f64_s_s_offsetm1:
 ; GCN:       ; %bb.0: ; %entry
 ; GCN-NEXT:    s_mov_b32 s0, s2
 ; GCN-NEXT:    s_mov_b32 s1, s3
-; GCN-NEXT:    s_add_i32 m0, s18, -1
+; GCN-NEXT:    s_sub_i32 m0, s18, 1
 ; GCN-NEXT:    s_mov_b32 s2, s4
 ; GCN-NEXT:    s_mov_b32 s3, s5
 ; GCN-NEXT:    s_mov_b32 s4, s6
 ; GCN-NEXT:    s_mov_b32 s5, s7
 ; GCN-NEXT:    s_mov_b32 s6, s8
 ; GCN-NEXT:    s_mov_b32 s7, s9
 ; GCN-NEXT:    s_mov_b32 s8, s10
 ; GCN-NEXT:    s_mov_b32 s9, s11
 ; GCN-NEXT:    s_mov_b32 s10, s12
 ; GCN-NEXT:    s_mov_b32 s11, s13
 ; GCN-NEXT:    s_mov_b32 s12, s14
 ; GCN-NEXT:    s_mov_b32 s13, s15
 ; GCN-NEXT:    s_mov_b32 s14, s16
 ; GCN-NEXT:    s_mov_b32 s15, s17
 ; GCN-NEXT:    s_movrels_b64 s[0:1], s[0:1]
 ; GCN-NEXT:    ; return to shader part epilog
 ;
 ; GFX10-LABEL: dyn_extract_v8f64_s_s_offsetm1:
 ; GFX10:       ; %bb.0: ; %entry
 ; GFX10-NEXT:    s_mov_b32 s0, s2
 ; GFX10-NEXT:    s_mov_b32 s1, s3
-; GFX10-NEXT:    s_add_i32 m0, s18, -1
+; GFX10-NEXT:    s_sub_i32 m0, s18, 1
 ; GFX10-NEXT:    s_mov_b32 s2, s4
 ; GFX10-NEXT:    s_mov_b32 s3, s5
 ; GFX10-NEXT:    s_mov_b32 s4, s6
 ; GFX10-NEXT:    s_mov_b32 s5, s7
 ; GFX10-NEXT:    s_mov_b32 s6, s8
 ; GFX10-NEXT:    s_mov_b32 s7, s9
 ; GFX10-NEXT:    s_mov_b32 s8, s10
 ; GFX10-NEXT:    s_mov_b32 s9, s11

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.memmove.ll

 ; LOOP-NEXT:  .LBB0_5: ; %copy_backwards_loop
 ; LOOP-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; LOOP-NEXT:    v_add_i32_e32 v6, vcc, v2, v4
 ; LOOP-NEXT:    v_addc_u32_e32 v7, vcc, v3, v5, vcc
 ; LOOP-NEXT:    s_waitcnt expcnt(0)
 ; LOOP-NEXT:    buffer_load_ubyte v8, v[6:7], s[0:3], 0 addr64
 ; LOOP-NEXT:    v_add_i32_e32 v6, vcc, v0, v4
 ; LOOP-NEXT:    v_addc_u32_e32 v7, vcc, v1, v5, vcc
-; LOOP-NEXT:    v_add_i32_e32 v4, vcc, -1, v4
-; LOOP-NEXT:    v_addc_u32_e32 v5, vcc, -1, v5, vcc
+; LOOP-NEXT:    v_subrev_i32_e32 v4, vcc, 1, v4
+; LOOP-NEXT:    v_subbrev_u32_e32 v5, vcc, 0, v5, vcc
 ; LOOP-NEXT:    v_cmp_eq_u32_e32 vcc, -1, v4
 ; LOOP-NEXT:    s_waitcnt vmcnt(0)
 ; LOOP-NEXT:    buffer_store_byte v8, v[6:7], s[0:3], 0 addr64
 ; LOOP-NEXT:    s_cbranch_vccz .LBB0_5
 ; LOOP-NEXT:  .LBB0_6: ; %memmove_done
 ; LOOP-NEXT:    s_endpgm
 ;
 ; UNROLL-LABEL: memmove_p1i8:

llvm/test/CodeGen/AMDGPU/GlobalISel/saddsat.ll

 ; GFX8-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX8-NEXT:    v_add_u32_e32 v2, vcc, v0, v1
 ; GFX8-NEXT:    v_bfe_i32 v3, v2, 0, 24
 ; GFX8-NEXT:    v_bfe_i32 v0, v0, 0, 24
 ; GFX8-NEXT:    v_cmp_lt_i32_e64 s[4:5], v3, v0
 ; GFX8-NEXT:    v_bfe_i32 v0, v1, 0, 24
 ; GFX8-NEXT:    v_cmp_gt_i32_e64 s[6:7], 0, v0
 ; GFX8-NEXT:    v_ashrrev_i32_e32 v0, 23, v3
-; GFX8-NEXT:    v_add_u32_e32 v0, vcc, 0xff800000, v0
+; GFX8-NEXT:    v_subrev_u32_e32 v0, vcc, 0x800000, v0
 ; GFX8-NEXT:    s_xor_b64 vcc, s[6:7], s[4:5]
 ; GFX8-NEXT:    v_cndmask_b32_e32 v0, v2, v0, vcc
 ; GFX8-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-LABEL: v_saddsat_i24:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    v_lshlrev_b32_e32 v0, 8, v0

 ; GFX8-NEXT:    s_bfe_i32 s0, s0, 0x180000
 ; GFX8-NEXT:    s_cmp_lt_i32 s3, s0
 ; GFX8-NEXT:    s_cselect_b32 s0, 1, 0
 ; GFX8-NEXT:    s_bfe_i32 s1, s1, 0x180000
 ; GFX8-NEXT:    s_cmp_lt_i32 s1, 0
 ; GFX8-NEXT:    s_cselect_b32 s1, 1, 0
 ; GFX8-NEXT:    s_xor_b32 s0, s1, s0
 ; GFX8-NEXT:    s_ashr_i32 s1, s3, 23
-; GFX8-NEXT:    s_add_i32 s1, s1, 0xff800000
+; GFX8-NEXT:    s_sub_i32 s1, s1, 0x800000
 ; GFX8-NEXT:    s_and_b32 s0, s0, 1
 ; GFX8-NEXT:    s_cmp_lg_u32 s0, 0
 ; GFX8-NEXT:    s_cselect_b32 s0, s1, s2
 ; GFX8-NEXT:    ; return to shader part epilog
 ;
 ; GFX9-LABEL: s_saddsat_i24:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_lshl_b32 s1, s1, 8

llvm/test/CodeGen/AMDGPU/GlobalISel/ssubsat.ll

 ; GFX8-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX8-NEXT:    v_sub_u32_e32 v2, vcc, v0, v1
 ; GFX8-NEXT:    v_bfe_i32 v3, v2, 0, 24
 ; GFX8-NEXT:    v_bfe_i32 v0, v0, 0, 24
 ; GFX8-NEXT:    v_cmp_lt_i32_e64 s[4:5], v3, v0
 ; GFX8-NEXT:    v_bfe_i32 v0, v1, 0, 24
 ; GFX8-NEXT:    v_cmp_lt_i32_e64 s[6:7], 0, v0
 ; GFX8-NEXT:    v_ashrrev_i32_e32 v0, 23, v3
-; GFX8-NEXT:    v_add_u32_e32 v0, vcc, 0xff800000, v0
+; GFX8-NEXT:    v_subrev_u32_e32 v0, vcc, 0x800000, v0
 ; GFX8-NEXT:    s_xor_b64 vcc, s[6:7], s[4:5]
 ; GFX8-NEXT:    v_cndmask_b32_e32 v0, v2, v0, vcc
 ; GFX8-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-LABEL: v_ssubsat_i24:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    v_lshlrev_b32_e32 v0, 8, v0

 ; GFX8-NEXT:    s_bfe_i32 s0, s0, 0x180000
 ; GFX8-NEXT:    s_cmp_lt_i32 s3, s0
 ; GFX8-NEXT:    s_cselect_b32 s0, 1, 0
 ; GFX8-NEXT:    s_bfe_i32 s1, s1, 0x180000
 ; GFX8-NEXT:    s_cmp_gt_i32 s1, 0
 ; GFX8-NEXT:    s_cselect_b32 s1, 1, 0
 ; GFX8-NEXT:    s_xor_b32 s0, s1, s0
 ; GFX8-NEXT:    s_ashr_i32 s1, s3, 23
-; GFX8-NEXT:    s_add_i32 s1, s1, 0xff800000
+; GFX8-NEXT:    s_sub_i32 s1, s1, 0x800000
 ; GFX8-NEXT:    s_and_b32 s0, s0, 1
 ; GFX8-NEXT:    s_cmp_lg_u32 s0, 0
 ; GFX8-NEXT:    s_cselect_b32 s0, s1, s2
 ; GFX8-NEXT:    ; return to shader part epilog
 ;
 ; GFX9-LABEL: s_ssubsat_i24:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_lshl_b32 s1, s1, 8