[AArch64][GlobalISel] Implement G_SELECT translate to min/max/abs

[yuntaopan]

Summary

We are porting gisel to our backend, but found that some optimization capabilities are weaker than SDISel. This is one of the optimizations G_SELECT translate to min/max/abs from SDISel

[yuntaopan]

This comment was removed by yuntaopan.

[aemerson]

Thanks for the patch. Unfortunately I see the code that this was ported from in SelectionDAGBuilder does the optimization there because it relies on ValueTracking's matchSelectPattern() infrastructure. @arsenm @foad do you think it's worth it here to replicate that functionality for generic MIR so we can avoid adding this to the IRTranslator?

[foad]

In D121085#3365851, @aemerson wrote:

Thanks for the patch. Unfortunately I see the code that this was ported from in SelectionDAGBuilder does the optimization there because it relies on ValueTracking's matchSelectPattern() infrastructure. @arsenm @foad do you think it's worth it here to replicate that functionality for generic MIR so we can avoid adding this to the IRTranslator?

Shouldn't the select -> min/max/abs be done in IR, so there would be no need for this patch? See D98152 for example.

[paquette]

IIRC I was working on something similar in the combiner but for FP stuff (D116702)

[yuntaopan]

In D121085#3366519, @foad wrote:

In D121085#3365851, @aemerson wrote:

Thanks for the patch. Unfortunately I see the code that this was ported from in SelectionDAGBuilder does the optimization there because it relies on ValueTracking's matchSelectPattern() infrastructure. @arsenm @foad do you think it's worth it here to replicate that functionality for generic MIR so we can avoid adding this to the IRTranslator?

Shouldn't the select -> min/max/abs be done in IR, so there would be no need for this patch? See D98152 for example.

Our development branch does not have that patch yet, I have tried that patch and it works well. it's more appropriate to handle it at the IR and no need for this patch. But I have a question，why gisel can not rely on ValueTracking infrastructure ? Because gisel is based on mir, but ValueTracking is an IR analysis ？@aemerson

[aemerson]

In D121085#3371597, @yuntaopan wrote:

In D121085#3366519, @foad wrote:

In D121085#3365851, @aemerson wrote:

Thanks for the patch. Unfortunately I see the code that this was ported from in SelectionDAGBuilder does the optimization there because it relies on ValueTracking's matchSelectPattern() infrastructure. @arsenm @foad do you think it's worth it here to replicate that functionality for generic MIR so we can avoid adding this to the IRTranslator?

Shouldn't the select -> min/max/abs be done in IR, so there would be no need for this patch? See D98152 for example.

Our development branch does not have that patch yet, I have tried that patch and it works well. it's more appropriate to handle it at the IR and no need for this patch. But I have a question，why gisel can not rely on ValueTracking infrastructure ? Because gisel is based on mir, but ValueTracking is an IR analysis ？@aemerson

There's nothing wrong with using the ValueTracking infrastructure, the problem is that the only place we can use IR analysis are during the IRTranslator phase, and we try to avoid doing optimizations there as much as possible. Sometimes that might not be practical, but where we can do like to perform optimizations on generic MIR in combine passes instead.

[arsenm]

Is this still relevant?

I think a native min/max matching combiner would be more helpful, the pattern can easily appear during legalization. InstCombine forms min/max intrinsics these days, so producing them at the start of codegen isn't really necessary

[yuntaopan]

Related optimizations have been done in InstCombine Pass, so there is no need to support such optimizations here

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

bool IRTranslator::translateSelect(const User &U,
   ArrayRef<Register> ResRegs = getOrCreateVRegs(U);
   ArrayRef<Register> Op0Regs = getOrCreateVRegs(*U.getOperand(1));
   ArrayRef<Register> Op1Regs = getOrCreateVRegs(*U.getOperand(2));
 
   uint16_t Flags = 0;
   if (const SelectInst *SI = dyn_cast<SelectInst>(&U))
     Flags = MachineInstr::copyFlagsFromInstruction(*SI);
 
+  Optional<unsigned> Opc;
+  bool IsUnaryAbs = false;
+  bool Negate = false;
+  if (EnableOpts) {
+    Value *LHS, *RHS;
+    auto SPR = matchSelectPattern(const_cast<User *>(&U), LHS, RHS);
+    switch (SPR.Flavor) {
+    case SPF_UMAX:
+      Opc = TargetOpcode::G_UMAX;
+      break;
+    case SPF_UMIN:
+      Opc = TargetOpcode::G_UMIN;
+      break;
+    case SPF_SMAX:
+      Opc = TargetOpcode::G_SMAX;
+      break;
+    case SPF_SMIN:
+      Opc = TargetOpcode::G_SMIN;
+      break;
+    case SPF_FMINNUM:
+      switch (SPR.NaNBehavior) {
+      case SPNB_NA:
+        llvm_unreachable("No NaN behavior for FP op?");
+      case SPNB_RETURNS_NAN:
+        Opc = TargetOpcode::G_FMINIMUM;
+        break;
+      // Treat ANY like OTHER
+      case SPNB_RETURNS_ANY:
+      case SPNB_RETURNS_OTHER:
+        Opc = TargetOpcode::G_FMINNUM;
+        break;
+      }
+      break;
+    case SPF_FMAXNUM:
+      switch (SPR.NaNBehavior) {
+      case SPNB_NA:
+        llvm_unreachable("No NaN behavior for FP op?");
+      case SPNB_RETURNS_NAN:
+        Opc = TargetOpcode::G_FMAXIMUM;
+        break;
+      // Treat ANY like OTHER
+      case SPNB_RETURNS_ANY:
+      case SPNB_RETURNS_OTHER:
+        Opc = TargetOpcode::G_FMAXNUM;
+        break;
+      }
+      break;
+    case SPF_NABS:
+      Negate = true;
+      LLVM_FALLTHROUGH;
+    case SPF_ABS:
+      IsUnaryAbs = true;
+      Opc = TargetOpcode::G_ABS;
+      break;
+    default:
+      break;
+    }
+
+    auto hasOnlySelectUsers = [](const Value *Cond) {
+      return llvm::all_of(Cond->users(),
+                          [](const Value *V) { return isa<SelectInst>(V); });
+    };
+
+    if (!IsUnaryAbs && Opc &&
+        // If the underlying comparison instruction is used by any other
+        // instruction, the consumed instructions won't be destroyed, so it is
+        // not profitable to convert to a min/max.
+        hasOnlySelectUsers(cast<SelectInst>(&U)->getCondition())) {
+      Op0Regs = getOrCreateVRegs(*LHS);
+      Op1Regs = getOrCreateVRegs(*RHS);
+    }
+
+    if (IsUnaryAbs) {
+      Op0Regs = getOrCreateVRegs(*LHS);
+    }
+  }
+
   for (unsigned i = 0; i < ResRegs.size(); ++i) {
+    if (!Opc.hasValue())
       MIRBuilder.buildSelect(ResRegs[i], Tst, Op0Regs[i], Op1Regs[i], Flags);
+    // convert to abs or abs + neg
+    else if (IsUnaryAbs) {
+      LLT Ty = MRI->getType(ResRegs[i]);
+      Register ABSOutReg =
+          Negate ? MRI->createGenericVirtualRegister(Ty) : ResRegs[i];
+      MIRBuilder.buildInstr(*Opc, {ABSOutReg}, {Op0Regs[i]});
+      if (Negate) {
+        MIRBuilder.buildInstr(TargetOpcode::G_SUB, {ResRegs[i]},
+                              {MIRBuilder.buildConstant(Ty, 0), ABSOutReg});
+      }
+      // convert to min/max
+    } else {
+      MIRBuilder.buildInstr(*Opc, {ResRegs[i]}, {Op0Regs[i], Op1Regs[i]});
+    }
   }
 
   return true;
 }
 
 bool IRTranslator::translateCopy(const User &U, const Value &V,
                                  MachineIRBuilder &MIRBuilder) {
   Register Src = getOrCreateVReg(V);

llvm/test/CodeGen/AArch64/GlobalISel/arm64-atomic.ll

 define i8 @atomicrmw_min_i8(i8* %ptr, i8 %rhs) {
 ; CHECK-NOLSE-O1-LABEL: atomicrmw_min_i8:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:  LBB33_1: ; %atomicrmw.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldaxrb w8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    sxtb w9, w8
 ; CHECK-NOLSE-O1-NEXT:    cmp w9, w1, sxtb
-; CHECK-NOLSE-O1-NEXT:    csel w9, w8, w1, le
+; CHECK-NOLSE-O1-NEXT:    csel w9, w8, w1, lt
 ; CHECK-NOLSE-O1-NEXT:    stxrb w10, w9, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w10, LBB33_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %atomicrmw.end
 ; CHECK-NOLSE-O1-NEXT:    mov w0, w8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ;
 ; CHECK-NOLSE-O0-LABEL: atomicrmw_min_i8:
 ; CHECK-NOLSE-O0:       ; %bb.0:

 ; CHECK-NOLSE-O1-LABEL: atomicrmw_umin_i8:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:    and w9, w1, #0xff
 ; CHECK-NOLSE-O1-NEXT:  LBB35_1: ; %atomicrmw.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldaxrb w8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    and w10, w8, #0xff
 ; CHECK-NOLSE-O1-NEXT:    cmp w10, w9
-; CHECK-NOLSE-O1-NEXT:    csel w10, w10, w9, ls
+; CHECK-NOLSE-O1-NEXT:    csel w10, w10, w9, lo
 ; CHECK-NOLSE-O1-NEXT:    stlxrb w11, w10, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w11, LBB35_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %atomicrmw.end
 ; CHECK-NOLSE-O1-NEXT:    mov w0, w8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ;
 ; CHECK-NOLSE-O0-LABEL: atomicrmw_umin_i8:
 ; CHECK-NOLSE-O0:       ; %bb.0:

 define i16 @atomicrmw_min_i16(i16* %ptr, i16 %rhs) {
 ; CHECK-NOLSE-O1-LABEL: atomicrmw_min_i16:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:  LBB43_1: ; %atomicrmw.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldaxrh w8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    sxth w9, w8
 ; CHECK-NOLSE-O1-NEXT:    cmp w9, w1, sxth
-; CHECK-NOLSE-O1-NEXT:    csel w9, w8, w1, le
+; CHECK-NOLSE-O1-NEXT:    csel w9, w8, w1, lt
 ; CHECK-NOLSE-O1-NEXT:    stxrh w10, w9, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w10, LBB43_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %atomicrmw.end
 ; CHECK-NOLSE-O1-NEXT:    mov w0, w8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ;
 ; CHECK-NOLSE-O0-LABEL: atomicrmw_min_i16:
 ; CHECK-NOLSE-O0:       ; %bb.0:

 ; CHECK-NOLSE-O1-LABEL: atomicrmw_umin_i16:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:    and w9, w1, #0xffff
 ; CHECK-NOLSE-O1-NEXT:  LBB45_1: ; %atomicrmw.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldaxrh w8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    and w10, w8, #0xffff
 ; CHECK-NOLSE-O1-NEXT:    cmp w10, w9
-; CHECK-NOLSE-O1-NEXT:    csel w10, w10, w9, ls
+; CHECK-NOLSE-O1-NEXT:    csel w10, w10, w9, lo
 ; CHECK-NOLSE-O1-NEXT:    stlxrh w11, w10, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w11, LBB45_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %atomicrmw.end
 ; CHECK-NOLSE-O1-NEXT:    mov w0, w8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ;
 ; CHECK-NOLSE-O0-LABEL: atomicrmw_umin_i16:
 ; CHECK-NOLSE-O0:       ; %bb.0:

llvm/test/CodeGen/AArch64/GlobalISel/irtranslator-select.ll

+; RUN: llc -O0 -global-isel -mtriple=aarch64-unknown-unknown -stop-after=irtranslator -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK-O0
+; RUN: llc -O2 -global-isel -mtriple=aarch64-unknown-unknown -stop-after=irtranslator -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK-O2
+
+define i32 @max_s32(i32 %a, i32 %b) {
+; CHECK-LABEL: name: max_s32
+; CHECK-O0: [[LHS:%.*]]:_(s32) = COPY $w0 
+; CHECK-O0: [[RHS:%.*]]:_(s32) = COPY $w1 
+; CHECK-O0: [[CMP:%.*]]:_(s1) = G_ICMP intpred(sge), [[LHS]](s32), [[RHS]]
+; CHECK-O0: [[RES:%.*]]:_(s32) = G_SELECT [[CMP]](s1), [[LHS]], [[RHS]]
+; CHECK-O0: $w0 = COPY [[RES]]
+; CHECK-O0: RET_ReallyLR implicit $w0
+
+; CHECK-O2: [[LHS:%.*]]:_(s32) = COPY $w0 
+; CHECK-O2: [[RHS:%.*]]:_(s32) = COPY $w1 
+; CHECK-O2: [[UNUSED:%.*]]:_(s1) = G_ICMP intpred(sge), [[LHS]](s32), [[RHS]]
+; CHECK-O2: [[RES:%.*]]:_(s32) = G_SMAX [[LHS]], [[RHS]]
+; CHECK-O2: $w0 = COPY [[RES]]
+; CHECK-O2: RET_ReallyLR implicit $w0
+  %cmp = icmp sge i32 %a, %b
+  %res = select i1 %cmp, i32 %a, i32 %b
+  ret i32 %res
+}
+
+; CHECK-LABEL: name: max_f32
+; CHECK-O0: [[LHS:%.*]]:_(s32) = COPY $s0 
+; CHECK-O0: [[RHS:%.*]]:_(s32) = G_FCONSTANT float 1.000000e+00 
+; CHECK-O0: [[CMP:%.*]]:_(s1) = G_FCMP floatpred(ogt), [[LHS]](s32), [[RHS]]
+; CHECK-O0: [[RES:%.*]]:_(s32) = G_SELECT [[CMP]](s1), [[LHS]], [[RHS]]
+; CHECK-O0: $s0 = COPY [[RES]]
+; CHECK-O0: RET_ReallyLR implicit $s0
+
+; CHECK-O2: [[LHS:%.*]]:_(s32) = COPY $s0 
+; CHECK-O2: [[RHS:%.*]]:_(s32) = G_FCONSTANT float 1.000000e+00 
+; CHECK-O2: [[UNUSED:%.*]]:_(s1) = G_FCMP floatpred(ogt), [[LHS]](s32), [[RHS]]
+; CHECK-O2: [[RES:%.*]]:_(s32) = G_FMAXNUM [[LHS]], [[RHS]]
+; CHECK-O2: $s0 = COPY [[RES]]
+; CHECK-O2: RET_ReallyLR implicit $s0
+define float @max_f32(float %a) {
+  %cmp = fcmp ogt float %a, 1.000000e+00
+  %narrow.sel = select i1 %cmp, float %a, float 1.000000e+00
+  ret float %narrow.sel
+}
+
+; CHECK-LABEL: name: min_s32
+; CHECK-O0: [[LHS:%.*]]:_(s32) = COPY $w0 
+; CHECK-O0: [[RHS:%.*]]:_(s32) = COPY $w1 
+; CHECK-O0: [[CMP:%.*]]:_(s1) = G_ICMP intpred(sge), [[LHS]](s32), [[RHS]]
+; CHECK-O0: [[RES:%.*]]:_(s32) = G_SELECT [[CMP]](s1), [[RHS]], [[LHS]]
+; CHECK-O0: $w0 = COPY [[RES]]
+; CHECK-O0: RET_ReallyLR implicit $w0
+
+; CHECK-O2: [[LHS:%.*]]:_(s32) = COPY $w0 
+; CHECK-O2: [[RHS:%.*]]:_(s32) = COPY $w1 
+; CHECK-O2: [[UNUSED:%.*]]:_(s1) = G_ICMP intpred(sge), [[LHS]](s32), [[RHS]]
+; CHECK-O2: [[RES:%.*]]:_(s32) = G_SMIN [[LHS]], [[RHS]]
+; CHECK-O2: $w0 = COPY [[RES]]
+; CHECK-O2: RET_ReallyLR implicit $w0
+define i32 @min_s32(i32 %a, i32 %b) {
+  %cmp = icmp sge i32 %a, %b
+  %res = select i1 %cmp, i32 %b, i32 %a
+  ret i32 %res
+}
+
+; CHECK-LABEL: name: min_f32
+; CHECK-O0: [[LHS:%.*]]:_(s32) = COPY $s0 
+; CHECK-O0: [[RHS:%.*]]:_(s32) = G_FCONSTANT float 1.000000e+00 
+; CHECK-O0: [[CMP:%.*]]:_(s1) = G_FCMP floatpred(ole), [[LHS]](s32), [[RHS]]
+; CHECK-O0: [[RES:%.*]]:_(s32) = G_SELECT [[CMP]](s1), [[LHS]], [[RHS]]
+; CHECK-O0: $s0 = COPY [[RES]]
+; CHECK-O0: RET_ReallyLR implicit $s0
+
+; CHECK-O2: [[LHS:%.*]]:_(s32) = COPY $s0 
+; CHECK-O2: [[RHS:%.*]]:_(s32) = G_FCONSTANT float 1.000000e+00 
+; CHECK-O2: [[UNUSED:%.*]]:_(s1) = G_FCMP floatpred(ole), [[LHS]](s32), [[RHS]]
+; CHECK-O2: [[RES:%.*]]:_(s32) = G_FMINNUM [[LHS]], [[RHS]]
+; CHECK-O2: $s0 = COPY [[RES]]
+; CHECK-O2: RET_ReallyLR implicit $s0
+define float @min_f32(float %a) {
+  %cmp = fcmp ole float %a, 1.000000e+00
+  %narrow.sel = select i1 %cmp, float %a, float 1.000000e+00
+  ret float %narrow.sel
+}
+
+; CHECK-LABEL: name: abs_s32
+; CHECK-O0: [[LHS:%.*]]:_(s32) = COPY $w0 
+; CHECK-O0: [[ZERO:%.*]]:_(s32) = G_CONSTANT i32 0
+; CHECK-O0: [[RHS:%.*]]:_(s32) = G_SUB [[ZERO]], [[LHS]]
+; CHECK-O0: [[CMP:%.*]]:_(s1) = G_ICMP intpred(slt), [[LHS]](s32), [[ZERO]]
+; CHECK-O0: [[RES:%.*]]:_(s32) = G_SELECT [[CMP]](s1), [[RHS]], [[LHS]]
+; CHECK-O0: $w0 = COPY [[RES]]
+; CHECK-O0: RET_ReallyLR implicit $w0
+
+; CHECK-O2: [[LHS:%.*]]:_(s32) = COPY $w0 
+; CHECK-O2: [[ZERO:%.*]]:_(s32) = G_CONSTANT i32 0
+; CHECK-O2: [[UNUSED_RHS:%.*]]:_(s32) = G_SUB [[ZERO]], [[LHS]]
+; CHECK-O2: [[UNUSED_CMP:%.*]]:_(s1) = G_ICMP intpred(slt), [[LHS]](s32), [[ZERO]]
+; CHECK-O2: [[RES:%.*]]:_(s32) = G_ABS [[LHS]]
+; CHECK-O2: $w0 = COPY [[RES]]
+; CHECK-O2: RET_ReallyLR implicit $w0
+define i32 @abs_s32(i32 %a) {
+  %tmp1neg = sub i32 0, %a
+  %b = icmp slt i32 %a, 0
+  %abs = select i1 %b, i32 %tmp1neg, i32 %a
+  ret i32 %abs
+}
+
+; CHECK-LABEL: name: abs_neg
+; CHECK-O0: [[LHS:%.*]]:_(s32) = COPY $w0 
+; CHECK-O0: [[ZERO:%.*]]:_(s32) = G_CONSTANT i32 0
+; CHECK-O0: [[RHS:%.*]]:_(s32) = G_SUB [[ZERO]], [[LHS]]
+; CHECK-O0: [[CMP:%.*]]:_(s1) = G_ICMP intpred(slt), [[LHS]](s32), [[ZERO]]
+; CHECK-O0: [[RES:%.*]]:_(s32) = G_SELECT [[CMP]](s1), [[LHS]], [[RHS]]
+; CHECK-O0: $w0 = COPY [[RES]]
+; CHECK-O0: RET_ReallyLR implicit $w0
+
+; CHECK-O2: [[LHS:%.*]]:_(s32) = COPY $w0 
+; CHECK-O2: [[ZERO:%.*]]:_(s32) = G_CONSTANT i32 0
+; CHECK-O2: [[UNUSED_RHS:%.*]]:_(s32) = G_SUB [[ZERO]], [[LHS]]
+; CHECK-O2: [[UNUSED_CMP:%.*]]:_(s1) = G_ICMP intpred(slt), [[LHS]](s32), [[ZERO]]
+; CHECK-O2: [[ABS_RES:%.*]]:_(s32) = G_ABS [[LHS]]
+; CHECK-O2: [[ZERO:%.*]]:_(s32) = G_CONSTANT i32 0
+; CHECK-O2: [[RES:%.*]]:_(s32) = G_SUB [[ZERO]], [[ABS_RES]]
+; CHECK-O2: $w0 = COPY [[RES]]
+; CHECK-O2: RET_ReallyLR implicit $w0
+define i32 @abs_neg(i32 %a) {
+  %tmp1neg = sub i32 0, %a
+  %b = icmp slt i32 %a, 0
+  %abs = select i1 %b, i32 %a, i32 %tmp1neg
+  ret i32 %abs
+}
+

llvm/test/CodeGen/AArch64/arm64-vabs.ll

 ; DAG:       // %bb.0:
 ; DAG-NEXT:    uabdl.4s v0, v0, v1
 ; DAG-NEXT:    addv.4s s0, v0
 ; DAG-NEXT:    fmov w0, s0
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: uabdl4s_rdx_i32:
 ; GISEL:       // %bb.0:
-; GISEL-NEXT:    movi.2d v2, #0000000000000000
-; GISEL-NEXT:    usubl.4s v0, v0, v1
-; GISEL-NEXT:    cmgt.4s v1, v2, v0
-; GISEL-NEXT:    neg.4s v2, v0
-; GISEL-NEXT:    shl.4s v1, v1, #31
-; GISEL-NEXT:    sshr.4s v1, v1, #31
-; GISEL-NEXT:    bit.16b v0, v2, v1
+; GISEL-NEXT:    uabdl.4s v0, v0, v1
 ; GISEL-NEXT:    addv.4s s0, v0
 ; GISEL-NEXT:    fmov w0, s0
 ; GISEL-NEXT:    ret
 
-; GISel doesn't match this pattern yet.
   %aext = zext <4 x i16> %a to <4 x i32>
   %bext = zext <4 x i16> %b to <4 x i32>
  %abdiff = sub nsw <4 x i32> %aext, %bext
   %abcmp = icmp slt <4 x i32> %abdiff, zeroinitializer
   %ababs = sub nsw <4 x i32> zeroinitializer, %abdiff
   %absel = select <4 x i1> %abcmp, <4 x i32> %ababs, <4 x i32> %abdiff
   %reduced_v = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> %absel)
   ret i32 %reduced_v

 ; DAG:       // %bb.0:
 ; DAG-NEXT:    uabdl.2d v0, v0, v1
 ; DAG-NEXT:    addp.2d d0, v0
 ; DAG-NEXT:    fmov x0, d0
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: uabdl2d_rdx_i64:
 ; GISEL:       // %bb.0:
-; GISEL-NEXT:    movi.2d v2, #0000000000000000
-; GISEL-NEXT:    usubl.2d v0, v0, v1
-; GISEL-NEXT:    cmgt.2d v1, v2, v0
-; GISEL-NEXT:    neg.2d v2, v0
-; GISEL-NEXT:    shl.2d v1, v1, #63
-; GISEL-NEXT:    sshr.2d v1, v1, #63
-; GISEL-NEXT:    bit.16b v0, v2, v1
+; GISEL-NEXT:    uabdl.2d v0, v0, v1
 ; GISEL-NEXT:    addp.2d d0, v0
 ; GISEL-NEXT:    fmov x0, d0
 ; GISEL-NEXT:    ret
 
 ; GISel doesn't match this pattern yet
   %aext = zext <2 x i32> %a to <2 x i64>
   %bext = zext <2 x i32> %b to <2 x i64>
   %abdiff = sub nsw <2 x i64> %aext, %bext

 define <2 x i32> @abspattern1(<2 x i32> %a) nounwind {
 ; DAG-LABEL: abspattern1:
 ; DAG:       // %bb.0:
 ; DAG-NEXT:    abs.2s v0, v0
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: abspattern1:
 ; GISEL:       // %bb.0:
-; GISEL-NEXT:    movi.2d v1, #0000000000000000
-; GISEL-NEXT:    neg.2s v2, v0
-; GISEL-NEXT:    cmge.2s v1, v0, v1
-; GISEL-NEXT:    shl.2s v1, v1, #31
-; GISEL-NEXT:    sshr.2s v1, v1, #31
-; GISEL-NEXT:    bif.8b v0, v2, v1
+; GISEL-NEXT:    abs.2s v0, v0
 ; GISEL-NEXT:    ret
 
   %tmp1neg = sub <2 x i32> zeroinitializer, %a
   %b = icmp sge <2 x i32> %a, zeroinitializer
   %abs = select <2 x i1> %b, <2 x i32> %a, <2 x i32> %tmp1neg
   ret <2 x i32> %abs
 }
 
 define <4 x i16> @abspattern2(<4 x i16> %a) nounwind {
 ; DAG-LABEL: abspattern2:
 ; DAG:       // %bb.0:
 ; DAG-NEXT:    abs.4h v0, v0
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: abspattern2:
 ; GISEL:       // %bb.0:
-; GISEL-NEXT:    movi.2d v1, #0000000000000000
-; GISEL-NEXT:    neg.4h v2, v0
-; GISEL-NEXT:    cmgt.4h v1, v0, v1
-; GISEL-NEXT:    shl.4h v1, v1, #15
-; GISEL-NEXT:    sshr.4h v1, v1, #15
-; GISEL-NEXT:    bif.8b v0, v2, v1
+; GISEL-NEXT:    abs.4h v0, v0
 ; GISEL-NEXT:    ret
-; For GlobalISel, this generates terrible code until we can pattern match this to abs.
 
   %tmp1neg = sub <4 x i16> zeroinitializer, %a
   %b = icmp sgt <4 x i16> %a, zeroinitializer
   %abs = select <4 x i1> %b, <4 x i16> %a, <4 x i16> %tmp1neg
   ret <4 x i16> %abs
 }
 
 define <8 x i8> @abspattern3(<8 x i8> %a) nounwind {
 ; DAG-LABEL: abspattern3:
 ; DAG:       // %bb.0:
 ; DAG-NEXT:    abs.8b v0, v0
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: abspattern3:
 ; GISEL:       // %bb.0:
-; GISEL-NEXT:    movi.2d v1, #0000000000000000
-; GISEL-NEXT:    neg.8b v2, v0
-; GISEL-NEXT:    cmgt.8b v1, v1, v0
-; GISEL-NEXT:    shl.8b v1, v1, #7
-; GISEL-NEXT:    sshr.8b v1, v1, #7
-; GISEL-NEXT:    bit.8b v0, v2, v1
+; GISEL-NEXT:    abs.8b v0, v0
 ; GISEL-NEXT:    ret
 
   %tmp1neg = sub <8 x i8> zeroinitializer, %a
   %b = icmp slt <8 x i8> %a, zeroinitializer
   %abs = select <8 x i1> %b, <8 x i8> %tmp1neg, <8 x i8> %a
   ret <8 x i8> %abs
 }
 
 define <4 x i32> @abspattern4(<4 x i32> %a) nounwind {
 ; DAG-LABEL: abspattern4:
 ; DAG:       // %bb.0:
 ; DAG-NEXT:    abs.4s v0, v0
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: abspattern4:
 ; GISEL:       // %bb.0:
-; GISEL-NEXT:    movi.2d v1, #0000000000000000
-; GISEL-NEXT:    neg.4s v2, v0
-; GISEL-NEXT:    cmge.4s v1, v0, v1
-; GISEL-NEXT:    shl.4s v1, v1, #31
-; GISEL-NEXT:    sshr.4s v1, v1, #31
-; GISEL-NEXT:    bif.16b v0, v2, v1
+; GISEL-NEXT:    abs.4s v0, v0
 ; GISEL-NEXT:    ret
 
   %tmp1neg = sub <4 x i32> zeroinitializer, %a
   %b = icmp sge <4 x i32> %a, zeroinitializer
   %abs = select <4 x i1> %b, <4 x i32> %a, <4 x i32> %tmp1neg
   ret <4 x i32> %abs
 }
 
 define <8 x i16> @abspattern5(<8 x i16> %a) nounwind {
 ; DAG-LABEL: abspattern5:
 ; DAG:       // %bb.0:
 ; DAG-NEXT:    abs.8h v0, v0
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: abspattern5:
 ; GISEL:       // %bb.0:
-; GISEL-NEXT:    movi.2d v1, #0000000000000000
-; GISEL-NEXT:    neg.8h v2, v0
-; GISEL-NEXT:    cmgt.8h v1, v0, v1
-; GISEL-NEXT:    shl.8h v1, v1, #15
-; GISEL-NEXT:    sshr.8h v1, v1, #15
-; GISEL-NEXT:    bif.16b v0, v2, v1
+; GISEL-NEXT:    abs.8h v0, v0
 ; GISEL-NEXT:    ret
 
   %tmp1neg = sub <8 x i16> zeroinitializer, %a
   %b = icmp sgt <8 x i16> %a, zeroinitializer
   %abs = select <8 x i1> %b, <8 x i16> %a, <8 x i16> %tmp1neg
   ret <8 x i16> %abs
 }
 
 define <16 x i8> @abspattern6(<16 x i8> %a) nounwind {
 ; DAG-LABEL: abspattern6:
 ; DAG:       // %bb.0:
 ; DAG-NEXT:    abs.16b v0, v0
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: abspattern6:
 ; GISEL:       // %bb.0:
-; GISEL-NEXT:    movi.2d v1, #0000000000000000
-; GISEL-NEXT:    neg.16b v2, v0
-; GISEL-NEXT:    cmgt.16b v1, v1, v0
-; GISEL-NEXT:    shl.16b v1, v1, #7
-; GISEL-NEXT:    sshr.16b v1, v1, #7
-; GISEL-NEXT:    bit.16b v0, v2, v1
+; GISEL-NEXT:    abs.16b v0, v0
 ; GISEL-NEXT:    ret
 
   %tmp1neg = sub <16 x i8> zeroinitializer, %a
   %b = icmp slt <16 x i8> %a, zeroinitializer
   %abs = select <16 x i1> %b, <16 x i8> %tmp1neg, <16 x i8> %a
   ret <16 x i8> %abs
 }
 
 define <2 x i64> @abspattern7(<2 x i64> %a) nounwind {
 ; DAG-LABEL: abspattern7:
 ; DAG:       // %bb.0:
 ; DAG-NEXT:    abs.2d v0, v0
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: abspattern7:
 ; GISEL:       // %bb.0:
 ; GISEL-NEXT:    movi.2d v1, #0000000000000000
 ; GISEL-NEXT:    neg.2d v2, v0
 ; GISEL-NEXT:    cmge.2d v1, v1, v0
 ; GISEL-NEXT:    shl.2d v1, v1, #63
 ; GISEL-NEXT:    sshr.2d v1, v1, #63
 ; GISEL-NEXT:    bit.16b v0, v2, v1
 ; GISEL-NEXT:    ret
+; the difference from dag-combine
 
   %tmp1neg = sub <2 x i64> zeroinitializer, %a
   %b = icmp sle <2 x i64> %a, zeroinitializer
   %abs = select <2 x i1> %b, <2 x i64> %tmp1neg, <2 x i64> %a
   ret <2 x i64> %abs
 }
 
 define <2 x i64> @uabd_i32(<2 x i32> %a, <2 x i32> %b) {
 ; DAG-LABEL: uabd_i32:
 ; DAG:       // %bb.0:
 ; DAG-NEXT:    sabdl.2d v0, v0, v1
 ; DAG-NEXT:    ret
 ;
 ; GISEL-LABEL: uabd_i32:
 ; GISEL:       // %bb.0:
-; GISEL-NEXT:    movi.2d v2, #0000000000000000
 ; GISEL-NEXT:    ssubl.2d v0, v0, v1
-; GISEL-NEXT:    cmgt.2d v1, v2, v0
-; GISEL-NEXT:    neg.2d v2, v0
-; GISEL-NEXT:    shl.2d v1, v1, #63
-; GISEL-NEXT:    sshr.2d v1, v1, #63
-; GISEL-NEXT:    bit.16b v0, v2, v1
+; GISEL-NEXT:    abs.2d  v0, v0
 ; GISEL-NEXT:    ret
+; the difference from dag-combine
   %aext = sext <2 x i32> %a to <2 x i64>
   %bext = sext <2 x i32> %b to <2 x i64>
   %abdiff = sub nsw <2 x i64> %aext, %bext
   %abcmp = icmp slt <2 x i64> %abdiff, zeroinitializer
   %ababs = sub nsw <2 x i64> zeroinitializer, %abdiff
   %absel = select <2 x i1> %abcmp, <2 x i64> %ababs, <2 x i64> %abdiff
   ret <2 x i64> %absel
 }