[AArch64][RISCV][SelectionDAG] Support VECREDUCE_ADD mask operations

[Jimerlife]

Summary

D125206 patch make CodeGen/AArch64/vecreduce-add-legalization.ll fail, So I revert it. Now I have update vecreduce-add-legalization.ll in this patch.

[paulwalker-arm]

Other than a minor nit the patch looks sensible. However, there's a clear regression that needs resolving first.

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

5315

This comment doesn't offer any real value. None of the canonicalisation above and below this one are commented so perhaps no comment it necessary.

[paulwalker-arm]

I've created D125605 that once landed should mean this patch will no longer negatively affect AArch64 code generation.

[Jimerlife]

address comment

[Jimerlife]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

5315

Thanks, I removed this comment

[reames]

Please go ahead and land the new tests, and then rebase so that changes in codegen are visible in review. Also, please rebase over the mentioned Aarch64 review. The test diff there currently looks like a regression because you haven't stacked the changes which was confusing at first.

The submission comment needs improved to describe what this is actually doing.

The patch seems generally reasonable. Once the above are done, should be an easy LGTM.

[Jimerlife]

rebase main. After D125605 landed, this patch will no longer negatively affect AArch64 code generation.

[Jimerlife]

In D125310#3515962, @reames wrote:

Please go ahead and land the new tests, and then rebase so that changes in codegen are visible in review. Also, please rebase over the mentioned Aarch64 review. The test diff there currently looks like a regression because you haven't stacked the changes which was confusing at first.

The submission comment needs improved to describe what this is actually doing.

The patch seems generally reasonable. Once the above are done, should be an easy LGTM.

Thanks for your advice. After I rebased main, this patch no longer affect AArch64 tests.

[reames]

Previous comments unaddressed

[craig.topper]

In D125310#3515962, @reames wrote:

Please go ahead and land the new tests, and then rebase so that changes in codegen are visible in review. Also, please rebase over the mentioned Aarch64 review. The test diff there currently looks like a regression because you haven't stacked the changes which was confusing at first.

The scalable vector tests crash right now so they can't be pre-comitted.

[reames]

In D125310#3517825, @craig.topper wrote:

In D125310#3515962, @reames wrote:

Please go ahead and land the new tests, and then rebase so that changes in codegen are visible in review. Also, please rebase over the mentioned Aarch64 review. The test diff there currently looks like a regression because you haven't stacked the changes which was confusing at first.

The scalable vector tests crash right now so they can't be pre-comitted.

Then the submission comment needs to mention that.

[Jimerlife]

In D125310#3517804, @Jimerlife wrote:

In D125310#3515962, @reames wrote:

Please go ahead and land the new tests, and then rebase so that changes in codegen are visible in review. Also, please rebase over the mentioned Aarch64 review. The test diff there currently looks like a regression because you haven't stacked the changes which was confusing at first.

The submission comment needs improved to describe what this is actually doing.

The patch seems generally reasonable. Once the above are done, should be an easy LGTM.

Thanks for your advice. After I rebased main, this patch no longer affect AArch64 tests.

I misunderstood what you meant before.In RISCV, VECREDUCE_ADD mask operations is unsupport, so I cann't precommit scalable vector tests. Before D125605 landed, my patch affect a AArch64 test(vecreduce-add-legalization.ll), but now this is no affect after D125605 landed.

[paulwalker-arm]

Given this all started by the revert of D125206 and the reason it was reverted has now been resolved, perhaps you should just re-land D125206?

[Jimerlife]

In D125310#3521676, @paulwalker-arm wrote:

Given this all started by the revert of D125206 and the reason it was reverted has now been resolved, perhaps you should just re-land D125206?

Thanks for your support. I will re-land D125206 and abandon this patch.

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

case ISD::CTPOP:
     if (Operand.getValueType().getScalarType() == MVT::i1)
       return Operand;
     break;
   case ISD::CTLZ:
   case ISD::CTTZ:
     if (Operand.getValueType().getScalarType() == MVT::i1)
       return getNOT(DL, Operand, Operand.getValueType());
     break;
+  case ISD::VECREDUCE_ADD:
+    if (Operand.getValueType().getScalarType() == MVT::i1)
+      return getNode(ISD::VECREDUCE_XOR, DL, VT, Operand);
+    break;
   case ISD::VECREDUCE_SMIN:
   case ISD::VECREDUCE_UMAX:
     if (Operand.getValueType().getScalarType() == MVT::i1)
       return getNode(ISD::VECREDUCE_OR, DL, VT, Operand);
     break;
   case ISD::VECREDUCE_SMAX:
   case ISD::VECREDUCE_UMIN:
     if (Operand.getValueType().getScalarType() == MVT::i1)

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-vreductions-mask.ll

 ; LMULMAX8-NEXT:    vsetvli zero, a0, e8, m4, ta, mu
 ; LMULMAX8-NEXT:    vcpop.m a0, v0
 ; LMULMAX8-NEXT:    snez a0, a0
 ; LMULMAX8-NEXT:    neg a0, a0
 ; LMULMAX8-NEXT:    ret
   %red = call i1 @llvm.vector.reduce.smin.v64i1(<64 x i1> %v)
   ret i1 %red
 }
+
+declare i1 @llvm.vector.reduce.add.v1i1(<1 x i1>)
+
+define signext i1 @vreduce_add_v1i1(<1 x i1> %v) {
+; CHECK-LABEL: vreduce_add_v1i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, zero, e8, mf8, ta, mu
+; CHECK-NEXT:    vmv.v.i v8, 0
+; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
+; CHECK-NEXT:    vmv.x.s a0, v8
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.v1i1(<1 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.v2i1(<2 x i1>)
+
+define signext i1 @vreduce_add_v2i1(<2 x i1> %v) {
+; CHECK-LABEL: vreduce_add_v2i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetivli zero, 2, e8, mf8, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.v2i1(<2 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.v4i1(<4 x i1>)
+
+define signext i1 @vreduce_add_v4i1(<4 x i1> %v) {
+; CHECK-LABEL: vreduce_add_v4i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.v4i1(<4 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.v8i1(<8 x i1>)
+
+define signext i1 @vreduce_add_v8i1(<8 x i1> %v) {
+; CHECK-LABEL: vreduce_add_v8i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.v8i1(<8 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.v16i1(<16 x i1>)
+
+define signext i1 @vreduce_add_v16i1(<16 x i1> %v) {
+; CHECK-LABEL: vreduce_add_v16i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.v16i1(<16 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.v32i1(<32 x i1>)
+
+define signext i1 @vreduce_add_v32i1(<32 x i1> %v) {
+; LMULMAX1-LABEL: vreduce_add_v32i1:
+; LMULMAX1:       # %bb.0:
+; LMULMAX1-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; LMULMAX1-NEXT:    vmxor.mm v8, v0, v8
+; LMULMAX1-NEXT:    vcpop.m a0, v8
+; LMULMAX1-NEXT:    andi a0, a0, 1
+; LMULMAX1-NEXT:    neg a0, a0
+; LMULMAX1-NEXT:    ret
+;
+; LMULMAX8-LABEL: vreduce_add_v32i1:
+; LMULMAX8:       # %bb.0:
+; LMULMAX8-NEXT:    li a0, 32
+; LMULMAX8-NEXT:    vsetvli zero, a0, e8, m2, ta, mu
+; LMULMAX8-NEXT:    vcpop.m a0, v0
+; LMULMAX8-NEXT:    andi a0, a0, 1
+; LMULMAX8-NEXT:    neg a0, a0
+; LMULMAX8-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.v32i1(<32 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.v64i1(<64 x i1>)
+
+define signext i1 @vreduce_add_v64i1(<64 x i1> %v) {
+; LMULMAX1-LABEL: vreduce_add_v64i1:
+; LMULMAX1:       # %bb.0:
+; LMULMAX1-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; LMULMAX1-NEXT:    vmxor.mm v8, v8, v10
+; LMULMAX1-NEXT:    vmxor.mm v9, v0, v9
+; LMULMAX1-NEXT:    vmxor.mm v8, v9, v8
+; LMULMAX1-NEXT:    vcpop.m a0, v8
+; LMULMAX1-NEXT:    andi a0, a0, 1
+; LMULMAX1-NEXT:    neg a0, a0
+; LMULMAX1-NEXT:    ret
+;
+; LMULMAX8-LABEL: vreduce_add_v64i1:
+; LMULMAX8:       # %bb.0:
+; LMULMAX8-NEXT:    li a0, 64
+; LMULMAX8-NEXT:    vsetvli zero, a0, e8, m4, ta, mu
+; LMULMAX8-NEXT:    vcpop.m a0, v0
+; LMULMAX8-NEXT:    andi a0, a0, 1
+; LMULMAX8-NEXT:    neg a0, a0
+; LMULMAX8-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.v64i1(<64 x i1> %v)
+  ret i1 %red
+}

llvm/test/CodeGen/RISCV/rvv/vreductions-mask.ll

 ; CHECK-NEXT:    vsetvli a0, zero, e8, m8, ta, mu
 ; CHECK-NEXT:    vcpop.m a0, v0
 ; CHECK-NEXT:    snez a0, a0
 ; CHECK-NEXT:    neg a0, a0
 ; CHECK-NEXT:    ret
   %red = call i1 @llvm.vector.reduce.smin.nxv64i1(<vscale x 64 x i1> %v)
   ret i1 %red
 }
+
+declare i1 @llvm.vector.reduce.add.nxv1i1(<vscale x 1 x i1>)
+
+define signext i1 @vreduce_add_nxv1i1(<vscale x 1 x i1> %v) {
+; CHECK-LABEL: vreduce_add_nxv1i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, zero, e8, mf8, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.nxv1i1(<vscale x 1 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.nxv2i1(<vscale x 2 x i1>)
+
+define signext i1 @vreduce_add_nxv2i1(<vscale x 2 x i1> %v) {
+; CHECK-LABEL: vreduce_add_nxv2i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, zero, e8, mf4, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.nxv2i1(<vscale x 2 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.nxv4i1(<vscale x 4 x i1>)
+
+define signext i1 @vreduce_add_nxv4i1(<vscale x 4 x i1> %v) {
+; CHECK-LABEL: vreduce_add_nxv4i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, zero, e8, mf2, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.nxv4i1(<vscale x 4 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.nxv8i1(<vscale x 8 x i1>)
+
+define signext i1 @vreduce_add_nxv8i1(<vscale x 8 x i1> %v) {
+; CHECK-LABEL: vreduce_add_nxv8i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, zero, e8, m1, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.nxv8i1(<vscale x 8 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.nxv16i1(<vscale x 16 x i1>)
+
+define signext i1 @vreduce_add_nxv16i1(<vscale x 16 x i1> %v) {
+; CHECK-LABEL: vreduce_add_nxv16i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, zero, e8, m2, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.nxv16i1(<vscale x 16 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.nxv32i1(<vscale x 32 x i1>)
+
+define signext i1 @vreduce_add_nxv32i1(<vscale x 32 x i1> %v) {
+; CHECK-LABEL: vreduce_add_nxv32i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, zero, e8, m4, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.nxv32i1(<vscale x 32 x i1> %v)
+  ret i1 %red
+}
+
+declare i1 @llvm.vector.reduce.add.nxv64i1(<vscale x 64 x i1>)
+
+define signext i1 @vreduce_add_nxv64i1(<vscale x 64 x i1> %v) {
+; CHECK-LABEL: vreduce_add_nxv64i1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, zero, e8, m8, ta, mu
+; CHECK-NEXT:    vcpop.m a0, v0
+; CHECK-NEXT:    andi a0, a0, 1
+; CHECK-NEXT:    neg a0, a0
+; CHECK-NEXT:    ret
+  %red = call i1 @llvm.vector.reduce.add.nxv64i1(<vscale x 64 x i1> %v)
+  ret i1 %red
+}