Do not lower VSETCC if operand is an f16 vector

[pirama]

Summary

SETCC with f16 vectors has OperationAction set to Expand but still gets
lowered to FCM* intrinsics based on its result type. This patch skips
lowering of VSETCC if the operand is an f16 vector.

v4 and v8 tests included.

[jmolloy]

Hi Pirama,

I'm slightly concerned that LowerSETCC is being called when the operation action is Expand. Could you please elaborate/investigate why this is the case before we go ahead with this patch?

Cheers,

James

[pirama]

Hi James,

During legalization, custom lowering is called based on the result type of a node and not its operand type. In this particular case, the caller is LegalizeOp in lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp (look for a switch statement close to the end of this function). Since the result type in these cases are i16 vectors, lowering still gets invoked.

As an aside, the legalizer queries OperationAction based on type of operand when legalizing operands and type of result when legalizing results. That is what necessitates these explicit checks/filters.

-Pirama

[pirama]

Ping... James, can you look at my earlier comment answering your question?

Thanks,
-Pirama

[jmolloy]

Hi Pirama,

Sorry for the long response time on this. It LGTM. Thanks for the explanation!

James

lib/Target/AArch64/AArch64ISelLowering.cpp

SDValue AArch64TargetLowering::LowerVSETCC(SDValue Op,
   if (LHS.getValueType().getVectorElementType().isInteger()) {
     assert(LHS.getValueType() == RHS.getValueType());
     AArch64CC::CondCode AArch64CC = changeIntCCToAArch64CC(CC);
     SDValue Cmp =
         EmitVectorComparison(LHS, RHS, AArch64CC, false, CmpVT, dl, DAG);
     return DAG.getSExtOrTrunc(Cmp, dl, Op.getValueType());
   }
 
+  if (LHS.getValueType().getVectorElementType() == MVT::f16)
+    return SDValue();
+
   assert(LHS.getValueType().getVectorElementType() == MVT::f32 ||
          LHS.getValueType().getVectorElementType() == MVT::f64);
 
   // Unfortunately, the mapping of LLVM FP CC's onto AArch64 CC's isn't totally
   // clean.  Some of them require two branches to implement.
   AArch64CC::CondCode CC1, CC2;
   bool ShouldInvert;
   changeVectorFPCCToAArch64CC(CC, CC1, CC2, ShouldInvert);

test/CodeGen/AArch64/fp16-v4-instructions.ll

 ; CHECK-LABEL: test_insert_at_zero:
 ; CHECK-NEXT: str d0, [x0]
 ; CHECK-NEXT: ret
   %1 = insertelement <4 x half> undef, half %a, i64 0
   store <4 x half> %1, <4 x half>* %b, align 4
   ret void
 }
 
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_une:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, ne
+; CHECK-DAG: csel {{.*}}, wzr, ne
+; CHECK-DAG: csel {{.*}}, wzr, ne
+; CHECK-DAG: csel {{.*}}, wzr, ne
+define <4 x i1> @test_fcmp_une(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp une <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_ueq:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, eq
+; CHECK-DAG: csel {{.*}}, wzr, eq
+; CHECK-DAG: csel {{.*}}, wzr, eq
+; CHECK-DAG: csel {{.*}}, wzr, eq
+; CHECK-DAG: csel {{.*}}, vs
+; CHECK-DAG: csel {{.*}}, vs
+; CHECK-DAG: csel {{.*}}, vs
+; CHECK-DAG: csel {{.*}}, vs
+define <4 x i1> @test_fcmp_ueq(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp ueq <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_ugt:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, hi
+; CHECK-DAG: csel {{.*}}, wzr, hi
+; CHECK-DAG: csel {{.*}}, wzr, hi
+; CHECK-DAG: csel {{.*}}, wzr, hi
+define <4 x i1> @test_fcmp_ugt(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp ugt <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_uge:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, pl
+; CHECK-DAG: csel {{.*}}, wzr, pl
+; CHECK-DAG: csel {{.*}}, wzr, pl
+; CHECK-DAG: csel {{.*}}, wzr, pl
+define <4 x i1> @test_fcmp_uge(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp uge <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_ult:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, lt
+; CHECK-DAG: csel {{.*}}, wzr, lt
+; CHECK-DAG: csel {{.*}}, wzr, lt
+; CHECK-DAG: csel {{.*}}, wzr, lt
+define <4 x i1> @test_fcmp_ult(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp ult <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_ule:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, le
+; CHECK-DAG: csel {{.*}}, wzr, le
+; CHECK-DAG: csel {{.*}}, wzr, le
+; CHECK-DAG: csel {{.*}}, wzr, le
+define <4 x i1> @test_fcmp_ule(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp ule <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_uno:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, vs
+; CHECK-DAG: csel {{.*}}, wzr, vs
+; CHECK-DAG: csel {{.*}}, wzr, vs
+; CHECK-DAG: csel {{.*}}, wzr, vs
+define <4 x i1> @test_fcmp_uno(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp uno <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_one:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, mi
+; CHECK-DAG: csel {{.*}}, wzr, mi
+; CHECK-DAG: csel {{.*}}, wzr, mi
+; CHECK-DAG: csel {{.*}}, wzr, mi
+; CHECK-DAG: csel {{.*}}, gt
+; CHECK-DAG: csel {{.*}}, gt
+; CHECK-DAG: csel {{.*}}, gt
+; CHECK-DAG: csel {{.*}}, gt
+define <4 x i1> @test_fcmp_one(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp one <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_oeq:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, eq
+; CHECK-DAG: csel {{.*}}, wzr, eq
+; CHECK-DAG: csel {{.*}}, wzr, eq
+; CHECK-DAG: csel {{.*}}, wzr, eq
+define <4 x i1> @test_fcmp_oeq(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp oeq <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_ogt:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, gt
+; CHECK-DAG: csel {{.*}}, wzr, gt
+; CHECK-DAG: csel {{.*}}, wzr, gt
+; CHECK-DAG: csel {{.*}}, wzr, gt
+define <4 x i1> @test_fcmp_ogt(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp ogt <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_oge:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, ge
+; CHECK-DAG: csel {{.*}}, wzr, ge
+; CHECK-DAG: csel {{.*}}, wzr, ge
+; CHECK-DAG: csel {{.*}}, wzr, ge
+define <4 x i1> @test_fcmp_oge(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp oge <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_olt:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, mi
+; CHECK-DAG: csel {{.*}}, wzr, mi
+; CHECK-DAG: csel {{.*}}, wzr, mi
+; CHECK-DAG: csel {{.*}}, wzr, mi
+define <4 x i1> @test_fcmp_olt(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp olt <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_ole:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, ls
+; CHECK-DAG: csel {{.*}}, wzr, ls
+; CHECK-DAG: csel {{.*}}, wzr, ls
+; CHECK-DAG: csel {{.*}}, wzr, ls
+define <4 x i1> @test_fcmp_ole(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp ole <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
+; Function Attrs: nounwind readnone
+; CHECK-LABEL: test_fcmp_ord:
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: fcvt
+; CHECK-DAG: csel {{.*}}, wzr, vc
+; CHECK-DAG: csel {{.*}}, wzr, vc
+; CHECK-DAG: csel {{.*}}, wzr, vc
+; CHECK-DAG: csel {{.*}}, wzr, vc
+define <4 x i1> @test_fcmp_ord(<4 x half> %a, <4 x half> %b) #0 {
+  %1 = fcmp ord <4 x half> %a, %b
+  ret <4 x i1> %1
+}
+
 attributes #0 = { nounwind }

test/CodeGen/AArch64/fp16-v8-instructions.ll

 ; CHECK-LABEL: test_insert_at_zero:
 ; CHECK-NEXT: str q0, [x0]
 ; CHECK-NEXT: ret
   %1 = insertelement <8 x half> undef, half %a, i64 0
   store <8 x half> %1, <8 x half>* %b, align 4
   ret void
 }
 
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_une(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp une <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 16 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_ueq(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp ueq <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_ugt(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp ugt <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_uge(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp uge <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_ult(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp ult <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_ule(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp ule <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_uno(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp uno <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_one(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp one <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_oeq(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp oeq <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_ogt(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp ogt <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_oge(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp oge <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_olt(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp olt <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_ole(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp ole <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
+; FileCheck checks are unwieldy with 16 fcvt and 8 csel tests.  Skipped.
+define <8 x i1> @test_fcmp_ord(<8 x half> %a, <8 x half> %b) #0 {
+  %1 = fcmp ord <8 x half> %a, %b
+  ret <8 x i1> %1
+}
+
 attributes #0 = { nounwind }