[AArc64] Handle ISD::LROUND and ISD::LLROUND

[zatrazz]

Summary

This patch optimizes ISD::LROUND and ISD::LROUND to fcvtas
instruction. It currently only handles the scalar version.

This patch depends on https://reviews.llvm.org/D61390.

[craig.topper]

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

769

Is this really legal? I don't see any patterns or tests for lround.i32.

[efriedma]

Missing testcase for lroundl.

[zatrazz]

Updated patch based on the previous comment. The main change adapt the testcases for D61390
update and handle lroundl/llroundl correctly.

[zatrazz]

Updated patch based on D61390 changes. The main change is it allows
index both lround/llround by returned type.

[zatrazz]

Ping again now that D61390 has been approved.

[SjoerdMeijer]

llvm/lib/Target/AArch64/AArch64InstrInfo.td

3076

I think this is supported too for f16s with Armv8.2.

[zatrazz]

llvm/lib/Target/AArch64/AArch64InstrInfo.td

3076

Indeed it does, but FPToIntegerPats does not define a pattern for f16 currently. I think one possible addition would to extends fp to/from int pattern to f16 as well, however, I also think it is out the scope of this specific patch.

[zatrazz]

So should I handle f16 in a different patch or should I also adapt it on this one?

[SjoerdMeijer]

Looks reasonable to me.

llvm/lib/Target/AArch64/AArch64InstrInfo.td

3076

Okay, I see, there's indeed a pattern missing for f16s in FPToIntegerPats, so that's indeed extra work that belongs to a follow up patch. Would be highly appreciated if you can follow this up with a patch, so that we don't run into this later.

[zatrazz]

llvm/lib/Target/AArch64/AArch64InstrInfo.td

3076

Ok, I will add it on my backlog. I plan to send a lrint patch similar to lround, and then I will take a look at f16 support on FPToIntegerPats.

llvm/include/llvm/Target/TargetSelectionDAG.td

 def flog2      : SDNode<"ISD::FLOG2"      , SDTFPUnaryOp>;
 def frint      : SDNode<"ISD::FRINT"      , SDTFPUnaryOp>;
 def ftrunc     : SDNode<"ISD::FTRUNC"     , SDTFPUnaryOp>;
 def fceil      : SDNode<"ISD::FCEIL"      , SDTFPUnaryOp>;
 def ffloor     : SDNode<"ISD::FFLOOR"     , SDTFPUnaryOp>;
 def fnearbyint : SDNode<"ISD::FNEARBYINT" , SDTFPUnaryOp>;
 def fround     : SDNode<"ISD::FROUND"     , SDTFPUnaryOp>;
 
+def lround     : SDNode<"ISD::LROUND"     , SDTFPToIntOp>;
+def llround    : SDNode<"ISD::LLROUND"    , SDTFPToIntOp>;
+
 def fpround    : SDNode<"ISD::FP_ROUND"   , SDTFPRoundOp>;
 def fpextend   : SDNode<"ISD::FP_EXTEND"  , SDTFPExtendOp>;
 def fcopysign  : SDNode<"ISD::FCOPYSIGN"  , SDTFPSignOp>;
 
 def sint_to_fp : SDNode<"ISD::SINT_TO_FP" , SDTIntToFPOp>;
 def uint_to_fp : SDNode<"ISD::UINT_TO_FP" , SDTIntToFPOp>;
 def fp_to_sint : SDNode<"ISD::FP_TO_SINT" , SDTFPToIntOp>;
 def fp_to_uint : SDNode<"ISD::FP_TO_UINT" , SDTFPToIntOp>;

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

if (Subtarget->hasFullFP16()) {
         setOperationAction(ISD::FCEIL, Ty, Legal);
         setOperationAction(ISD::FRINT, Ty, Legal);
         setOperationAction(ISD::FTRUNC, Ty, Legal);
         setOperationAction(ISD::FROUND, Ty, Legal);
       }
     }
 
     setTruncStoreAction(MVT::v4i16, MVT::v4i8, Custom);
+
+    setOperationAction(ISD::LROUND,  MVT::i32, Legal);
+    setOperationAction(ISD::LROUND,  MVT::i64, Legal);
+    setOperationAction(ISD::LLROUND, MVT::i64, Legal);
   }
 
   PredictableSelectIsExpensive = Subtarget->predictableSelectIsExpensive();
 }
 
 void AArch64TargetLowering::addTypeForNEON(MVT VT, MVT PromotedBitwiseVT) {
   assert(VT.isVector() && "VT should be a vector type");
 

llvm/lib/Target/AArch64/AArch64InstrInfo.td

 defm : FPToIntegerPats<fp_to_uint, fceil,  "FCVTPU">;
 defm : FPToIntegerPats<fp_to_sint, ffloor, "FCVTMS">;
 defm : FPToIntegerPats<fp_to_uint, ffloor, "FCVTMU">;
 defm : FPToIntegerPats<fp_to_sint, ftrunc, "FCVTZS">;
 defm : FPToIntegerPats<fp_to_uint, ftrunc, "FCVTZU">;
 defm : FPToIntegerPats<fp_to_sint, fround, "FCVTAS">;
 defm : FPToIntegerPats<fp_to_uint, fround, "FCVTAU">;
 
+def : Pat<(i64 (lround f32:$Rn)),
+          (!cast<Instruction>(FCVTASUXSr) f32:$Rn)>;
+def : Pat<(i64 (lround f64:$Rn)),
+          (!cast<Instruction>(FCVTASUXDr) f64:$Rn)>;
+def : Pat<(i64 (llround f32:$Rn)),
+          (!cast<Instruction>(FCVTASUXSr) f32:$Rn)>;
+def : Pat<(i64 (llround f64:$Rn)),
+          (!cast<Instruction>(FCVTASUXDr) f64:$Rn)>;
+
 //===----------------------------------------------------------------------===//
 // Scaled integer to floating point conversion instructions.
 //===----------------------------------------------------------------------===//
 
 defm SCVTF : IntegerToFP<0, "scvtf", sint_to_fp>;
 defm UCVTF : IntegerToFP<1, "ucvtf", uint_to_fp>;
 
 //===----------------------------------------------------------------------===//

llvm/test/CodeGen/AArch64/llround-conv.ll

 ; CHECK-LABEL: testmuxd:
 ; CHECK:       b       llround
 define i64 @testmuxd(double %x) {
 entry:
   %call = tail call i64 @llround(double %x)
   ret i64 %call
 }
 
+; CHECK-LABEL: testmsws_builtin:
+; CHECK:       fcvtas  x0, s0
+; CHECK:       ret
+define i32 @testmsws_builtin(float %x) {
+entry:
+  %0 = tail call i64 @llvm.llround.f32(float %x)
+  %conv = trunc i64 %0 to i32
+  ret i32 %conv
+}
+
+; CHECK-LABEL: testmsxs_builtin:
+; CHECK:       fcvtas  x0, s0
+; CHECK-NEXT:  ret
+define i64 @testmsxs_builtin(float %x) {
+entry:
+  %0 = tail call i64 @llvm.llround.f32(float %x)
+  ret i64 %0
+}
+
+; CHECK-LABEL: testmswd_builtin:
+; CHECK:       fcvtas  x0, d0
+; CHECK:       ret
+define i32 @testmswd_builtin(double %x) {
+entry:
+  %0 = tail call i64 @llvm.llround.f64(double %x)
+  %conv = trunc i64 %0 to i32
+  ret i32 %conv
+}
+
+; CHECK-LABEL: testmsxd_builtin:
+; CHECK:       fcvtas  x0, d0
+; CHECK-NEXT:  ret
+define i64 @testmsxd_builtin(double %x) {
+entry:
+  %0 = tail call i64 @llvm.llround.f64(double %x)
+  ret i64 %0
+}
+
+; CHECK-LABEL: testmuws_builtin:
+; CHECK:       fcvtas  x0, s0
+; CHECK:       ret
+define i32 @testmuws_builtin(float %x) {
+entry:
+  %0 = tail call i64 @llvm.llround.f32(float %x)
+  %conv = trunc i64 %0 to i32
+  ret i32 %conv
+}
+
+; CHECK-LABEL: testmuxs_builtin:
+; CHECK:       fcvtas  x0, s0
+; CHECK-NEXT:  ret
+define i64 @testmuxs_builtin(float %x) {
+entry:
+  %0 = tail call i64 @llvm.llround.f32(float %x)
+  ret i64 %0
+}
+
+; CHECK-LABEL: testmuwd_builtin:
+; CHECK:       fcvtas  x0, d0
+; CHECK:       ret
+define i32 @testmuwd_builtin(double %x) {
+entry:
+  %0 = tail call i64 @llvm.llround.f64(double %x)
+  %conv = trunc i64 %0 to i32
+  ret i32 %conv
+}
+
+; CHECK-LABEL: testmuxd_builtin:
+; XCHECK:       fcvtas  x0, d0
+; XCHECK-NEXT:  ret
+define i64 @testmuxd_builtin(double %x) {
+entry:
+  %0 = tail call i64 @llvm.llround.f64(double %x)
+  ret i64 %0
+}
+
 declare i64 @llroundf(float) nounwind
 declare i64 @llround(double) nounwind
+declare i64 @llvm.llround.f32(float) nounwind readnone
+declare i64 @llvm.llround.f64(double) nounwind readnone

llvm/test/CodeGen/AArch64/lround-conv.ll

 ; CHECK-LABEL: testmuxd:
 ; CHECK:       b       lround
 define i64 @testmuxd(double %x) {
 entry:
   %call = tail call i64 @lround(double %x)
   ret i64 %call
 }
 
+; CHECK-LABEL: testmsws_builtin:
+; CHECK:       fcvtas  x0, s0
+; CHECK:       ret
+define i32 @testmsws_builtin(float %x) {
+entry:
+  %0 = tail call i64 @llvm.lround.i64.f32(float %x)
+  %conv = trunc i64 %0 to i32
+  ret i32 %conv
+}
+
+; CHECK-LABEL: testmsxs_builtin:
+; CHECK:       fcvtas  x0, s0
+; CHECK-NEXT:  ret
+define i64 @testmsxs_builtin(float %x) {
+entry:
+  %0 = tail call i64 @llvm.lround.i64.f32(float %x)
+  ret i64 %0
+}
+
+; CHECK-LABEL: testmswd_builtin:
+; CHECK:       fcvtas  x0, d0
+; CHECK:       ret
+define i32 @testmswd_builtin(double %x) {
+entry:
+  %0 = tail call i64 @llvm.lround.i64.f64(double %x)
+  %conv = trunc i64 %0 to i32
+  ret i32 %conv
+}
+
+; CHECK-LABEL: testmsxd_builtin:
+; CHECK:       fcvtas  x0, d0
+; CHECK-NEXT:  ret
+define i64 @testmsxd_builtin(double %x) {
+entry:
+  %0 = tail call i64 @llvm.lround.i64.f64(double %x)
+  ret i64 %0
+}
+
+; CHECK-LABEL: testmuws_builtin:
+; CHECK:       fcvtas  x0, s0
+; CHECK:       ret
+define i32 @testmuws_builtin(float %x) {
+entry:
+  %0 = tail call i64 @llvm.lround.i64.f32(float %x)
+  %conv = trunc i64 %0 to i32
+  ret i32 %conv
+}
+
+; CHECK-LABEL: testmuxs_builtin:
+; CHECK:       fcvtas  x0, s0
+; CHECK-NEXT:  ret
+define i64 @testmuxs_builtin(float %x) {
+entry:
+  %0 = tail call i64 @llvm.lround.i64.f32(float %x)
+  ret i64 %0
+}
+
+; CHECK-LABEL: testmuwd_builtin:
+; CHECK:       fcvtas  x0, d0
+; CHECK:       ret
+define i32 @testmuwd_builtin(double %x) {
+entry:
+  %0 = tail call i64 @llvm.lround.i64.f64(double %x)
+  %conv = trunc i64 %0 to i32
+  ret i32 %conv
+}
+
+; CHECK-LABEL: testmuxd_builtin:
+; CHECK:       fcvtas  x0, d0
+; CHECK-NEXT:  ret
+define i64 @testmuxd_builtin(double %x) {
+entry:
+  %0 = tail call i64 @llvm.lround.i64.f64(double %x)
+  ret i64 %0
+}
+
 declare i64 @lroundf(float) nounwind
 declare i64 @lround(double) nounwind
+declare i64 @llvm.lround.i64.f32(float) nounwind readnone
+declare i64 @llvm.lround.i64.f64(double) nounwind readnone