This is an archive of the discontinued LLVM Phabricator instance.

Differential D113601

[AArch64] Use custom lowering for {U,S}INT_TO_FP with i8.
ClosedPublic

Authored by fhahn on Nov 10 2021, 11:55 AM.

Details

Reviewers
dmgreen
ab
aemerson
Commits
rGc2ed9fd05479: [AArch64] Use custom lowering for {U,S}INT_TO_FP with i8.
Summary

With fullfp16, it is cheaper to cast the {U,S}INT_TO_FP operand to i16
first, rather than promoting it to i32. The custom lowering for
{U,S}INT_TO_FP already supports that, it just needs to be used.

Diff Detail

Event Timeline

fhahn created this revision.Nov 10 2021, 11:55 AM
Herald added subscribers: hiraditya, kristof.beyls. · View Herald TranscriptNov 10 2021, 11:56 AM
fhahn requested review of this revision.Nov 10 2021, 11:56 AM
Herald added a project: Restricted Project. · View Herald TranscriptNov 10 2021, 11:56 AM
Harbormaster completed remote builds in B133556: Diff 386275.Nov 10 2021, 1:31 PM
dmgreen accepted this revision.Nov 10 2021, 10:16 PM

Thanks. LGTM

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
1017

-> Otherwise

1034

It might be worth combining into this if block, to keep the similar code together.

This revision is now accepted and ready to land.Nov 10 2021, 10:16 PM
Closed by commit rGc2ed9fd05479: [AArch64] Use custom lowering for {U,S}INT_TO_FP with i8. (authored by fhahn). · Explain WhyNov 11 2021, 12:47 AM
This revision was automatically updated to reflect the committed changes.
fhahn added a commit: rGc2ed9fd05479: [AArch64] Use custom lowering for {U,S}INT_TO_FP with i8..
fhahn marked 2 inline comments as done.Nov 11 2021, 12:49 AM
fhahn added inline comments.
llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
1017

Thanks, the comment change is gone after moving to the existing if/else.

1034

Good point, I moved it to the existing if/else in the committed version. Thanks!

Revision Contents

PathSize
llvm/
lib/
Target/
AArch64/
12 lines
test/
CodeGen/
AArch64/
148 lines

Diff 386441

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 1,008 Lines • ▼ Show 20 Lines if (Subtarget->hasNEON()) {
setOperationAction(ISD::FP_TO_SINT_SAT, MVT::v1i64, Expand); setOperationAction(ISD::FP_TO_SINT_SAT, MVT::v1i64, Expand);
setOperationAction(ISD::FP_TO_UINT_SAT, MVT::v1i64, Expand); setOperationAction(ISD::FP_TO_UINT_SAT, MVT::v1i64, Expand);
setOperationAction(ISD::MUL, MVT::v1i64, Expand); setOperationAction(ISD::MUL, MVT::v1i64, Expand);
// AArch64 doesn't have a direct vector ->f32 conversion instructions for // AArch64 doesn't have a direct vector ->f32 conversion instructions for
// elements smaller than i32, so promote the input to i32 first. // elements smaller than i32, so promote the input to i32 first.
setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v4i8, MVT::v4i32); setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v4i8, MVT::v4i32);
setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v4i8, MVT::v4i32); setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v4i8, MVT::v4i32);
dmgreenUnsubmitted
Not Done
ReplyInline Actions

-> Otherwise

dmgreen: -> Otherwise
fhahnAuthorUnsubmitted
Done
ReplyInline Actions

Thanks, the comment change is gone after moving to the existing if/else.

fhahn: Thanks, the comment change is gone after moving to the existing if/else.
setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v8i8, MVT::v8i32);
setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v8i8, MVT::v8i32);
setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v16i8, MVT::v16i32);
setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v16i8, MVT::v16i32);
// Similarly, there is no direct i32 -> f64 vector conversion instruction. // Similarly, there is no direct i32 -> f64 vector conversion instruction.
setOperationAction(ISD::SINT_TO_FP, MVT::v2i32, Custom); setOperationAction(ISD::SINT_TO_FP, MVT::v2i32, Custom);
setOperationAction(ISD::UINT_TO_FP, MVT::v2i32, Custom); setOperationAction(ISD::UINT_TO_FP, MVT::v2i32, Custom);
setOperationAction(ISD::SINT_TO_FP, MVT::v2i64, Custom); setOperationAction(ISD::SINT_TO_FP, MVT::v2i64, Custom);
setOperationAction(ISD::UINT_TO_FP, MVT::v2i64, Custom); setOperationAction(ISD::UINT_TO_FP, MVT::v2i64, Custom);
// Or, direct i32 -> f16 vector conversion. Set it so custom, so the // Or, direct i32 -> f16 vector conversion. Set it so custom, so the
// conversion happens in two steps: v4i32 -> v4f32 -> v4f16 // conversion happens in two steps: v4i32 -> v4f32 -> v4f16
setOperationAction(ISD::SINT_TO_FP, MVT::v4i32, Custom); setOperationAction(ISD::SINT_TO_FP, MVT::v4i32, Custom);
setOperationAction(ISD::UINT_TO_FP, MVT::v4i32, Custom); setOperationAction(ISD::UINT_TO_FP, MVT::v4i32, Custom);
if (Subtarget->hasFullFP16()) { if (Subtarget->hasFullFP16()) {
setOperationAction(ISD::SINT_TO_FP, MVT::v8i8, Custom);
setOperationAction(ISD::UINT_TO_FP, MVT::v8i8, Custom);
setOperationAction(ISD::SINT_TO_FP, MVT::v16i8, Custom);
setOperationAction(ISD::UINT_TO_FP, MVT::v16i8, Custom);
setOperationAction(ISD::SINT_TO_FP, MVT::v4i16, Custom); setOperationAction(ISD::SINT_TO_FP, MVT::v4i16, Custom);
dmgreenUnsubmitted
Not Done
ReplyInline Actions

It might be worth combining into this if block, to keep the similar code together.

dmgreen: It might be worth combining into this if block, to keep the similar code together.
fhahnAuthorUnsubmitted
Done
ReplyInline Actions

Good point, I moved it to the existing if/else in the committed version. Thanks!

fhahn: Good point, I moved it to the existing if/else in the committed version. Thanks!
setOperationAction(ISD::UINT_TO_FP, MVT::v4i16, Custom); setOperationAction(ISD::UINT_TO_FP, MVT::v4i16, Custom);
setOperationAction(ISD::SINT_TO_FP, MVT::v8i16, Custom); setOperationAction(ISD::SINT_TO_FP, MVT::v8i16, Custom);
setOperationAction(ISD::UINT_TO_FP, MVT::v8i16, Custom); setOperationAction(ISD::UINT_TO_FP, MVT::v8i16, Custom);
} else { } else {
// when AArch64 doesn't have fullfp16 support, promote the input // when AArch64 doesn't have fullfp16 support, promote the input
// to i32 first. // to i32 first.
setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v8i8, MVT::v8i32);
setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v8i8, MVT::v8i32);
setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v16i8, MVT::v16i32);
setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v16i8, MVT::v16i32);
setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v4i16, MVT::v4i32); setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v4i16, MVT::v4i32);
setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v4i16, MVT::v4i32); setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v4i16, MVT::v4i32);
setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v8i16, MVT::v8i32); setOperationPromotedToType(ISD::SINT_TO_FP, MVT::v8i16, MVT::v8i32);
setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v8i16, MVT::v8i32); setOperationPromotedToType(ISD::UINT_TO_FP, MVT::v8i16, MVT::v8i32);
} }
setOperationAction(ISD::CTLZ, MVT::v1i64, Expand); setOperationAction(ISD::CTLZ, MVT::v1i64, Expand);
setOperationAction(ISD::CTLZ, MVT::v2i64, Expand); setOperationAction(ISD::CTLZ, MVT::v2i64, Expand);
▲ Show 20 Lines • Show All 18,348 Lines • Show Last 20 Lines

llvm/test/CodeGen/AArch64/fp16-v8-instructions.ll

Show First 20 Lines • Show All 408 Lines • ▼ Show 20 Lines
; CHECK-FP16-NEXT: sshr v0.4h, v0.4h, #8 ; CHECK-FP16-NEXT: sshr v0.4h, v0.4h, #8
; CHECK-FP16-NEXT: scvtf v0.4h, v0.4h ; CHECK-FP16-NEXT: scvtf v0.4h, v0.4h
; CHECK-FP16-NEXT: ret ; CHECK-FP16-NEXT: ret
%1 = sitofp <4 x i8> %a to <4 x half> %1 = sitofp <4 x i8> %a to <4 x half>
ret <4 x half> %1 ret <4 x half> %1
} }
define <8 x half> @sitofp_v8i8(<8 x i8> %a) #0 { define <8 x half> @sitofp_v8i8(<8 x i8> %a) #0 {
; CHECK-LABEL: sitofp_v8i8: ; CHECK-CVT-LABEL: sitofp_v8i8:
; CHECK: // %bb.0: ; CHECK-CVT: // %bb.0:
; CHECK-NEXT: sshll v0.8h, v0.8b, #0 ; CHECK-CVT-NEXT: sshll v0.8h, v0.8b, #0
; CHECK-NEXT: sshll2 v1.4s, v0.8h, #0 ; CHECK-CVT-NEXT: sshll2 v1.4s, v0.8h, #0
; CHECK-NEXT: sshll v0.4s, v0.4h, #0 ; CHECK-CVT-NEXT: sshll v0.4s, v0.4h, #0
; CHECK-NEXT: scvtf v1.4s, v1.4s ; CHECK-CVT-NEXT: scvtf v1.4s, v1.4s
; CHECK-NEXT: scvtf v0.4s, v0.4s ; CHECK-CVT-NEXT: scvtf v0.4s, v0.4s
; CHECK-NEXT: fcvtn v1.4h, v1.4s ; CHECK-CVT-NEXT: fcvtn v1.4h, v1.4s
; CHECK-NEXT: fcvtn v0.4h, v0.4s ; CHECK-CVT-NEXT: fcvtn v0.4h, v0.4s
; CHECK-NEXT: mov v0.d[1], v1.d[0] ; CHECK-CVT-NEXT: mov v0.d[1], v1.d[0]
; CHECK-NEXT: ret ; CHECK-CVT-NEXT: ret
;
; CHECK-FP16-LABEL: sitofp_v8i8:
; CHECK-FP16: // %bb.0:
; CHECK-FP16-NEXT: sshll v0.8h, v0.8b, #0
; CHECK-FP16-NEXT: scvtf v0.8h, v0.8h
; CHECK-FP16-NEXT: ret
%1 = sitofp <8 x i8> %a to <8 x half> %1 = sitofp <8 x i8> %a to <8 x half>
ret <8 x half> %1 ret <8 x half> %1
} }
define <16 x half> @sitofp_v16i8(<16 x i8> %a) #0 { define <16 x half> @sitofp_v16i8(<16 x i8> %a) #0 {
; CHECK-LABEL: sitofp_v16i8: ; CHECK-CVT-LABEL: sitofp_v16i8:
; CHECK: // %bb.0: ; CHECK-CVT: // %bb.0:
; CHECK-NEXT: sshll2 v1.8h, v0.16b, #0 ; CHECK-CVT-NEXT: sshll2 v1.8h, v0.16b, #0
; CHECK-NEXT: sshll v0.8h, v0.8b, #0 ; CHECK-CVT-NEXT: sshll v0.8h, v0.8b, #0
; CHECK-NEXT: sshll2 v2.4s, v1.8h, #0 ; CHECK-CVT-NEXT: sshll2 v2.4s, v1.8h, #0
; CHECK-NEXT: sshll v1.4s, v1.4h, #0 ; CHECK-CVT-NEXT: sshll v1.4s, v1.4h, #0
; CHECK-NEXT: sshll2 v3.4s, v0.8h, #0 ; CHECK-CVT-NEXT: sshll2 v3.4s, v0.8h, #0
; CHECK-NEXT: sshll v0.4s, v0.4h, #0 ; CHECK-CVT-NEXT: sshll v0.4s, v0.4h, #0
; CHECK-NEXT: scvtf v2.4s, v2.4s ; CHECK-CVT-NEXT: scvtf v2.4s, v2.4s
; CHECK-NEXT: scvtf v1.4s, v1.4s ; CHECK-CVT-NEXT: scvtf v1.4s, v1.4s
; CHECK-NEXT: scvtf v3.4s, v3.4s ; CHECK-CVT-NEXT: scvtf v3.4s, v3.4s
; CHECK-NEXT: scvtf v0.4s, v0.4s ; CHECK-CVT-NEXT: scvtf v0.4s, v0.4s
; CHECK-NEXT: fcvtn v2.4h, v2.4s ; CHECK-CVT-NEXT: fcvtn v2.4h, v2.4s
; CHECK-NEXT: fcvtn v1.4h, v1.4s ; CHECK-CVT-NEXT: fcvtn v1.4h, v1.4s
; CHECK-NEXT: fcvtn v3.4h, v3.4s ; CHECK-CVT-NEXT: fcvtn v3.4h, v3.4s
; CHECK-NEXT: fcvtn v0.4h, v0.4s ; CHECK-CVT-NEXT: fcvtn v0.4h, v0.4s
; CHECK-NEXT: mov v1.d[1], v2.d[0] ; CHECK-CVT-NEXT: mov v1.d[1], v2.d[0]
; CHECK-NEXT: mov v0.d[1], v3.d[0] ; CHECK-CVT-NEXT: mov v0.d[1], v3.d[0]
; CHECK-NEXT: ret ; CHECK-CVT-NEXT: ret
;
; CHECK-FP16-LABEL: sitofp_v16i8:
; CHECK-FP16: // %bb.0:
; CHECK-FP16-NEXT: sshll2 v1.8h, v0.16b, #0
; CHECK-FP16-NEXT: sshll v0.8h, v0.8b, #0
; CHECK-FP16-NEXT: scvtf v1.8h, v1.8h
; CHECK-FP16-NEXT: scvtf v0.8h, v0.8h
; CHECK-FP16-NEXT: ret
%1 = sitofp <16 x i8> %a to <16 x half> %1 = sitofp <16 x i8> %a to <16 x half>
ret <16 x half> %1 ret <16 x half> %1
} }
define <8 x half> @sitofp_i16(<8 x i16> %a) #0 { define <8 x half> @sitofp_i16(<8 x i16> %a) #0 {
; CHECK-CVT-LABEL: sitofp_i16: ; CHECK-CVT-LABEL: sitofp_i16:
; CHECK-CVT: // %bb.0: ; CHECK-CVT: // %bb.0:
; CHECK-CVT-NEXT: sshll2 v1.4s, v0.8h, #0 ; CHECK-CVT-NEXT: sshll2 v1.4s, v0.8h, #0
▲ Show 20 Lines • Show All 60 Lines • ▼ Show 20 Lines
; CHECK-FP16-NEXT: bic v0.4h, #255, lsl #8 ; CHECK-FP16-NEXT: bic v0.4h, #255, lsl #8
; CHECK-FP16-NEXT: ucvtf v0.4h, v0.4h ; CHECK-FP16-NEXT: ucvtf v0.4h, v0.4h
; CHECK-FP16-NEXT: ret ; CHECK-FP16-NEXT: ret
%1 = uitofp <4 x i8> %a to <4 x half> %1 = uitofp <4 x i8> %a to <4 x half>
ret <4 x half> %1 ret <4 x half> %1
} }
define <8 x half> @uitofp_v8i8(<8 x i8> %a) #0 { define <8 x half> @uitofp_v8i8(<8 x i8> %a) #0 {
; CHECK-LABEL: uitofp_v8i8: ; CHECK-CVT-LABEL: uitofp_v8i8:
; CHECK: // %bb.0: ; CHECK-CVT: // %bb.0:
; CHECK-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-CVT-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-NEXT: ushll2 v1.4s, v0.8h, #0 ; CHECK-CVT-NEXT: ushll2 v1.4s, v0.8h, #0
; CHECK-NEXT: ushll v0.4s, v0.4h, #0 ; CHECK-CVT-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-NEXT: ucvtf v1.4s, v1.4s ; CHECK-CVT-NEXT: ucvtf v1.4s, v1.4s
; CHECK-NEXT: ucvtf v0.4s, v0.4s ; CHECK-CVT-NEXT: ucvtf v0.4s, v0.4s
; CHECK-NEXT: fcvtn v1.4h, v1.4s ; CHECK-CVT-NEXT: fcvtn v1.4h, v1.4s
; CHECK-NEXT: fcvtn v0.4h, v0.4s ; CHECK-CVT-NEXT: fcvtn v0.4h, v0.4s
; CHECK-NEXT: mov v0.d[1], v1.d[0] ; CHECK-CVT-NEXT: mov v0.d[1], v1.d[0]
; CHECK-NEXT: ret ; CHECK-CVT-NEXT: ret
;
; CHECK-FP16-LABEL: uitofp_v8i8:
; CHECK-FP16: // %bb.0:
; CHECK-FP16-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-FP16-NEXT: ucvtf v0.8h, v0.8h
; CHECK-FP16-NEXT: ret
%1 = uitofp <8 x i8> %a to <8 x half> %1 = uitofp <8 x i8> %a to <8 x half>
ret <8 x half> %1 ret <8 x half> %1
} }
define <16 x half> @uitofp_v16i8(<16 x i8> %a) #0 { define <16 x half> @uitofp_v16i8(<16 x i8> %a) #0 {
; CHECK-LABEL: uitofp_v16i8: ; CHECK-CVT-LABEL: uitofp_v16i8:
; CHECK: // %bb.0: ; CHECK-CVT: // %bb.0:
; CHECK-NEXT: ushll2 v1.8h, v0.16b, #0 ; CHECK-CVT-NEXT: ushll2 v1.8h, v0.16b, #0
; CHECK-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-CVT-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-NEXT: ushll2 v2.4s, v1.8h, #0 ; CHECK-CVT-NEXT: ushll2 v2.4s, v1.8h, #0
; CHECK-NEXT: ushll v1.4s, v1.4h, #0 ; CHECK-CVT-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-NEXT: ushll2 v3.4s, v0.8h, #0 ; CHECK-CVT-NEXT: ushll2 v3.4s, v0.8h, #0
; CHECK-NEXT: ushll v0.4s, v0.4h, #0 ; CHECK-CVT-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-NEXT: ucvtf v2.4s, v2.4s ; CHECK-CVT-NEXT: ucvtf v2.4s, v2.4s
; CHECK-NEXT: ucvtf v1.4s, v1.4s ; CHECK-CVT-NEXT: ucvtf v1.4s, v1.4s
; CHECK-NEXT: ucvtf v3.4s, v3.4s ; CHECK-CVT-NEXT: ucvtf v3.4s, v3.4s
; CHECK-NEXT: ucvtf v0.4s, v0.4s ; CHECK-CVT-NEXT: ucvtf v0.4s, v0.4s
; CHECK-NEXT: fcvtn v2.4h, v2.4s ; CHECK-CVT-NEXT: fcvtn v2.4h, v2.4s
; CHECK-NEXT: fcvtn v1.4h, v1.4s ; CHECK-CVT-NEXT: fcvtn v1.4h, v1.4s
; CHECK-NEXT: fcvtn v3.4h, v3.4s ; CHECK-CVT-NEXT: fcvtn v3.4h, v3.4s
; CHECK-NEXT: fcvtn v0.4h, v0.4s ; CHECK-CVT-NEXT: fcvtn v0.4h, v0.4s
; CHECK-NEXT: mov v1.d[1], v2.d[0] ; CHECK-CVT-NEXT: mov v1.d[1], v2.d[0]
; CHECK-NEXT: mov v0.d[1], v3.d[0] ; CHECK-CVT-NEXT: mov v0.d[1], v3.d[0]
; CHECK-NEXT: ret ; CHECK-CVT-NEXT: ret
;
; CHECK-FP16-LABEL: uitofp_v16i8:
; CHECK-FP16: // %bb.0:
; CHECK-FP16-NEXT: ushll2 v1.8h, v0.16b, #0
; CHECK-FP16-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-FP16-NEXT: ucvtf v1.8h, v1.8h
; CHECK-FP16-NEXT: ucvtf v0.8h, v0.8h
; CHECK-FP16-NEXT: ret
%1 = uitofp <16 x i8> %a to <16 x half> %1 = uitofp <16 x i8> %a to <16 x half>
ret <16 x half> %1 ret <16 x half> %1
} }
define <8 x half> @uitofp_i16(<8 x i16> %a) #0 { define <8 x half> @uitofp_i16(<8 x i16> %a) #0 {
; CHECK-CVT-LABEL: uitofp_i16: ; CHECK-CVT-LABEL: uitofp_i16:
; CHECK-CVT: // %bb.0: ; CHECK-CVT: // %bb.0:
▲ Show 20 Lines • Show All 1,147 Lines • Show Last 20 Lines