This is an archive of the discontinued LLVM Phabricator instance.

Differential D88011

[ARM] Add patterns for vmulh
ClosedPublic

Authored by dmgreen on Sep 21 2020, 3:17 AM.

Details

Reviewers
SjoerdMeijer
samtebbs
samparker
efriedma
simon_tatham
Commits
rG2cf0e52b8548: [ARM] Add patterns for vmulh
Summary

Now that vmulh can be selected, this adds the MVE patterns to make it legal and generate instructions.

Diff Detail

Event Timeline

dmgreen created this revision.Sep 21 2020, 3:17 AM
Herald added a project: Restricted Project. · View Herald TranscriptSep 21 2020, 3:17 AM
Herald added subscribers: danielkiss, hiraditya, kristof.beyls. · View Herald Transcript
dmgreen requested review of this revision.Sep 21 2020, 3:17 AM
samparker added inline comments.Sep 21 2020, 4:22 AM
llvm/lib/Target/ARM/ARMISelLowering.cpp
762

So this doesn't break for non-vector targets then? Do they get expanded automatically because no vector types are supported?

dmgreen added inline comments.Sep 21 2020, 7:36 AM
llvm/lib/Target/ARM/ARMISelLowering.cpp
762

Do you mean like a cortex-m33? Where there is no MVE or NEON? I admit I didn't even consider that - but I think you are right. Everything will end up being scalarized anyway, and i32 mulhs/mulhu are handled appropriately.

samparker accepted this revision.Sep 22 2020, 12:50 AM

Thanks.

llvm/lib/Target/ARM/ARMISelLowering.cpp
762

Makes sense.

This revision is now accepted and ready to land.Sep 22 2020, 12:50 AM
dmgreen added a parent revision: D91921: [ARM] Introduce MVETRUNC ISel lowering.Dec 6 2020, 12:22 AM
Closed by commit rG2cf0e52b8548: [ARM] Add patterns for vmulh (authored by dmgreen). · Explain WhyMay 26 2021, 1:22 AM
This revision was automatically updated to reflect the committed changes.
dmgreen added a commit: rG2cf0e52b8548: [ARM] Add patterns for vmulh.

Revision Contents

PathSize
llvm/
lib/
Target/
ARM/
7 lines
9 lines
test/
CodeGen/
Thumb2/
351 lines

Diff 293119

llvm/lib/Target/ARM/ARMISelLowering.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 753 Lines • ▼ Show 20 LinesARMTargetLowering::ARMTargetLowering(const TargetMachine &TM,
} }
for (MVT VT : MVT::fixedlen_vector_valuetypes()) { for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
for (MVT InnerVT : MVT::fixedlen_vector_valuetypes()) { for (MVT InnerVT : MVT::fixedlen_vector_valuetypes()) {
setTruncStoreAction(VT, InnerVT, Expand); setTruncStoreAction(VT, InnerVT, Expand);
addAllExtLoads(VT, InnerVT, Expand); addAllExtLoads(VT, InnerVT, Expand);
} }
setOperationAction(ISD::MULHS, VT, Expand);
samparkerUnsubmitted
Not Done
ReplyInline Actions

So this doesn't break for non-vector targets then? Do they get expanded automatically because no vector types are supported?

samparker: So this doesn't break for non-vector targets then? Do they get expanded automatically because…
dmgreenAuthorUnsubmitted
Done
ReplyInline Actions

Do you mean like a cortex-m33? Where there is no MVE or NEON? I admit I didn't even consider that - but I think you are right. Everything will end up being scalarized anyway, and i32 mulhs/mulhu are handled appropriately.

dmgreen: Do you mean like a cortex-m33? Where there is no MVE or NEON? I admit I didn't even consider…
samparkerUnsubmitted
Not Done
ReplyInline Actions

Makes sense.

samparker: Makes sense.
setOperationAction(ISD::SMUL_LOHI, VT, Expand); setOperationAction(ISD::SMUL_LOHI, VT, Expand);
setOperationAction(ISD::MULHU, VT, Expand);
setOperationAction(ISD::UMUL_LOHI, VT, Expand); setOperationAction(ISD::UMUL_LOHI, VT, Expand);
setOperationAction(ISD::BSWAP, VT, Expand); setOperationAction(ISD::BSWAP, VT, Expand);
} }
setOperationAction(ISD::ConstantFP, MVT::f32, Custom); setOperationAction(ISD::ConstantFP, MVT::f32, Custom);
setOperationAction(ISD::ConstantFP, MVT::f64, Custom); setOperationAction(ISD::ConstantFP, MVT::f64, Custom);
▲ Show 20 Lines • Show All 162 Lines • ▼ Show 20 Lines if (Subtarget->hasNEON()) {
setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v2i32, Custom); setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v2i32, Custom);
setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v1i64, Custom); setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v1i64, Custom);
setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v16i8, Custom); setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v16i8, Custom);
setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v8i16, Custom); setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v8i16, Custom);
setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v4i32, Custom); setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v4i32, Custom);
setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v2i64, Custom); setOperationAction(ISD::CTTZ_ZERO_UNDEF, MVT::v2i64, Custom);
for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
setOperationAction(ISD::MULHS, VT, Expand);
setOperationAction(ISD::MULHU, VT, Expand);
}
// NEON only has FMA instructions as of VFP4. // NEON only has FMA instructions as of VFP4.
if (!Subtarget->hasVFP4Base()) { if (!Subtarget->hasVFP4Base()) {
setOperationAction(ISD::FMA, MVT::v2f32, Expand); setOperationAction(ISD::FMA, MVT::v2f32, Expand);
setOperationAction(ISD::FMA, MVT::v4f32, Expand); setOperationAction(ISD::FMA, MVT::v4f32, Expand);
} }
setTargetDAGCombine(ISD::SHL); setTargetDAGCombine(ISD::SHL);
setTargetDAGCombine(ISD::SRL); setTargetDAGCombine(ISD::SRL);
▲ Show 20 Lines • Show All 18,107 Lines • Show Last 20 Lines

llvm/lib/Target/ARM/ARMInstrMVE.td

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 4,701 Lines • ▼ Show 20 Lines let Predicates = [HasMVEInt] in {
// Predicated multiply returning high bits // Predicated multiply returning high bits
def : Pat<(VTI.Vec (pred_int (VTI.Vec MQPR:$Qm), (VTI.Vec MQPR:$Qn), def : Pat<(VTI.Vec (pred_int (VTI.Vec MQPR:$Qm), (VTI.Vec MQPR:$Qn),
(i32 VTI.Unsigned), (VTI.Pred VCCR:$mask), (i32 VTI.Unsigned), (VTI.Pred VCCR:$mask),
(VTI.Vec MQPR:$inactive))), (VTI.Vec MQPR:$inactive))),
(VTI.Vec (Inst (VTI.Vec MQPR:$Qm), (VTI.Vec MQPR:$Qn), (VTI.Vec (Inst (VTI.Vec MQPR:$Qm), (VTI.Vec MQPR:$Qn),
ARMVCCThen, (VTI.Pred VCCR:$mask), ARMVCCThen, (VTI.Pred VCCR:$mask),
(VTI.Vec MQPR:$inactive)))>; (VTI.Vec MQPR:$inactive)))>;
// ISD::MULHU/MULHS
if !eq(round, 0b0) then {
defvar mulh = !if(VTI.Unsigned, mulhu, mulhs);
let Predicates = [HasMVEInt] in {
def : Pat<(VTI.Vec (mulh (VTI.Vec MQPR:$a), (VTI.Vec MQPR:$b))),
(VTI.Vec (Inst MQPR:$a, MQPR:$b))>;
}
}
} }
} }
multiclass MVE_VMULT<string iname, MVEVectorVTInfo VTI, bit round> multiclass MVE_VMULT<string iname, MVEVectorVTInfo VTI, bit round>
: MVE_VxMULH_m<iname, VTI, !if(round, int_arm_mve_vrmulh, int_arm_mve_vmulh), : MVE_VxMULH_m<iname, VTI, !if(round, int_arm_mve_vrmulh, int_arm_mve_vmulh),
!if(round, int_arm_mve_rmulh_predicated, !if(round, int_arm_mve_rmulh_predicated,
int_arm_mve_mulh_predicated), int_arm_mve_mulh_predicated),
round>; round>;
▲ Show 20 Lines • Show All 2,616 Lines • Show Last 20 Lines

llvm/test/CodeGen/Thumb2/mve-vmulh.ll

Show First 20 Lines • Show All 42 Lines • ▼ Show 20 Linesentry:
%s = lshr <2 x i64> %m, <i64 32, i64 32> %s = lshr <2 x i64> %m, <i64 32, i64 32>
%s2 = trunc <2 x i64> %s to <2 x i32> %s2 = trunc <2 x i64> %s to <2 x i32>
ret <2 x i32> %s2 ret <2 x i32> %s2
} }
define arm_aapcs_vfpcc <4 x i32> @vmulhs_v4i32(<4 x i32> %s0, <4 x i32> %s1) { define arm_aapcs_vfpcc <4 x i32> @vmulhs_v4i32(<4 x i32> %s0, <4 x i32> %s1) {
; CHECK-LABEL: vmulhs_v4i32: ; CHECK-LABEL: vmulhs_v4i32:
; CHECK: @ %bb.0: @ %entry ; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vmov.f32 s8, s4 ; CHECK-NEXT: vmulh.s32 q0, q0, q1
; CHECK-NEXT: vmov.f32 s12, s0
; CHECK-NEXT: vmov.f32 s14, s1
; CHECK-NEXT: vmov.f32 s10, s5
; CHECK-NEXT: vmov r2, s12
; CHECK-NEXT: vmov r1, s14
; CHECK-NEXT: vmov r0, s10
; CHECK-NEXT: vmov.f32 s12, s6
; CHECK-NEXT: vmov.f32 s14, s7
; CHECK-NEXT: vmov.f32 s4, s2
; CHECK-NEXT: vmov.f32 s6, s3
; CHECK-NEXT: vmullb.s32 q0, q1, q3
; CHECK-NEXT: smmul r0, r1, r0
; CHECK-NEXT: vmov r1, s8
; CHECK-NEXT: smmul r1, r2, r1
; CHECK-NEXT: vmov.32 q2[0], r1
; CHECK-NEXT: vmov.32 q2[1], r0
; CHECK-NEXT: vmov r0, s1
; CHECK-NEXT: vmov.32 q2[2], r0
; CHECK-NEXT: vmov r0, s3
; CHECK-NEXT: vmov.32 q2[3], r0
; CHECK-NEXT: vmov q0, q2
; CHECK-NEXT: bx lr ; CHECK-NEXT: bx lr
entry: entry:
%s0s = sext <4 x i32> %s0 to <4 x i64> %s0s = sext <4 x i32> %s0 to <4 x i64>
%s1s = sext <4 x i32> %s1 to <4 x i64> %s1s = sext <4 x i32> %s1 to <4 x i64>
%m = mul <4 x i64> %s0s, %s1s %m = mul <4 x i64> %s0s, %s1s
%s = ashr <4 x i64> %m, <i64 32, i64 32, i64 32, i64 32> %s = ashr <4 x i64> %m, <i64 32, i64 32, i64 32, i64 32>
%s2 = trunc <4 x i64> %s to <4 x i32> %s2 = trunc <4 x i64> %s to <4 x i32>
ret <4 x i32> %s2 ret <4 x i32> %s2
} }
define arm_aapcs_vfpcc <4 x i32> @vmulhu_v4i32(<4 x i32> %s0, <4 x i32> %s1) { define arm_aapcs_vfpcc <4 x i32> @vmulhu_v4i32(<4 x i32> %s0, <4 x i32> %s1) {
; CHECK-LABEL: vmulhu_v4i32: ; CHECK-LABEL: vmulhu_v4i32:
; CHECK: @ %bb.0: @ %entry ; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: .vsave {d8, d9} ; CHECK-NEXT: vmulh.u32 q0, q0, q1
; CHECK-NEXT: vpush {d8, d9}
; CHECK-NEXT: vmov.f32 s12, s6
; CHECK-NEXT: vmov.f32 s16, s2
; CHECK-NEXT: vmov.f32 s14, s7
; CHECK-NEXT: vmov.f32 s18, s3
; CHECK-NEXT: vmov.f32 s6, s5
; CHECK-NEXT: vmullb.u32 q2, q4, q3
; CHECK-NEXT: vmov.f32 s2, s1
; CHECK-NEXT: vmullb.u32 q3, q0, q1
; CHECK-NEXT: vmov.f32 s0, s13
; CHECK-NEXT: vmov.f32 s1, s15
; CHECK-NEXT: vmov.f32 s2, s9
; CHECK-NEXT: vmov.f32 s3, s11
; CHECK-NEXT: vpop {d8, d9}
; CHECK-NEXT: bx lr ; CHECK-NEXT: bx lr
entry: entry:
%s0s = zext <4 x i32> %s0 to <4 x i64> %s0s = zext <4 x i32> %s0 to <4 x i64>
%s1s = zext <4 x i32> %s1 to <4 x i64> %s1s = zext <4 x i32> %s1 to <4 x i64>
%m = mul <4 x i64> %s0s, %s1s %m = mul <4 x i64> %s0s, %s1s
%s = lshr <4 x i64> %m, <i64 32, i64 32, i64 32, i64 32> %s = lshr <4 x i64> %m, <i64 32, i64 32, i64 32, i64 32>
%s2 = trunc <4 x i64> %s to <4 x i32> %s2 = trunc <4 x i64> %s to <4 x i32>
ret <4 x i32> %s2 ret <4 x i32> %s2
Show All 27 Linesentry:
%s = lshr <4 x i32> %m, <i32 16, i32 16, i32 16, i32 16> %s = lshr <4 x i32> %m, <i32 16, i32 16, i32 16, i32 16>
%s2 = trunc <4 x i32> %s to <4 x i16> %s2 = trunc <4 x i32> %s to <4 x i16>
ret <4 x i16> %s2 ret <4 x i16> %s2
} }
define arm_aapcs_vfpcc <8 x i16> @vmulhs_v8i16(<8 x i16> %s0, <8 x i16> %s1) { define arm_aapcs_vfpcc <8 x i16> @vmulhs_v8i16(<8 x i16> %s0, <8 x i16> %s1) {
; CHECK-LABEL: vmulhs_v8i16: ; CHECK-LABEL: vmulhs_v8i16:
; CHECK: @ %bb.0: @ %entry ; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vmov.u16 r0, q1[0] ; CHECK-NEXT: vmulh.s16 q0, q0, q1
; CHECK-NEXT: vmov.32 q2[0], r0
; CHECK-NEXT: vmov.u16 r0, q1[1]
; CHECK-NEXT: vmov.32 q2[1], r0
; CHECK-NEXT: vmov.u16 r0, q1[2]
; CHECK-NEXT: vmov.32 q2[2], r0
; CHECK-NEXT: vmov.u16 r0, q1[3]
; CHECK-NEXT: vmov.32 q2[3], r0
; CHECK-NEXT: vmov.u16 r0, q0[0]
; CHECK-NEXT: vmov.32 q3[0], r0
; CHECK-NEXT: vmov.u16 r0, q0[1]
; CHECK-NEXT: vmov.32 q3[1], r0
; CHECK-NEXT: vmov.u16 r0, q0[2]
; CHECK-NEXT: vmov.32 q3[2], r0
; CHECK-NEXT: vmov.u16 r0, q0[3]
; CHECK-NEXT: vmov.32 q3[3], r0
; CHECK-NEXT: vmullb.s16 q2, q3, q2
; CHECK-NEXT: vshr.s32 q3, q2, #16
; CHECK-NEXT: vmov r0, s12
; CHECK-NEXT: vmov.16 q2[0], r0
; CHECK-NEXT: vmov r0, s13
; CHECK-NEXT: vmov.16 q2[1], r0
; CHECK-NEXT: vmov r0, s14
; CHECK-NEXT: vmov.16 q2[2], r0
; CHECK-NEXT: vmov r0, s15
; CHECK-NEXT: vmov.16 q2[3], r0
; CHECK-NEXT: vmov.u16 r0, q1[4]
; CHECK-NEXT: vmov.32 q3[0], r0
; CHECK-NEXT: vmov.u16 r0, q1[5]
; CHECK-NEXT: vmov.32 q3[1], r0
; CHECK-NEXT: vmov.u16 r0, q1[6]
; CHECK-NEXT: vmov.32 q3[2], r0
; CHECK-NEXT: vmov.u16 r0, q1[7]
; CHECK-NEXT: vmov.32 q3[3], r0
; CHECK-NEXT: vmov.u16 r0, q0[4]
; CHECK-NEXT: vmov.32 q1[0], r0
; CHECK-NEXT: vmov.u16 r0, q0[5]
; CHECK-NEXT: vmov.32 q1[1], r0
; CHECK-NEXT: vmov.u16 r0, q0[6]
; CHECK-NEXT: vmov.32 q1[2], r0
; CHECK-NEXT: vmov.u16 r0, q0[7]
; CHECK-NEXT: vmov.32 q1[3], r0
; CHECK-NEXT: vmullb.s16 q0, q1, q3
; CHECK-NEXT: vshr.s32 q0, q0, #16
; CHECK-NEXT: vmov r0, s0
; CHECK-NEXT: vmov.16 q2[4], r0
; CHECK-NEXT: vmov r0, s1
; CHECK-NEXT: vmov.16 q2[5], r0
; CHECK-NEXT: vmov r0, s2
; CHECK-NEXT: vmov.16 q2[6], r0
; CHECK-NEXT: vmov r0, s3
; CHECK-NEXT: vmov.16 q2[7], r0
; CHECK-NEXT: vmov q0, q2
; CHECK-NEXT: bx lr ; CHECK-NEXT: bx lr
entry: entry:
%s0s = sext <8 x i16> %s0 to <8 x i32> %s0s = sext <8 x i16> %s0 to <8 x i32>
%s1s = sext <8 x i16> %s1 to <8 x i32> %s1s = sext <8 x i16> %s1 to <8 x i32>
%m = mul <8 x i32> %s0s, %s1s %m = mul <8 x i32> %s0s, %s1s
%s = ashr <8 x i32> %m, <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16> %s = ashr <8 x i32> %m, <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16>
%s2 = trunc <8 x i32> %s to <8 x i16> %s2 = trunc <8 x i32> %s to <8 x i16>
ret <8 x i16> %s2 ret <8 x i16> %s2
} }
define arm_aapcs_vfpcc <8 x i16> @vmulhu_v8i16(<8 x i16> %s0, <8 x i16> %s1) { define arm_aapcs_vfpcc <8 x i16> @vmulhu_v8i16(<8 x i16> %s0, <8 x i16> %s1) {
; CHECK-LABEL: vmulhu_v8i16: ; CHECK-LABEL: vmulhu_v8i16:
; CHECK: @ %bb.0: @ %entry ; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vmov.u16 r0, q1[0] ; CHECK-NEXT: vmulh.u16 q0, q0, q1
; CHECK-NEXT: vmov.32 q2[0], r0
; CHECK-NEXT: vmov.u16 r0, q1[1]
; CHECK-NEXT: vmov.32 q2[1], r0
; CHECK-NEXT: vmov.u16 r0, q1[2]
; CHECK-NEXT: vmov.32 q2[2], r0
; CHECK-NEXT: vmov.u16 r0, q1[3]
; CHECK-NEXT: vmov.32 q2[3], r0
; CHECK-NEXT: vmov.u16 r0, q0[0]
; CHECK-NEXT: vmov.32 q3[0], r0
; CHECK-NEXT: vmov.u16 r0, q0[1]
; CHECK-NEXT: vmov.32 q3[1], r0
; CHECK-NEXT: vmov.u16 r0, q0[2]
; CHECK-NEXT: vmov.32 q3[2], r0
; CHECK-NEXT: vmov.u16 r0, q0[3]
; CHECK-NEXT: vmov.32 q3[3], r0
; CHECK-NEXT: vmullb.u16 q2, q3, q2
; CHECK-NEXT: vshr.u32 q3, q2, #16
; CHECK-NEXT: vmov r0, s12
; CHECK-NEXT: vmov.16 q2[0], r0
; CHECK-NEXT: vmov r0, s13
; CHECK-NEXT: vmov.16 q2[1], r0
; CHECK-NEXT: vmov r0, s14
; CHECK-NEXT: vmov.16 q2[2], r0
; CHECK-NEXT: vmov r0, s15
; CHECK-NEXT: vmov.16 q2[3], r0
; CHECK-NEXT: vmov.u16 r0, q1[4]
; CHECK-NEXT: vmov.32 q3[0], r0
; CHECK-NEXT: vmov.u16 r0, q1[5]
; CHECK-NEXT: vmov.32 q3[1], r0
; CHECK-NEXT: vmov.u16 r0, q1[6]
; CHECK-NEXT: vmov.32 q3[2], r0
; CHECK-NEXT: vmov.u16 r0, q1[7]
; CHECK-NEXT: vmov.32 q3[3], r0
; CHECK-NEXT: vmov.u16 r0, q0[4]
; CHECK-NEXT: vmov.32 q1[0], r0
; CHECK-NEXT: vmov.u16 r0, q0[5]
; CHECK-NEXT: vmov.32 q1[1], r0
; CHECK-NEXT: vmov.u16 r0, q0[6]
; CHECK-NEXT: vmov.32 q1[2], r0
; CHECK-NEXT: vmov.u16 r0, q0[7]
; CHECK-NEXT: vmov.32 q1[3], r0
; CHECK-NEXT: vmullb.u16 q0, q1, q3
; CHECK-NEXT: vshr.u32 q0, q0, #16
; CHECK-NEXT: vmov r0, s0
; CHECK-NEXT: vmov.16 q2[4], r0
; CHECK-NEXT: vmov r0, s1
; CHECK-NEXT: vmov.16 q2[5], r0
; CHECK-NEXT: vmov r0, s2
; CHECK-NEXT: vmov.16 q2[6], r0
; CHECK-NEXT: vmov r0, s3
; CHECK-NEXT: vmov.16 q2[7], r0
; CHECK-NEXT: vmov q0, q2
; CHECK-NEXT: bx lr ; CHECK-NEXT: bx lr
entry: entry:
%s0s = zext <8 x i16> %s0 to <8 x i32> %s0s = zext <8 x i16> %s0 to <8 x i32>
%s1s = zext <8 x i16> %s1 to <8 x i32> %s1s = zext <8 x i16> %s1 to <8 x i32>
%m = mul <8 x i32> %s0s, %s1s %m = mul <8 x i32> %s0s, %s1s
%s = lshr <8 x i32> %m, <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16> %s = lshr <8 x i32> %m, <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16>
%s2 = trunc <8 x i32> %s to <8 x i16> %s2 = trunc <8 x i32> %s to <8 x i16>
ret <8 x i16> %s2 ret <8 x i16> %s2
Show All 27 Linesentry:
%s = lshr <8 x i16> %m, <i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8> %s = lshr <8 x i16> %m, <i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8>
%s2 = trunc <8 x i16> %s to <8 x i8> %s2 = trunc <8 x i16> %s to <8 x i8>
ret <8 x i8> %s2 ret <8 x i8> %s2
} }
define arm_aapcs_vfpcc <16 x i8> @vmulhs_v16i8(<16 x i8> %s0, <16 x i8> %s1) { define arm_aapcs_vfpcc <16 x i8> @vmulhs_v16i8(<16 x i8> %s0, <16 x i8> %s1) {
; CHECK-LABEL: vmulhs_v16i8: ; CHECK-LABEL: vmulhs_v16i8:
; CHECK: @ %bb.0: @ %entry ; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vmov.u8 r0, q1[0] ; CHECK-NEXT: vmulh.s8 q0, q0, q1
; CHECK-NEXT: vmov.16 q2[0], r0
; CHECK-NEXT: vmov.u8 r0, q1[1]
; CHECK-NEXT: vmov.16 q2[1], r0
; CHECK-NEXT: vmov.u8 r0, q1[2]
; CHECK-NEXT: vmov.16 q2[2], r0
; CHECK-NEXT: vmov.u8 r0, q1[3]
; CHECK-NEXT: vmov.16 q2[3], r0
; CHECK-NEXT: vmov.u8 r0, q1[4]
; CHECK-NEXT: vmov.16 q2[4], r0
; CHECK-NEXT: vmov.u8 r0, q1[5]
; CHECK-NEXT: vmov.16 q2[5], r0
; CHECK-NEXT: vmov.u8 r0, q1[6]
; CHECK-NEXT: vmov.16 q2[6], r0
; CHECK-NEXT: vmov.u8 r0, q1[7]
; CHECK-NEXT: vmov.16 q2[7], r0
; CHECK-NEXT: vmov.u8 r0, q0[0]
; CHECK-NEXT: vmov.16 q3[0], r0
; CHECK-NEXT: vmov.u8 r0, q0[1]
; CHECK-NEXT: vmov.16 q3[1], r0
; CHECK-NEXT: vmov.u8 r0, q0[2]
; CHECK-NEXT: vmov.16 q3[2], r0
; CHECK-NEXT: vmov.u8 r0, q0[3]
; CHECK-NEXT: vmov.16 q3[3], r0
; CHECK-NEXT: vmov.u8 r0, q0[4]
; CHECK-NEXT: vmov.16 q3[4], r0
; CHECK-NEXT: vmov.u8 r0, q0[5]
; CHECK-NEXT: vmov.16 q3[5], r0
; CHECK-NEXT: vmov.u8 r0, q0[6]
; CHECK-NEXT: vmov.16 q3[6], r0
; CHECK-NEXT: vmov.u8 r0, q0[7]
; CHECK-NEXT: vmov.16 q3[7], r0
; CHECK-NEXT: vmullb.s8 q2, q3, q2
; CHECK-NEXT: vshr.s16 q3, q2, #8
; CHECK-NEXT: vmov.u16 r0, q3[0]
; CHECK-NEXT: vmov.8 q2[0], r0
; CHECK-NEXT: vmov.u16 r0, q3[1]
; CHECK-NEXT: vmov.8 q2[1], r0
; CHECK-NEXT: vmov.u16 r0, q3[2]
; CHECK-NEXT: vmov.8 q2[2], r0
; CHECK-NEXT: vmov.u16 r0, q3[3]
; CHECK-NEXT: vmov.8 q2[3], r0
; CHECK-NEXT: vmov.u16 r0, q3[4]
; CHECK-NEXT: vmov.8 q2[4], r0
; CHECK-NEXT: vmov.u16 r0, q3[5]
; CHECK-NEXT: vmov.8 q2[5], r0
; CHECK-NEXT: vmov.u16 r0, q3[6]
; CHECK-NEXT: vmov.8 q2[6], r0
; CHECK-NEXT: vmov.u16 r0, q3[7]
; CHECK-NEXT: vmov.8 q2[7], r0
; CHECK-NEXT: vmov.u8 r0, q1[8]
; CHECK-NEXT: vmov.16 q3[0], r0
; CHECK-NEXT: vmov.u8 r0, q1[9]
; CHECK-NEXT: vmov.16 q3[1], r0
; CHECK-NEXT: vmov.u8 r0, q1[10]
; CHECK-NEXT: vmov.16 q3[2], r0
; CHECK-NEXT: vmov.u8 r0, q1[11]
; CHECK-NEXT: vmov.16 q3[3], r0
; CHECK-NEXT: vmov.u8 r0, q1[12]
; CHECK-NEXT: vmov.16 q3[4], r0
; CHECK-NEXT: vmov.u8 r0, q1[13]
; CHECK-NEXT: vmov.16 q3[5], r0
; CHECK-NEXT: vmov.u8 r0, q1[14]
; CHECK-NEXT: vmov.16 q3[6], r0
; CHECK-NEXT: vmov.u8 r0, q1[15]
; CHECK-NEXT: vmov.16 q3[7], r0
; CHECK-NEXT: vmov.u8 r0, q0[8]
; CHECK-NEXT: vmov.16 q1[0], r0
; CHECK-NEXT: vmov.u8 r0, q0[9]
; CHECK-NEXT: vmov.16 q1[1], r0
; CHECK-NEXT: vmov.u8 r0, q0[10]
; CHECK-NEXT: vmov.16 q1[2], r0
; CHECK-NEXT: vmov.u8 r0, q0[11]
; CHECK-NEXT: vmov.16 q1[3], r0
; CHECK-NEXT: vmov.u8 r0, q0[12]
; CHECK-NEXT: vmov.16 q1[4], r0
; CHECK-NEXT: vmov.u8 r0, q0[13]
; CHECK-NEXT: vmov.16 q1[5], r0
; CHECK-NEXT: vmov.u8 r0, q0[14]
; CHECK-NEXT: vmov.16 q1[6], r0
; CHECK-NEXT: vmov.u8 r0, q0[15]
; CHECK-NEXT: vmov.16 q1[7], r0
; CHECK-NEXT: vmullb.s8 q0, q1, q3
; CHECK-NEXT: vshr.s16 q0, q0, #8
; CHECK-NEXT: vmov.u16 r0, q0[0]
; CHECK-NEXT: vmov.8 q2[8], r0
; CHECK-NEXT: vmov.u16 r0, q0[1]
; CHECK-NEXT: vmov.8 q2[9], r0
; CHECK-NEXT: vmov.u16 r0, q0[2]
; CHECK-NEXT: vmov.8 q2[10], r0
; CHECK-NEXT: vmov.u16 r0, q0[3]
; CHECK-NEXT: vmov.8 q2[11], r0
; CHECK-NEXT: vmov.u16 r0, q0[4]
; CHECK-NEXT: vmov.8 q2[12], r0
; CHECK-NEXT: vmov.u16 r0, q0[5]
; CHECK-NEXT: vmov.8 q2[13], r0
; CHECK-NEXT: vmov.u16 r0, q0[6]
; CHECK-NEXT: vmov.8 q2[14], r0
; CHECK-NEXT: vmov.u16 r0, q0[7]
; CHECK-NEXT: vmov.8 q2[15], r0
; CHECK-NEXT: vmov q0, q2
; CHECK-NEXT: bx lr ; CHECK-NEXT: bx lr
entry: entry:
%s0s = sext <16 x i8> %s0 to <16 x i16> %s0s = sext <16 x i8> %s0 to <16 x i16>
%s1s = sext <16 x i8> %s1 to <16 x i16> %s1s = sext <16 x i8> %s1 to <16 x i16>
%m = mul <16 x i16> %s0s, %s1s %m = mul <16 x i16> %s0s, %s1s
%s = ashr <16 x i16> %m, <i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8> %s = ashr <16 x i16> %m, <i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8>
%s2 = trunc <16 x i16> %s to <16 x i8> %s2 = trunc <16 x i16> %s to <16 x i8>
ret <16 x i8> %s2 ret <16 x i8> %s2
} }
define arm_aapcs_vfpcc <16 x i8> @vmulhu_v16i8(<16 x i8> %s0, <16 x i8> %s1) { define arm_aapcs_vfpcc <16 x i8> @vmulhu_v16i8(<16 x i8> %s0, <16 x i8> %s1) {
; CHECK-LABEL: vmulhu_v16i8: ; CHECK-LABEL: vmulhu_v16i8:
; CHECK: @ %bb.0: @ %entry ; CHECK: @ %bb.0: @ %entry
; CHECK-NEXT: vmov.u8 r0, q1[0] ; CHECK-NEXT: vmulh.u8 q0, q0, q1
; CHECK-NEXT: vmov.16 q2[0], r0
; CHECK-NEXT: vmov.u8 r0, q1[1]
; CHECK-NEXT: vmov.16 q2[1], r0
; CHECK-NEXT: vmov.u8 r0, q1[2]
; CHECK-NEXT: vmov.16 q2[2], r0
; CHECK-NEXT: vmov.u8 r0, q1[3]
; CHECK-NEXT: vmov.16 q2[3], r0
; CHECK-NEXT: vmov.u8 r0, q1[4]
; CHECK-NEXT: vmov.16 q2[4], r0
; CHECK-NEXT: vmov.u8 r0, q1[5]
; CHECK-NEXT: vmov.16 q2[5], r0
; CHECK-NEXT: vmov.u8 r0, q1[6]
; CHECK-NEXT: vmov.16 q2[6], r0
; CHECK-NEXT: vmov.u8 r0, q1[7]
; CHECK-NEXT: vmov.16 q2[7], r0
; CHECK-NEXT: vmov.u8 r0, q0[0]
; CHECK-NEXT: vmov.16 q3[0], r0
; CHECK-NEXT: vmov.u8 r0, q0[1]
; CHECK-NEXT: vmov.16 q3[1], r0
; CHECK-NEXT: vmov.u8 r0, q0[2]
; CHECK-NEXT: vmov.16 q3[2], r0
; CHECK-NEXT: vmov.u8 r0, q0[3]
; CHECK-NEXT: vmov.16 q3[3], r0
; CHECK-NEXT: vmov.u8 r0, q0[4]
; CHECK-NEXT: vmov.16 q3[4], r0
; CHECK-NEXT: vmov.u8 r0, q0[5]
; CHECK-NEXT: vmov.16 q3[5], r0
; CHECK-NEXT: vmov.u8 r0, q0[6]
; CHECK-NEXT: vmov.16 q3[6], r0
; CHECK-NEXT: vmov.u8 r0, q0[7]
; CHECK-NEXT: vmov.16 q3[7], r0
; CHECK-NEXT: vmullb.u8 q2, q3, q2
; CHECK-NEXT: vshr.u16 q3, q2, #8
; CHECK-NEXT: vmov.u16 r0, q3[0]
; CHECK-NEXT: vmov.8 q2[0], r0
; CHECK-NEXT: vmov.u16 r0, q3[1]
; CHECK-NEXT: vmov.8 q2[1], r0
; CHECK-NEXT: vmov.u16 r0, q3[2]
; CHECK-NEXT: vmov.8 q2[2], r0
; CHECK-NEXT: vmov.u16 r0, q3[3]
; CHECK-NEXT: vmov.8 q2[3], r0
; CHECK-NEXT: vmov.u16 r0, q3[4]
; CHECK-NEXT: vmov.8 q2[4], r0
; CHECK-NEXT: vmov.u16 r0, q3[5]
; CHECK-NEXT: vmov.8 q2[5], r0
; CHECK-NEXT: vmov.u16 r0, q3[6]
; CHECK-NEXT: vmov.8 q2[6], r0
; CHECK-NEXT: vmov.u16 r0, q3[7]
; CHECK-NEXT: vmov.8 q2[7], r0
; CHECK-NEXT: vmov.u8 r0, q1[8]
; CHECK-NEXT: vmov.16 q3[0], r0
; CHECK-NEXT: vmov.u8 r0, q1[9]
; CHECK-NEXT: vmov.16 q3[1], r0
; CHECK-NEXT: vmov.u8 r0, q1[10]
; CHECK-NEXT: vmov.16 q3[2], r0
; CHECK-NEXT: vmov.u8 r0, q1[11]
; CHECK-NEXT: vmov.16 q3[3], r0
; CHECK-NEXT: vmov.u8 r0, q1[12]
; CHECK-NEXT: vmov.16 q3[4], r0
; CHECK-NEXT: vmov.u8 r0, q1[13]
; CHECK-NEXT: vmov.16 q3[5], r0
; CHECK-NEXT: vmov.u8 r0, q1[14]
; CHECK-NEXT: vmov.16 q3[6], r0
; CHECK-NEXT: vmov.u8 r0, q1[15]
; CHECK-NEXT: vmov.16 q3[7], r0
; CHECK-NEXT: vmov.u8 r0, q0[8]
; CHECK-NEXT: vmov.16 q1[0], r0
; CHECK-NEXT: vmov.u8 r0, q0[9]
; CHECK-NEXT: vmov.16 q1[1], r0
; CHECK-NEXT: vmov.u8 r0, q0[10]
; CHECK-NEXT: vmov.16 q1[2], r0
; CHECK-NEXT: vmov.u8 r0, q0[11]
; CHECK-NEXT: vmov.16 q1[3], r0
; CHECK-NEXT: vmov.u8 r0, q0[12]
; CHECK-NEXT: vmov.16 q1[4], r0
; CHECK-NEXT: vmov.u8 r0, q0[13]
; CHECK-NEXT: vmov.16 q1[5], r0
; CHECK-NEXT: vmov.u8 r0, q0[14]
; CHECK-NEXT: vmov.16 q1[6], r0
; CHECK-NEXT: vmov.u8 r0, q0[15]
; CHECK-NEXT: vmov.16 q1[7], r0
; CHECK-NEXT: vmullb.u8 q0, q1, q3
; CHECK-NEXT: vshr.u16 q0, q0, #8
; CHECK-NEXT: vmov.u16 r0, q0[0]
; CHECK-NEXT: vmov.8 q2[8], r0
; CHECK-NEXT: vmov.u16 r0, q0[1]
; CHECK-NEXT: vmov.8 q2[9], r0
; CHECK-NEXT: vmov.u16 r0, q0[2]
; CHECK-NEXT: vmov.8 q2[10], r0
; CHECK-NEXT: vmov.u16 r0, q0[3]
; CHECK-NEXT: vmov.8 q2[11], r0
; CHECK-NEXT: vmov.u16 r0, q0[4]
; CHECK-NEXT: vmov.8 q2[12], r0
; CHECK-NEXT: vmov.u16 r0, q0[5]
; CHECK-NEXT: vmov.8 q2[13], r0
; CHECK-NEXT: vmov.u16 r0, q0[6]
; CHECK-NEXT: vmov.8 q2[14], r0
; CHECK-NEXT: vmov.u16 r0, q0[7]
; CHECK-NEXT: vmov.8 q2[15], r0
; CHECK-NEXT: vmov q0, q2
; CHECK-NEXT: bx lr ; CHECK-NEXT: bx lr
entry: entry:
%s0s = zext <16 x i8> %s0 to <16 x i16> %s0s = zext <16 x i8> %s0 to <16 x i16>
%s1s = zext <16 x i8> %s1 to <16 x i16> %s1s = zext <16 x i8> %s1 to <16 x i16>
%m = mul <16 x i16> %s0s, %s1s %m = mul <16 x i16> %s0s, %s1s
%s = lshr <16 x i16> %m, <i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8> %s = lshr <16 x i16> %m, <i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8, i16 8>
%s2 = trunc <16 x i16> %s to <16 x i8> %s2 = trunc <16 x i16> %s to <16 x i8>
ret <16 x i8> %s2 ret <16 x i8> %s2
} }