This is an archive of the discontinued LLVM Phabricator instance.

Differential D79652

[X86] Remove the v16i8->v16i16 path for MULHS with AVX2.
ClosedPublic

Authored by craig.topper on May 8 2020, 2:28 PM.

Details

Reviewers
RKSimon
spatel
andreadb
Commits
rG01636c1eeace: [X86] Remove the v16i8->v16i16 path for MULHS with AVX2.
Summary

We have a couple main strategies for legalizing MULH.

-If the vXi16 type is legal, extend to do the full i16 multiply
and then shift and truncate the results.
-Use unpcks to split each 128 bit lane into high and low halves.a

For signed we have an extra case to split a v32i8 to v16i8 and then
use the extending to v16i16 strategy.

This patch proposes to use the unpck strategy instead. Which is
what we already do for unsigned.

This seems to be 1 instruction shorter when the RHS is constant
like the idiv case. It's 1 instruction longer for the smulo case.
But we're trading cross lane shuffles for inlane shuffles and a
shift.

I'm not sure this is the right answer, so I wanted to put it out
there for discussion.

Diff Detail

Event Timeline

craig.topper created this revision.May 8 2020, 2:28 PM
Herald added a project: Restricted Project. · View Herald TranscriptMay 8 2020, 2:28 PM
Herald added a subscriber: hiraditya. · View Herald Transcript
Harbormaster completed remote builds in B56200: Diff 262959.May 8 2020, 3:03 PM
RKSimon added a comment.May 9 2020, 3:34 AM

llvm-mca suggests this is beneficial (znver1/skx) or neutral (haswell): https://godbolt.org/z/Pi3Mqb - and its closer to what we do for regular vXi8 multiplies.

llvm/lib/Target/X86/X86ISelLowering.cpp
26563–26564

Update the comment

craig.topper updated this revision to Diff 263023.May 9 2020, 9:58 AM

Update comment

Harbormaster completed remote builds in B56238: Diff 263023.May 9 2020, 11:40 AM
craig.topper updated this revision to Diff 263038.May 9 2020, 2:00 PM

Rebase after the punpck false dependency fix

Harbormaster failed remote builds in B56248: Diff 263038!May 9 2020, 3:57 PM
RKSimon accepted this revision.May 12 2020, 2:44 AM

LGTM with one minor comment

llvm/test/CodeGen/X86/vec_smulo.ll
2842

How come this can't be a broadcast of a smaller constant?

This revision is now accepted and ready to land.May 12 2020, 2:44 AM
craig.topper marked an inline comment as done.May 12 2020, 10:38 AM
craig.topper added inline comments.
llvm/test/CodeGen/X86/vec_smulo.ll
2842

We ended up here which doesn't know to broadcast i16 for some reason.

// On Sandybridge (no AVX2), it is still better to load a constant vector
// from the constant pool and not to broadcast it from a scalar.
// But override that restriction when optimizing for size.
// TODO: Check if splatting is recommended for other AVX-capable CPUs.
if (ConstSplatVal && (Subtarget.hasAVX2() || OptForSize)) {
  EVT CVT = Ld.getValueType();
  assert(!CVT.isVector() && "Must not broadcast a vector type");

  // Splat f32, i32, v4f64, v4i64 in all cases with AVX2.
  // For size optimization, also splat v2f64 and v2i64, and for size opt
  // with AVX2, also splat i8 and i16.
  // With pattern matching, the VBROADCAST node may become a VMOVDDUP.
  if (ScalarSize == 32 || (IsGE256 && ScalarSize == 64) ||
      (OptForSize && (ScalarSize == 64 || Subtarget.hasAVX2()))) {
Closed by commit rG01636c1eeace: [X86] Remove the v16i8->v16i16 path for MULHS with AVX2. (authored by craig.topper). · Explain WhyMay 12 2020, 10:44 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathSize
llvm/
lib/
Target/
X86/
38 lines
test/
CodeGen/
X86/
147 lines
32 lines
108 lines

Diff 263473

llvm/lib/Target/X86/X86ISelLowering.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 26,554 Lines • ▼ Show 20 Lines if ((VT == MVT::v16i8 && Subtarget.hasInt256()) ||
(VT == MVT::v32i8 && Subtarget.canExtendTo512BW())) { (VT == MVT::v32i8 && Subtarget.canExtendTo512BW())) {
MVT ExVT = MVT::getVectorVT(MVT::i16, NumElts); MVT ExVT = MVT::getVectorVT(MVT::i16, NumElts);
SDValue ExA = DAG.getNode(ExAVX, dl, ExVT, A); SDValue ExA = DAG.getNode(ExAVX, dl, ExVT, A);
SDValue ExB = DAG.getNode(ExAVX, dl, ExVT, B); SDValue ExB = DAG.getNode(ExAVX, dl, ExVT, B);
SDValue Mul = DAG.getNode(ISD::MUL, dl, ExVT, ExA, ExB); SDValue Mul = DAG.getNode(ISD::MUL, dl, ExVT, ExA, ExB);
Mul = getTargetVShiftByConstNode(X86ISD::VSRLI, dl, ExVT, Mul, 8, DAG); Mul = getTargetVShiftByConstNode(X86ISD::VSRLI, dl, ExVT, Mul, 8, DAG);
return DAG.getNode(ISD::TRUNCATE, dl, VT, Mul); return DAG.getNode(ISD::TRUNCATE, dl, VT, Mul);
} }
// For signed 512-bit vectors, split into 256-bit vectors to allow the // For vXi8 we will unpack the low and high half of each 128 bit lane to widen
RKSimonUnsubmitted
Not Done
ReplyInline Actions

Update the comment

RKSimon: Update the comment
// sign-extension to occur. // to a vXi16 type. Do the multiplies, shift the results and pack the half
if (VT == MVT::v64i8 && IsSigned) // lane results back together.
return splitVectorIntBinary(Op, DAG);
// Signed AVX2 implementation - extend xmm subvectors to ymm.
if (VT == MVT::v32i8 && IsSigned) {
MVT ExVT = MVT::v16i16;
SDValue ALo = extract128BitVector(A, 0, DAG, dl);
SDValue BLo = extract128BitVector(B, 0, DAG, dl);
SDValue AHi = extract128BitVector(A, NumElts / 2, DAG, dl);
SDValue BHi = extract128BitVector(B, NumElts / 2, DAG, dl);
ALo = DAG.getNode(ExAVX, dl, ExVT, ALo);
BLo = DAG.getNode(ExAVX, dl, ExVT, BLo);
AHi = DAG.getNode(ExAVX, dl, ExVT, AHi);
BHi = DAG.getNode(ExAVX, dl, ExVT, BHi);
SDValue Lo = DAG.getNode(ISD::MUL, dl, ExVT, ALo, BLo);
SDValue Hi = DAG.getNode(ISD::MUL, dl, ExVT, AHi, BHi);
Lo = getTargetVShiftByConstNode(X86ISD::VSRLI, dl, ExVT, Lo, 8, DAG);
Hi = getTargetVShiftByConstNode(X86ISD::VSRLI, dl, ExVT, Hi, 8, DAG);
// Bitcast back to VT and then pack all the even elements from Lo and Hi.
// Shuffle lowering should turn this into PACKUS+PERMQ
Lo = DAG.getBitcast(VT, Lo);
Hi = DAG.getBitcast(VT, Hi);
return DAG.getVectorShuffle(VT, dl, Lo, Hi,
{ 0, 2, 4, 6, 8, 10, 12, 14,
16, 18, 20, 22, 24, 26, 28, 30,
32, 34, 36, 38, 40, 42, 44, 46,
48, 50, 52, 54, 56, 58, 60, 62});
}
// For signed v16i8 and all unsigned vXi8 we will unpack the low and high
// half of each 128 bit lane to widen to a vXi16 type. Do the multiplies,
// shift the results and pack the half lane results back together.
MVT ExVT = MVT::getVectorVT(MVT::i16, NumElts / 2); MVT ExVT = MVT::getVectorVT(MVT::i16, NumElts / 2);
static const int PSHUFDMask[] = { 8, 9, 10, 11, 12, 13, 14, 15, static const int PSHUFDMask[] = { 8, 9, 10, 11, 12, 13, 14, 15,
-1, -1, -1, -1, -1, -1, -1, -1}; -1, -1, -1, -1, -1, -1, -1, -1};
// Extract the lo parts and zero/sign extend to i16. // Extract the lo parts and zero/sign extend to i16.
// Only use SSE4.1 instructions for signed v16i8 where using unpack requires // Only use SSE4.1 instructions for signed v16i8 where using unpack requires
▲ Show 20 Lines • Show All 22,321 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/vec_smulo.ll

Show First 20 Lines • Show All 1,904 Lines • ▼ Show 20 Lines
; AVX2-NEXT: vpand %ymm3, %ymm2, %ymm2 ; AVX2-NEXT: vpand %ymm3, %ymm2, %ymm2
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm5 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm5 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpmullw %ymm4, %ymm5, %ymm4 ; AVX2-NEXT: vpmullw %ymm4, %ymm5, %ymm4
; AVX2-NEXT: vpand %ymm3, %ymm4, %ymm3 ; AVX2-NEXT: vpand %ymm3, %ymm4, %ymm3
; AVX2-NEXT: vpackuswb %ymm2, %ymm3, %ymm4 ; AVX2-NEXT: vpackuswb %ymm2, %ymm3, %ymm4
; AVX2-NEXT: vpxor %xmm2, %xmm2, %xmm2 ; AVX2-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX2-NEXT: vpcmpgtb %ymm4, %ymm2, %ymm2 ; AVX2-NEXT: vpcmpgtb %ymm4, %ymm2, %ymm2
; AVX2-NEXT: vextracti128 $1, %ymm1, %xmm3 ; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpmovsxbw %xmm3, %ymm3 ; AVX2-NEXT: vpsraw $8, %ymm3, %ymm3
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm5 ; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm5 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpmovsxbw %xmm5, %ymm5 ; AVX2-NEXT: vpsraw $8, %ymm5, %ymm5
; AVX2-NEXT: vpmullw %ymm3, %ymm5, %ymm3 ; AVX2-NEXT: vpmullw %ymm3, %ymm5, %ymm3
; AVX2-NEXT: vpsrlw $8, %ymm3, %ymm3 ; AVX2-NEXT: vpsrlw $8, %ymm3, %ymm3
; AVX2-NEXT: vpmovsxbw %xmm1, %ymm1 ; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpmovsxbw %xmm0, %ymm0 ; AVX2-NEXT: vpsraw $8, %ymm1, %ymm1
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpsraw $8, %ymm0, %ymm0
; AVX2-NEXT: vpmullw %ymm1, %ymm0, %ymm0 ; AVX2-NEXT: vpmullw %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0 ; AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX2-NEXT: vpackuswb %ymm3, %ymm0, %ymm0 ; AVX2-NEXT: vpackuswb %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpermq {{.*#+}} ymm0 = ymm0[0,2,1,3]
; AVX2-NEXT: vpcmpeqb %ymm2, %ymm0, %ymm0 ; AVX2-NEXT: vpcmpeqb %ymm2, %ymm0, %ymm0
; AVX2-NEXT: vpcmpeqd %ymm1, %ymm1, %ymm1 ; AVX2-NEXT: vpcmpeqd %ymm1, %ymm1, %ymm1
; AVX2-NEXT: vpxor %ymm1, %ymm0, %ymm1 ; AVX2-NEXT: vpxor %ymm1, %ymm0, %ymm1
; AVX2-NEXT: vpmovsxbd %xmm1, %ymm0 ; AVX2-NEXT: vpmovsxbd %xmm1, %ymm0
; AVX2-NEXT: vextracti128 $1, %ymm1, %xmm3 ; AVX2-NEXT: vextracti128 $1, %ymm1, %xmm3
; AVX2-NEXT: vpmovsxbd %xmm3, %ymm2 ; AVX2-NEXT: vpmovsxbd %xmm3, %ymm2
; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[2,3,0,1] ; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[2,3,0,1]
; AVX2-NEXT: vpmovsxbd %xmm1, %ymm1 ; AVX2-NEXT: vpmovsxbd %xmm1, %ymm1
▲ Show 20 Lines • Show All 811 Lines • ▼ Show 20 Lines
; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm5 = ymm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm5 = ymm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpmullw %ymm4, %ymm5, %ymm4 ; AVX2-NEXT: vpmullw %ymm4, %ymm5, %ymm4
; AVX2-NEXT: vmovdqa {{.*#+}} ymm5 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255] ; AVX2-NEXT: vmovdqa {{.*#+}} ymm5 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
; AVX2-NEXT: vpand %ymm5, %ymm4, %ymm4 ; AVX2-NEXT: vpand %ymm5, %ymm4, %ymm4
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm6 = ymm3[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm6 = ymm3[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm7 = ymm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm7 = ymm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpmullw %ymm6, %ymm7, %ymm6 ; AVX2-NEXT: vpmullw %ymm6, %ymm7, %ymm6
; AVX2-NEXT: vpand %ymm5, %ymm6, %ymm6 ; AVX2-NEXT: vpand %ymm5, %ymm6, %ymm6
; AVX2-NEXT: vpackuswb %ymm4, %ymm6, %ymm9 ; AVX2-NEXT: vpackuswb %ymm4, %ymm6, %ymm10
; AVX2-NEXT: vpxor %xmm8, %xmm8, %xmm8 ; AVX2-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX2-NEXT: vpcmpgtb %ymm9, %ymm8, %ymm7 ; AVX2-NEXT: vpcmpgtb %ymm10, %ymm6, %ymm7
; AVX2-NEXT: vextracti128 $1, %ymm3, %xmm6 ; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm8 = ymm3[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpmovsxbw %xmm6, %ymm6 ; AVX2-NEXT: vpsraw $8, %ymm8, %ymm8
; AVX2-NEXT: vextracti128 $1, %ymm1, %xmm4 ; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm9 = ymm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpmovsxbw %xmm4, %ymm4 ; AVX2-NEXT: vpsraw $8, %ymm9, %ymm9
; AVX2-NEXT: vpmullw %ymm6, %ymm4, %ymm4 ; AVX2-NEXT: vpmullw %ymm8, %ymm9, %ymm8
; AVX2-NEXT: vpsrlw $8, %ymm4, %ymm4 ; AVX2-NEXT: vpsrlw $8, %ymm8, %ymm8
; AVX2-NEXT: vpmovsxbw %xmm3, %ymm3 ; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm3 = ymm3[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpmovsxbw %xmm1, %ymm1 ; AVX2-NEXT: vpsraw $8, %ymm3, %ymm3
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpsraw $8, %ymm1, %ymm1
; AVX2-NEXT: vpmullw %ymm3, %ymm1, %ymm1 ; AVX2-NEXT: vpmullw %ymm3, %ymm1, %ymm1
; AVX2-NEXT: vpsrlw $8, %ymm1, %ymm1 ; AVX2-NEXT: vpsrlw $8, %ymm1, %ymm1
; AVX2-NEXT: vpackuswb %ymm4, %ymm1, %ymm1 ; AVX2-NEXT: vpackuswb %ymm8, %ymm1, %ymm1
; AVX2-NEXT: vpermq {{.*#+}} ymm1 = ymm1[0,2,1,3]
; AVX2-NEXT: vpcmpeqb %ymm7, %ymm1, %ymm1 ; AVX2-NEXT: vpcmpeqb %ymm7, %ymm1, %ymm1
; AVX2-NEXT: vpcmpeqd %ymm4, %ymm4, %ymm4 ; AVX2-NEXT: vpcmpeqd %ymm7, %ymm7, %ymm7
; AVX2-NEXT: vpxor %ymm4, %ymm1, %ymm1 ; AVX2-NEXT: vpxor %ymm7, %ymm1, %ymm1
; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm6 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm8 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpmullw %ymm3, %ymm6, %ymm3 ; AVX2-NEXT: vpmullw %ymm3, %ymm8, %ymm3
; AVX2-NEXT: vpand %ymm5, %ymm3, %ymm3 ; AVX2-NEXT: vpand %ymm5, %ymm3, %ymm3
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm6 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm8 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm7 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm9 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpmullw %ymm6, %ymm7, %ymm6 ; AVX2-NEXT: vpmullw %ymm8, %ymm9, %ymm8
; AVX2-NEXT: vpand %ymm5, %ymm6, %ymm5 ; AVX2-NEXT: vpand %ymm5, %ymm8, %ymm5
; AVX2-NEXT: vpackuswb %ymm3, %ymm5, %ymm3 ; AVX2-NEXT: vpackuswb %ymm3, %ymm5, %ymm3
; AVX2-NEXT: vpcmpgtb %ymm3, %ymm8, %ymm5 ; AVX2-NEXT: vpcmpgtb %ymm3, %ymm6, %ymm5
; AVX2-NEXT: vextracti128 $1, %ymm2, %xmm6 ; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm6 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpmovsxbw %xmm6, %ymm6 ; AVX2-NEXT: vpsraw $8, %ymm6, %ymm6
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm7 ; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm8 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpmovsxbw %xmm7, %ymm7 ; AVX2-NEXT: vpsraw $8, %ymm8, %ymm8
; AVX2-NEXT: vpmullw %ymm6, %ymm7, %ymm6 ; AVX2-NEXT: vpmullw %ymm6, %ymm8, %ymm6
; AVX2-NEXT: vpsrlw $8, %ymm6, %ymm6 ; AVX2-NEXT: vpsrlw $8, %ymm6, %ymm6
; AVX2-NEXT: vpmovsxbw %xmm2, %ymm2 ; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm2 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpmovsxbw %xmm0, %ymm0 ; AVX2-NEXT: vpsraw $8, %ymm2, %ymm2
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpsraw $8, %ymm0, %ymm0
; AVX2-NEXT: vpmullw %ymm2, %ymm0, %ymm0 ; AVX2-NEXT: vpmullw %ymm2, %ymm0, %ymm0
; AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0 ; AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX2-NEXT: vpackuswb %ymm6, %ymm0, %ymm0 ; AVX2-NEXT: vpackuswb %ymm6, %ymm0, %ymm0
; AVX2-NEXT: vpermq {{.*#+}} ymm0 = ymm0[0,2,1,3]
; AVX2-NEXT: vpcmpeqb %ymm5, %ymm0, %ymm0 ; AVX2-NEXT: vpcmpeqb %ymm5, %ymm0, %ymm0
; AVX2-NEXT: vpxor %ymm4, %ymm0, %ymm0 ; AVX2-NEXT: vpxor %ymm7, %ymm0, %ymm0
; AVX2-NEXT: vpshufd {{.*#+}} xmm2 = xmm0[2,3,0,1] ; AVX2-NEXT: vpshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
; AVX2-NEXT: vpmovsxbd %xmm2, %ymm8 ; AVX2-NEXT: vpmovsxbd %xmm2, %ymm8
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm4 ; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm5
; AVX2-NEXT: vpshufd {{.*#+}} xmm5 = xmm4[2,3,0,1] ; AVX2-NEXT: vpshufd {{.*#+}} xmm6 = xmm5[2,3,0,1]
; AVX2-NEXT: vpmovsxbd %xmm5, %ymm5
; AVX2-NEXT: vpshufd {{.*#+}} xmm6 = xmm1[2,3,0,1]
; AVX2-NEXT: vpmovsxbd %xmm6, %ymm6 ; AVX2-NEXT: vpmovsxbd %xmm6, %ymm6
; AVX2-NEXT: vextracti128 $1, %ymm1, %xmm7 ; AVX2-NEXT: vpshufd {{.*#+}} xmm7 = xmm1[2,3,0,1]
; AVX2-NEXT: vpshufd {{.*#+}} xmm2 = xmm7[2,3,0,1] ; AVX2-NEXT: vpmovsxbd %xmm7, %ymm7
; AVX2-NEXT: vextracti128 $1, %ymm1, %xmm4
; AVX2-NEXT: vpshufd {{.*#+}} xmm2 = xmm4[2,3,0,1]
; AVX2-NEXT: vpmovsxbd %xmm2, %ymm2 ; AVX2-NEXT: vpmovsxbd %xmm2, %ymm2
; AVX2-NEXT: vpmovsxbd %xmm0, %ymm0 ; AVX2-NEXT: vpmovsxbd %xmm0, %ymm0
; AVX2-NEXT: vpmovsxbd %xmm4, %ymm4 ; AVX2-NEXT: vpmovsxbd %xmm5, %ymm5
; AVX2-NEXT: vpmovsxbd %xmm1, %ymm1 ; AVX2-NEXT: vpmovsxbd %xmm1, %ymm1
; AVX2-NEXT: vpmovsxbd %xmm7, %ymm7 ; AVX2-NEXT: vpmovsxbd %xmm4, %ymm4
; AVX2-NEXT: vmovdqa %ymm9, 32(%rsi) ; AVX2-NEXT: vmovdqa %ymm10, 32(%rsi)
; AVX2-NEXT: vmovdqa %ymm3, (%rsi) ; AVX2-NEXT: vmovdqa %ymm3, (%rsi)
; AVX2-NEXT: vmovdqa %ymm7, 192(%rdi) ; AVX2-NEXT: vmovdqa %ymm4, 192(%rdi)
; AVX2-NEXT: vmovdqa %ymm1, 128(%rdi) ; AVX2-NEXT: vmovdqa %ymm1, 128(%rdi)
; AVX2-NEXT: vmovdqa %ymm4, 64(%rdi) ; AVX2-NEXT: vmovdqa %ymm5, 64(%rdi)
; AVX2-NEXT: vmovdqa %ymm0, (%rdi) ; AVX2-NEXT: vmovdqa %ymm0, (%rdi)
; AVX2-NEXT: vmovdqa %ymm2, 224(%rdi) ; AVX2-NEXT: vmovdqa %ymm2, 224(%rdi)
; AVX2-NEXT: vmovdqa %ymm6, 160(%rdi) ; AVX2-NEXT: vmovdqa %ymm7, 160(%rdi)
; AVX2-NEXT: vmovdqa %ymm5, 96(%rdi) ; AVX2-NEXT: vmovdqa %ymm6, 96(%rdi)
; AVX2-NEXT: vmovdqa %ymm8, 32(%rdi) ; AVX2-NEXT: vmovdqa %ymm8, 32(%rdi)
; AVX2-NEXT: vzeroupper ; AVX2-NEXT: vzeroupper
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: smulo_v64i8: ; AVX512-LABEL: smulo_v64i8:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpunpckhbw {{.*#+}} zmm2 = zmm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63] ; AVX512-NEXT: vpunpckhbw {{.*#+}} zmm2 = zmm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512-NEXT: vpsraw $8, %zmm2, %zmm2
; AVX512-NEXT: vpunpckhbw {{.*#+}} zmm3 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63] ; AVX512-NEXT: vpunpckhbw {{.*#+}} zmm3 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512-NEXT: vpsraw $8, %zmm3, %zmm3
; AVX512-NEXT: vpmullw %zmm2, %zmm3, %zmm2 ; AVX512-NEXT: vpmullw %zmm2, %zmm3, %zmm2
; AVX512-NEXT: vmovdqa64 {{.*#+}} zmm3 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255] ; AVX512-NEXT: vpsrlw $8, %zmm2, %zmm2
; AVX512-NEXT: vpandq %zmm3, %zmm2, %zmm2 ; AVX512-NEXT: vpunpcklbw {{.*#+}} zmm3 = zmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512-NEXT: vpunpcklbw {{.*#+}} zmm4 = zmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55] ; AVX512-NEXT: vpsraw $8, %zmm3, %zmm3
; AVX512-NEXT: vpunpcklbw {{.*#+}} zmm5 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55] ; AVX512-NEXT: vpunpcklbw {{.*#+}} zmm4 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512-NEXT: vpmullw %zmm4, %zmm5, %zmm4 ; AVX512-NEXT: vpsraw $8, %zmm4, %zmm4
; AVX512-NEXT: vpandq %zmm3, %zmm4, %zmm3 ; AVX512-NEXT: vpmullw %zmm3, %zmm4, %zmm3
; AVX512-NEXT: vpackuswb %zmm2, %zmm3, %zmm4
; AVX512-NEXT: vpmovb2m %zmm4, %k0
; AVX512-NEXT: vpmovm2b %k0, %zmm2
; AVX512-NEXT: vpmovsxbw %ymm1, %zmm3
; AVX512-NEXT: vpmovsxbw %ymm0, %zmm5
; AVX512-NEXT: vpmullw %zmm3, %zmm5, %zmm3
; AVX512-NEXT: vpsrlw $8, %zmm3, %zmm3 ; AVX512-NEXT: vpsrlw $8, %zmm3, %zmm3
; AVX512-NEXT: vpmovwb %zmm3, %ymm3 ; AVX512-NEXT: vpackuswb %zmm2, %zmm3, %zmm2
; AVX512-NEXT: vextracti64x4 $1, %zmm1, %ymm1 ; AVX512-NEXT: vpunpckhbw {{.*#+}} zmm3 = zmm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512-NEXT: vpmovsxbw %ymm1, %zmm1 ; AVX512-NEXT: vpunpckhbw {{.*#+}} zmm4 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512-NEXT: vextracti64x4 $1, %zmm0, %ymm0 ; AVX512-NEXT: vpmullw %zmm3, %zmm4, %zmm3
; AVX512-NEXT: vpmovsxbw %ymm0, %zmm0 ; AVX512-NEXT: vmovdqa64 {{.*#+}} zmm4 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
RKSimonUnsubmitted
Not Done
ReplyInline Actions

How come this can't be a broadcast of a smaller constant?

RKSimon: How come this can't be a broadcast of a smaller constant?
craig.topperAuthorUnsubmitted
Done
ReplyInline Actions

We ended up here which doesn't know to broadcast i16 for some reason.

// On Sandybridge (no AVX2), it is still better to load a constant vector
// from the constant pool and not to broadcast it from a scalar.
// But override that restriction when optimizing for size.
// TODO: Check if splatting is recommended for other AVX-capable CPUs.
if (ConstSplatVal && (Subtarget.hasAVX2() || OptForSize)) {
  EVT CVT = Ld.getValueType();
  assert(!CVT.isVector() && "Must not broadcast a vector type");

  // Splat f32, i32, v4f64, v4i64 in all cases with AVX2.
  // For size optimization, also splat v2f64 and v2i64, and for size opt
  // with AVX2, also splat i8 and i16.
  // With pattern matching, the VBROADCAST node may become a VMOVDDUP.
  if (ScalarSize == 32 || (IsGE256 && ScalarSize == 64) ||
      (OptForSize && (ScalarSize == 64 || Subtarget.hasAVX2()))) {
craig.topper: We ended up here which doesn't know to broadcast i16 for some reason. // On Sandybridge (no…
; AVX512-NEXT: vpandq %zmm4, %zmm3, %zmm3
; AVX512-NEXT: vpunpcklbw {{.*#+}} zmm1 = zmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512-NEXT: vpunpcklbw {{.*#+}} zmm0 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512-NEXT: vpmullw %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vpmullw %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vpsrlw $8, %zmm0, %zmm0 ; AVX512-NEXT: vpandq %zmm4, %zmm0, %zmm0
; AVX512-NEXT: vpmovwb %zmm0, %ymm0 ; AVX512-NEXT: vpackuswb %zmm3, %zmm0, %zmm4
; AVX512-NEXT: vinserti64x4 $1, %ymm0, %zmm3, %zmm0 ; AVX512-NEXT: vpmovb2m %zmm4, %k0
; AVX512-NEXT: vpcmpneqb %zmm2, %zmm0, %k1 ; AVX512-NEXT: vpmovm2b %k0, %zmm0
; AVX512-NEXT: vpcmpneqb %zmm0, %zmm2, %k1
; AVX512-NEXT: vpternlogd $255, %zmm0, %zmm0, %zmm0 {%k1} {z} ; AVX512-NEXT: vpternlogd $255, %zmm0, %zmm0, %zmm0 {%k1} {z}
; AVX512-NEXT: kshiftrd $16, %k1, %k2 ; AVX512-NEXT: kshiftrd $16, %k1, %k2
; AVX512-NEXT: vpternlogd $255, %zmm1, %zmm1, %zmm1 {%k2} {z} ; AVX512-NEXT: vpternlogd $255, %zmm1, %zmm1, %zmm1 {%k2} {z}
; AVX512-NEXT: kshiftrq $32, %k1, %k1 ; AVX512-NEXT: kshiftrq $32, %k1, %k1
; AVX512-NEXT: vpternlogd $255, %zmm2, %zmm2, %zmm2 {%k1} {z} ; AVX512-NEXT: vpternlogd $255, %zmm2, %zmm2, %zmm2 {%k1} {z}
; AVX512-NEXT: kshiftrd $16, %k1, %k1 ; AVX512-NEXT: kshiftrd $16, %k1, %k1
; AVX512-NEXT: vpternlogd $255, %zmm3, %zmm3, %zmm3 {%k1} {z} ; AVX512-NEXT: vpternlogd $255, %zmm3, %zmm3, %zmm3 {%k1} {z}
; AVX512-NEXT: vmovdqa64 %zmm4, (%rdi) ; AVX512-NEXT: vmovdqa64 %zmm4, (%rdi)
▲ Show 20 Lines • Show All 1,136 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/vector-idiv-sdiv-256.ll

Show First 20 Lines • Show All 194 Lines • ▼ Show 20 Lines
; AVX1-NEXT: vpxor %xmm6, %xmm0, %xmm0 ; AVX1-NEXT: vpxor %xmm6, %xmm0, %xmm0
; AVX1-NEXT: vpaddb %xmm2, %xmm0, %xmm0 ; AVX1-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpsubb %xmm6, %xmm0, %xmm0 ; AVX1-NEXT: vpsubb %xmm6, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0 ; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2NOBW-LABEL: test_div7_32i8: ; AVX2NOBW-LABEL: test_div7_32i8:
; AVX2NOBW: # %bb.0: ; AVX2NOBW: # %bb.0:
; AVX2NOBW-NEXT: vextracti128 $1, %ymm0, %xmm1 ; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2NOBW-NEXT: vpmovsxbw %xmm1, %ymm1 ; AVX2NOBW-NEXT: vpsraw $8, %ymm1, %ymm1
; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm2 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427] ; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm2 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427]
; AVX2NOBW-NEXT: vpmullw %ymm2, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpmullw %ymm2, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpsrlw $8, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpmovsxbw %xmm0, %ymm3 ; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm3 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2NOBW-NEXT: vpsraw $8, %ymm3, %ymm3
; AVX2NOBW-NEXT: vpmullw %ymm2, %ymm3, %ymm2 ; AVX2NOBW-NEXT: vpmullw %ymm2, %ymm3, %ymm2
; AVX2NOBW-NEXT: vpsrlw $8, %ymm2, %ymm2 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX2NOBW-NEXT: vpackuswb %ymm1, %ymm2, %ymm1 ; AVX2NOBW-NEXT: vpackuswb %ymm1, %ymm2, %ymm1
; AVX2NOBW-NEXT: vpermq {{.*#+}} ymm1 = ymm1[0,2,1,3]
; AVX2NOBW-NEXT: vpaddb %ymm0, %ymm1, %ymm0 ; AVX2NOBW-NEXT: vpaddb %ymm0, %ymm1, %ymm0
; AVX2NOBW-NEXT: vpsrlw $2, %ymm0, %ymm1 ; AVX2NOBW-NEXT: vpsrlw $2, %ymm0, %ymm1
; AVX2NOBW-NEXT: vpand {{.*}}(%rip), %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpand {{.*}}(%rip), %ymm1, %ymm1
; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm2 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32] ; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm2 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32]
; AVX2NOBW-NEXT: vpxor %ymm2, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpxor %ymm2, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpsrlw $7, %ymm0, %ymm0 ; AVX2NOBW-NEXT: vpsrlw $7, %ymm0, %ymm0
; AVX2NOBW-NEXT: vpand {{.*}}(%rip), %ymm0, %ymm0 ; AVX2NOBW-NEXT: vpand {{.*}}(%rip), %ymm0, %ymm0
; AVX2NOBW-NEXT: vpaddb %ymm0, %ymm1, %ymm0 ; AVX2NOBW-NEXT: vpaddb %ymm0, %ymm1, %ymm0
▲ Show 20 Lines • Show All 94 Lines • ▼ Show 20 Lines
; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1
; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm2 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255] ; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm2 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
; AVX2NOBW-NEXT: vpand %ymm2, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpand %ymm2, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm3 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm3 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3
; AVX2NOBW-NEXT: vpand %ymm2, %ymm3, %ymm2 ; AVX2NOBW-NEXT: vpand %ymm2, %ymm3, %ymm2
; AVX2NOBW-NEXT: vpackuswb %ymm1, %ymm2, %ymm1 ; AVX2NOBW-NEXT: vpackuswb %ymm1, %ymm2, %ymm1
; AVX2NOBW-NEXT: vextracti128 $1, %ymm0, %xmm2 ; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2NOBW-NEXT: vpmovsxbw %xmm2, %ymm2 ; AVX2NOBW-NEXT: vpsraw $8, %ymm2, %ymm2
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm2, %ymm2 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm2, %ymm2
; AVX2NOBW-NEXT: vpsrlw $8, %ymm2, %ymm2 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX2NOBW-NEXT: vpmovsxbw %xmm0, %ymm0 ; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2NOBW-NEXT: vpsraw $8, %ymm0, %ymm0
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm0, %ymm0 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm0, %ymm0
; AVX2NOBW-NEXT: vpsrlw $8, %ymm0, %ymm0 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX2NOBW-NEXT: vpackuswb %ymm2, %ymm0, %ymm0 ; AVX2NOBW-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX2NOBW-NEXT: vpermq {{.*#+}} ymm0 = ymm0[0,2,1,3]
; AVX2NOBW-NEXT: vpaddb %ymm1, %ymm0, %ymm0 ; AVX2NOBW-NEXT: vpaddb %ymm1, %ymm0, %ymm0
; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2NOBW-NEXT: vpsraw $8, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpsraw $8, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1
; AVX2NOBW-NEXT: vpsrlw $8, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm2 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm2 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2NOBW-NEXT: vpsraw $8, %ymm2, %ymm2 ; AVX2NOBW-NEXT: vpsraw $8, %ymm2, %ymm2
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm2, %ymm2 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm2, %ymm2
▲ Show 20 Lines • Show All 275 Lines • ▼ Show 20 Lines
; AVX1-NEXT: vpand %xmm5, %xmm3, %xmm3 ; AVX1-NEXT: vpand %xmm5, %xmm3, %xmm3
; AVX1-NEXT: vpsubb %xmm3, %xmm2, %xmm2 ; AVX1-NEXT: vpsubb %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vpaddb %xmm2, %xmm0, %xmm0 ; AVX1-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0 ; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2NOBW-LABEL: test_rem7_32i8: ; AVX2NOBW-LABEL: test_rem7_32i8:
; AVX2NOBW: # %bb.0: ; AVX2NOBW: # %bb.0:
; AVX2NOBW-NEXT: vextracti128 $1, %ymm0, %xmm1 ; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2NOBW-NEXT: vpmovsxbw %xmm1, %ymm1 ; AVX2NOBW-NEXT: vpsraw $8, %ymm1, %ymm1
; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm2 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427] ; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm2 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427]
; AVX2NOBW-NEXT: vpmullw %ymm2, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpmullw %ymm2, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpsrlw $8, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpmovsxbw %xmm0, %ymm3 ; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm3 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2NOBW-NEXT: vpsraw $8, %ymm3, %ymm3
; AVX2NOBW-NEXT: vpmullw %ymm2, %ymm3, %ymm2 ; AVX2NOBW-NEXT: vpmullw %ymm2, %ymm3, %ymm2
; AVX2NOBW-NEXT: vpsrlw $8, %ymm2, %ymm2 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX2NOBW-NEXT: vpackuswb %ymm1, %ymm2, %ymm1 ; AVX2NOBW-NEXT: vpackuswb %ymm1, %ymm2, %ymm1
; AVX2NOBW-NEXT: vpermq {{.*#+}} ymm1 = ymm1[0,2,1,3]
; AVX2NOBW-NEXT: vpaddb %ymm0, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpaddb %ymm0, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpsrlw $2, %ymm1, %ymm2 ; AVX2NOBW-NEXT: vpsrlw $2, %ymm1, %ymm2
; AVX2NOBW-NEXT: vpand {{.*}}(%rip), %ymm2, %ymm2 ; AVX2NOBW-NEXT: vpand {{.*}}(%rip), %ymm2, %ymm2
; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm3 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32] ; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm3 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32]
; AVX2NOBW-NEXT: vpxor %ymm3, %ymm2, %ymm2 ; AVX2NOBW-NEXT: vpxor %ymm3, %ymm2, %ymm2
; AVX2NOBW-NEXT: vpsrlw $7, %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpsrlw $7, %ymm1, %ymm1
; AVX2NOBW-NEXT: vpand {{.*}}(%rip), %ymm1, %ymm1 ; AVX2NOBW-NEXT: vpand {{.*}}(%rip), %ymm1, %ymm1
; AVX2NOBW-NEXT: vpaddb %ymm1, %ymm2, %ymm1 ; AVX2NOBW-NEXT: vpaddb %ymm1, %ymm2, %ymm1
▲ Show 20 Lines • Show All 118 Lines • ▼ Show 20 Lines
; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm2 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm2
; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm1 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255] ; AVX2NOBW-NEXT: vmovdqa {{.*#+}} ymm1 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
; AVX2NOBW-NEXT: vpand %ymm1, %ymm2, %ymm2 ; AVX2NOBW-NEXT: vpand %ymm1, %ymm2, %ymm2
; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm3 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm3 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3
; AVX2NOBW-NEXT: vpand %ymm1, %ymm3, %ymm3 ; AVX2NOBW-NEXT: vpand %ymm1, %ymm3, %ymm3
; AVX2NOBW-NEXT: vpackuswb %ymm2, %ymm3, %ymm2 ; AVX2NOBW-NEXT: vpackuswb %ymm2, %ymm3, %ymm2
; AVX2NOBW-NEXT: vextracti128 $1, %ymm0, %xmm3 ; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2NOBW-NEXT: vpmovsxbw %xmm3, %ymm3 ; AVX2NOBW-NEXT: vpsraw $8, %ymm3, %ymm3
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3
; AVX2NOBW-NEXT: vpsrlw $8, %ymm3, %ymm3 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm3, %ymm3
; AVX2NOBW-NEXT: vpmovsxbw %xmm0, %ymm4 ; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2NOBW-NEXT: vpsraw $8, %ymm4, %ymm4
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4
; AVX2NOBW-NEXT: vpsrlw $8, %ymm4, %ymm4 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm4, %ymm4
; AVX2NOBW-NEXT: vpackuswb %ymm3, %ymm4, %ymm3 ; AVX2NOBW-NEXT: vpackuswb %ymm3, %ymm4, %ymm3
; AVX2NOBW-NEXT: vpermq {{.*#+}} ymm3 = ymm3[0,2,1,3]
; AVX2NOBW-NEXT: vpaddb %ymm2, %ymm3, %ymm2 ; AVX2NOBW-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX2NOBW-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2NOBW-NEXT: vpsraw $8, %ymm3, %ymm3 ; AVX2NOBW-NEXT: vpsraw $8, %ymm3, %ymm3
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3
; AVX2NOBW-NEXT: vpsrlw $8, %ymm3, %ymm3 ; AVX2NOBW-NEXT: vpsrlw $8, %ymm3, %ymm3
; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX2NOBW-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2NOBW-NEXT: vpsraw $8, %ymm4, %ymm4 ; AVX2NOBW-NEXT: vpsraw $8, %ymm4, %ymm4
; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4 ; AVX2NOBW-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4
Show All 39 Lines

llvm/test/CodeGen/X86/vector-idiv-sdiv-512.ll

Show First 20 Lines • Show All 124 Lines • ▼ Show 20 Lines; AVX512BW-NEXT: retq
%res = sdiv <32 x i16> %a, <i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7> %res = sdiv <32 x i16> %a, <i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7>
ret <32 x i16> %res ret <32 x i16> %res
} }
define <64 x i8> @test_div7_64i8(<64 x i8> %a) nounwind { define <64 x i8> @test_div7_64i8(<64 x i8> %a) nounwind {
; AVX512F-LABEL: test_div7_64i8: ; AVX512F-LABEL: test_div7_64i8:
; AVX512F: # %bb.0: ; AVX512F: # %bb.0:
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512F-NEXT: vextracti128 $1, %ymm1, %xmm2 ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmovsxbw %xmm2, %ymm2 ; AVX512F-NEXT: vpsraw $8, %ymm2, %ymm2
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm3 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm3 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427]
; AVX512F-NEXT: vpmullw %ymm3, %ymm2, %ymm2 ; AVX512F-NEXT: vpmullw %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2 ; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512F-NEXT: vpmovsxbw %xmm1, %ymm4 ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpmullw %ymm3, %ymm4, %ymm4 ; AVX512F-NEXT: vpmullw %ymm3, %ymm4, %ymm4
; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4 ; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpackuswb %ymm2, %ymm4, %ymm2 ; AVX512F-NEXT: vpackuswb %ymm2, %ymm4, %ymm2
; AVX512F-NEXT: vpermq {{.*#+}} ymm2 = ymm2[0,2,1,3]
; AVX512F-NEXT: vpaddb %ymm1, %ymm2, %ymm1 ; AVX512F-NEXT: vpaddb %ymm1, %ymm2, %ymm1
; AVX512F-NEXT: vpsrlw $7, %ymm1, %ymm2 ; AVX512F-NEXT: vpsrlw $7, %ymm1, %ymm2
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm4 = [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm4 = [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]
; AVX512F-NEXT: vpand %ymm4, %ymm2, %ymm2 ; AVX512F-NEXT: vpand %ymm4, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $2, %ymm1, %ymm1 ; AVX512F-NEXT: vpsrlw $2, %ymm1, %ymm1
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm5 = [63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm5 = [63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63]
; AVX512F-NEXT: vpand %ymm5, %ymm1, %ymm1 ; AVX512F-NEXT: vpand %ymm5, %ymm1, %ymm1
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm6 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm6 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32]
; AVX512F-NEXT: vpxor %ymm6, %ymm1, %ymm1 ; AVX512F-NEXT: vpxor %ymm6, %ymm1, %ymm1
; AVX512F-NEXT: vpaddb %ymm2, %ymm1, %ymm1 ; AVX512F-NEXT: vpaddb %ymm2, %ymm1, %ymm1
; AVX512F-NEXT: vpsubb %ymm6, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubb %ymm6, %ymm1, %ymm1
; AVX512F-NEXT: vextracti128 $1, %ymm0, %xmm2 ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmovsxbw %xmm2, %ymm2 ; AVX512F-NEXT: vpsraw $8, %ymm2, %ymm2
; AVX512F-NEXT: vpmullw %ymm3, %ymm2, %ymm2 ; AVX512F-NEXT: vpmullw %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2 ; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512F-NEXT: vpmovsxbw %xmm0, %ymm7 ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm7 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm7, %ymm7
; AVX512F-NEXT: vpmullw %ymm3, %ymm7, %ymm3 ; AVX512F-NEXT: vpmullw %ymm3, %ymm7, %ymm3
; AVX512F-NEXT: vpsrlw $8, %ymm3, %ymm3 ; AVX512F-NEXT: vpsrlw $8, %ymm3, %ymm3
; AVX512F-NEXT: vpackuswb %ymm2, %ymm3, %ymm2 ; AVX512F-NEXT: vpackuswb %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vpermq {{.*#+}} ymm2 = ymm2[0,2,1,3]
; AVX512F-NEXT: vpaddb %ymm0, %ymm2, %ymm0 ; AVX512F-NEXT: vpaddb %ymm0, %ymm2, %ymm0
; AVX512F-NEXT: vpsrlw $7, %ymm0, %ymm2 ; AVX512F-NEXT: vpsrlw $7, %ymm0, %ymm2
; AVX512F-NEXT: vpand %ymm4, %ymm2, %ymm2 ; AVX512F-NEXT: vpand %ymm4, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $2, %ymm0, %ymm0 ; AVX512F-NEXT: vpsrlw $2, %ymm0, %ymm0
; AVX512F-NEXT: vpand %ymm5, %ymm0, %ymm0 ; AVX512F-NEXT: vpand %ymm5, %ymm0, %ymm0
; AVX512F-NEXT: vpxor %ymm6, %ymm0, %ymm0 ; AVX512F-NEXT: vpxor %ymm6, %ymm0, %ymm0
; AVX512F-NEXT: vpaddb %ymm2, %ymm0, %ymm0 ; AVX512F-NEXT: vpaddb %ymm2, %ymm0, %ymm0
; AVX512F-NEXT: vpsubb %ymm6, %ymm0, %ymm0 ; AVX512F-NEXT: vpsubb %ymm6, %ymm0, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512BW-LABEL: test_div7_64i8: ; AVX512BW-LABEL: test_div7_64i8:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vpmovsxbw %ymm0, %zmm1 ; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm1 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512BW-NEXT: vpsraw $8, %zmm1, %zmm1
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427] ; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427]
; AVX512BW-NEXT: vpmullw %zmm2, %zmm1, %zmm1 ; AVX512BW-NEXT: vpmullw %zmm2, %zmm1, %zmm1
; AVX512BW-NEXT: vpsrlw $8, %zmm1, %zmm1 ; AVX512BW-NEXT: vpsrlw $8, %zmm1, %zmm1
; AVX512BW-NEXT: vpmovwb %zmm1, %ymm1 ; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm3 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512BW-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512BW-NEXT: vpsraw $8, %zmm3, %zmm3
; AVX512BW-NEXT: vpmovsxbw %ymm3, %zmm3
; AVX512BW-NEXT: vpmullw %zmm2, %zmm3, %zmm2 ; AVX512BW-NEXT: vpmullw %zmm2, %zmm3, %zmm2
; AVX512BW-NEXT: vpsrlw $8, %zmm2, %zmm2 ; AVX512BW-NEXT: vpsrlw $8, %zmm2, %zmm2
; AVX512BW-NEXT: vpmovwb %zmm2, %ymm2 ; AVX512BW-NEXT: vpackuswb %zmm1, %zmm2, %zmm1
; AVX512BW-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512BW-NEXT: vpaddb %zmm0, %zmm1, %zmm0 ; AVX512BW-NEXT: vpaddb %zmm0, %zmm1, %zmm0
; AVX512BW-NEXT: vpsrlw $2, %zmm0, %zmm1 ; AVX512BW-NEXT: vpsrlw $2, %zmm0, %zmm1
; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm1, %zmm1 ; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm1, %zmm1
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32] ; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32]
; AVX512BW-NEXT: vpxorq %zmm2, %zmm1, %zmm1 ; AVX512BW-NEXT: vpxorq %zmm2, %zmm1, %zmm1
; AVX512BW-NEXT: vpsrlw $7, %zmm0, %zmm0 ; AVX512BW-NEXT: vpsrlw $7, %zmm0, %zmm0
; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm0, %zmm0 ; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm0, %zmm0
; AVX512BW-NEXT: vpaddb %zmm0, %zmm1, %zmm0 ; AVX512BW-NEXT: vpaddb %zmm0, %zmm1, %zmm0
Show All 14 Lines
; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm3 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm3
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm1 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm1 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
; AVX512F-NEXT: vpand %ymm1, %ymm3, %ymm3 ; AVX512F-NEXT: vpand %ymm1, %ymm3, %ymm3
; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4
; AVX512F-NEXT: vpand %ymm1, %ymm4, %ymm4 ; AVX512F-NEXT: vpand %ymm1, %ymm4, %ymm4
; AVX512F-NEXT: vpackuswb %ymm3, %ymm4, %ymm3 ; AVX512F-NEXT: vpackuswb %ymm3, %ymm4, %ymm3
; AVX512F-NEXT: vextracti128 $1, %ymm2, %xmm4 ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm4 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmovsxbw %xmm4, %ymm4 ; AVX512F-NEXT: vpsraw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4
; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4 ; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpmovsxbw %xmm2, %ymm2 ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm2 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm2, %ymm2
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm2, %ymm2 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2 ; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512F-NEXT: vpackuswb %ymm4, %ymm2, %ymm2 ; AVX512F-NEXT: vpackuswb %ymm4, %ymm2, %ymm2
; AVX512F-NEXT: vpermq {{.*#+}} ymm2 = ymm2[0,2,1,3]
; AVX512F-NEXT: vpaddb %ymm3, %ymm2, %ymm2 ; AVX512F-NEXT: vpaddb %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpsraw $8, %ymm3, %ymm3 ; AVX512F-NEXT: vpsraw $8, %ymm3, %ymm3
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3
; AVX512F-NEXT: vpsrlw $8, %ymm3, %ymm3 ; AVX512F-NEXT: vpsrlw $8, %ymm3, %ymm3
; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm4, %ymm4 ; AVX512F-NEXT: vpsraw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4
; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4 ; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpackuswb %ymm3, %ymm4, %ymm3 ; AVX512F-NEXT: vpackuswb %ymm3, %ymm4, %ymm3
; AVX512F-NEXT: vpsrlw $7, %ymm2, %ymm2 ; AVX512F-NEXT: vpsrlw $7, %ymm2, %ymm2
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm4 = [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm4 = [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]
; AVX512F-NEXT: vpand %ymm4, %ymm2, %ymm2 ; AVX512F-NEXT: vpand %ymm4, %ymm2, %ymm2
; AVX512F-NEXT: vpaddb %ymm2, %ymm3, %ymm2 ; AVX512F-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3
; AVX512F-NEXT: vpand %ymm1, %ymm3, %ymm3 ; AVX512F-NEXT: vpand %ymm1, %ymm3, %ymm3
; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm5 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm5 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5
; AVX512F-NEXT: vpand %ymm1, %ymm5, %ymm1 ; AVX512F-NEXT: vpand %ymm1, %ymm5, %ymm1
; AVX512F-NEXT: vpackuswb %ymm3, %ymm1, %ymm1 ; AVX512F-NEXT: vpackuswb %ymm3, %ymm1, %ymm1
; AVX512F-NEXT: vextracti128 $1, %ymm0, %xmm3 ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmovsxbw %xmm3, %ymm3 ; AVX512F-NEXT: vpsraw $8, %ymm3, %ymm3
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3
; AVX512F-NEXT: vpsrlw $8, %ymm3, %ymm3 ; AVX512F-NEXT: vpsrlw $8, %ymm3, %ymm3
; AVX512F-NEXT: vpmovsxbw %xmm0, %ymm0 ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm0, %ymm0
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm0, %ymm0 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm0, %ymm0
; AVX512F-NEXT: vpsrlw $8, %ymm0, %ymm0 ; AVX512F-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX512F-NEXT: vpackuswb %ymm3, %ymm0, %ymm0 ; AVX512F-NEXT: vpackuswb %ymm3, %ymm0, %ymm0
; AVX512F-NEXT: vpermq {{.*#+}} ymm0 = ymm0[0,2,1,3]
; AVX512F-NEXT: vpaddb %ymm1, %ymm0, %ymm0 ; AVX512F-NEXT: vpaddb %ymm1, %ymm0, %ymm0
; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpsraw $8, %ymm1, %ymm1 ; AVX512F-NEXT: vpsraw $8, %ymm1, %ymm1
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1
; AVX512F-NEXT: vpsrlw $8, %ymm1, %ymm1 ; AVX512F-NEXT: vpsrlw $8, %ymm1, %ymm1
; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm3 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm3 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm3, %ymm3 ; AVX512F-NEXT: vpsraw $8, %ymm3, %ymm3
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm3, %ymm3
Show All 10 Lines
; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm1 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63] ; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm1 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm1, %zmm1 ; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm1, %zmm1
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255] ; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
; AVX512BW-NEXT: vpandq %zmm2, %zmm1, %zmm1 ; AVX512BW-NEXT: vpandq %zmm2, %zmm1, %zmm1
; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm3 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55] ; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm3 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm3, %zmm3 ; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm3, %zmm3
; AVX512BW-NEXT: vpandq %zmm2, %zmm3, %zmm2 ; AVX512BW-NEXT: vpandq %zmm2, %zmm3, %zmm2
; AVX512BW-NEXT: vpackuswb %zmm1, %zmm2, %zmm1 ; AVX512BW-NEXT: vpackuswb %zmm1, %zmm2, %zmm1
; AVX512BW-NEXT: vpmovsxbw %ymm0, %zmm2 ; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm2 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512BW-NEXT: vpsraw $8, %zmm2, %zmm2
; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm2, %zmm2 ; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm2, %zmm2
; AVX512BW-NEXT: vpsrlw $8, %zmm2, %zmm2 ; AVX512BW-NEXT: vpsrlw $8, %zmm2, %zmm2
; AVX512BW-NEXT: vpmovwb %zmm2, %ymm2 ; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm0 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512BW-NEXT: vextracti64x4 $1, %zmm0, %ymm0 ; AVX512BW-NEXT: vpsraw $8, %zmm0, %zmm0
; AVX512BW-NEXT: vpmovsxbw %ymm0, %zmm0
; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm0, %zmm0 ; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlw $8, %zmm0, %zmm0 ; AVX512BW-NEXT: vpsrlw $8, %zmm0, %zmm0
; AVX512BW-NEXT: vpmovwb %zmm0, %ymm0 ; AVX512BW-NEXT: vpackuswb %zmm2, %zmm0, %zmm0
; AVX512BW-NEXT: vinserti64x4 $1, %ymm0, %zmm2, %zmm0
; AVX512BW-NEXT: vpaddb %zmm1, %zmm0, %zmm0 ; AVX512BW-NEXT: vpaddb %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm1 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63] ; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm1 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512BW-NEXT: vpsraw $8, %zmm1, %zmm1 ; AVX512BW-NEXT: vpsraw $8, %zmm1, %zmm1
; AVX512BW-NEXT: vpsllvw {{.*}}(%rip), %zmm1, %zmm1 ; AVX512BW-NEXT: vpsllvw {{.*}}(%rip), %zmm1, %zmm1
; AVX512BW-NEXT: vpsrlw $8, %zmm1, %zmm1 ; AVX512BW-NEXT: vpsrlw $8, %zmm1, %zmm1
; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm2 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55] ; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm2 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512BW-NEXT: vpsraw $8, %zmm2, %zmm2 ; AVX512BW-NEXT: vpsraw $8, %zmm2, %zmm2
; AVX512BW-NEXT: vpsllvw {{.*}}(%rip), %zmm2, %zmm2 ; AVX512BW-NEXT: vpsllvw {{.*}}(%rip), %zmm2, %zmm2
▲ Show 20 Lines • Show All 172 Lines • ▼ Show 20 Lines; AVX512BW-NEXT: retq
%res = srem <32 x i16> %a, <i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7> %res = srem <32 x i16> %a, <i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7>
ret <32 x i16> %res ret <32 x i16> %res
} }
define <64 x i8> @test_rem7_64i8(<64 x i8> %a) nounwind { define <64 x i8> @test_rem7_64i8(<64 x i8> %a) nounwind {
; AVX512F-LABEL: test_rem7_64i8: ; AVX512F-LABEL: test_rem7_64i8:
; AVX512F: # %bb.0: ; AVX512F: # %bb.0:
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512F-NEXT: vextracti128 $1, %ymm1, %xmm2 ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmovsxbw %xmm2, %ymm2 ; AVX512F-NEXT: vpsraw $8, %ymm2, %ymm2
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm3 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm3 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427]
; AVX512F-NEXT: vpmullw %ymm3, %ymm2, %ymm2 ; AVX512F-NEXT: vpmullw %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2 ; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512F-NEXT: vpmovsxbw %xmm1, %ymm4 ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpmullw %ymm3, %ymm4, %ymm4 ; AVX512F-NEXT: vpmullw %ymm3, %ymm4, %ymm4
; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4 ; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpackuswb %ymm2, %ymm4, %ymm2 ; AVX512F-NEXT: vpackuswb %ymm2, %ymm4, %ymm2
; AVX512F-NEXT: vpermq {{.*#+}} ymm2 = ymm2[0,2,1,3]
; AVX512F-NEXT: vpaddb %ymm1, %ymm2, %ymm2 ; AVX512F-NEXT: vpaddb %ymm1, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $7, %ymm2, %ymm4 ; AVX512F-NEXT: vpsrlw $7, %ymm2, %ymm4
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm5 = [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm5 = [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]
; AVX512F-NEXT: vpand %ymm5, %ymm4, %ymm4 ; AVX512F-NEXT: vpand %ymm5, %ymm4, %ymm4
; AVX512F-NEXT: vpsrlw $2, %ymm2, %ymm2 ; AVX512F-NEXT: vpsrlw $2, %ymm2, %ymm2
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm6 = [63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm6 = [63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63,63]
; AVX512F-NEXT: vpand %ymm6, %ymm2, %ymm2 ; AVX512F-NEXT: vpand %ymm6, %ymm2, %ymm2
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm7 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm7 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32]
; AVX512F-NEXT: vpxor %ymm7, %ymm2, %ymm2 ; AVX512F-NEXT: vpxor %ymm7, %ymm2, %ymm2
; AVX512F-NEXT: vpaddb %ymm4, %ymm2, %ymm2 ; AVX512F-NEXT: vpaddb %ymm4, %ymm2, %ymm2
; AVX512F-NEXT: vpsubb %ymm7, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubb %ymm7, %ymm2, %ymm2
; AVX512F-NEXT: vpsllw $3, %ymm2, %ymm4 ; AVX512F-NEXT: vpsllw $3, %ymm2, %ymm4
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm8 = [248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm8 = [248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248,248]
; AVX512F-NEXT: vpand %ymm4, %ymm8, %ymm4 ; AVX512F-NEXT: vpand %ymm4, %ymm8, %ymm4
; AVX512F-NEXT: vpsubb %ymm4, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubb %ymm4, %ymm2, %ymm2
; AVX512F-NEXT: vpaddb %ymm2, %ymm1, %ymm1 ; AVX512F-NEXT: vpaddb %ymm2, %ymm1, %ymm1
; AVX512F-NEXT: vextracti128 $1, %ymm0, %xmm2 ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmovsxbw %xmm2, %ymm2 ; AVX512F-NEXT: vpsraw $8, %ymm2, %ymm2
; AVX512F-NEXT: vpmullw %ymm3, %ymm2, %ymm2 ; AVX512F-NEXT: vpmullw %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2 ; AVX512F-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512F-NEXT: vpmovsxbw %xmm0, %ymm4 ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpmullw %ymm3, %ymm4, %ymm3 ; AVX512F-NEXT: vpmullw %ymm3, %ymm4, %ymm3
; AVX512F-NEXT: vpsrlw $8, %ymm3, %ymm3 ; AVX512F-NEXT: vpsrlw $8, %ymm3, %ymm3
; AVX512F-NEXT: vpackuswb %ymm2, %ymm3, %ymm2 ; AVX512F-NEXT: vpackuswb %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vpermq {{.*#+}} ymm2 = ymm2[0,2,1,3]
; AVX512F-NEXT: vpaddb %ymm0, %ymm2, %ymm2 ; AVX512F-NEXT: vpaddb %ymm0, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $7, %ymm2, %ymm3 ; AVX512F-NEXT: vpsrlw $7, %ymm2, %ymm3
; AVX512F-NEXT: vpand %ymm5, %ymm3, %ymm3 ; AVX512F-NEXT: vpand %ymm5, %ymm3, %ymm3
; AVX512F-NEXT: vpsrlw $2, %ymm2, %ymm2 ; AVX512F-NEXT: vpsrlw $2, %ymm2, %ymm2
; AVX512F-NEXT: vpand %ymm6, %ymm2, %ymm2 ; AVX512F-NEXT: vpand %ymm6, %ymm2, %ymm2
; AVX512F-NEXT: vpxor %ymm7, %ymm2, %ymm2 ; AVX512F-NEXT: vpxor %ymm7, %ymm2, %ymm2
; AVX512F-NEXT: vpaddb %ymm3, %ymm2, %ymm2 ; AVX512F-NEXT: vpaddb %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpsubb %ymm7, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubb %ymm7, %ymm2, %ymm2
; AVX512F-NEXT: vpsllw $3, %ymm2, %ymm3 ; AVX512F-NEXT: vpsllw $3, %ymm2, %ymm3
; AVX512F-NEXT: vpand %ymm3, %ymm8, %ymm3 ; AVX512F-NEXT: vpand %ymm3, %ymm8, %ymm3
; AVX512F-NEXT: vpsubb %ymm3, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubb %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpaddb %ymm2, %ymm0, %ymm0 ; AVX512F-NEXT: vpaddb %ymm2, %ymm0, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512BW-LABEL: test_rem7_64i8: ; AVX512BW-LABEL: test_rem7_64i8:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vpmovsxbw %ymm0, %zmm1 ; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm1 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512BW-NEXT: vpsraw $8, %zmm1, %zmm1
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427] ; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427,65427]
; AVX512BW-NEXT: vpmullw %zmm2, %zmm1, %zmm1 ; AVX512BW-NEXT: vpmullw %zmm2, %zmm1, %zmm1
; AVX512BW-NEXT: vpsrlw $8, %zmm1, %zmm1 ; AVX512BW-NEXT: vpsrlw $8, %zmm1, %zmm1
; AVX512BW-NEXT: vpmovwb %zmm1, %ymm1 ; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm3 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512BW-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512BW-NEXT: vpsraw $8, %zmm3, %zmm3
; AVX512BW-NEXT: vpmovsxbw %ymm3, %zmm3
; AVX512BW-NEXT: vpmullw %zmm2, %zmm3, %zmm2 ; AVX512BW-NEXT: vpmullw %zmm2, %zmm3, %zmm2
; AVX512BW-NEXT: vpsrlw $8, %zmm2, %zmm2 ; AVX512BW-NEXT: vpsrlw $8, %zmm2, %zmm2
; AVX512BW-NEXT: vpmovwb %zmm2, %ymm2 ; AVX512BW-NEXT: vpackuswb %zmm1, %zmm2, %zmm1
; AVX512BW-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512BW-NEXT: vpaddb %zmm0, %zmm1, %zmm1 ; AVX512BW-NEXT: vpaddb %zmm0, %zmm1, %zmm1
; AVX512BW-NEXT: vpsrlw $2, %zmm1, %zmm2 ; AVX512BW-NEXT: vpsrlw $2, %zmm1, %zmm2
; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm2, %zmm2 ; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm2, %zmm2
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm3 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32] ; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm3 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32]
; AVX512BW-NEXT: vpxorq %zmm3, %zmm2, %zmm2 ; AVX512BW-NEXT: vpxorq %zmm3, %zmm2, %zmm2
; AVX512BW-NEXT: vpsrlw $7, %zmm1, %zmm1 ; AVX512BW-NEXT: vpsrlw $7, %zmm1, %zmm1
; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm1, %zmm1 ; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm1, %zmm1
; AVX512BW-NEXT: vpaddb %zmm1, %zmm2, %zmm1 ; AVX512BW-NEXT: vpaddb %zmm1, %zmm2, %zmm1
Show All 18 Lines
; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm1 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm3 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm3
; AVX512F-NEXT: vmovdqa {{.*#+}} ymm1 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255] ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm1 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
; AVX512F-NEXT: vpand %ymm1, %ymm3, %ymm3 ; AVX512F-NEXT: vpand %ymm1, %ymm3, %ymm3
; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm4 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4
; AVX512F-NEXT: vpand %ymm1, %ymm4, %ymm4 ; AVX512F-NEXT: vpand %ymm1, %ymm4, %ymm4
; AVX512F-NEXT: vpackuswb %ymm3, %ymm4, %ymm3 ; AVX512F-NEXT: vpackuswb %ymm3, %ymm4, %ymm3
; AVX512F-NEXT: vextracti128 $1, %ymm2, %xmm4 ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm4 = ymm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmovsxbw %xmm4, %ymm4 ; AVX512F-NEXT: vpsraw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4
; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4 ; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpmovsxbw %xmm2, %ymm5 ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm5 = ymm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm5, %ymm5
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5
; AVX512F-NEXT: vpsrlw $8, %ymm5, %ymm5 ; AVX512F-NEXT: vpsrlw $8, %ymm5, %ymm5
; AVX512F-NEXT: vpackuswb %ymm4, %ymm5, %ymm4 ; AVX512F-NEXT: vpackuswb %ymm4, %ymm5, %ymm4
; AVX512F-NEXT: vpermq {{.*#+}} ymm4 = ymm4[0,2,1,3]
; AVX512F-NEXT: vpaddb %ymm3, %ymm4, %ymm3 ; AVX512F-NEXT: vpaddb %ymm3, %ymm4, %ymm3
; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm4 = ymm3[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm4 = ymm3[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpsraw $8, %ymm4, %ymm4 ; AVX512F-NEXT: vpsraw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4
; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4 ; AVX512F-NEXT: vpsrlw $8, %ymm4, %ymm4
; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm5 = ymm3[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm5 = ymm3[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm5, %ymm5 ; AVX512F-NEXT: vpsraw $8, %ymm5, %ymm5
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5
Show All 13 Lines
; AVX512F-NEXT: vpsubb %ymm4, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubb %ymm4, %ymm2, %ymm2
; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm4 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm4 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm4, %ymm4
; AVX512F-NEXT: vpand %ymm1, %ymm4, %ymm4 ; AVX512F-NEXT: vpand %ymm1, %ymm4, %ymm4
; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm5 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm5 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5
; AVX512F-NEXT: vpand %ymm1, %ymm5, %ymm5 ; AVX512F-NEXT: vpand %ymm1, %ymm5, %ymm5
; AVX512F-NEXT: vpackuswb %ymm4, %ymm5, %ymm4 ; AVX512F-NEXT: vpackuswb %ymm4, %ymm5, %ymm4
; AVX512F-NEXT: vextracti128 $1, %ymm0, %xmm5 ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm5 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpmovsxbw %xmm5, %ymm5 ; AVX512F-NEXT: vpsraw $8, %ymm5, %ymm5
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5
; AVX512F-NEXT: vpsrlw $8, %ymm5, %ymm5 ; AVX512F-NEXT: vpsrlw $8, %ymm5, %ymm5
; AVX512F-NEXT: vpmovsxbw %xmm0, %ymm6 ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm6 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm6, %ymm6
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm6, %ymm6 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm6, %ymm6
; AVX512F-NEXT: vpsrlw $8, %ymm6, %ymm6 ; AVX512F-NEXT: vpsrlw $8, %ymm6, %ymm6
; AVX512F-NEXT: vpackuswb %ymm5, %ymm6, %ymm5 ; AVX512F-NEXT: vpackuswb %ymm5, %ymm6, %ymm5
; AVX512F-NEXT: vpermq {{.*#+}} ymm5 = ymm5[0,2,1,3]
; AVX512F-NEXT: vpaddb %ymm4, %ymm5, %ymm4 ; AVX512F-NEXT: vpaddb %ymm4, %ymm5, %ymm4
; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm5 = ymm4[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31] ; AVX512F-NEXT: vpunpckhbw {{.*#+}} ymm5 = ymm4[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512F-NEXT: vpsraw $8, %ymm5, %ymm5 ; AVX512F-NEXT: vpsraw $8, %ymm5, %ymm5
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm5, %ymm5
; AVX512F-NEXT: vpsrlw $8, %ymm5, %ymm5 ; AVX512F-NEXT: vpsrlw $8, %ymm5, %ymm5
; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm6 = ymm4[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23] ; AVX512F-NEXT: vpunpcklbw {{.*#+}} ymm6 = ymm4[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512F-NEXT: vpsraw $8, %ymm6, %ymm6 ; AVX512F-NEXT: vpsraw $8, %ymm6, %ymm6
; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm6, %ymm6 ; AVX512F-NEXT: vpmullw {{.*}}(%rip), %ymm6, %ymm6
Show All 18 Lines
; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm1 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63] ; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm1 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm1, %zmm1 ; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm1, %zmm1
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255] ; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
; AVX512BW-NEXT: vpandq %zmm2, %zmm1, %zmm1 ; AVX512BW-NEXT: vpandq %zmm2, %zmm1, %zmm1
; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm3 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55] ; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm3 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm3, %zmm3 ; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm3, %zmm3
; AVX512BW-NEXT: vpandq %zmm2, %zmm3, %zmm3 ; AVX512BW-NEXT: vpandq %zmm2, %zmm3, %zmm3
; AVX512BW-NEXT: vpackuswb %zmm1, %zmm3, %zmm1 ; AVX512BW-NEXT: vpackuswb %zmm1, %zmm3, %zmm1
; AVX512BW-NEXT: vpmovsxbw %ymm0, %zmm3 ; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm3 = zmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512BW-NEXT: vpsraw $8, %zmm3, %zmm3
; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm3, %zmm3 ; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm3, %zmm3
; AVX512BW-NEXT: vpsrlw $8, %zmm3, %zmm3 ; AVX512BW-NEXT: vpsrlw $8, %zmm3, %zmm3
; AVX512BW-NEXT: vpmovwb %zmm3, %ymm3 ; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm4 = zmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512BW-NEXT: vextracti64x4 $1, %zmm0, %ymm4 ; AVX512BW-NEXT: vpsraw $8, %zmm4, %zmm4
; AVX512BW-NEXT: vpmovsxbw %ymm4, %zmm4
; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm4, %zmm4 ; AVX512BW-NEXT: vpmullw {{.*}}(%rip), %zmm4, %zmm4
; AVX512BW-NEXT: vpsrlw $8, %zmm4, %zmm4 ; AVX512BW-NEXT: vpsrlw $8, %zmm4, %zmm4
; AVX512BW-NEXT: vpmovwb %zmm4, %ymm4 ; AVX512BW-NEXT: vpackuswb %zmm3, %zmm4, %zmm3
; AVX512BW-NEXT: vinserti64x4 $1, %ymm4, %zmm3, %zmm3
; AVX512BW-NEXT: vpaddb %zmm1, %zmm3, %zmm1 ; AVX512BW-NEXT: vpaddb %zmm1, %zmm3, %zmm1
; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm3 = zmm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63] ; AVX512BW-NEXT: vpunpckhbw {{.*#+}} zmm3 = zmm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31,40,40,41,41,42,42,43,43,44,44,45,45,46,46,47,47,56,56,57,57,58,58,59,59,60,60,61,61,62,62,63,63]
; AVX512BW-NEXT: vpsraw $8, %zmm3, %zmm3 ; AVX512BW-NEXT: vpsraw $8, %zmm3, %zmm3
; AVX512BW-NEXT: vpsllvw {{.*}}(%rip), %zmm3, %zmm3 ; AVX512BW-NEXT: vpsllvw {{.*}}(%rip), %zmm3, %zmm3
; AVX512BW-NEXT: vpsrlw $8, %zmm3, %zmm3 ; AVX512BW-NEXT: vpsrlw $8, %zmm3, %zmm3
; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm4 = zmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55] ; AVX512BW-NEXT: vpunpcklbw {{.*#+}} zmm4 = zmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23,32,32,33,33,34,34,35,35,36,36,37,37,38,38,39,39,48,48,49,49,50,50,51,51,52,52,53,53,54,54,55,55]
; AVX512BW-NEXT: vpsraw $8, %zmm4, %zmm4 ; AVX512BW-NEXT: vpsraw $8, %zmm4, %zmm4
; AVX512BW-NEXT: vpsllvw {{.*}}(%rip), %zmm4, %zmm4 ; AVX512BW-NEXT: vpsllvw {{.*}}(%rip), %zmm4, %zmm4
Show All 17 Lines