This is an archive of the discontinued LLVM Phabricator instance.

Differential D61806

[DAGCombiner] try to move bitcast after extract_subvector
ClosedPublic

Authored by spatel on May 10 2019, 2:58 PM.

Details

Reviewers
craig.topper
RKSimon
efriedma
Commits
rZORG933eb64941b2: [DAGCombiner] try to move bitcast after extract_subvector
rZORGd5e3f88f9d83: [DAGCombiner] try to move bitcast after extract_subvector
rG933eb64941b2: [DAGCombiner] try to move bitcast after extract_subvector
rGd5e3f88f9d83: [DAGCombiner] try to move bitcast after extract_subvector
rGa09e6868218b: [DAGCombiner] try to move bitcast after extract_subvector
rL360541: [DAGCombiner] try to move bitcast after extract_subvector
Summary

I noticed that we were failing to narrow an x86 ymm math op in a case similar to the 'madd' test diff. That is because a bitcast is sitting between the math and the extract subvector and thwarting our pattern matching for narrowing:

     t56: v8i32 = add t59, t58
    t68: v4i64 = bitcast t56
  t73: v2i64 = extract_subvector t68, Constant:i64<2>
t96: v4i32 = bitcast t73

There are a few wins and neutral diffs in the other tests.

Diff Detail

Repository
rL LLVM

Event Timeline

spatel created this revision.May 10 2019, 2:58 PM
Herald added a project: Restricted Project. · View Herald TranscriptMay 10 2019, 2:58 PM
Herald added subscribers: hiraditya, mcrosier. · View Herald Transcript
RKSimon added inline comments.May 11 2019, 9:09 AM
llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
17595 ↗(On Diff #199087)

Its typically safer to do (SrcNumElts % DestNumElts) == 0

llvm/test/CodeGen/X86/sad.ll
1452 ↗(On Diff #199087)

weird - if this is all xmm code now why hasn't the vzeroupper gone?

spatel marked an inline comment as done.May 12 2019, 6:09 AM
spatel added inline comments.
llvm/test/CodeGen/X86/sad.ll
1452 ↗(On Diff #199087)

There seem to be several potential missed opportunities to squash this, but we have the following heading into isel:

      t142: v4i32 = scalar_to_vector Constant:i32<1>
    t143: v8i32 = insert_subvector undef:v8i32, t142, Constant:i64<0>
  t141: v8i32 = X86ISD::VZEXT_MOVL t143
t178: v4i32 = extract_subvector t141, Constant:i64<0>
spatel updated this revision to Diff 199167.May 12 2019, 6:12 AM
spatel marked an inline comment as done.

Patch updated:
Use modulo of element counts for safety.

RKSimon accepted this revision.May 12 2019, 6:29 AM

LGTM - cheers.

llvm/test/CodeGen/X86/sad.ll
1452 ↗(On Diff #199087)

Thanks for checking - that should be easy enough to fix.

This revision is now accepted and ready to land.May 12 2019, 6:29 AM
Closed by commit rL360541: [DAGCombiner] try to move bitcast after extract_subvector (authored by spatel). · Explain WhyMay 12 2019, 7:40 AM
This revision was automatically updated to reflect the committed changes.
RKSimon mentioned this in rL360543: [X86][AVX] Split VZEXT_MOVL ymm/zmm if the upper elements are not demanded..May 12 2019, 8:14 AM
RKSimon mentioned this in rGa7fc7630829b: [X86][AVX] Split VZEXT_MOVL ymm/zmm if the upper elements are not demanded..May 12 2019, 8:17 AM
sidorovd mentioned this in rG2de363410c76: [X86][AVX] Split VZEXT_MOVL ymm/zmm if the upper elements are not demanded..May 30 2019, 9:12 AM
sidorovd mentioned this in rGc8d4fc480355: [X86][AVX] Split VZEXT_MOVL ymm/zmm if the upper elements are not demanded..May 30 2019, 10:12 AM
dmgreen mentioned this in D126782: [AArch64] Look through bitcast when looking for extract_high subvector .Jun 1 2022, 7:48 AM
dmgreen mentioned this in rG963c0a014700: [AArch64] Look through bitcast when looking for extract_high subvector.Jun 12 2022, 2:59 AM

Revision Contents

PathSize
llvm/
trunk/
lib/
CodeGen/
SelectionDAG/
24 lines
test/
CodeGen/
X86/
6 lines
43 lines
71 lines
17 lines
4 lines
4 lines
4 lines
4 lines
4 lines
2 lines
8 lines

Diff 199168

llvm/trunk/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 17,577 Lines • ▼ Show 20 Lines if (isNullConstant(Index) && V.getOpcode() == ISD::EXTRACT_SUBVECTOR &&
if (TLI.isExtractSubvectorCheap(NVT, V.getOperand(0).getValueType(), if (TLI.isExtractSubvectorCheap(NVT, V.getOperand(0).getValueType(),
V.getConstantOperandVal(1)) && V.getConstantOperandVal(1)) &&
TLI.isOperationLegalOrCustom(ISD::EXTRACT_SUBVECTOR, NVT)) { TLI.isOperationLegalOrCustom(ISD::EXTRACT_SUBVECTOR, NVT)) {
return DAG.getNode(ISD::EXTRACT_SUBVECTOR, SDLoc(N), NVT, V.getOperand(0), return DAG.getNode(ISD::EXTRACT_SUBVECTOR, SDLoc(N), NVT, V.getOperand(0),
V.getOperand(1)); V.getOperand(1));
} }
} }
// Try to move vector bitcast after extract_subv by scaling extraction index:
// extract_subv (bitcast X), Index --> bitcast (extract_subv X, Index')
if (isa<ConstantSDNode>(Index) && V.getOpcode() == ISD::BITCAST &&
V.getOperand(0).getValueType().isVector()) {
SDValue SrcOp = V.getOperand(0);
EVT SrcVT = SrcOp.getValueType();
unsigned SrcNumElts = SrcVT.getVectorNumElements();
unsigned DestNumElts = V.getValueType().getVectorNumElements();
if ((SrcNumElts % DestNumElts) == 0) {
unsigned SrcDestRatio = SrcNumElts / DestNumElts;
unsigned NewExtNumElts = NVT.getVectorNumElements() * SrcDestRatio;
EVT NewExtVT = EVT::getVectorVT(*DAG.getContext(), SrcVT.getScalarType(),
NewExtNumElts);
if (TLI.isOperationLegalOrCustom(ISD::EXTRACT_SUBVECTOR, NewExtVT)) {
unsigned IndexValScaled = N->getConstantOperandVal(1) * SrcDestRatio;
SDLoc DL(N);
SDValue NewIndex = DAG.getIntPtrConstant(IndexValScaled, DL);
SDValue NewExtract = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, NewExtVT,
V.getOperand(0), NewIndex);
return DAG.getBitcast(NVT, NewExtract);
}
}
}
// Combine: // Combine:
// (extract_subvec (concat V1, V2, ...), i) // (extract_subvec (concat V1, V2, ...), i)
// Into: // Into:
// Vi if possible // Vi if possible
// Only operand 0 is checked as 'concat' assumes all inputs of the same // Only operand 0 is checked as 'concat' assumes all inputs of the same
// type. // type.
if (V.getOpcode() == ISD::CONCAT_VECTORS && isa<ConstantSDNode>(Index) && if (V.getOpcode() == ISD::CONCAT_VECTORS && isa<ConstantSDNode>(Index) &&
V.getOperand(0).getValueType() == NVT) { V.getOperand(0).getValueType() == NVT) {
▲ Show 20 Lines • Show All 2,538 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/extractelement-fp.ll

Show All 20 Lines; X86-NEXT: retl
%v = fneg <4 x float> %x %v = fneg <4 x float> %x
%r = extractelement <4 x float> %v, i32 0 %r = extractelement <4 x float> %v, i32 0
ret float %r ret float %r
} }
define double @fneg_v4f64(<4 x double> %x) nounwind { define double @fneg_v4f64(<4 x double> %x) nounwind {
; X64-LABEL: fneg_v4f64: ; X64-LABEL: fneg_v4f64:
; X64: # %bb.0: ; X64: # %bb.0:
; X64-NEXT: vbroadcastsd {{.*#+}} ymm1 = [-0.0E+0,-0.0E+0,-0.0E+0,-0.0E+0] ; X64-NEXT: vmovddup {{.*#+}} xmm1 = [-0.0E+0,-0.0E+0]
; X64-NEXT: # xmm1 = mem[0,0]
; X64-NEXT: vxorps %xmm1, %xmm0, %xmm0 ; X64-NEXT: vxorps %xmm1, %xmm0, %xmm0
; X64-NEXT: vzeroupper ; X64-NEXT: vzeroupper
; X64-NEXT: retq ; X64-NEXT: retq
; ;
; X86-LABEL: fneg_v4f64: ; X86-LABEL: fneg_v4f64:
; X86: # %bb.0: ; X86: # %bb.0:
; X86-NEXT: pushl %ebp ; X86-NEXT: pushl %ebp
; X86-NEXT: movl %esp, %ebp ; X86-NEXT: movl %esp, %ebp
; X86-NEXT: andl $-8, %esp ; X86-NEXT: andl $-8, %esp
; X86-NEXT: subl $8, %esp ; X86-NEXT: subl $8, %esp
; X86-NEXT: vbroadcastsd {{.*#+}} ymm1 = [-0.0E+0,-0.0E+0,-0.0E+0,-0.0E+0] ; X86-NEXT: vmovddup {{.*#+}} xmm1 = [-0.0E+0,-0.0E+0]
; X86-NEXT: # xmm1 = mem[0,0]
; X86-NEXT: vxorps %xmm1, %xmm0, %xmm0 ; X86-NEXT: vxorps %xmm1, %xmm0, %xmm0
; X86-NEXT: vmovlps %xmm0, (%esp) ; X86-NEXT: vmovlps %xmm0, (%esp)
; X86-NEXT: fldl (%esp) ; X86-NEXT: fldl (%esp)
; X86-NEXT: movl %ebp, %esp ; X86-NEXT: movl %ebp, %esp
; X86-NEXT: popl %ebp ; X86-NEXT: popl %ebp
; X86-NEXT: vzeroupper ; X86-NEXT: vzeroupper
; X86-NEXT: retl ; X86-NEXT: retl
%v = fneg <4 x double> %x %v = fneg <4 x double> %x
▲ Show 20 Lines • Show All 1,170 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/madd.ll

Show First 20 Lines • Show All 2,667 Lines • ▼ Show 20 Lines
; SSE2-NEXT: paddd %xmm1, %xmm2 ; SSE2-NEXT: paddd %xmm1, %xmm2
; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm2[2,3,0,1] ; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm2[2,3,0,1]
; SSE2-NEXT: paddd %xmm2, %xmm0 ; SSE2-NEXT: paddd %xmm2, %xmm0
; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; SSE2-NEXT: paddd %xmm0, %xmm1 ; SSE2-NEXT: paddd %xmm0, %xmm1
; SSE2-NEXT: movd %xmm1, %eax ; SSE2-NEXT: movd %xmm1, %eax
; SSE2-NEXT: retq ; SSE2-NEXT: retq
; ;
; AVX1-LABEL: madd_double_reduction: ; AVX-LABEL: madd_double_reduction:
; AVX1: # %bb.0: ; AVX: # %bb.0:
; AVX1-NEXT: vmovdqu (%rdi), %xmm0 ; AVX-NEXT: vmovdqu (%rdi), %xmm0
; AVX1-NEXT: vmovdqu (%rdx), %xmm1 ; AVX-NEXT: vmovdqu (%rdx), %xmm1
; AVX1-NEXT: vpmaddwd (%rsi), %xmm0, %xmm0 ; AVX-NEXT: vpmaddwd (%rsi), %xmm0, %xmm0
; AVX1-NEXT: vpmaddwd (%rcx), %xmm1, %xmm1 ; AVX-NEXT: vpmaddwd (%rcx), %xmm1, %xmm1
; AVX1-NEXT: vpaddd %xmm0, %xmm1, %xmm0 ; AVX-NEXT: vpaddd %xmm0, %xmm1, %xmm0
; AVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1] ; AVX-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX1-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; AVX-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX1-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vmovd %xmm0, %eax ; AVX-NEXT: vmovd %xmm0, %eax
; AVX1-NEXT: retq ; AVX-NEXT: retq
;
; AVX256-LABEL: madd_double_reduction:
; AVX256: # %bb.0:
; AVX256-NEXT: vmovdqu (%rdi), %xmm0
; AVX256-NEXT: vmovdqu (%rdx), %xmm1
; AVX256-NEXT: vpmaddwd (%rsi), %xmm0, %xmm0
; AVX256-NEXT: vpmaddwd (%rcx), %xmm1, %xmm1
; AVX256-NEXT: vpaddd %ymm0, %ymm1, %ymm0
; AVX256-NEXT: vextracti128 $1, %ymm0, %xmm1
; AVX256-NEXT: vpor %xmm1, %xmm0, %xmm0
; AVX256-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX256-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX256-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX256-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX256-NEXT: vmovd %xmm0, %eax
; AVX256-NEXT: vzeroupper
; AVX256-NEXT: retq
%tmp = load <8 x i16>, <8 x i16>* %arg, align 1 %tmp = load <8 x i16>, <8 x i16>* %arg, align 1
%tmp6 = load <8 x i16>, <8 x i16>* %arg1, align 1 %tmp6 = load <8 x i16>, <8 x i16>* %arg1, align 1
%tmp7 = sext <8 x i16> %tmp to <8 x i32> %tmp7 = sext <8 x i16> %tmp to <8 x i32>
%tmp17 = sext <8 x i16> %tmp6 to <8 x i32> %tmp17 = sext <8 x i16> %tmp6 to <8 x i32>
%tmp19 = mul nsw <8 x i32> %tmp7, %tmp17 %tmp19 = mul nsw <8 x i32> %tmp7, %tmp17
%tmp20 = load <8 x i16>, <8 x i16>* %arg2, align 1 %tmp20 = load <8 x i16>, <8 x i16>* %arg2, align 1
%tmp21 = load <8 x i16>, <8 x i16>* %arg3, align 1 %tmp21 = load <8 x i16>, <8 x i16>* %arg3, align 1
%tmp22 = sext <8 x i16> %tmp20 to <8 x i32> %tmp22 = sext <8 x i16> %tmp20 to <8 x i32>
Show All 12 Lines

llvm/trunk/test/CodeGen/X86/oddshuffles.ll

Show First 20 Lines • Show All 588 Lines • ▼ Show 20 Lines
; SSE42-NEXT: retq ; SSE42-NEXT: retq
; ;
; AVX1-LABEL: v12i32: ; AVX1-LABEL: v12i32:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1] ; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1]
; AVX1-NEXT: vmovsldup {{.*#+}} ymm2 = ymm2[0,0,2,2,4,4,6,6] ; AVX1-NEXT: vmovsldup {{.*#+}} ymm2 = ymm2[0,0,2,2,4,4,6,6]
; AVX1-NEXT: vpermilps {{.*#+}} ymm3 = ymm0[0,u,u,1,5,u,u,6] ; AVX1-NEXT: vpermilps {{.*#+}} ymm3 = ymm0[0,u,u,1,5,u,u,6]
; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm3[0],ymm2[1],ymm3[2,3,4,5],ymm2[6],ymm3[7] ; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm3[0],ymm2[1],ymm3[2,3,4,5],ymm2[6],ymm3[7]
; AVX1-NEXT: vmovddup {{.*#+}} xmm3 = xmm1[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm3 = xmm1[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3 ; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3
; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; AVX1-NEXT: vshufps {{.*#+}} xmm0 = xmm0[0,3],xmm3[3,3] ; AVX1-NEXT: vshufps {{.*#+}} xmm0 = xmm0[0,3],xmm3[3,3]
; AVX1-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,1] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,2,3]
; AVX1-NEXT: vblendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2],xmm1[3] ; AVX1-NEXT: vblendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2],xmm1[3]
; AVX1-NEXT: vmovaps %xmm0, 32(%rdi) ; AVX1-NEXT: vmovaps %xmm0, 32(%rdi)
; AVX1-NEXT: vmovaps %ymm2, (%rdi) ; AVX1-NEXT: vmovaps %ymm2, (%rdi)
; AVX1-NEXT: vzeroupper ; AVX1-NEXT: vzeroupper
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-SLOW-LABEL: v12i32: ; AVX2-SLOW-LABEL: v12i32:
; AVX2-SLOW: # %bb.0: ; AVX2-SLOW: # %bb.0:
Show All 24 Lines
; AVX2-FAST-NEXT: vmovaps %ymm2, (%rdi) ; AVX2-FAST-NEXT: vmovaps %ymm2, (%rdi)
; AVX2-FAST-NEXT: vzeroupper ; AVX2-FAST-NEXT: vzeroupper
; AVX2-FAST-NEXT: retq ; AVX2-FAST-NEXT: retq
; ;
; XOP-LABEL: v12i32: ; XOP-LABEL: v12i32:
; XOP: # %bb.0: ; XOP: # %bb.0:
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1] ; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1]
; XOP-NEXT: vpermil2ps {{.*#+}} ymm2 = ymm0[0],ymm2[0],ymm0[u,1,5,u],ymm2[6],ymm0[6] ; XOP-NEXT: vpermil2ps {{.*#+}} ymm2 = ymm0[0],ymm2[0],ymm0[u,1,5,u],ymm2[6],ymm0[6]
; XOP-NEXT: vmovddup {{.*#+}} xmm3 = xmm1[0,0] ; XOP-NEXT: vpermilps {{.*#+}} xmm3 = xmm1[0,1,0,1]
; XOP-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3 ; XOP-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3
; XOP-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7]
; XOP-NEXT: vextractf128 $1, %ymm0, %xmm3 ; XOP-NEXT: vextractf128 $1, %ymm0, %xmm3
; XOP-NEXT: vshufps {{.*#+}} xmm0 = xmm0[0,3],xmm3[3,3] ; XOP-NEXT: vshufps {{.*#+}} xmm0 = xmm0[0,3],xmm3[3,3]
; XOP-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,1] ; XOP-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,2,3]
; XOP-NEXT: vblendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2],xmm1[3] ; XOP-NEXT: vblendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2],xmm1[3]
; XOP-NEXT: vmovaps %xmm0, 32(%rdi) ; XOP-NEXT: vmovaps %xmm0, 32(%rdi)
; XOP-NEXT: vmovaps %ymm2, (%rdi) ; XOP-NEXT: vmovaps %ymm2, (%rdi)
; XOP-NEXT: vzeroupper ; XOP-NEXT: vzeroupper
; XOP-NEXT: retq ; XOP-NEXT: retq
%r = shufflevector <8 x i32> %a, <8 x i32> %b, <12 x i32> <i32 0, i32 4, i32 8, i32 1, i32 5, i32 9, i32 2, i32 6, i32 10, i32 3, i32 7, i32 11> %r = shufflevector <8 x i32> %a, <8 x i32> %b, <12 x i32> <i32 0, i32 4, i32 8, i32 1, i32 5, i32 9, i32 2, i32 6, i32 10, i32 3, i32 7, i32 11>
store <12 x i32> %r, <12 x i32>* %p store <12 x i32> %r, <12 x i32>* %p
ret void ret void
▲ Show 20 Lines • Show All 694 Lines • ▼ Show 20 Lines
; AVX1-NEXT: vblendps {{.*#+}} ymm4 = ymm5[0,1,2,3,4,5],ymm4[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm4 = ymm5[0,1,2,3,4,5],ymm4[6,7]
; AVX1-NEXT: vblendps {{.*#+}} xmm5 = xmm3[0,1],xmm2[2],xmm3[3] ; AVX1-NEXT: vblendps {{.*#+}} xmm5 = xmm3[0,1],xmm2[2],xmm3[3]
; AVX1-NEXT: vpermilps {{.*#+}} xmm5 = xmm5[0,0,3,2] ; AVX1-NEXT: vpermilps {{.*#+}} xmm5 = xmm5[0,0,3,2]
; AVX1-NEXT: vinsertf128 $1, %xmm5, %ymm0, %ymm5 ; AVX1-NEXT: vinsertf128 $1, %xmm5, %ymm0, %ymm5
; AVX1-NEXT: vblendps {{.*#+}} ymm6 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm6 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7]
; AVX1-NEXT: vextractf128 $1, %ymm6, %xmm7 ; AVX1-NEXT: vextractf128 $1, %ymm6, %xmm7
; AVX1-NEXT: vblendps {{.*#+}} xmm6 = xmm7[0],xmm6[1,2],xmm7[3] ; AVX1-NEXT: vblendps {{.*#+}} xmm6 = xmm7[0],xmm6[1,2],xmm7[3]
; AVX1-NEXT: vpermilps {{.*#+}} xmm6 = xmm6[1,0,3,2] ; AVX1-NEXT: vpermilps {{.*#+}} xmm6 = xmm6[1,0,3,2]
; AVX1-NEXT: vmovshdup {{.*#+}} xmm7 = xmm7[1,1,3,3] ; AVX1-NEXT: vpermilps {{.*#+}} xmm7 = xmm7[1,1,3,3]
; AVX1-NEXT: vinsertf128 $1, %xmm7, %ymm6, %ymm6 ; AVX1-NEXT: vinsertf128 $1, %xmm7, %ymm6, %ymm6
; AVX1-NEXT: vblendps {{.*#+}} ymm5 = ymm6[0,1,2,3,4],ymm5[5,6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm5 = ymm6[0,1,2,3,4],ymm5[5,6,7]
; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm3[0,1],xmm2[0,3] ; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm3[0,1],xmm2[0,3]
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm2 ; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm2
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0,1],ymm0[2],ymm1[3,4],ymm0[5],ymm1[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0,1],ymm0[2],ymm1[3,4],ymm0[5],ymm1[6,7]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX1-NEXT: vblendps {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2,3] ; AVX1-NEXT: vblendps {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2,3]
; AVX1-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[2,1,0,3] ; AVX1-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[2,1,0,3]
; AVX1-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0 ; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4],ymm2[5,6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4],ymm2[5,6,7]
; AVX1-NEXT: vmovups %ymm4, (%rsi) ; AVX1-NEXT: vmovups %ymm4, (%rsi)
; AVX1-NEXT: vmovups %ymm5, (%rdx) ; AVX1-NEXT: vmovups %ymm5, (%rdx)
; AVX1-NEXT: vmovups %ymm0, (%rcx) ; AVX1-NEXT: vmovups %ymm0, (%rcx)
; AVX1-NEXT: vzeroupper ; AVX1-NEXT: vzeroupper
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
▲ Show 20 Lines • Show All 74 Lines • ▼ Show 20 Lines
; XOP-NEXT: vblendps {{.*#+}} ymm4 = ymm5[0,1,2,3,4,5],ymm4[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm4 = ymm5[0,1,2,3,4,5],ymm4[6,7]
; XOP-NEXT: vblendps {{.*#+}} xmm5 = xmm3[0,1],xmm2[2],xmm3[3] ; XOP-NEXT: vblendps {{.*#+}} xmm5 = xmm3[0,1],xmm2[2],xmm3[3]
; XOP-NEXT: vpermilps {{.*#+}} xmm5 = xmm5[0,0,3,2] ; XOP-NEXT: vpermilps {{.*#+}} xmm5 = xmm5[0,0,3,2]
; XOP-NEXT: vinsertf128 $1, %xmm5, %ymm0, %ymm5 ; XOP-NEXT: vinsertf128 $1, %xmm5, %ymm0, %ymm5
; XOP-NEXT: vblendps {{.*#+}} ymm6 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm6 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7]
; XOP-NEXT: vextractf128 $1, %ymm6, %xmm7 ; XOP-NEXT: vextractf128 $1, %ymm6, %xmm7
; XOP-NEXT: vblendps {{.*#+}} xmm6 = xmm7[0],xmm6[1,2],xmm7[3] ; XOP-NEXT: vblendps {{.*#+}} xmm6 = xmm7[0],xmm6[1,2],xmm7[3]
; XOP-NEXT: vpermilps {{.*#+}} xmm6 = xmm6[1,0,3,2] ; XOP-NEXT: vpermilps {{.*#+}} xmm6 = xmm6[1,0,3,2]
; XOP-NEXT: vmovshdup {{.*#+}} xmm7 = xmm7[1,1,3,3] ; XOP-NEXT: vpermilps {{.*#+}} xmm7 = xmm7[1,1,3,3]
; XOP-NEXT: vinsertf128 $1, %xmm7, %ymm6, %ymm6 ; XOP-NEXT: vinsertf128 $1, %xmm7, %ymm6, %ymm6
; XOP-NEXT: vblendps {{.*#+}} ymm5 = ymm6[0,1,2,3,4],ymm5[5,6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm5 = ymm6[0,1,2,3,4],ymm5[5,6,7]
; XOP-NEXT: vshufps {{.*#+}} xmm2 = xmm3[0,1],xmm2[0,3] ; XOP-NEXT: vshufps {{.*#+}} xmm2 = xmm3[0,1],xmm2[0,3]
; XOP-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm2 ; XOP-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm2
; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0,1],ymm0[2],ymm1[3,4],ymm0[5],ymm1[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0,1],ymm0[2],ymm1[3,4],ymm0[5],ymm1[6,7]
; XOP-NEXT: vextractf128 $1, %ymm0, %xmm1 ; XOP-NEXT: vextractf128 $1, %ymm0, %xmm1
; XOP-NEXT: vblendps {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2,3] ; XOP-NEXT: vblendps {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2,3]
; XOP-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[2,1,0,3] ; XOP-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[2,1,0,3]
; XOP-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; XOP-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,0,1]
; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0 ; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4],ymm2[5,6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4],ymm2[5,6,7]
; XOP-NEXT: vmovups %ymm4, (%rsi) ; XOP-NEXT: vmovups %ymm4, (%rsi)
; XOP-NEXT: vmovups %ymm5, (%rdx) ; XOP-NEXT: vmovups %ymm5, (%rdx)
; XOP-NEXT: vmovups %ymm0, (%rcx) ; XOP-NEXT: vmovups %ymm0, (%rcx)
; XOP-NEXT: vzeroupper ; XOP-NEXT: vzeroupper
; XOP-NEXT: retq ; XOP-NEXT: retq
%wide.vec = load <24 x i32>, <24 x i32>* %p, align 4 %wide.vec = load <24 x i32>, <24 x i32>* %p, align 4
▲ Show 20 Lines • Show All 89 Lines • ▼ Show 20 Lines
; SSE42-NEXT: movdqu %xmm0, 64(%rdi) ; SSE42-NEXT: movdqu %xmm0, 64(%rdi)
; SSE42-NEXT: movdqu %xmm7, 80(%rdi) ; SSE42-NEXT: movdqu %xmm7, 80(%rdi)
; SSE42-NEXT: movdqu %xmm1, (%rdi) ; SSE42-NEXT: movdqu %xmm1, (%rdi)
; SSE42-NEXT: retq ; SSE42-NEXT: retq
; ;
; AVX1-LABEL: interleave_24i32_in: ; AVX1-LABEL: interleave_24i32_in:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovupd (%rsi), %ymm0 ; AVX1-NEXT: vmovupd (%rsi), %ymm0
; AVX1-NEXT: vmovupd (%rcx), %ymm1 ; AVX1-NEXT: vmovups 16(%rcx), %xmm1
; AVX1-NEXT: vmovups 16(%rcx), %xmm2 ; AVX1-NEXT: vmovups (%rdx), %xmm2
; AVX1-NEXT: vmovups (%rdx), %xmm3 ; AVX1-NEXT: vmovups 16(%rdx), %xmm3
; AVX1-NEXT: vmovups 16(%rdx), %xmm4 ; AVX1-NEXT: vshufps {{.*#+}} xmm4 = xmm3[3,0],xmm1[3,0]
; AVX1-NEXT: vshufps {{.*#+}} xmm5 = xmm4[3,0],xmm2[3,0] ; AVX1-NEXT: vshufps {{.*#+}} xmm4 = xmm1[2,1],xmm4[0,2]
; AVX1-NEXT: vshufps {{.*#+}} xmm5 = xmm2[2,1],xmm5[0,2] ; AVX1-NEXT: vshufps {{.*#+}} xmm1 = xmm1[1,0],xmm3[1,0]
; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm2[1,0],xmm4[1,0] ; AVX1-NEXT: vshufps {{.*#+}} xmm1 = xmm1[2,0],xmm3[2,2]
; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm2[2,0],xmm4[2,2] ; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm1, %ymm1
; AVX1-NEXT: vinsertf128 $1, %xmm5, %ymm2, %ymm2 ; AVX1-NEXT: vpermilpd {{.*#+}} ymm3 = ymm0[1,1,3,3]
; AVX1-NEXT: vpermilpd {{.*#+}} ymm4 = ymm0[1,1,3,3] ; AVX1-NEXT: vperm2f128 {{.*#+}} ymm3 = ymm3[2,3,2,3]
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm4 = ymm4[2,3,2,3] ; AVX1-NEXT: vblendps {{.*#+}} ymm1 = ymm1[0,1],ymm3[2],ymm1[3,4],ymm3[5],ymm1[6,7]
; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm4[2],ymm2[3,4],ymm4[5],ymm2[6,7] ; AVX1-NEXT: vmovups (%rsi), %xmm3
; AVX1-NEXT: vmovups (%rsi), %xmm4 ; AVX1-NEXT: vshufps {{.*#+}} xmm4 = xmm3[2,0],xmm2[2,0]
; AVX1-NEXT: vshufps {{.*#+}} xmm5 = xmm4[2,0],xmm3[2,0] ; AVX1-NEXT: vshufps {{.*#+}} xmm4 = xmm2[1,1],xmm4[0,2]
; AVX1-NEXT: vshufps {{.*#+}} xmm5 = xmm3[1,1],xmm5[0,2] ; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm2[0,0],xmm3[0,0]
; AVX1-NEXT: vshufps {{.*#+}} xmm3 = xmm3[0,0],xmm4[0,0] ; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm2[2,0],xmm3[2,1]
; AVX1-NEXT: vshufps {{.*#+}} xmm3 = xmm3[2,0],xmm4[2,1] ; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm2, %ymm2
; AVX1-NEXT: vinsertf128 $1, %xmm5, %ymm3, %ymm3 ; AVX1-NEXT: vpermilps {{.*#+}} xmm3 = mem[0,1,0,1]
; AVX1-NEXT: vmovddup {{.*#+}} xmm4 = xmm1[0,0] ; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3
; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm4, %ymm4 ; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7]
; AVX1-NEXT: vblendps {{.*#+}} ymm3 = ymm3[0,1],ymm4[2],ymm3[3,4],ymm4[5],ymm3[6,7]
; AVX1-NEXT: vpermilpd {{.*#+}} ymm0 = ymm0[1,0,2,2] ; AVX1-NEXT: vpermilpd {{.*#+}} ymm0 = ymm0[1,0,2,2]
; AVX1-NEXT: vpermilpd {{.*#+}} ymm1 = ymm1[1,1,2,2] ; AVX1-NEXT: vpermilpd {{.*#+}} ymm3 = mem[1,1,2,2]
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0],ymm0[1],ymm1[2,3],ymm0[4],ymm1[5,6],ymm0[7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm3[0],ymm0[1],ymm3[2,3],ymm0[4],ymm3[5,6],ymm0[7]
; AVX1-NEXT: vpermilps {{.*#+}} ymm1 = mem[0,0,3,3,4,4,7,7] ; AVX1-NEXT: vpermilps {{.*#+}} ymm3 = mem[0,0,3,3,4,4,7,7]
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm3[2],ymm0[3,4],ymm3[5],ymm0[6,7]
; AVX1-NEXT: vmovups %ymm0, 32(%rdi) ; AVX1-NEXT: vmovups %ymm0, 32(%rdi)
; AVX1-NEXT: vmovups %ymm3, (%rdi) ; AVX1-NEXT: vmovups %ymm2, (%rdi)
; AVX1-NEXT: vmovups %ymm2, 64(%rdi) ; AVX1-NEXT: vmovups %ymm1, 64(%rdi)
; AVX1-NEXT: vzeroupper ; AVX1-NEXT: vzeroupper
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-SLOW-LABEL: interleave_24i32_in: ; AVX2-SLOW-LABEL: interleave_24i32_in:
; AVX2-SLOW: # %bb.0: ; AVX2-SLOW: # %bb.0:
; AVX2-SLOW-NEXT: vmovups (%rsi), %ymm0 ; AVX2-SLOW-NEXT: vmovups (%rsi), %ymm0
; AVX2-SLOW-NEXT: vmovups (%rdx), %ymm1 ; AVX2-SLOW-NEXT: vmovups (%rdx), %ymm1
; AVX2-SLOW-NEXT: vmovups (%rcx), %ymm2 ; AVX2-SLOW-NEXT: vmovups (%rcx), %ymm2
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines
; XOP-NEXT: vperm2f128 {{.*#+}} ymm4 = ymm4[2,3,2,3] ; XOP-NEXT: vperm2f128 {{.*#+}} ymm4 = ymm4[2,3,2,3]
; XOP-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm4[2],ymm2[3,4],ymm4[5],ymm2[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm4[2],ymm2[3,4],ymm4[5],ymm2[6,7]
; XOP-NEXT: vmovups (%rsi), %xmm4 ; XOP-NEXT: vmovups (%rsi), %xmm4
; XOP-NEXT: vshufps {{.*#+}} xmm5 = xmm4[2,0],xmm3[2,0] ; XOP-NEXT: vshufps {{.*#+}} xmm5 = xmm4[2,0],xmm3[2,0]
; XOP-NEXT: vshufps {{.*#+}} xmm5 = xmm3[1,1],xmm5[0,2] ; XOP-NEXT: vshufps {{.*#+}} xmm5 = xmm3[1,1],xmm5[0,2]
; XOP-NEXT: vshufps {{.*#+}} xmm3 = xmm3[0,0],xmm4[0,0] ; XOP-NEXT: vshufps {{.*#+}} xmm3 = xmm3[0,0],xmm4[0,0]
; XOP-NEXT: vshufps {{.*#+}} xmm3 = xmm3[2,0],xmm4[2,1] ; XOP-NEXT: vshufps {{.*#+}} xmm3 = xmm3[2,0],xmm4[2,1]
; XOP-NEXT: vinsertf128 $1, %xmm5, %ymm3, %ymm3 ; XOP-NEXT: vinsertf128 $1, %xmm5, %ymm3, %ymm3
; XOP-NEXT: vmovddup {{.*#+}} xmm4 = xmm1[0,0] ; XOP-NEXT: vpermilps {{.*#+}} xmm4 = mem[0,1,0,1]
; XOP-NEXT: vinsertf128 $1, %xmm4, %ymm4, %ymm4 ; XOP-NEXT: vinsertf128 $1, %xmm4, %ymm4, %ymm4
; XOP-NEXT: vblendps {{.*#+}} ymm3 = ymm3[0,1],ymm4[2],ymm3[3,4],ymm4[5],ymm3[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm3 = ymm3[0,1],ymm4[2],ymm3[3,4],ymm4[5],ymm3[6,7]
; XOP-NEXT: vpermil2ps {{.*#+}} ymm0 = ymm1[2],ymm0[3],ymm1[2,3],ymm0[4],ymm1[5,4],ymm0[5] ; XOP-NEXT: vpermil2ps {{.*#+}} ymm0 = ymm1[2],ymm0[3],ymm1[2,3],ymm0[4],ymm1[5,4],ymm0[5]
; XOP-NEXT: vpermilps {{.*#+}} ymm1 = mem[0,0,3,3,4,4,7,7] ; XOP-NEXT: vpermilps {{.*#+}} ymm1 = mem[0,0,3,3,4,4,7,7]
; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7]
; XOP-NEXT: vmovups %ymm0, 32(%rdi) ; XOP-NEXT: vmovups %ymm0, 32(%rdi)
; XOP-NEXT: vmovups %ymm3, (%rdi) ; XOP-NEXT: vmovups %ymm3, (%rdi)
; XOP-NEXT: vmovups %ymm2, 64(%rdi) ; XOP-NEXT: vmovups %ymm2, 64(%rdi)
▲ Show 20 Lines • Show All 113 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/sad.ll

Show First 20 Lines • Show All 1,432 Lines • ▼ Show 20 Lines
; AVX1-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX1-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vmovd %xmm0, %eax ; AVX1-NEXT: vmovd %xmm0, %eax
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: sad_unroll_nonzero_initial: ; AVX2-LABEL: sad_unroll_nonzero_initial:
; AVX2: # %bb.0: # %bb ; AVX2: # %bb.0: # %bb
; AVX2-NEXT: vmovdqu (%rdi), %xmm0 ; AVX2-NEXT: vmovdqu (%rdi), %xmm0
; AVX2-NEXT: vpsadbw (%rsi), %xmm0, %xmm0 ; AVX2-NEXT: vpsadbw (%rsi), %xmm0, %xmm0
; AVX2-NEXT: vmovdqu (%rdx), %xmm1
; AVX2-NEXT: vpsadbw (%rcx), %xmm1, %xmm1
; AVX2-NEXT: movl $1, %eax ; AVX2-NEXT: movl $1, %eax
; AVX2-NEXT: vmovd %eax, %xmm2 ; AVX2-NEXT: vmovd %eax, %xmm1
; AVX2-NEXT: vpaddd %ymm2, %ymm1, %ymm1 ; AVX2-NEXT: vmovdqu (%rdx), %xmm2
; AVX2-NEXT: vpaddd %ymm1, %ymm0, %ymm0 ; AVX2-NEXT: vpsadbw (%rcx), %xmm2, %xmm2
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm1 ; AVX2-NEXT: vpaddd %xmm1, %xmm2, %xmm1
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1] ; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vmovd %xmm0, %eax ; AVX2-NEXT: vmovd %xmm0, %eax
; AVX2-NEXT: vzeroupper ; AVX2-NEXT: vzeroupper
; AVX2-NEXT: retq ; AVX2-NEXT: retq
▲ Show 20 Lines • Show All 84 Lines • ▼ Show 20 Lines
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: sad_double_reduction: ; AVX2-LABEL: sad_double_reduction:
; AVX2: # %bb.0: # %bb ; AVX2: # %bb.0: # %bb
; AVX2-NEXT: vmovdqu (%rdi), %xmm0 ; AVX2-NEXT: vmovdqu (%rdi), %xmm0
; AVX2-NEXT: vmovdqu (%rdx), %xmm1 ; AVX2-NEXT: vmovdqu (%rdx), %xmm1
; AVX2-NEXT: vpsadbw (%rsi), %xmm0, %xmm0 ; AVX2-NEXT: vpsadbw (%rsi), %xmm0, %xmm0
; AVX2-NEXT: vpsadbw (%rcx), %xmm1, %xmm1 ; AVX2-NEXT: vpsadbw (%rcx), %xmm1, %xmm1
; AVX2-NEXT: vpaddd %ymm0, %ymm1, %ymm0 ; AVX2-NEXT: vpaddd %xmm0, %xmm1, %xmm0
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm1
; AVX2-NEXT: vpor %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1] ; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vmovd %xmm0, %eax ; AVX2-NEXT: vmovd %xmm0, %eax
; AVX2-NEXT: vzeroupper
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: sad_double_reduction: ; AVX512-LABEL: sad_double_reduction:
; AVX512: # %bb.0: # %bb ; AVX512: # %bb.0: # %bb
; AVX512-NEXT: vmovdqu (%rdi), %xmm0 ; AVX512-NEXT: vmovdqu (%rdi), %xmm0
; AVX512-NEXT: vmovdqu (%rdx), %xmm1 ; AVX512-NEXT: vmovdqu (%rdx), %xmm1
; AVX512-NEXT: vpsadbw (%rsi), %xmm0, %xmm0 ; AVX512-NEXT: vpsadbw (%rsi), %xmm0, %xmm0
; AVX512-NEXT: vpsadbw (%rcx), %xmm1, %xmm1 ; AVX512-NEXT: vpsadbw (%rcx), %xmm1, %xmm1
; AVX512-NEXT: vpaddd %zmm0, %zmm1, %zmm0 ; AVX512-NEXT: vpaddd %zmm0, %zmm1, %zmm0
; AVX512-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vpaddd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextracti128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextracti128 $1, %ymm0, %xmm1
; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1] ; AVX512-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; AVX512-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovd %xmm0, %eax ; AVX512-NEXT: vmovd %xmm0, %eax
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
Show All 30 Lines

llvm/trunk/test/CodeGen/X86/vector-fshl-256.ll

Show First 20 Lines • Show All 900 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Shifts ; Uniform Variable Shifts
; ;
define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i64> %amt) nounwind { define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i64> %amt) nounwind {
; AVX1-LABEL: splatvar_funnnel_v4i64: ; AVX1-LABEL: splatvar_funnnel_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm2[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm2 = xmm2[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2 ; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2
; AVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2 ; AVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; AVX1-NEXT: vpsllq %xmm2, %xmm3, %xmm3 ; AVX1-NEXT: vpsllq %xmm2, %xmm3, %xmm3
; AVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm4 ; AVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm4
; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3 ; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3
; AVX1-NEXT: vextractf128 $1, %ymm2, %xmm4 ; AVX1-NEXT: vextractf128 $1, %ymm2, %xmm4
; AVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64]
▲ Show 20 Lines • Show All 105 Lines • ▼ Show 20 Lines
; AVX512VLVBMI2-LABEL: splatvar_funnnel_v4i64: ; AVX512VLVBMI2-LABEL: splatvar_funnnel_v4i64:
; AVX512VLVBMI2: # %bb.0: ; AVX512VLVBMI2: # %bb.0:
; AVX512VLVBMI2-NEXT: vpbroadcastq %xmm2, %ymm2 ; AVX512VLVBMI2-NEXT: vpbroadcastq %xmm2, %ymm2
; AVX512VLVBMI2-NEXT: vpshldvq %ymm2, %ymm1, %ymm0 ; AVX512VLVBMI2-NEXT: vpshldvq %ymm2, %ymm1, %ymm0
; AVX512VLVBMI2-NEXT: retq ; AVX512VLVBMI2-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_funnnel_v4i64: ; XOPAVX1-LABEL: splatvar_funnnel_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm2[0,0] ; XOPAVX1-NEXT: vpermilps {{.*#+}} xmm2 = xmm2[0,1,0,1]
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2
; XOPAVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2 ; XOPAVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; XOPAVX1-NEXT: vpsllq %xmm2, %xmm3, %xmm3 ; XOPAVX1-NEXT: vpsllq %xmm2, %xmm3, %xmm3
; XOPAVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm4 ; XOPAVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm4
; XOPAVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3
; XOPAVX1-NEXT: vextractf128 $1, %ymm2, %xmm4 ; XOPAVX1-NEXT: vextractf128 $1, %ymm2, %xmm4
; XOPAVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64] ; XOPAVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64]
▲ Show 20 Lines • Show All 1,497 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/vector-fshl-rot-256.ll

Show First 20 Lines • Show All 513 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Shifts ; Uniform Variable Shifts
; ;
define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %amt) nounwind { define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %amt) nounwind {
; AVX1-LABEL: splatvar_funnnel_v4i64: ; AVX1-LABEL: splatvar_funnnel_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; AVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2 ; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm2 ; AVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm2
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [63,63] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [63,63]
; AVX1-NEXT: vpand %xmm3, %xmm2, %xmm2 ; AVX1-NEXT: vpand %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; AVX1-NEXT: vpsrlq %xmm2, %xmm4, %xmm5 ; AVX1-NEXT: vpsrlq %xmm2, %xmm4, %xmm5
; AVX1-NEXT: vpshufd {{.*#+}} xmm6 = xmm2[2,3,0,1] ; AVX1-NEXT: vpshufd {{.*#+}} xmm6 = xmm2[2,3,0,1]
; AVX1-NEXT: vpsrlq %xmm6, %xmm4, %xmm7 ; AVX1-NEXT: vpsrlq %xmm6, %xmm4, %xmm7
▲ Show 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
; AVX512VLBW-LABEL: splatvar_funnnel_v4i64: ; AVX512VLBW-LABEL: splatvar_funnnel_v4i64:
; AVX512VLBW: # %bb.0: ; AVX512VLBW: # %bb.0:
; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1 ; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1
; AVX512VLBW-NEXT: vprolvq %ymm1, %ymm0, %ymm0 ; AVX512VLBW-NEXT: vprolvq %ymm1, %ymm0, %ymm0
; AVX512VLBW-NEXT: retq ; AVX512VLBW-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_funnnel_v4i64: ; XOPAVX1-LABEL: splatvar_funnnel_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; XOPAVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq ; XOPAVX1-NEXT: retq
; ;
; XOPAVX2-LABEL: splatvar_funnnel_v4i64: ; XOPAVX2-LABEL: splatvar_funnnel_v4i64:
; XOPAVX2: # %bb.0: ; XOPAVX2: # %bb.0:
▲ Show 20 Lines • Show All 933 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/vector-fshr-256.ll

Show First 20 Lines • Show All 904 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Shifts ; Uniform Variable Shifts
; ;
define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i64> %amt) nounwind { define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i64> %amt) nounwind {
; AVX1-LABEL: splatvar_funnnel_v4i64: ; AVX1-LABEL: splatvar_funnnel_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm2[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm2 = xmm2[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2 ; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2
; AVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2 ; AVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm3 ; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm3
; AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3 ; AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3
; AVX1-NEXT: vpsrlq %xmm2, %xmm1, %xmm4 ; AVX1-NEXT: vpsrlq %xmm2, %xmm1, %xmm4
; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3 ; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3
; AVX1-NEXT: vextractf128 $1, %ymm2, %xmm4 ; AVX1-NEXT: vextractf128 $1, %ymm2, %xmm4
; AVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64]
▲ Show 20 Lines • Show All 104 Lines • ▼ Show 20 Lines
; AVX512VLVBMI2: # %bb.0: ; AVX512VLVBMI2: # %bb.0:
; AVX512VLVBMI2-NEXT: vpbroadcastq %xmm2, %ymm2 ; AVX512VLVBMI2-NEXT: vpbroadcastq %xmm2, %ymm2
; AVX512VLVBMI2-NEXT: vpshrdvq %ymm2, %ymm0, %ymm1 ; AVX512VLVBMI2-NEXT: vpshrdvq %ymm2, %ymm0, %ymm1
; AVX512VLVBMI2-NEXT: vmovdqa %ymm1, %ymm0 ; AVX512VLVBMI2-NEXT: vmovdqa %ymm1, %ymm0
; AVX512VLVBMI2-NEXT: retq ; AVX512VLVBMI2-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_funnnel_v4i64: ; XOPAVX1-LABEL: splatvar_funnnel_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm2[0,0] ; XOPAVX1-NEXT: vpermilps {{.*#+}} xmm2 = xmm2[0,1,0,1]
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2
; XOPAVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2 ; XOPAVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm1, %xmm3 ; XOPAVX1-NEXT: vextractf128 $1, %ymm1, %xmm3
; XOPAVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3 ; XOPAVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3
; XOPAVX1-NEXT: vpsrlq %xmm2, %xmm1, %xmm4 ; XOPAVX1-NEXT: vpsrlq %xmm2, %xmm1, %xmm4
; XOPAVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3
; XOPAVX1-NEXT: vextractf128 $1, %ymm2, %xmm4 ; XOPAVX1-NEXT: vextractf128 $1, %ymm2, %xmm4
; XOPAVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64] ; XOPAVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64]
▲ Show 20 Lines • Show All 1,506 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/vector-fshr-rot-256.ll

Show First 20 Lines • Show All 558 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Shifts ; Uniform Variable Shifts
; ;
define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %amt) nounwind { define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %amt) nounwind {
; AVX1-LABEL: splatvar_funnnel_v4i64: ; AVX1-LABEL: splatvar_funnnel_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; AVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2 ; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm2 ; AVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm2
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [63,63] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [63,63]
; AVX1-NEXT: vpand %xmm3, %xmm2, %xmm2 ; AVX1-NEXT: vpand %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; AVX1-NEXT: vpsllq %xmm2, %xmm4, %xmm5 ; AVX1-NEXT: vpsllq %xmm2, %xmm4, %xmm5
; AVX1-NEXT: vpshufd {{.*#+}} xmm6 = xmm2[2,3,0,1] ; AVX1-NEXT: vpshufd {{.*#+}} xmm6 = xmm2[2,3,0,1]
; AVX1-NEXT: vpsllq %xmm6, %xmm4, %xmm7 ; AVX1-NEXT: vpsllq %xmm6, %xmm4, %xmm7
▲ Show 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
; AVX512VLBW-LABEL: splatvar_funnnel_v4i64: ; AVX512VLBW-LABEL: splatvar_funnnel_v4i64:
; AVX512VLBW: # %bb.0: ; AVX512VLBW: # %bb.0:
; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1 ; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1
; AVX512VLBW-NEXT: vprorvq %ymm1, %ymm0, %ymm0 ; AVX512VLBW-NEXT: vprorvq %ymm1, %ymm0, %ymm0
; AVX512VLBW-NEXT: retq ; AVX512VLBW-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_funnnel_v4i64: ; XOPAVX1-LABEL: splatvar_funnnel_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; XOPAVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2 ; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; XOPAVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm1 ; XOPAVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq ; XOPAVX1-NEXT: retq
; ;
▲ Show 20 Lines • Show All 965 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/vector-rotate-256.ll

Show First 20 Lines • Show All 506 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Rotates ; Uniform Variable Rotates
; ;
define <4 x i64> @splatvar_rotate_v4i64(<4 x i64> %a, <4 x i64> %b) nounwind { define <4 x i64> @splatvar_rotate_v4i64(<4 x i64> %a, <4 x i64> %b) nounwind {
; AVX1-LABEL: splatvar_rotate_v4i64: ; AVX1-LABEL: splatvar_rotate_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm1[0,0] ; AVX1-NEXT: vpshufd {{.*#+}} xmm2 = xmm1[0,1,0,1]
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [64,64] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [64,64]
; AVX1-NEXT: vpsubq %xmm2, %xmm3, %xmm2 ; AVX1-NEXT: vpsubq %xmm2, %xmm3, %xmm2
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; AVX1-NEXT: vpsllq %xmm1, %xmm3, %xmm4 ; AVX1-NEXT: vpsllq %xmm1, %xmm3, %xmm4
; AVX1-NEXT: vpsllq %xmm1, %xmm0, %xmm1 ; AVX1-NEXT: vpsllq %xmm1, %xmm0, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm1, %ymm1 ; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm1, %ymm1
; AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3 ; AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3
; AVX1-NEXT: vpsrlq %xmm2, %xmm0, %xmm0 ; AVX1-NEXT: vpsrlq %xmm2, %xmm0, %xmm0
Show All 35 Lines
; AVX512VLBW-LABEL: splatvar_rotate_v4i64: ; AVX512VLBW-LABEL: splatvar_rotate_v4i64:
; AVX512VLBW: # %bb.0: ; AVX512VLBW: # %bb.0:
; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1 ; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1
; AVX512VLBW-NEXT: vprolvq %ymm1, %ymm0, %ymm0 ; AVX512VLBW-NEXT: vprolvq %ymm1, %ymm0, %ymm0
; AVX512VLBW-NEXT: retq ; AVX512VLBW-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_rotate_v4i64: ; XOPAVX1-LABEL: splatvar_rotate_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; XOPAVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq ; XOPAVX1-NEXT: retq
; ;
; XOPAVX2-LABEL: splatvar_rotate_v4i64: ; XOPAVX2-LABEL: splatvar_rotate_v4i64:
; XOPAVX2: # %bb.0: ; XOPAVX2: # %bb.0:
▲ Show 20 Lines • Show All 1,273 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/vector-shift-ashr-256.ll

Show First 20 Lines • Show All 671 Lines • ▼ Show 20 Lines
; AVX2-NEXT: vpsrlq %xmm1, %ymm2, %ymm2 ; AVX2-NEXT: vpsrlq %xmm1, %ymm2, %ymm2
; AVX2-NEXT: vpsrlq %xmm1, %ymm0, %ymm0 ; AVX2-NEXT: vpsrlq %xmm1, %ymm0, %ymm0
; AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0 ; AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0
; AVX2-NEXT: vpsubq %ymm2, %ymm0, %ymm0 ; AVX2-NEXT: vpsubq %ymm2, %ymm0, %ymm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_shift_v4i64: ; XOPAVX1-LABEL: splatvar_shift_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; XOPAVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2 ; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; XOPAVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm1 ; XOPAVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm2, %xmm2 ; XOPAVX1-NEXT: vpshaq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm0, %xmm0 ; XOPAVX1-NEXT: vpshaq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq ; XOPAVX1-NEXT: retq
; ;
▲ Show 20 Lines • Show All 979 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/vector-shuffle-256-v4.ll

Show First 20 Lines • Show All 683 Lines • ▼ Show 20 Lines
; AVX512VL-FAST-NEXT: retq ; AVX512VL-FAST-NEXT: retq
%1 = shufflevector <4 x double> %a, <4 x double> <double 0.000000e+00, double undef, double undef, double undef>, <4 x i32> <i32 1, i32 4, i32 2, i32 4> %1 = shufflevector <4 x double> %a, <4 x double> <double 0.000000e+00, double undef, double undef, double undef>, <4 x i32> <i32 1, i32 4, i32 2, i32 4>
ret <4 x double> %1 ret <4 x double> %1
} }
define <4 x i64> @shuffle_v4i64_0000(<4 x i64> %a, <4 x i64> %b) { define <4 x i64> @shuffle_v4i64_0000(<4 x i64> %a, <4 x i64> %b) {
; AVX1-LABEL: shuffle_v4i64_0000: ; AVX1-LABEL: shuffle_v4i64_0000:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm0 = xmm0[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0 ; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: shuffle_v4i64_0000: ; AVX2-LABEL: shuffle_v4i64_0000:
; AVX2: # %bb.0: ; AVX2: # %bb.0:
; AVX2-NEXT: vbroadcastsd %xmm0, %ymm0 ; AVX2-NEXT: vbroadcastsd %xmm0, %ymm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
▲ Show 20 Lines • Show All 189 Lines • ▼ Show 20 Lines
; AVX512VL-NEXT: retq ; AVX512VL-NEXT: retq
%shuffle = shufflevector <4 x i64> %a, <4 x i64> %b, <4 x i32> <i32 0, i32 2, i32 1, i32 3> %shuffle = shufflevector <4 x i64> %a, <4 x i64> %b, <4 x i32> <i32 0, i32 2, i32 1, i32 3>
ret <4 x i64> %shuffle ret <4 x i64> %shuffle
} }
define <4 x i64> @shuffle_v4i64_0124(<4 x i64> %a, <4 x i64> %b) { define <4 x i64> @shuffle_v4i64_0124(<4 x i64> %a, <4 x i64> %b) {
; AVX1-LABEL: shuffle_v4i64_0124: ; AVX1-LABEL: shuffle_v4i64_0124:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm1 ; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm1
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4,5],ymm1[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4,5],ymm1[6,7]
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: shuffle_v4i64_0124: ; AVX2-LABEL: shuffle_v4i64_0124:
; AVX2: # %bb.0: ; AVX2: # %bb.0:
; AVX2-NEXT: vbroadcastsd %xmm1, %ymm1 ; AVX2-NEXT: vbroadcastsd %xmm1, %ymm1
; AVX2-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4,5],ymm1[6,7] ; AVX2-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4,5],ymm1[6,7]
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
; AVX512VL-FAST-NEXT: retq ; AVX512VL-FAST-NEXT: retq
%shuffle = shufflevector <4 x i64> %a, <4 x i64> %b, <4 x i32> <i32 0, i32 1, i32 4, i32 2> %shuffle = shufflevector <4 x i64> %a, <4 x i64> %b, <4 x i32> <i32 0, i32 1, i32 4, i32 2>
ret <4 x i64> %shuffle ret <4 x i64> %shuffle
} }
define <4 x i64> @shuffle_v4i64_0412(<4 x i64> %a, <4 x i64> %b) { define <4 x i64> @shuffle_v4i64_0412(<4 x i64> %a, <4 x i64> %b) {
; AVX1-LABEL: shuffle_v4i64_0412: ; AVX1-LABEL: shuffle_v4i64_0412:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[0,1,0,1]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; AVX1-NEXT: vpalignr {{.*#+}} xmm2 = xmm0[8,9,10,11,12,13,14,15],xmm2[0,1,2,3,4,5,6,7] ; AVX1-NEXT: vpalignr {{.*#+}} xmm2 = xmm0[8,9,10,11,12,13,14,15],xmm2[0,1,2,3,4,5,6,7]
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2,3],ymm0[4,5,6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2,3],ymm0[4,5,6,7]
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: shuffle_v4i64_0412: ; AVX2-LABEL: shuffle_v4i64_0412:
; AVX2: # %bb.0: ; AVX2: # %bb.0:
▲ Show 20 Lines • Show All 137 Lines • ▼ Show 20 Lines; AVX512VL-FAST-NEXT: retq
ret <4 x i64> %shuffle ret <4 x i64> %shuffle
} }
define <4 x i64> @shuffle_v4i64_1251(<4 x i64> %a, <4 x i64> %b) { define <4 x i64> @shuffle_v4i64_1251(<4 x i64> %a, <4 x i64> %b) {
; AVX1-LABEL: shuffle_v4i64_1251: ; AVX1-LABEL: shuffle_v4i64_1251:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1] ; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1]
; AVX1-NEXT: vshufpd {{.*#+}} ymm0 = ymm0[1],ymm2[0],ymm0[2],ymm2[3] ; AVX1-NEXT: vshufpd {{.*#+}} ymm0 = ymm0[1],ymm2[0],ymm0[2],ymm2[3]
; AVX1-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm1 ; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm1
; AVX1-NEXT: vblendpd {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3] ; AVX1-NEXT: vblendpd {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3]
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: shuffle_v4i64_1251: ; AVX2-LABEL: shuffle_v4i64_1251:
; AVX2: # %bb.0: ; AVX2: # %bb.0:
; AVX2-NEXT: vpermpd {{.*#+}} ymm1 = ymm1[0,1,1,3] ; AVX2-NEXT: vpermpd {{.*#+}} ymm1 = ymm1[0,1,1,3]
; AVX2-NEXT: vpermpd {{.*#+}} ymm0 = ymm0[1,2,2,1] ; AVX2-NEXT: vpermpd {{.*#+}} ymm0 = ymm0[1,2,2,1]
▲ Show 20 Lines • Show All 844 Lines • Show Last 20 Lines