This is an archive of the discontinued LLVM Phabricator instance.

Differential D61806

[DAGCombiner] try to move bitcast after extract_subvector
ClosedPublic

Authored by spatel on May 10 2019, 2:58 PM.

Details

Reviewers
craig.topper
RKSimon
efriedma
Commits
rZORG933eb64941b2: [DAGCombiner] try to move bitcast after extract_subvector
rZORGd5e3f88f9d83: [DAGCombiner] try to move bitcast after extract_subvector
rG933eb64941b2: [DAGCombiner] try to move bitcast after extract_subvector
rGd5e3f88f9d83: [DAGCombiner] try to move bitcast after extract_subvector
rGa09e6868218b: [DAGCombiner] try to move bitcast after extract_subvector
rL360541: [DAGCombiner] try to move bitcast after extract_subvector
Summary

I noticed that we were failing to narrow an x86 ymm math op in a case similar to the 'madd' test diff. That is because a bitcast is sitting between the math and the extract subvector and thwarting our pattern matching for narrowing:

     t56: v8i32 = add t59, t58
    t68: v4i64 = bitcast t56
  t73: v2i64 = extract_subvector t68, Constant:i64<2>
t96: v4i32 = bitcast t73

There are a few wins and neutral diffs in the other tests.

Diff Detail

Event Timeline

spatel created this revision.May 10 2019, 2:58 PM
Herald added a project: Restricted Project. · View Herald TranscriptMay 10 2019, 2:58 PM
Herald added subscribers: hiraditya, mcrosier. · View Herald Transcript
RKSimon added inline comments.May 11 2019, 9:09 AM
llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
17595

Its typically safer to do (SrcNumElts % DestNumElts) == 0

llvm/test/CodeGen/X86/sad.ll
1452

weird - if this is all xmm code now why hasn't the vzeroupper gone?

spatel marked an inline comment as done.May 12 2019, 6:09 AM
spatel added inline comments.
llvm/test/CodeGen/X86/sad.ll
1452

There seem to be several potential missed opportunities to squash this, but we have the following heading into isel:

      t142: v4i32 = scalar_to_vector Constant:i32<1>
    t143: v8i32 = insert_subvector undef:v8i32, t142, Constant:i64<0>
  t141: v8i32 = X86ISD::VZEXT_MOVL t143
t178: v4i32 = extract_subvector t141, Constant:i64<0>
spatel updated this revision to Diff 199167.May 12 2019, 6:12 AM
spatel marked an inline comment as done.

Patch updated:
Use modulo of element counts for safety.

RKSimon accepted this revision.May 12 2019, 6:29 AM

LGTM - cheers.

llvm/test/CodeGen/X86/sad.ll
1452

Thanks for checking - that should be easy enough to fix.

This revision is now accepted and ready to land.May 12 2019, 6:29 AM
Closed by commit rL360541: [DAGCombiner] try to move bitcast after extract_subvector (authored by spatel). · Explain WhyMay 12 2019, 7:40 AM
This revision was automatically updated to reflect the committed changes.
RKSimon mentioned this in rL360543: [X86][AVX] Split VZEXT_MOVL ymm/zmm if the upper elements are not demanded..May 12 2019, 8:14 AM
RKSimon mentioned this in rGa7fc7630829b: [X86][AVX] Split VZEXT_MOVL ymm/zmm if the upper elements are not demanded..May 12 2019, 8:17 AM
sidorovd mentioned this in rG2de363410c76: [X86][AVX] Split VZEXT_MOVL ymm/zmm if the upper elements are not demanded..May 30 2019, 9:12 AM
sidorovd mentioned this in rGc8d4fc480355: [X86][AVX] Split VZEXT_MOVL ymm/zmm if the upper elements are not demanded..May 30 2019, 10:12 AM
dmgreen mentioned this in D126782: [AArch64] Look through bitcast when looking for extract_high subvector .Jun 1 2022, 7:48 AM
dmgreen mentioned this in rG963c0a014700: [AArch64] Look through bitcast when looking for extract_high subvector.Jun 12 2022, 2:59 AM

Revision Contents

PathSize
llvm/
lib/
CodeGen/
SelectionDAG/
24 lines
test/
CodeGen/
X86/
6 lines
43 lines
71 lines
17 lines
4 lines
4 lines
4 lines
4 lines
4 lines
2 lines
8 lines

Diff 199087

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 17,578 Lines • ▼ Show 20 Lines if (isNullConstant(Index) && V.getOpcode() == ISD::EXTRACT_SUBVECTOR &&
if (TLI.isExtractSubvectorCheap(NVT, V.getOperand(0).getValueType(), if (TLI.isExtractSubvectorCheap(NVT, V.getOperand(0).getValueType(),
V.getConstantOperandVal(1)) && V.getConstantOperandVal(1)) &&
TLI.isOperationLegalOrCustom(ISD::EXTRACT_SUBVECTOR, NVT)) { TLI.isOperationLegalOrCustom(ISD::EXTRACT_SUBVECTOR, NVT)) {
return DAG.getNode(ISD::EXTRACT_SUBVECTOR, SDLoc(N), NVT, V.getOperand(0), return DAG.getNode(ISD::EXTRACT_SUBVECTOR, SDLoc(N), NVT, V.getOperand(0),
V.getOperand(1)); V.getOperand(1));
} }
} }
// Try to move vector bitcast after extract_subv by scaling extraction index:
// extract_subv (bitcast X), Index --> bitcast (extract_subv X, Index')
if (isa<ConstantSDNode>(Index) && V.getOpcode() == ISD::BITCAST &&
V.getOperand(0).getValueType().isVector()) {
SDValue SrcOp = V.getOperand(0);
EVT SrcVT = SrcOp.getValueType();
unsigned SrcNumElts = SrcVT.getVectorNumElements();
unsigned DestNumElts = V.getValueType().getVectorNumElements();
if (SrcNumElts >= DestNumElts) {
RKSimonUnsubmitted
Done
ReplyInline Actions

Its typically safer to do (SrcNumElts % DestNumElts) == 0

RKSimon: Its typically safer to do (SrcNumElts % DestNumElts) == 0
unsigned SrcDestRatio = SrcNumElts / DestNumElts;
unsigned NewExtNumElts = NVT.getVectorNumElements() * SrcDestRatio;
EVT NewExtVT = EVT::getVectorVT(*DAG.getContext(), SrcVT.getScalarType(),
NewExtNumElts);
if (TLI.isOperationLegalOrCustom(ISD::EXTRACT_SUBVECTOR, NewExtVT)) {
unsigned IndexValScaled = N->getConstantOperandVal(1) * SrcDestRatio;
SDLoc DL(N);
SDValue NewIndex = DAG.getIntPtrConstant(IndexValScaled, DL);
SDValue NewExtract = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, NewExtVT,
V.getOperand(0), NewIndex);
return DAG.getBitcast(NVT, NewExtract);
}
}
}
// Combine: // Combine:
// (extract_subvec (concat V1, V2, ...), i) // (extract_subvec (concat V1, V2, ...), i)
// Into: // Into:
// Vi if possible // Vi if possible
// Only operand 0 is checked as 'concat' assumes all inputs of the same // Only operand 0 is checked as 'concat' assumes all inputs of the same
// type. // type.
if (V.getOpcode() == ISD::CONCAT_VECTORS && isa<ConstantSDNode>(Index) && if (V.getOpcode() == ISD::CONCAT_VECTORS && isa<ConstantSDNode>(Index) &&
V.getOperand(0).getValueType() == NVT) { V.getOperand(0).getValueType() == NVT) {
▲ Show 20 Lines • Show All 2,538 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/extractelement-fp.ll

Show All 20 Lines; X86-NEXT: retl
%v = fneg <4 x float> %x %v = fneg <4 x float> %x
%r = extractelement <4 x float> %v, i32 0 %r = extractelement <4 x float> %v, i32 0
ret float %r ret float %r
} }
define double @fneg_v4f64(<4 x double> %x) nounwind { define double @fneg_v4f64(<4 x double> %x) nounwind {
; X64-LABEL: fneg_v4f64: ; X64-LABEL: fneg_v4f64:
; X64: # %bb.0: ; X64: # %bb.0:
; X64-NEXT: vbroadcastsd {{.*#+}} ymm1 = [-0.0E+0,-0.0E+0,-0.0E+0,-0.0E+0] ; X64-NEXT: vmovddup {{.*#+}} xmm1 = [-0.0E+0,-0.0E+0]
; X64-NEXT: # xmm1 = mem[0,0]
; X64-NEXT: vxorps %xmm1, %xmm0, %xmm0 ; X64-NEXT: vxorps %xmm1, %xmm0, %xmm0
; X64-NEXT: vzeroupper ; X64-NEXT: vzeroupper
; X64-NEXT: retq ; X64-NEXT: retq
; ;
; X86-LABEL: fneg_v4f64: ; X86-LABEL: fneg_v4f64:
; X86: # %bb.0: ; X86: # %bb.0:
; X86-NEXT: pushl %ebp ; X86-NEXT: pushl %ebp
; X86-NEXT: movl %esp, %ebp ; X86-NEXT: movl %esp, %ebp
; X86-NEXT: andl $-8, %esp ; X86-NEXT: andl $-8, %esp
; X86-NEXT: subl $8, %esp ; X86-NEXT: subl $8, %esp
; X86-NEXT: vbroadcastsd {{.*#+}} ymm1 = [-0.0E+0,-0.0E+0,-0.0E+0,-0.0E+0] ; X86-NEXT: vmovddup {{.*#+}} xmm1 = [-0.0E+0,-0.0E+0]
; X86-NEXT: # xmm1 = mem[0,0]
; X86-NEXT: vxorps %xmm1, %xmm0, %xmm0 ; X86-NEXT: vxorps %xmm1, %xmm0, %xmm0
; X86-NEXT: vmovlps %xmm0, (%esp) ; X86-NEXT: vmovlps %xmm0, (%esp)
; X86-NEXT: fldl (%esp) ; X86-NEXT: fldl (%esp)
; X86-NEXT: movl %ebp, %esp ; X86-NEXT: movl %ebp, %esp
; X86-NEXT: popl %ebp ; X86-NEXT: popl %ebp
; X86-NEXT: vzeroupper ; X86-NEXT: vzeroupper
; X86-NEXT: retl ; X86-NEXT: retl
%v = fneg <4 x double> %x %v = fneg <4 x double> %x
▲ Show 20 Lines • Show All 1,170 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/madd.ll

Show First 20 Lines • Show All 2,667 Lines • ▼ Show 20 Lines
; SSE2-NEXT: paddd %xmm1, %xmm2 ; SSE2-NEXT: paddd %xmm1, %xmm2
; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm2[2,3,0,1] ; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm2[2,3,0,1]
; SSE2-NEXT: paddd %xmm2, %xmm0 ; SSE2-NEXT: paddd %xmm2, %xmm0
; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; SSE2-NEXT: paddd %xmm0, %xmm1 ; SSE2-NEXT: paddd %xmm0, %xmm1
; SSE2-NEXT: movd %xmm1, %eax ; SSE2-NEXT: movd %xmm1, %eax
; SSE2-NEXT: retq ; SSE2-NEXT: retq
; ;
; AVX1-LABEL: madd_double_reduction: ; AVX-LABEL: madd_double_reduction:
; AVX1: # %bb.0: ; AVX: # %bb.0:
; AVX1-NEXT: vmovdqu (%rdi), %xmm0 ; AVX-NEXT: vmovdqu (%rdi), %xmm0
; AVX1-NEXT: vmovdqu (%rdx), %xmm1 ; AVX-NEXT: vmovdqu (%rdx), %xmm1
; AVX1-NEXT: vpmaddwd (%rsi), %xmm0, %xmm0 ; AVX-NEXT: vpmaddwd (%rsi), %xmm0, %xmm0
; AVX1-NEXT: vpmaddwd (%rcx), %xmm1, %xmm1 ; AVX-NEXT: vpmaddwd (%rcx), %xmm1, %xmm1
; AVX1-NEXT: vpaddd %xmm0, %xmm1, %xmm0 ; AVX-NEXT: vpaddd %xmm0, %xmm1, %xmm0
; AVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1] ; AVX-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX1-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; AVX-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX1-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vmovd %xmm0, %eax ; AVX-NEXT: vmovd %xmm0, %eax
; AVX1-NEXT: retq ; AVX-NEXT: retq
;
; AVX256-LABEL: madd_double_reduction:
; AVX256: # %bb.0:
; AVX256-NEXT: vmovdqu (%rdi), %xmm0
; AVX256-NEXT: vmovdqu (%rdx), %xmm1
; AVX256-NEXT: vpmaddwd (%rsi), %xmm0, %xmm0
; AVX256-NEXT: vpmaddwd (%rcx), %xmm1, %xmm1
; AVX256-NEXT: vpaddd %ymm0, %ymm1, %ymm0
; AVX256-NEXT: vextracti128 $1, %ymm0, %xmm1
; AVX256-NEXT: vpor %xmm1, %xmm0, %xmm0
; AVX256-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX256-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX256-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX256-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX256-NEXT: vmovd %xmm0, %eax
; AVX256-NEXT: vzeroupper
; AVX256-NEXT: retq
%tmp = load <8 x i16>, <8 x i16>* %arg, align 1 %tmp = load <8 x i16>, <8 x i16>* %arg, align 1
%tmp6 = load <8 x i16>, <8 x i16>* %arg1, align 1 %tmp6 = load <8 x i16>, <8 x i16>* %arg1, align 1
%tmp7 = sext <8 x i16> %tmp to <8 x i32> %tmp7 = sext <8 x i16> %tmp to <8 x i32>
%tmp17 = sext <8 x i16> %tmp6 to <8 x i32> %tmp17 = sext <8 x i16> %tmp6 to <8 x i32>
%tmp19 = mul nsw <8 x i32> %tmp7, %tmp17 %tmp19 = mul nsw <8 x i32> %tmp7, %tmp17
%tmp20 = load <8 x i16>, <8 x i16>* %arg2, align 1 %tmp20 = load <8 x i16>, <8 x i16>* %arg2, align 1
%tmp21 = load <8 x i16>, <8 x i16>* %arg3, align 1 %tmp21 = load <8 x i16>, <8 x i16>* %arg3, align 1
%tmp22 = sext <8 x i16> %tmp20 to <8 x i32> %tmp22 = sext <8 x i16> %tmp20 to <8 x i32>
Show All 12 Lines

llvm/test/CodeGen/X86/oddshuffles.ll

Show First 20 Lines • Show All 588 Lines • ▼ Show 20 Lines
; SSE42-NEXT: retq ; SSE42-NEXT: retq
; ;
; AVX1-LABEL: v12i32: ; AVX1-LABEL: v12i32:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1] ; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1]
; AVX1-NEXT: vmovsldup {{.*#+}} ymm2 = ymm2[0,0,2,2,4,4,6,6] ; AVX1-NEXT: vmovsldup {{.*#+}} ymm2 = ymm2[0,0,2,2,4,4,6,6]
; AVX1-NEXT: vpermilps {{.*#+}} ymm3 = ymm0[0,u,u,1,5,u,u,6] ; AVX1-NEXT: vpermilps {{.*#+}} ymm3 = ymm0[0,u,u,1,5,u,u,6]
; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm3[0],ymm2[1],ymm3[2,3,4,5],ymm2[6],ymm3[7] ; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm3[0],ymm2[1],ymm3[2,3,4,5],ymm2[6],ymm3[7]
; AVX1-NEXT: vmovddup {{.*#+}} xmm3 = xmm1[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm3 = xmm1[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3 ; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3
; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; AVX1-NEXT: vshufps {{.*#+}} xmm0 = xmm0[0,3],xmm3[3,3] ; AVX1-NEXT: vshufps {{.*#+}} xmm0 = xmm0[0,3],xmm3[3,3]
; AVX1-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,1] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,2,3]
; AVX1-NEXT: vblendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2],xmm1[3] ; AVX1-NEXT: vblendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2],xmm1[3]
; AVX1-NEXT: vmovaps %xmm0, 32(%rdi) ; AVX1-NEXT: vmovaps %xmm0, 32(%rdi)
; AVX1-NEXT: vmovaps %ymm2, (%rdi) ; AVX1-NEXT: vmovaps %ymm2, (%rdi)
; AVX1-NEXT: vzeroupper ; AVX1-NEXT: vzeroupper
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-SLOW-LABEL: v12i32: ; AVX2-SLOW-LABEL: v12i32:
; AVX2-SLOW: # %bb.0: ; AVX2-SLOW: # %bb.0:
Show All 24 Lines
; AVX2-FAST-NEXT: vmovaps %ymm2, (%rdi) ; AVX2-FAST-NEXT: vmovaps %ymm2, (%rdi)
; AVX2-FAST-NEXT: vzeroupper ; AVX2-FAST-NEXT: vzeroupper
; AVX2-FAST-NEXT: retq ; AVX2-FAST-NEXT: retq
; ;
; XOP-LABEL: v12i32: ; XOP-LABEL: v12i32:
; XOP: # %bb.0: ; XOP: # %bb.0:
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1] ; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1]
; XOP-NEXT: vpermil2ps {{.*#+}} ymm2 = ymm0[0],ymm2[0],ymm0[u,1,5,u],ymm2[6],ymm0[6] ; XOP-NEXT: vpermil2ps {{.*#+}} ymm2 = ymm0[0],ymm2[0],ymm0[u,1,5,u],ymm2[6],ymm0[6]
; XOP-NEXT: vmovddup {{.*#+}} xmm3 = xmm1[0,0] ; XOP-NEXT: vpermilps {{.*#+}} xmm3 = xmm1[0,1,0,1]
; XOP-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3 ; XOP-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3
; XOP-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7]
; XOP-NEXT: vextractf128 $1, %ymm0, %xmm3 ; XOP-NEXT: vextractf128 $1, %ymm0, %xmm3
; XOP-NEXT: vshufps {{.*#+}} xmm0 = xmm0[0,3],xmm3[3,3] ; XOP-NEXT: vshufps {{.*#+}} xmm0 = xmm0[0,3],xmm3[3,3]
; XOP-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,1] ; XOP-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,2,3]
; XOP-NEXT: vblendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2],xmm1[3] ; XOP-NEXT: vblendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2],xmm1[3]
; XOP-NEXT: vmovaps %xmm0, 32(%rdi) ; XOP-NEXT: vmovaps %xmm0, 32(%rdi)
; XOP-NEXT: vmovaps %ymm2, (%rdi) ; XOP-NEXT: vmovaps %ymm2, (%rdi)
; XOP-NEXT: vzeroupper ; XOP-NEXT: vzeroupper
; XOP-NEXT: retq ; XOP-NEXT: retq
%r = shufflevector <8 x i32> %a, <8 x i32> %b, <12 x i32> <i32 0, i32 4, i32 8, i32 1, i32 5, i32 9, i32 2, i32 6, i32 10, i32 3, i32 7, i32 11> %r = shufflevector <8 x i32> %a, <8 x i32> %b, <12 x i32> <i32 0, i32 4, i32 8, i32 1, i32 5, i32 9, i32 2, i32 6, i32 10, i32 3, i32 7, i32 11>
store <12 x i32> %r, <12 x i32>* %p store <12 x i32> %r, <12 x i32>* %p
ret void ret void
▲ Show 20 Lines • Show All 694 Lines • ▼ Show 20 Lines
; AVX1-NEXT: vblendps {{.*#+}} ymm4 = ymm5[0,1,2,3,4,5],ymm4[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm4 = ymm5[0,1,2,3,4,5],ymm4[6,7]
; AVX1-NEXT: vblendps {{.*#+}} xmm5 = xmm3[0,1],xmm2[2],xmm3[3] ; AVX1-NEXT: vblendps {{.*#+}} xmm5 = xmm3[0,1],xmm2[2],xmm3[3]
; AVX1-NEXT: vpermilps {{.*#+}} xmm5 = xmm5[0,0,3,2] ; AVX1-NEXT: vpermilps {{.*#+}} xmm5 = xmm5[0,0,3,2]
; AVX1-NEXT: vinsertf128 $1, %xmm5, %ymm0, %ymm5 ; AVX1-NEXT: vinsertf128 $1, %xmm5, %ymm0, %ymm5
; AVX1-NEXT: vblendps {{.*#+}} ymm6 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm6 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7]
; AVX1-NEXT: vextractf128 $1, %ymm6, %xmm7 ; AVX1-NEXT: vextractf128 $1, %ymm6, %xmm7
; AVX1-NEXT: vblendps {{.*#+}} xmm6 = xmm7[0],xmm6[1,2],xmm7[3] ; AVX1-NEXT: vblendps {{.*#+}} xmm6 = xmm7[0],xmm6[1,2],xmm7[3]
; AVX1-NEXT: vpermilps {{.*#+}} xmm6 = xmm6[1,0,3,2] ; AVX1-NEXT: vpermilps {{.*#+}} xmm6 = xmm6[1,0,3,2]
; AVX1-NEXT: vmovshdup {{.*#+}} xmm7 = xmm7[1,1,3,3] ; AVX1-NEXT: vpermilps {{.*#+}} xmm7 = xmm7[1,1,3,3]
; AVX1-NEXT: vinsertf128 $1, %xmm7, %ymm6, %ymm6 ; AVX1-NEXT: vinsertf128 $1, %xmm7, %ymm6, %ymm6
; AVX1-NEXT: vblendps {{.*#+}} ymm5 = ymm6[0,1,2,3,4],ymm5[5,6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm5 = ymm6[0,1,2,3,4],ymm5[5,6,7]
; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm3[0,1],xmm2[0,3] ; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm3[0,1],xmm2[0,3]
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm2 ; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm2
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0,1],ymm0[2],ymm1[3,4],ymm0[5],ymm1[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0,1],ymm0[2],ymm1[3,4],ymm0[5],ymm1[6,7]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX1-NEXT: vblendps {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2,3] ; AVX1-NEXT: vblendps {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2,3]
; AVX1-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[2,1,0,3] ; AVX1-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[2,1,0,3]
; AVX1-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0 ; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4],ymm2[5,6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4],ymm2[5,6,7]
; AVX1-NEXT: vmovups %ymm4, (%rsi) ; AVX1-NEXT: vmovups %ymm4, (%rsi)
; AVX1-NEXT: vmovups %ymm5, (%rdx) ; AVX1-NEXT: vmovups %ymm5, (%rdx)
; AVX1-NEXT: vmovups %ymm0, (%rcx) ; AVX1-NEXT: vmovups %ymm0, (%rcx)
; AVX1-NEXT: vzeroupper ; AVX1-NEXT: vzeroupper
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
▲ Show 20 Lines • Show All 74 Lines • ▼ Show 20 Lines
; XOP-NEXT: vblendps {{.*#+}} ymm4 = ymm5[0,1,2,3,4,5],ymm4[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm4 = ymm5[0,1,2,3,4,5],ymm4[6,7]
; XOP-NEXT: vblendps {{.*#+}} xmm5 = xmm3[0,1],xmm2[2],xmm3[3] ; XOP-NEXT: vblendps {{.*#+}} xmm5 = xmm3[0,1],xmm2[2],xmm3[3]
; XOP-NEXT: vpermilps {{.*#+}} xmm5 = xmm5[0,0,3,2] ; XOP-NEXT: vpermilps {{.*#+}} xmm5 = xmm5[0,0,3,2]
; XOP-NEXT: vinsertf128 $1, %xmm5, %ymm0, %ymm5 ; XOP-NEXT: vinsertf128 $1, %xmm5, %ymm0, %ymm5
; XOP-NEXT: vblendps {{.*#+}} ymm6 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm6 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7]
; XOP-NEXT: vextractf128 $1, %ymm6, %xmm7 ; XOP-NEXT: vextractf128 $1, %ymm6, %xmm7
; XOP-NEXT: vblendps {{.*#+}} xmm6 = xmm7[0],xmm6[1,2],xmm7[3] ; XOP-NEXT: vblendps {{.*#+}} xmm6 = xmm7[0],xmm6[1,2],xmm7[3]
; XOP-NEXT: vpermilps {{.*#+}} xmm6 = xmm6[1,0,3,2] ; XOP-NEXT: vpermilps {{.*#+}} xmm6 = xmm6[1,0,3,2]
; XOP-NEXT: vmovshdup {{.*#+}} xmm7 = xmm7[1,1,3,3] ; XOP-NEXT: vpermilps {{.*#+}} xmm7 = xmm7[1,1,3,3]
; XOP-NEXT: vinsertf128 $1, %xmm7, %ymm6, %ymm6 ; XOP-NEXT: vinsertf128 $1, %xmm7, %ymm6, %ymm6
; XOP-NEXT: vblendps {{.*#+}} ymm5 = ymm6[0,1,2,3,4],ymm5[5,6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm5 = ymm6[0,1,2,3,4],ymm5[5,6,7]
; XOP-NEXT: vshufps {{.*#+}} xmm2 = xmm3[0,1],xmm2[0,3] ; XOP-NEXT: vshufps {{.*#+}} xmm2 = xmm3[0,1],xmm2[0,3]
; XOP-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm2 ; XOP-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm2
; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0,1],ymm0[2],ymm1[3,4],ymm0[5],ymm1[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0,1],ymm0[2],ymm1[3,4],ymm0[5],ymm1[6,7]
; XOP-NEXT: vextractf128 $1, %ymm0, %xmm1 ; XOP-NEXT: vextractf128 $1, %ymm0, %xmm1
; XOP-NEXT: vblendps {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2,3] ; XOP-NEXT: vblendps {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2,3]
; XOP-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[2,1,0,3] ; XOP-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[2,1,0,3]
; XOP-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; XOP-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,0,1]
; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0 ; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4],ymm2[5,6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4],ymm2[5,6,7]
; XOP-NEXT: vmovups %ymm4, (%rsi) ; XOP-NEXT: vmovups %ymm4, (%rsi)
; XOP-NEXT: vmovups %ymm5, (%rdx) ; XOP-NEXT: vmovups %ymm5, (%rdx)
; XOP-NEXT: vmovups %ymm0, (%rcx) ; XOP-NEXT: vmovups %ymm0, (%rcx)
; XOP-NEXT: vzeroupper ; XOP-NEXT: vzeroupper
; XOP-NEXT: retq ; XOP-NEXT: retq
%wide.vec = load <24 x i32>, <24 x i32>* %p, align 4 %wide.vec = load <24 x i32>, <24 x i32>* %p, align 4
▲ Show 20 Lines • Show All 89 Lines • ▼ Show 20 Lines
; SSE42-NEXT: movdqu %xmm0, 64(%rdi) ; SSE42-NEXT: movdqu %xmm0, 64(%rdi)
; SSE42-NEXT: movdqu %xmm7, 80(%rdi) ; SSE42-NEXT: movdqu %xmm7, 80(%rdi)
; SSE42-NEXT: movdqu %xmm1, (%rdi) ; SSE42-NEXT: movdqu %xmm1, (%rdi)
; SSE42-NEXT: retq ; SSE42-NEXT: retq
; ;
; AVX1-LABEL: interleave_24i32_in: ; AVX1-LABEL: interleave_24i32_in:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovupd (%rsi), %ymm0 ; AVX1-NEXT: vmovupd (%rsi), %ymm0
; AVX1-NEXT: vmovupd (%rcx), %ymm1 ; AVX1-NEXT: vmovups 16(%rcx), %xmm1
; AVX1-NEXT: vmovups 16(%rcx), %xmm2 ; AVX1-NEXT: vmovups (%rdx), %xmm2
; AVX1-NEXT: vmovups (%rdx), %xmm3 ; AVX1-NEXT: vmovups 16(%rdx), %xmm3
; AVX1-NEXT: vmovups 16(%rdx), %xmm4 ; AVX1-NEXT: vshufps {{.*#+}} xmm4 = xmm3[3,0],xmm1[3,0]
; AVX1-NEXT: vshufps {{.*#+}} xmm5 = xmm4[3,0],xmm2[3,0] ; AVX1-NEXT: vshufps {{.*#+}} xmm4 = xmm1[2,1],xmm4[0,2]
; AVX1-NEXT: vshufps {{.*#+}} xmm5 = xmm2[2,1],xmm5[0,2] ; AVX1-NEXT: vshufps {{.*#+}} xmm1 = xmm1[1,0],xmm3[1,0]
; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm2[1,0],xmm4[1,0] ; AVX1-NEXT: vshufps {{.*#+}} xmm1 = xmm1[2,0],xmm3[2,2]
; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm2[2,0],xmm4[2,2] ; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm1, %ymm1
; AVX1-NEXT: vinsertf128 $1, %xmm5, %ymm2, %ymm2 ; AVX1-NEXT: vpermilpd {{.*#+}} ymm3 = ymm0[1,1,3,3]
; AVX1-NEXT: vpermilpd {{.*#+}} ymm4 = ymm0[1,1,3,3] ; AVX1-NEXT: vperm2f128 {{.*#+}} ymm3 = ymm3[2,3,2,3]
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm4 = ymm4[2,3,2,3] ; AVX1-NEXT: vblendps {{.*#+}} ymm1 = ymm1[0,1],ymm3[2],ymm1[3,4],ymm3[5],ymm1[6,7]
; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm4[2],ymm2[3,4],ymm4[5],ymm2[6,7] ; AVX1-NEXT: vmovups (%rsi), %xmm3
; AVX1-NEXT: vmovups (%rsi), %xmm4 ; AVX1-NEXT: vshufps {{.*#+}} xmm4 = xmm3[2,0],xmm2[2,0]
; AVX1-NEXT: vshufps {{.*#+}} xmm5 = xmm4[2,0],xmm3[2,0] ; AVX1-NEXT: vshufps {{.*#+}} xmm4 = xmm2[1,1],xmm4[0,2]
; AVX1-NEXT: vshufps {{.*#+}} xmm5 = xmm3[1,1],xmm5[0,2] ; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm2[0,0],xmm3[0,0]
; AVX1-NEXT: vshufps {{.*#+}} xmm3 = xmm3[0,0],xmm4[0,0] ; AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm2[2,0],xmm3[2,1]
; AVX1-NEXT: vshufps {{.*#+}} xmm3 = xmm3[2,0],xmm4[2,1] ; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm2, %ymm2
; AVX1-NEXT: vinsertf128 $1, %xmm5, %ymm3, %ymm3 ; AVX1-NEXT: vpermilps {{.*#+}} xmm3 = mem[0,1,0,1]
; AVX1-NEXT: vmovddup {{.*#+}} xmm4 = xmm1[0,0] ; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm3, %ymm3
; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm4, %ymm4 ; AVX1-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm3[2],ymm2[3,4],ymm3[5],ymm2[6,7]
; AVX1-NEXT: vblendps {{.*#+}} ymm3 = ymm3[0,1],ymm4[2],ymm3[3,4],ymm4[5],ymm3[6,7]
; AVX1-NEXT: vpermilpd {{.*#+}} ymm0 = ymm0[1,0,2,2] ; AVX1-NEXT: vpermilpd {{.*#+}} ymm0 = ymm0[1,0,2,2]
; AVX1-NEXT: vpermilpd {{.*#+}} ymm1 = ymm1[1,1,2,2] ; AVX1-NEXT: vpermilpd {{.*#+}} ymm3 = mem[1,1,2,2]
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0],ymm0[1],ymm1[2,3],ymm0[4],ymm1[5,6],ymm0[7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm3[0],ymm0[1],ymm3[2,3],ymm0[4],ymm3[5,6],ymm0[7]
; AVX1-NEXT: vpermilps {{.*#+}} ymm1 = mem[0,0,3,3,4,4,7,7] ; AVX1-NEXT: vpermilps {{.*#+}} ymm3 = mem[0,0,3,3,4,4,7,7]
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm3[2],ymm0[3,4],ymm3[5],ymm0[6,7]
; AVX1-NEXT: vmovups %ymm0, 32(%rdi) ; AVX1-NEXT: vmovups %ymm0, 32(%rdi)
; AVX1-NEXT: vmovups %ymm3, (%rdi) ; AVX1-NEXT: vmovups %ymm2, (%rdi)
; AVX1-NEXT: vmovups %ymm2, 64(%rdi) ; AVX1-NEXT: vmovups %ymm1, 64(%rdi)
; AVX1-NEXT: vzeroupper ; AVX1-NEXT: vzeroupper
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-SLOW-LABEL: interleave_24i32_in: ; AVX2-SLOW-LABEL: interleave_24i32_in:
; AVX2-SLOW: # %bb.0: ; AVX2-SLOW: # %bb.0:
; AVX2-SLOW-NEXT: vmovups (%rsi), %ymm0 ; AVX2-SLOW-NEXT: vmovups (%rsi), %ymm0
; AVX2-SLOW-NEXT: vmovups (%rdx), %ymm1 ; AVX2-SLOW-NEXT: vmovups (%rdx), %ymm1
; AVX2-SLOW-NEXT: vmovups (%rcx), %ymm2 ; AVX2-SLOW-NEXT: vmovups (%rcx), %ymm2
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines
; XOP-NEXT: vperm2f128 {{.*#+}} ymm4 = ymm4[2,3,2,3] ; XOP-NEXT: vperm2f128 {{.*#+}} ymm4 = ymm4[2,3,2,3]
; XOP-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm4[2],ymm2[3,4],ymm4[5],ymm2[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm2 = ymm2[0,1],ymm4[2],ymm2[3,4],ymm4[5],ymm2[6,7]
; XOP-NEXT: vmovups (%rsi), %xmm4 ; XOP-NEXT: vmovups (%rsi), %xmm4
; XOP-NEXT: vshufps {{.*#+}} xmm5 = xmm4[2,0],xmm3[2,0] ; XOP-NEXT: vshufps {{.*#+}} xmm5 = xmm4[2,0],xmm3[2,0]
; XOP-NEXT: vshufps {{.*#+}} xmm5 = xmm3[1,1],xmm5[0,2] ; XOP-NEXT: vshufps {{.*#+}} xmm5 = xmm3[1,1],xmm5[0,2]
; XOP-NEXT: vshufps {{.*#+}} xmm3 = xmm3[0,0],xmm4[0,0] ; XOP-NEXT: vshufps {{.*#+}} xmm3 = xmm3[0,0],xmm4[0,0]
; XOP-NEXT: vshufps {{.*#+}} xmm3 = xmm3[2,0],xmm4[2,1] ; XOP-NEXT: vshufps {{.*#+}} xmm3 = xmm3[2,0],xmm4[2,1]
; XOP-NEXT: vinsertf128 $1, %xmm5, %ymm3, %ymm3 ; XOP-NEXT: vinsertf128 $1, %xmm5, %ymm3, %ymm3
; XOP-NEXT: vmovddup {{.*#+}} xmm4 = xmm1[0,0] ; XOP-NEXT: vpermilps {{.*#+}} xmm4 = mem[0,1,0,1]
; XOP-NEXT: vinsertf128 $1, %xmm4, %ymm4, %ymm4 ; XOP-NEXT: vinsertf128 $1, %xmm4, %ymm4, %ymm4
; XOP-NEXT: vblendps {{.*#+}} ymm3 = ymm3[0,1],ymm4[2],ymm3[3,4],ymm4[5],ymm3[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm3 = ymm3[0,1],ymm4[2],ymm3[3,4],ymm4[5],ymm3[6,7]
; XOP-NEXT: vpermil2ps {{.*#+}} ymm0 = ymm1[2],ymm0[3],ymm1[2,3],ymm0[4],ymm1[5,4],ymm0[5] ; XOP-NEXT: vpermil2ps {{.*#+}} ymm0 = ymm1[2],ymm0[3],ymm1[2,3],ymm0[4],ymm1[5,4],ymm0[5]
; XOP-NEXT: vpermilps {{.*#+}} ymm1 = mem[0,0,3,3,4,4,7,7] ; XOP-NEXT: vpermilps {{.*#+}} ymm1 = mem[0,0,3,3,4,4,7,7]
; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7] ; XOP-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3,4],ymm1[5],ymm0[6,7]
; XOP-NEXT: vmovups %ymm0, 32(%rdi) ; XOP-NEXT: vmovups %ymm0, 32(%rdi)
; XOP-NEXT: vmovups %ymm3, (%rdi) ; XOP-NEXT: vmovups %ymm3, (%rdi)
; XOP-NEXT: vmovups %ymm2, 64(%rdi) ; XOP-NEXT: vmovups %ymm2, 64(%rdi)
▲ Show 20 Lines • Show All 113 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/sad.ll

Show First 20 Lines • Show All 1,432 Lines • ▼ Show 20 Lines
; AVX1-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX1-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vmovd %xmm0, %eax ; AVX1-NEXT: vmovd %xmm0, %eax
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: sad_unroll_nonzero_initial: ; AVX2-LABEL: sad_unroll_nonzero_initial:
; AVX2: # %bb.0: # %bb ; AVX2: # %bb.0: # %bb
; AVX2-NEXT: vmovdqu (%rdi), %xmm0 ; AVX2-NEXT: vmovdqu (%rdi), %xmm0
; AVX2-NEXT: vpsadbw (%rsi), %xmm0, %xmm0 ; AVX2-NEXT: vpsadbw (%rsi), %xmm0, %xmm0
; AVX2-NEXT: vmovdqu (%rdx), %xmm1
; AVX2-NEXT: vpsadbw (%rcx), %xmm1, %xmm1
; AVX2-NEXT: movl $1, %eax ; AVX2-NEXT: movl $1, %eax
; AVX2-NEXT: vmovd %eax, %xmm2 ; AVX2-NEXT: vmovd %eax, %xmm1
; AVX2-NEXT: vpaddd %ymm2, %ymm1, %ymm1 ; AVX2-NEXT: vmovdqu (%rdx), %xmm2
; AVX2-NEXT: vpaddd %ymm1, %ymm0, %ymm0 ; AVX2-NEXT: vpsadbw (%rcx), %xmm2, %xmm2
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm1 ; AVX2-NEXT: vpaddd %xmm1, %xmm2, %xmm1
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1] ; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vmovd %xmm0, %eax ; AVX2-NEXT: vmovd %xmm0, %eax
; AVX2-NEXT: vzeroupper ; AVX2-NEXT: vzeroupper
RKSimonUnsubmitted
Not Done
ReplyInline Actions

weird - if this is all xmm code now why hasn't the vzeroupper gone?

RKSimon: weird - if this is all xmm code now why hasn't the vzeroupper gone?
spatelAuthorUnsubmitted
Done
ReplyInline Actions

There seem to be several potential missed opportunities to squash this, but we have the following heading into isel:

      t142: v4i32 = scalar_to_vector Constant:i32<1>
    t143: v8i32 = insert_subvector undef:v8i32, t142, Constant:i64<0>
  t141: v8i32 = X86ISD::VZEXT_MOVL t143
t178: v4i32 = extract_subvector t141, Constant:i64<0>
spatel: There seem to be several potential missed opportunities to squash this, but we have the…
RKSimonUnsubmitted
Not Done
ReplyInline Actions

Thanks for checking - that should be easy enough to fix.

RKSimon: Thanks for checking - that should be easy enough to fix.
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: sad_unroll_nonzero_initial: ; AVX512-LABEL: sad_unroll_nonzero_initial:
; AVX512: # %bb.0: # %bb ; AVX512: # %bb.0: # %bb
; AVX512-NEXT: vmovdqu (%rdi), %xmm0 ; AVX512-NEXT: vmovdqu (%rdi), %xmm0
; AVX512-NEXT: vpsadbw (%rsi), %xmm0, %xmm0 ; AVX512-NEXT: vpsadbw (%rsi), %xmm0, %xmm0
; AVX512-NEXT: vmovdqu (%rdx), %xmm1 ; AVX512-NEXT: vmovdqu (%rdx), %xmm1
; AVX512-NEXT: vpsadbw (%rcx), %xmm1, %xmm1 ; AVX512-NEXT: vpsadbw (%rcx), %xmm1, %xmm1
▲ Show 20 Lines • Show All 77 Lines • ▼ Show 20 Lines
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: sad_double_reduction: ; AVX2-LABEL: sad_double_reduction:
; AVX2: # %bb.0: # %bb ; AVX2: # %bb.0: # %bb
; AVX2-NEXT: vmovdqu (%rdi), %xmm0 ; AVX2-NEXT: vmovdqu (%rdi), %xmm0
; AVX2-NEXT: vmovdqu (%rdx), %xmm1 ; AVX2-NEXT: vmovdqu (%rdx), %xmm1
; AVX2-NEXT: vpsadbw (%rsi), %xmm0, %xmm0 ; AVX2-NEXT: vpsadbw (%rsi), %xmm0, %xmm0
; AVX2-NEXT: vpsadbw (%rcx), %xmm1, %xmm1 ; AVX2-NEXT: vpsadbw (%rcx), %xmm1, %xmm1
; AVX2-NEXT: vpaddd %ymm0, %ymm1, %ymm0 ; AVX2-NEXT: vpaddd %xmm0, %xmm1, %xmm0
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm1
; AVX2-NEXT: vpor %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1] ; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vmovd %xmm0, %eax ; AVX2-NEXT: vmovd %xmm0, %eax
; AVX2-NEXT: vzeroupper
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: sad_double_reduction: ; AVX512-LABEL: sad_double_reduction:
; AVX512: # %bb.0: # %bb ; AVX512: # %bb.0: # %bb
; AVX512-NEXT: vmovdqu (%rdi), %xmm0 ; AVX512-NEXT: vmovdqu (%rdi), %xmm0
; AVX512-NEXT: vmovdqu (%rdx), %xmm1 ; AVX512-NEXT: vmovdqu (%rdx), %xmm1
; AVX512-NEXT: vpsadbw (%rsi), %xmm0, %xmm0 ; AVX512-NEXT: vpsadbw (%rsi), %xmm0, %xmm0
; AVX512-NEXT: vpsadbw (%rcx), %xmm1, %xmm1 ; AVX512-NEXT: vpsadbw (%rcx), %xmm1, %xmm1
; AVX512-NEXT: vpaddd %zmm0, %zmm1, %zmm0 ; AVX512-NEXT: vpaddd %zmm0, %zmm1, %zmm0
; AVX512-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vpaddd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextracti128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextracti128 $1, %ymm0, %xmm1
; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1] ; AVX512-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3] ; AVX512-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[1,1,2,3]
; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vpaddd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovd %xmm0, %eax ; AVX512-NEXT: vmovd %xmm0, %eax
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
Show All 30 Lines

llvm/test/CodeGen/X86/vector-fshl-256.ll

Show First 20 Lines • Show All 900 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Shifts ; Uniform Variable Shifts
; ;
define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i64> %amt) nounwind { define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i64> %amt) nounwind {
; AVX1-LABEL: splatvar_funnnel_v4i64: ; AVX1-LABEL: splatvar_funnnel_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm2[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm2 = xmm2[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2 ; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2
; AVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2 ; AVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; AVX1-NEXT: vpsllq %xmm2, %xmm3, %xmm3 ; AVX1-NEXT: vpsllq %xmm2, %xmm3, %xmm3
; AVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm4 ; AVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm4
; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3 ; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3
; AVX1-NEXT: vextractf128 $1, %ymm2, %xmm4 ; AVX1-NEXT: vextractf128 $1, %ymm2, %xmm4
; AVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64]
▲ Show 20 Lines • Show All 105 Lines • ▼ Show 20 Lines
; AVX512VLVBMI2-LABEL: splatvar_funnnel_v4i64: ; AVX512VLVBMI2-LABEL: splatvar_funnnel_v4i64:
; AVX512VLVBMI2: # %bb.0: ; AVX512VLVBMI2: # %bb.0:
; AVX512VLVBMI2-NEXT: vpbroadcastq %xmm2, %ymm2 ; AVX512VLVBMI2-NEXT: vpbroadcastq %xmm2, %ymm2
; AVX512VLVBMI2-NEXT: vpshldvq %ymm2, %ymm1, %ymm0 ; AVX512VLVBMI2-NEXT: vpshldvq %ymm2, %ymm1, %ymm0
; AVX512VLVBMI2-NEXT: retq ; AVX512VLVBMI2-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_funnnel_v4i64: ; XOPAVX1-LABEL: splatvar_funnnel_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm2[0,0] ; XOPAVX1-NEXT: vpermilps {{.*#+}} xmm2 = xmm2[0,1,0,1]
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2
; XOPAVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2 ; XOPAVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; XOPAVX1-NEXT: vpsllq %xmm2, %xmm3, %xmm3 ; XOPAVX1-NEXT: vpsllq %xmm2, %xmm3, %xmm3
; XOPAVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm4 ; XOPAVX1-NEXT: vpsllq %xmm2, %xmm0, %xmm4
; XOPAVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3
; XOPAVX1-NEXT: vextractf128 $1, %ymm2, %xmm4 ; XOPAVX1-NEXT: vextractf128 $1, %ymm2, %xmm4
; XOPAVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64] ; XOPAVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64]
▲ Show 20 Lines • Show All 1,497 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/vector-fshl-rot-256.ll

Show First 20 Lines • Show All 513 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Shifts ; Uniform Variable Shifts
; ;
define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %amt) nounwind { define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %amt) nounwind {
; AVX1-LABEL: splatvar_funnnel_v4i64: ; AVX1-LABEL: splatvar_funnnel_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; AVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2 ; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm2 ; AVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm2
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [63,63] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [63,63]
; AVX1-NEXT: vpand %xmm3, %xmm2, %xmm2 ; AVX1-NEXT: vpand %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; AVX1-NEXT: vpsrlq %xmm2, %xmm4, %xmm5 ; AVX1-NEXT: vpsrlq %xmm2, %xmm4, %xmm5
; AVX1-NEXT: vpshufd {{.*#+}} xmm6 = xmm2[2,3,0,1] ; AVX1-NEXT: vpshufd {{.*#+}} xmm6 = xmm2[2,3,0,1]
; AVX1-NEXT: vpsrlq %xmm6, %xmm4, %xmm7 ; AVX1-NEXT: vpsrlq %xmm6, %xmm4, %xmm7
▲ Show 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
; AVX512VLBW-LABEL: splatvar_funnnel_v4i64: ; AVX512VLBW-LABEL: splatvar_funnnel_v4i64:
; AVX512VLBW: # %bb.0: ; AVX512VLBW: # %bb.0:
; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1 ; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1
; AVX512VLBW-NEXT: vprolvq %ymm1, %ymm0, %ymm0 ; AVX512VLBW-NEXT: vprolvq %ymm1, %ymm0, %ymm0
; AVX512VLBW-NEXT: retq ; AVX512VLBW-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_funnnel_v4i64: ; XOPAVX1-LABEL: splatvar_funnnel_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; XOPAVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq ; XOPAVX1-NEXT: retq
; ;
; XOPAVX2-LABEL: splatvar_funnnel_v4i64: ; XOPAVX2-LABEL: splatvar_funnnel_v4i64:
; XOPAVX2: # %bb.0: ; XOPAVX2: # %bb.0:
▲ Show 20 Lines • Show All 933 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/vector-fshr-256.ll

Show First 20 Lines • Show All 904 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Shifts ; Uniform Variable Shifts
; ;
define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i64> %amt) nounwind { define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %y, <4 x i64> %amt) nounwind {
; AVX1-LABEL: splatvar_funnnel_v4i64: ; AVX1-LABEL: splatvar_funnnel_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm2[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm2 = xmm2[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2 ; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2
; AVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2 ; AVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm3 ; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm3
; AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3 ; AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3
; AVX1-NEXT: vpsrlq %xmm2, %xmm1, %xmm4 ; AVX1-NEXT: vpsrlq %xmm2, %xmm1, %xmm4
; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3 ; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3
; AVX1-NEXT: vextractf128 $1, %ymm2, %xmm4 ; AVX1-NEXT: vextractf128 $1, %ymm2, %xmm4
; AVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64]
▲ Show 20 Lines • Show All 104 Lines • ▼ Show 20 Lines
; AVX512VLVBMI2: # %bb.0: ; AVX512VLVBMI2: # %bb.0:
; AVX512VLVBMI2-NEXT: vpbroadcastq %xmm2, %ymm2 ; AVX512VLVBMI2-NEXT: vpbroadcastq %xmm2, %ymm2
; AVX512VLVBMI2-NEXT: vpshrdvq %ymm2, %ymm0, %ymm1 ; AVX512VLVBMI2-NEXT: vpshrdvq %ymm2, %ymm0, %ymm1
; AVX512VLVBMI2-NEXT: vmovdqa %ymm1, %ymm0 ; AVX512VLVBMI2-NEXT: vmovdqa %ymm1, %ymm0
; AVX512VLVBMI2-NEXT: retq ; AVX512VLVBMI2-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_funnnel_v4i64: ; XOPAVX1-LABEL: splatvar_funnnel_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm2[0,0] ; XOPAVX1-NEXT: vpermilps {{.*#+}} xmm2 = xmm2[0,1,0,1]
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm2, %ymm2
; XOPAVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2 ; XOPAVX1-NEXT: vandps {{.*}}(%rip), %ymm2, %ymm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm1, %xmm3 ; XOPAVX1-NEXT: vextractf128 $1, %ymm1, %xmm3
; XOPAVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3 ; XOPAVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3
; XOPAVX1-NEXT: vpsrlq %xmm2, %xmm1, %xmm4 ; XOPAVX1-NEXT: vpsrlq %xmm2, %xmm1, %xmm4
; XOPAVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm3, %ymm4, %ymm3
; XOPAVX1-NEXT: vextractf128 $1, %ymm2, %xmm4 ; XOPAVX1-NEXT: vextractf128 $1, %ymm2, %xmm4
; XOPAVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64] ; XOPAVX1-NEXT: vmovdqa {{.*#+}} xmm5 = [64,64]
▲ Show 20 Lines • Show All 1,506 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/vector-fshr-rot-256.ll

Show First 20 Lines • Show All 558 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Shifts ; Uniform Variable Shifts
; ;
define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %amt) nounwind { define <4 x i64> @splatvar_funnnel_v4i64(<4 x i64> %x, <4 x i64> %amt) nounwind {
; AVX1-LABEL: splatvar_funnnel_v4i64: ; AVX1-LABEL: splatvar_funnnel_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; AVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2 ; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm2 ; AVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm2
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [63,63] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [63,63]
; AVX1-NEXT: vpand %xmm3, %xmm2, %xmm2 ; AVX1-NEXT: vpand %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; AVX1-NEXT: vpsllq %xmm2, %xmm4, %xmm5 ; AVX1-NEXT: vpsllq %xmm2, %xmm4, %xmm5
; AVX1-NEXT: vpshufd {{.*#+}} xmm6 = xmm2[2,3,0,1] ; AVX1-NEXT: vpshufd {{.*#+}} xmm6 = xmm2[2,3,0,1]
; AVX1-NEXT: vpsllq %xmm6, %xmm4, %xmm7 ; AVX1-NEXT: vpsllq %xmm6, %xmm4, %xmm7
▲ Show 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
; AVX512VLBW-LABEL: splatvar_funnnel_v4i64: ; AVX512VLBW-LABEL: splatvar_funnnel_v4i64:
; AVX512VLBW: # %bb.0: ; AVX512VLBW: # %bb.0:
; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1 ; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1
; AVX512VLBW-NEXT: vprorvq %ymm1, %ymm0, %ymm0 ; AVX512VLBW-NEXT: vprorvq %ymm1, %ymm0, %ymm0
; AVX512VLBW-NEXT: retq ; AVX512VLBW-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_funnnel_v4i64: ; XOPAVX1-LABEL: splatvar_funnnel_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; XOPAVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2 ; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; XOPAVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm1 ; XOPAVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq ; XOPAVX1-NEXT: retq
; ;
▲ Show 20 Lines • Show All 965 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/vector-rotate-256.ll

Show First 20 Lines • Show All 506 Lines • ▼ Show 20 Lines
; ;
; Uniform Variable Rotates ; Uniform Variable Rotates
; ;
define <4 x i64> @splatvar_rotate_v4i64(<4 x i64> %a, <4 x i64> %b) nounwind { define <4 x i64> @splatvar_rotate_v4i64(<4 x i64> %a, <4 x i64> %b) nounwind {
; AVX1-LABEL: splatvar_rotate_v4i64: ; AVX1-LABEL: splatvar_rotate_v4i64:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm2 = xmm1[0,0] ; AVX1-NEXT: vpshufd {{.*#+}} xmm2 = xmm1[0,1,0,1]
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [64,64] ; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [64,64]
; AVX1-NEXT: vpsubq %xmm2, %xmm3, %xmm2 ; AVX1-NEXT: vpsubq %xmm2, %xmm3, %xmm2
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; AVX1-NEXT: vpsllq %xmm1, %xmm3, %xmm4 ; AVX1-NEXT: vpsllq %xmm1, %xmm3, %xmm4
; AVX1-NEXT: vpsllq %xmm1, %xmm0, %xmm1 ; AVX1-NEXT: vpsllq %xmm1, %xmm0, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm1, %ymm1 ; AVX1-NEXT: vinsertf128 $1, %xmm4, %ymm1, %ymm1
; AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3 ; AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm3
; AVX1-NEXT: vpsrlq %xmm2, %xmm0, %xmm0 ; AVX1-NEXT: vpsrlq %xmm2, %xmm0, %xmm0
Show All 35 Lines
; AVX512VLBW-LABEL: splatvar_rotate_v4i64: ; AVX512VLBW-LABEL: splatvar_rotate_v4i64:
; AVX512VLBW: # %bb.0: ; AVX512VLBW: # %bb.0:
; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1 ; AVX512VLBW-NEXT: vpbroadcastq %xmm1, %ymm1
; AVX512VLBW-NEXT: vprolvq %ymm1, %ymm0, %ymm0 ; AVX512VLBW-NEXT: vprolvq %ymm1, %ymm0, %ymm0
; AVX512VLBW-NEXT: retq ; AVX512VLBW-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_rotate_v4i64: ; XOPAVX1-LABEL: splatvar_rotate_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; XOPAVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0 ; XOPAVX1-NEXT: vprotq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq ; XOPAVX1-NEXT: retq
; ;
; XOPAVX2-LABEL: splatvar_rotate_v4i64: ; XOPAVX2-LABEL: splatvar_rotate_v4i64:
; XOPAVX2: # %bb.0: ; XOPAVX2: # %bb.0:
▲ Show 20 Lines • Show All 1,273 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/vector-shift-ashr-256.ll

Show First 20 Lines • Show All 671 Lines • ▼ Show 20 Lines
; AVX2-NEXT: vpsrlq %xmm1, %ymm2, %ymm2 ; AVX2-NEXT: vpsrlq %xmm1, %ymm2, %ymm2
; AVX2-NEXT: vpsrlq %xmm1, %ymm0, %ymm0 ; AVX2-NEXT: vpsrlq %xmm1, %ymm0, %ymm0
; AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0 ; AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0
; AVX2-NEXT: vpsubq %ymm2, %ymm0, %ymm0 ; AVX2-NEXT: vpsubq %ymm2, %ymm0, %ymm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; XOPAVX1-LABEL: splatvar_shift_v4i64: ; XOPAVX1-LABEL: splatvar_shift_v4i64:
; XOPAVX1: # %bb.0: ; XOPAVX1: # %bb.0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; XOPAVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,1]
; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2 ; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; XOPAVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm1 ; XOPAVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm2, %xmm2 ; XOPAVX1-NEXT: vpshaq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm0, %xmm0 ; XOPAVX1-NEXT: vpshaq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq ; XOPAVX1-NEXT: retq
; ;
▲ Show 20 Lines • Show All 979 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/vector-shuffle-256-v4.ll

Show First 20 Lines • Show All 683 Lines • ▼ Show 20 Lines
; AVX512VL-FAST-NEXT: retq ; AVX512VL-FAST-NEXT: retq
%1 = shufflevector <4 x double> %a, <4 x double> <double 0.000000e+00, double undef, double undef, double undef>, <4 x i32> <i32 1, i32 4, i32 2, i32 4> %1 = shufflevector <4 x double> %a, <4 x double> <double 0.000000e+00, double undef, double undef, double undef>, <4 x i32> <i32 1, i32 4, i32 2, i32 4>
ret <4 x double> %1 ret <4 x double> %1
} }
define <4 x i64> @shuffle_v4i64_0000(<4 x i64> %a, <4 x i64> %b) { define <4 x i64> @shuffle_v4i64_0000(<4 x i64> %a, <4 x i64> %b) {
; AVX1-LABEL: shuffle_v4i64_0000: ; AVX1-LABEL: shuffle_v4i64_0000:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm0 = xmm0[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0 ; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: shuffle_v4i64_0000: ; AVX2-LABEL: shuffle_v4i64_0000:
; AVX2: # %bb.0: ; AVX2: # %bb.0:
; AVX2-NEXT: vbroadcastsd %xmm0, %ymm0 ; AVX2-NEXT: vbroadcastsd %xmm0, %ymm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
▲ Show 20 Lines • Show All 189 Lines • ▼ Show 20 Lines
; AVX512VL-NEXT: retq ; AVX512VL-NEXT: retq
%shuffle = shufflevector <4 x i64> %a, <4 x i64> %b, <4 x i32> <i32 0, i32 2, i32 1, i32 3> %shuffle = shufflevector <4 x i64> %a, <4 x i64> %b, <4 x i32> <i32 0, i32 2, i32 1, i32 3>
ret <4 x i64> %shuffle ret <4 x i64> %shuffle
} }
define <4 x i64> @shuffle_v4i64_0124(<4 x i64> %a, <4 x i64> %b) { define <4 x i64> @shuffle_v4i64_0124(<4 x i64> %a, <4 x i64> %b) {
; AVX1-LABEL: shuffle_v4i64_0124: ; AVX1-LABEL: shuffle_v4i64_0124:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[0,1,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm1 ; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm1
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4,5],ymm1[6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4,5],ymm1[6,7]
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: shuffle_v4i64_0124: ; AVX2-LABEL: shuffle_v4i64_0124:
; AVX2: # %bb.0: ; AVX2: # %bb.0:
; AVX2-NEXT: vbroadcastsd %xmm1, %ymm1 ; AVX2-NEXT: vbroadcastsd %xmm1, %ymm1
; AVX2-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4,5],ymm1[6,7] ; AVX2-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1,2,3,4,5],ymm1[6,7]
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
; AVX512VL-FAST-NEXT: retq ; AVX512VL-FAST-NEXT: retq
%shuffle = shufflevector <4 x i64> %a, <4 x i64> %b, <4 x i32> <i32 0, i32 1, i32 4, i32 2> %shuffle = shufflevector <4 x i64> %a, <4 x i64> %b, <4 x i32> <i32 0, i32 1, i32 4, i32 2>
ret <4 x i64> %shuffle ret <4 x i64> %shuffle
} }
define <4 x i64> @shuffle_v4i64_0412(<4 x i64> %a, <4 x i64> %b) { define <4 x i64> @shuffle_v4i64_0412(<4 x i64> %a, <4 x i64> %b) {
; AVX1-LABEL: shuffle_v4i64_0412: ; AVX1-LABEL: shuffle_v4i64_0412:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[0,1,0,1]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2 ; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; AVX1-NEXT: vpalignr {{.*#+}} xmm2 = xmm0[8,9,10,11,12,13,14,15],xmm2[0,1,2,3,4,5,6,7] ; AVX1-NEXT: vpalignr {{.*#+}} xmm2 = xmm0[8,9,10,11,12,13,14,15],xmm2[0,1,2,3,4,5,6,7]
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0 ; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2,3],ymm0[4,5,6,7] ; AVX1-NEXT: vblendps {{.*#+}} ymm0 = ymm0[0,1],ymm1[2,3],ymm0[4,5,6,7]
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: shuffle_v4i64_0412: ; AVX2-LABEL: shuffle_v4i64_0412:
; AVX2: # %bb.0: ; AVX2: # %bb.0:
▲ Show 20 Lines • Show All 137 Lines • ▼ Show 20 Lines; AVX512VL-FAST-NEXT: retq
ret <4 x i64> %shuffle ret <4 x i64> %shuffle
} }
define <4 x i64> @shuffle_v4i64_1251(<4 x i64> %a, <4 x i64> %b) { define <4 x i64> @shuffle_v4i64_1251(<4 x i64> %a, <4 x i64> %b) {
; AVX1-LABEL: shuffle_v4i64_1251: ; AVX1-LABEL: shuffle_v4i64_1251:
; AVX1: # %bb.0: ; AVX1: # %bb.0:
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1] ; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,0,1]
; AVX1-NEXT: vshufpd {{.*#+}} ymm0 = ymm0[1],ymm2[0],ymm0[2],ymm2[3] ; AVX1-NEXT: vshufpd {{.*#+}} ymm0 = ymm0[1],ymm2[0],ymm0[2],ymm2[3]
; AVX1-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[2,3,0,1]
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm1 ; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm1
; AVX1-NEXT: vblendpd {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3] ; AVX1-NEXT: vblendpd {{.*#+}} ymm0 = ymm0[0,1],ymm1[2],ymm0[3]
; AVX1-NEXT: retq ; AVX1-NEXT: retq
; ;
; AVX2-LABEL: shuffle_v4i64_1251: ; AVX2-LABEL: shuffle_v4i64_1251:
; AVX2: # %bb.0: ; AVX2: # %bb.0:
; AVX2-NEXT: vpermpd {{.*#+}} ymm1 = ymm1[0,1,1,3] ; AVX2-NEXT: vpermpd {{.*#+}} ymm1 = ymm1[0,1,1,3]
; AVX2-NEXT: vpermpd {{.*#+}} ymm0 = ymm0[1,2,2,1] ; AVX2-NEXT: vpermpd {{.*#+}} ymm0 = ymm0[1,2,2,1]
▲ Show 20 Lines • Show All 844 Lines • Show Last 20 Lines