This is an archive of the discontinued LLVM Phabricator instance.

Differential D29690

[X86] Fix EXTRACT_VECTOR_ELT with variable index.
ClosedPublic

Authored by igorb on Feb 7 2017, 3:16 PM.

Details

Reviewers
RKSimon
zvi
delena
Commits
rGfda32d266a07: [X86] Fix EXTRACT_VECTOR_ELT with variable index from v32i16 and v64i8 vector.
rL295660: [X86] Fix EXTRACT_VECTOR_ELT with variable index from v32i16 and v64i8 vector.
Summary

Fix EXTRACT_VECTOR_ELT with variable index from v32i16 and v64i8 vector.
Its more profitable to goes through memory (1 cycles throughput)
than use VMOVD + VPERMV/PSHUFB sequence ( 2/3 cycles throughput) to implement EXTRACT_VECTOR_ELT with variable index.
IACA tool was used to get performace estimation (https://software.intel.com/en-us/articles/intel-architecture-code-analyzer)
For example for var_shuffle_v16i8_v16i8_xxxxxxxxxxxxxxxx_i8 test from vector-shuffle-variable-128.ll I get 26 cycles vs 79 cycles.
Tests on real hardware get similar results.

Removing the VINSERT node , we don't need it any more.

First patch to fix Bug 31731

Diff Detail

Repository
rL LLVM

Event Timeline

igorb created this revision.Feb 7 2017, 3:16 PM
zvi edited edge metadata.Feb 7 2017, 11:53 PM

Does it make sense to split this into 2 patches?

  1. Replace X86ISD::VINSERT with BUILD_VECTOR
  2. Implement missing functionality for variable-index extractelt?
lib/Target/X86/X86ISelLowering.cpp
13658 ↗(On Diff #87532)

Make this a static function or consider making a member of X86Subtarget

igorb updated this revision to Diff 87639.Feb 8 2017, 5:39 AM
igorb retitled this revision from [AVX512] Fix EXTRACT_VECTOR_ELT for v2i1/v4i1/v32i1/v64i1 with variable index. to [AVX512] Improve EXTRACT_VECTOR_ELT with variable index. .
igorb edited the summary of this revision. (Show Details)
igorb marked an inline comment as done.
delena added inline comments.Feb 8 2017, 6:14 AM
test/CodeGen/X86/avx512-insert-extract.ll
1463 ↗(On Diff #87639)

You can use VPERMQ here.
Permute qwords in ymm3/m256/m64bcst using indexes in ymm2 and store the result in ymm1.
VPERMQ ymm1 {k1}{z}, ymm2, ymm3/m256/m64bcst

1595 ↗(On Diff #87639)

varible -> variable

1609 ↗(On Diff #87639)

You don't need stack. Use VPEMD, for the both KNL and SKX.

1719 ↗(On Diff #87639)

Please use VPSHUFB instead of stack.

RKSimon added a subscriber: RKSimon.Feb 8 2017, 6:36 AM
RKSimon added inline comments.
lib/Target/X86/X86ISelLowering.cpp
13658 ↗(On Diff #87639)

// VPERMD/PS

13664 ↗(On Diff #87639)

// VPERMW

igorb updated this revision to Diff 87661.Feb 8 2017, 8:07 AM
igorb marked 6 inline comments as done.
igorb added a reviewer: RKSimon.

Fix according to comments ,
Thanks!

RKSimon edited edge metadata.Feb 8 2017, 8:58 AM

Add the new tests to trunk now to show current codegen?

lib/Target/X86/X86ISelLowering.cpp
13698 ↗(On Diff #87661)

SmallVector<SDValue, 8> Ops(NumElts, DAG.getConstant(0, dl, MaskEltVT));

26982 ↗(On Diff #87661)

Haven't you've lost the hasVLX() check by using isVPERMVsupported?

zvi added a comment.Feb 8 2017, 10:16 PM
In D29690#670860, @RKSimon wrote:

Add the new tests to trunk now to show current codegen?

According to pr31731 some cases should miscompile on ToT

delena added inline comments.Feb 8 2017, 11:03 PM
test/CodeGen/X86/avx512-insert-extract.ll
1505 ↗(On Diff #87661)

vmovslq + vmovq can be replaced with vmovd

1520 ↗(On Diff #87661)

It should be vpermpd.

1693 ↗(On Diff #87661)

I propose to extend v8i16 to v8i32 and use vpermd.

1723 ↗(On Diff #87661)

Again, you can extend to v16i32.

1780 ↗(On Diff #87661)

The result will be wrong. If you have '1' in MSB, the destination will be zeroed. You should mask the %index with 0xF:
and $15, %edi
vmovd %edi, %xmm

1840 ↗(On Diff #87661)

You can extend to v32i16 and use VPERMW

igorb updated this revision to Diff 87793.Feb 9 2017, 4:36 AM
igorb marked 2 inline comments as done.
igorb edited the summary of this revision. (Show Details)
igorb added inline comments.Feb 9 2017, 4:38 AM
lib/Target/X86/X86ISelLowering.cpp
26982 ↗(On Diff #87661)

I added patterns to TD file to handle 128/256 bit vector in case NoVLX using 512bit.
I think the check here only to insure VPERMV can be used.

test/CodeGen/X86/avx512-insert-extract.ll
1520 ↗(On Diff #87661)

Thanks!

1693 ↗(On Diff #87661)

Thanks Elena,
I prefer to handle all cases that can be extend in separate path.

1780 ↗(On Diff #87661)

Max 5 bits are used , according to LLVM doc "If idx exceeds the length of val, the results are undefined."

igorb added a comment.Feb 9 2017, 4:39 AM
In D29690#670860, @RKSimon wrote:

Add the new tests to trunk now to show current codegen?

Done,
Thanks.

RKSimon added inline comments.Feb 9 2017, 9:48 AM
lib/Target/X86/X86InstrAVX512.td
4844 ↗(On Diff #87793)

Very minor but is it worth renaming this if its not just being used for shifts/rotates?

igorb updated this revision to Diff 89090.Feb 20 2017, 1:55 AM
igorb retitled this revision from [AVX512] Improve EXTRACT_VECTOR_ELT with variable index. to [AVX512] Fix EXTRACT_VECTOR_ELT with variable index. .
igorb edited the summary of this revision. (Show Details)
delena accepted this revision.Feb 20 2017, 2:50 AM

It's not AVX-512 specific any more, please re-title the review.
Please specify that you are removing the VINSERT together with the code that using it and we don't need this node any more.

LGTM after the minor changes in the comments.

lib/Target/X86/X86ISelLowering.cpp
13779 ↗(On Diff #89090)

It's more profitable to go ..
than using ..

This revision is now accepted and ready to land.Feb 20 2017, 2:50 AM
igorb retitled this revision from [AVX512] Fix EXTRACT_VECTOR_ELT with variable index. to [X86] Fix EXTRACT_VECTOR_ELT with variable index. .Feb 20 2017, 4:27 AM
igorb edited the summary of this revision. (Show Details)
zvi accepted this revision.Feb 20 2017, 5:15 AM

LGTM with a minor comment

lib/Target/X86/X86ISelLowering.cpp
13782 ↗(On Diff #89090)

Would be nice if you added in the comment the two generated code sequences you compared in IACA and their IACA analysis.

Closed by commit rL295660: [X86] Fix EXTRACT_VECTOR_ELT with variable index from v32i16 and v64i8 vector. (authored by ibreger). · Explain WhyFeb 20 2017, 6:28 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathSize
llvm/
trunk/
lib/
Target/
X86/
3 lines
49 lines
13 lines
3 lines
13 lines
test/
CodeGen/
X86/
526 lines
34 lines
101 lines

Diff 89112

llvm/trunk/lib/Target/X86/X86ISelLowering.h

Show First 20 Lines • Show All 440 Lines • ▼ Show 20 Lines enum NodeType : unsigned {
// Broadcast scalar to vector. // Broadcast scalar to vector.
VBROADCAST, VBROADCAST,
// Broadcast mask to vector. // Broadcast mask to vector.
VBROADCASTM, VBROADCASTM,
// Broadcast subvector to vector. // Broadcast subvector to vector.
SUBV_BROADCAST, SUBV_BROADCAST,
// Insert/Extract vector element. // Extract vector element.
VINSERT,
VEXTRACT, VEXTRACT,
/// SSE4A Extraction and Insertion. /// SSE4A Extraction and Insertion.
EXTRQI, INSERTQI, EXTRQI, INSERTQI,
// XOP variable/immediate rotations. // XOP variable/immediate rotations.
VPROT, VPROTI, VPROT, VPROTI,
// XOP arithmetic/logical shifts. // XOP arithmetic/logical shifts.
▲ Show 20 Lines • Show All 927 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/X86/X86ISelLowering.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 13,770 Lines • ▼ Show 20 LinesX86TargetLowering::LowerEXTRACT_VECTOR_ELT(SDValue Op,
SDValue Vec = Op.getOperand(0); SDValue Vec = Op.getOperand(0);
MVT VecVT = Vec.getSimpleValueType(); MVT VecVT = Vec.getSimpleValueType();
SDValue Idx = Op.getOperand(1); SDValue Idx = Op.getOperand(1);
if (Op.getSimpleValueType() == MVT::i1) if (Op.getSimpleValueType() == MVT::i1)
return ExtractBitFromMaskVector(Op, DAG); return ExtractBitFromMaskVector(Op, DAG);
if (!isa<ConstantSDNode>(Idx)) { if (!isa<ConstantSDNode>(Idx)) {
if (VecVT.is512BitVector() || // Its more profitable to go through memory (1 cycles throughput)
(VecVT.is256BitVector() && Subtarget.hasInt256() && // than using VMOVD + VPERMV/PSHUFB sequence ( 2/3 cycles throughput)
VecVT.getScalarSizeInBits() == 32)) { // IACA tool was used to get performace estimation
// (https://software.intel.com/en-us/articles/intel-architecture-code-analyzer)
MVT MaskEltVT = //
MVT::getIntegerVT(VecVT.getScalarSizeInBits()); // exmample : extractelement <16 x i8> %a, i32 %i
MVT MaskVT = MVT::getVectorVT(MaskEltVT, VecVT.getSizeInBits() / //
MaskEltVT.getSizeInBits()); // Block Throughput: 3.00 Cycles
// Throughput Bottleneck: Port5
//
// | Num Of | Ports pressure in cycles | |
// | Uops | 0 - DV | 5 | 6 | 7 | |
// ---------------------------------------------
// | 1 | | 1.0 | | | CP | vmovd xmm1, edi
// | 1 | | 1.0 | | | CP | vpshufb xmm0, xmm0, xmm1
// | 2 | 1.0 | 1.0 | | | CP | vpextrb eax, xmm0, 0x0
// Total Num Of Uops: 4
//
//
// Block Throughput: 1.00 Cycles
// Throughput Bottleneck: PORT2_AGU, PORT3_AGU, Port4
//
// | | Ports pressure in cycles | |
// |Uops| 1 | 2 - D |3 - D | 4 | 5 | |
// ---------------------------------------------------------
// |2^ | | 0.5 | 0.5 |1.0| |CP| vmovaps xmmword ptr [rsp-0x18], xmm0
// |1 |0.5| | | |0.5| | lea rax, ptr [rsp-0x18]
// |1 | |0.5, 0.5|0.5, 0.5| | |CP| mov al, byte ptr [rdi+rax*1]
// Total Num Of Uops: 4
Idx = DAG.getZExtOrTrunc(Idx, dl, MaskEltVT);
auto PtrVT = getPointerTy(DAG.getDataLayout());
SDValue Mask = DAG.getNode(X86ISD::VINSERT, dl, MaskVT,
getZeroVector(MaskVT, Subtarget, DAG, dl), Idx,
DAG.getConstant(0, dl, PtrVT));
SDValue Perm = DAG.getNode(X86ISD::VPERMV, dl, VecVT, Mask, Vec);
return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, dl, Op.getValueType(), Perm,
DAG.getConstant(0, dl, PtrVT));
}
return SDValue(); return SDValue();
} }
unsigned IdxVal = cast<ConstantSDNode>(Idx)->getZExtValue(); unsigned IdxVal = cast<ConstantSDNode>(Idx)->getZExtValue();
// If this is a 256-bit vector result, first extract the 128-bit vector and // If this is a 256-bit vector result, first extract the 128-bit vector and
// then extract the element from the 128-bit vector. // then extract the element from the 128-bit vector.
if (VecVT.is256BitVector() || VecVT.is512BitVector()) { if (VecVT.is256BitVector() || VecVT.is512BitVector()) {
▲ Show 20 Lines • Show All 10,127 Lines • ▼ Show 20 Linesconst char *X86TargetLowering::getTargetNodeName(unsigned Opcode) const {
case X86ISD::VSEXT: return "X86ISD::VSEXT"; case X86ISD::VSEXT: return "X86ISD::VSEXT";
case X86ISD::VTRUNC: return "X86ISD::VTRUNC"; case X86ISD::VTRUNC: return "X86ISD::VTRUNC";
case X86ISD::VTRUNCS: return "X86ISD::VTRUNCS"; case X86ISD::VTRUNCS: return "X86ISD::VTRUNCS";
case X86ISD::VTRUNCUS: return "X86ISD::VTRUNCUS"; case X86ISD::VTRUNCUS: return "X86ISD::VTRUNCUS";
case X86ISD::VTRUNCSTORES: return "X86ISD::VTRUNCSTORES"; case X86ISD::VTRUNCSTORES: return "X86ISD::VTRUNCSTORES";
case X86ISD::VTRUNCSTOREUS: return "X86ISD::VTRUNCSTOREUS"; case X86ISD::VTRUNCSTOREUS: return "X86ISD::VTRUNCSTOREUS";
case X86ISD::VMTRUNCSTORES: return "X86ISD::VMTRUNCSTORES"; case X86ISD::VMTRUNCSTORES: return "X86ISD::VMTRUNCSTORES";
case X86ISD::VMTRUNCSTOREUS: return "X86ISD::VMTRUNCSTOREUS"; case X86ISD::VMTRUNCSTOREUS: return "X86ISD::VMTRUNCSTOREUS";
case X86ISD::VINSERT: return "X86ISD::VINSERT";
case X86ISD::VFPEXT: return "X86ISD::VFPEXT"; case X86ISD::VFPEXT: return "X86ISD::VFPEXT";
case X86ISD::VFPEXT_RND: return "X86ISD::VFPEXT_RND"; case X86ISD::VFPEXT_RND: return "X86ISD::VFPEXT_RND";
case X86ISD::VFPEXTS_RND: return "X86ISD::VFPEXTS_RND"; case X86ISD::VFPEXTS_RND: return "X86ISD::VFPEXTS_RND";
case X86ISD::VFPROUND: return "X86ISD::VFPROUND"; case X86ISD::VFPROUND: return "X86ISD::VFPROUND";
case X86ISD::VFPROUND_RND: return "X86ISD::VFPROUND_RND"; case X86ISD::VFPROUND_RND: return "X86ISD::VFPROUND_RND";
case X86ISD::VFPROUNDS_RND: return "X86ISD::VFPROUNDS_RND"; case X86ISD::VFPROUNDS_RND: return "X86ISD::VFPROUNDS_RND";
case X86ISD::CVT2MASK: return "X86ISD::CVT2MASK"; case X86ISD::CVT2MASK: return "X86ISD::CVT2MASK";
case X86ISD::VSHLDQ: return "X86ISD::VSHLDQ"; case X86ISD::VSHLDQ: return "X86ISD::VSHLDQ";
▲ Show 20 Lines • Show All 11,426 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/X86/X86InstrAVX512.td

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 3,574 Lines • ▼ Show 20 Lines def : Pat<(v16i32 (X86vzmovl (insert_subvector undef,
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src), sub_xmm)>; (SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src), sub_xmm)>;
// Use regular 128-bit instructions to match 512-bit scalar_to_vec+zext. // Use regular 128-bit instructions to match 512-bit scalar_to_vec+zext.
def : Pat<(v16i32 (X86vzload addr:$src)), def : Pat<(v16i32 (X86vzload addr:$src)),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrm addr:$src), sub_xmm)>; (SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrm addr:$src), sub_xmm)>;
def : Pat<(v8i64 (X86vzload addr:$src)), def : Pat<(v8i64 (X86vzload addr:$src)),
(SUBREG_TO_REG (i64 0), (VMOVQI2PQIZrm addr:$src), sub_xmm)>; (SUBREG_TO_REG (i64 0), (VMOVQI2PQIZrm addr:$src), sub_xmm)>;
} }
def : Pat<(v16i32 (X86Vinsert (v16i32 immAllZerosV), GR32:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src2), sub_xmm)>;
def : Pat<(v8i64 (X86Vinsert (bc_v8i64 (v16i32 immAllZerosV)), GR64:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOV64toPQIZrr GR64:$src2), sub_xmm)>;
def : Pat<(v16i32 (X86Vinsert undef, GR32:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src2), sub_xmm)>;
def : Pat<(v8i64 (X86Vinsert undef, GR64:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOV64toPQIZrr GR64:$src2), sub_xmm)>;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// AVX-512 - Non-temporals // AVX-512 - Non-temporals
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
let SchedRW = [WriteLoad] in { let SchedRW = [WriteLoad] in {
def VMOVNTDQAZrm : AVX512PI<0x2A, MRMSrcMem, (outs VR512:$dst), def VMOVNTDQAZrm : AVX512PI<0x2A, MRMSrcMem, (outs VR512:$dst),
(ins i512mem:$src), "vmovntdqa\t{$src, $dst|$dst, $src}", (ins i512mem:$src), "vmovntdqa\t{$src, $dst|$dst, $src}",
[(set VR512:$dst, (int_x86_avx512_movntdqa addr:$src))], [(set VR512:$dst, (int_x86_avx512_movntdqa addr:$src))],
SSEPackedInt>, EVEX, T8PD, EVEX_V512, SSEPackedInt>, EVEX, T8PD, EVEX_V512,
▲ Show 20 Lines • Show All 5,760 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/X86/X86InstrFragmentsSIMD.td

Show First 20 Lines • Show All 447 Lines • ▼ Show 20 Linesdef X86Vfpclasss : SDNode<"X86ISD::VFPCLASSS",
SDTCisFP<1>, SDTCisVT<2, i32>]>,[]>; SDTCisFP<1>, SDTCisVT<2, i32>]>,[]>;
def X86SubVBroadcast : SDNode<"X86ISD::SUBV_BROADCAST", def X86SubVBroadcast : SDNode<"X86ISD::SUBV_BROADCAST",
SDTypeProfile<1, 1, [SDTCisVec<0>, SDTCisVec<1>, SDTypeProfile<1, 1, [SDTCisVec<0>, SDTCisVec<1>,
SDTCisSubVecOfVec<1, 0>]>, []>; SDTCisSubVecOfVec<1, 0>]>, []>;
def X86VBroadcast : SDNode<"X86ISD::VBROADCAST", SDTVBroadcast>; def X86VBroadcast : SDNode<"X86ISD::VBROADCAST", SDTVBroadcast>;
def X86VBroadcastm : SDNode<"X86ISD::VBROADCASTM", SDTVBroadcastm>; def X86VBroadcastm : SDNode<"X86ISD::VBROADCASTM", SDTVBroadcastm>;
def X86Vinsert : SDNode<"X86ISD::VINSERT", SDTypeProfile<1, 3,
[SDTCisSameAs<0, 1>, SDTCisEltOfVec<2, 1>,
SDTCisPtrTy<3>]>, []>;
def X86Vextract : SDNode<"X86ISD::VEXTRACT", SDTypeProfile<1, 2, def X86Vextract : SDNode<"X86ISD::VEXTRACT", SDTypeProfile<1, 2,
[SDTCisEltOfVec<0, 1>, SDTCisVec<1>, [SDTCisEltOfVec<0, 1>, SDTCisVec<1>,
SDTCisPtrTy<2>]>, []>; SDTCisPtrTy<2>]>, []>;
def X86Blendi : SDNode<"X86ISD::BLENDI", SDTBlend>; def X86Blendi : SDNode<"X86ISD::BLENDI", SDTBlend>;
def X86Addsub : SDNode<"X86ISD::ADDSUB", SDTFPBinOp>; def X86Addsub : SDNode<"X86ISD::ADDSUB", SDTFPBinOp>;
▲ Show 20 Lines • Show All 660 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/X86/X86InstrSSE.td

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 4,696 Lines • ▼ Show 20 Linesdef MOVPDI2DIrr : S2I<0x7E, MRMDestReg, (outs GR32:$dst), (ins VR128:$src),
(iPTR 0)))], IIC_SSE_MOVD_ToGP>, (iPTR 0)))], IIC_SSE_MOVD_ToGP>,
Sched<[WriteMove]>; Sched<[WriteMove]>;
def MOVPDI2DImr : S2I<0x7E, MRMDestMem, (outs), (ins i32mem:$dst, VR128:$src), def MOVPDI2DImr : S2I<0x7E, MRMDestMem, (outs), (ins i32mem:$dst, VR128:$src),
"movd\t{$src, $dst|$dst, $src}", "movd\t{$src, $dst|$dst, $src}",
[(store (i32 (extractelt (v4i32 VR128:$src), [(store (i32 (extractelt (v4i32 VR128:$src),
(iPTR 0))), addr:$dst)], (iPTR 0))), addr:$dst)],
IIC_SSE_MOVDQ>, Sched<[WriteStore]>; IIC_SSE_MOVDQ>, Sched<[WriteStore]>;
} // ExeDomain = SSEPackedInt } // ExeDomain = SSEPackedInt
def : Pat<(v8i32 (X86Vinsert (v8i32 immAllZerosV), GR32:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIrr GR32:$src2), sub_xmm)>;
def : Pat<(v4i64 (X86Vinsert (bc_v4i64 (v8i32 immAllZerosV)), GR64:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOV64toPQIrr GR64:$src2), sub_xmm)>;
def : Pat<(v8i32 (X86Vinsert undef, GR32:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIrr GR32:$src2), sub_xmm)>;
def : Pat<(v4i64 (X86Vinsert undef, GR64:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOV64toPQIrr GR64:$src2), sub_xmm)>;
//===---------------------------------------------------------------------===// //===---------------------------------------------------------------------===//
// Move Packed Doubleword Int first element to Doubleword Int // Move Packed Doubleword Int first element to Doubleword Int
// //
let ExeDomain = SSEPackedInt in { let ExeDomain = SSEPackedInt in {
let SchedRW = [WriteMove] in { let SchedRW = [WriteMove] in {
def VMOVPQIto64rr : VRS2I<0x7E, MRMDestReg, (outs GR64:$dst), (ins VR128:$src), def VMOVPQIto64rr : VRS2I<0x7E, MRMDestReg, (outs GR64:$dst), (ins VR128:$src),
"movq\t{$src, $dst|$dst, $src}", "movq\t{$src, $dst|$dst, $src}",
[(set GR64:$dst, (extractelt (v2i64 VR128:$src), [(set GR64:$dst, (extractelt (v2i64 VR128:$src),
▲ Show 20 Lines • Show All 4,066 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/avx512-insert-extract.ll

Show First 20 Lines • Show All 118 Lines • ▼ Show 20 Lines; SKX-NEXT: retq
%ef = extractelement <4 x float> %x, i32 3 %ef = extractelement <4 x float> %x, i32 3
store float %ef, float* %out, align 4 store float %ef, float* %out, align 4
ret void ret void
} }
define float @test7(<16 x float> %x, i32 %ind) nounwind { define float @test7(<16 x float> %x, i32 %ind) nounwind {
; KNL-LABEL: test7: ; KNL-LABEL: test7:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: vmovd %edi, %xmm1 ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vpermps %zmm0, %zmm1, %zmm0 ; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: ## kill: %XMM0<def> %XMM0<kill> %ZMM0<kill> ; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %zmm0, (%rsp)
; KNL-NEXT: andl $15, %edi
; KNL-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test7: ; SKX-LABEL: test7:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: vmovd %edi, %xmm1 ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vpermps %zmm0, %zmm1, %zmm0 ; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: ## kill: %XMM0<def> %XMM0<kill> %ZMM0<kill> ; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %zmm0, (%rsp)
; SKX-NEXT: andl $15, %edi
; SKX-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%e = extractelement <16 x float> %x, i32 %ind %e = extractelement <16 x float> %x, i32 %ind
ret float %e ret float %e
} }
define double @test8(<8 x double> %x, i32 %ind) nounwind { define double @test8(<8 x double> %x, i32 %ind) nounwind {
; KNL-LABEL: test8: ; KNL-LABEL: test8:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: movslq %edi, %rax ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vmovq %rax, %xmm1 ; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: vpermpd %zmm0, %zmm1, %zmm0 ; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: ## kill: %XMM0<def> %XMM0<kill> %ZMM0<kill> ; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %zmm0, (%rsp)
; KNL-NEXT: andl $7, %edi
; KNL-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test8: ; SKX-LABEL: test8:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: movslq %edi, %rax ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vmovq %rax, %xmm1 ; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: vpermpd %zmm0, %zmm1, %zmm0 ; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: ## kill: %XMM0<def> %XMM0<kill> %ZMM0<kill> ; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %zmm0, (%rsp)
; SKX-NEXT: andl $7, %edi
; SKX-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%e = extractelement <8 x double> %x, i32 %ind %e = extractelement <8 x double> %x, i32 %ind
ret double %e ret double %e
} }
define float @test9(<8 x float> %x, i32 %ind) nounwind { define float @test9(<8 x float> %x, i32 %ind) nounwind {
; KNL-LABEL: test9: ; KNL-LABEL: test9:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: vmovd %edi, %xmm1 ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vpermps %ymm0, %ymm1, %ymm0 ; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: ## kill: %XMM0<def> %XMM0<kill> %YMM0<kill> ; KNL-NEXT: andq $-32, %rsp
; KNL-NEXT: subq $64, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %ymm0, (%rsp)
; KNL-NEXT: andl $7, %edi
; KNL-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test9: ; SKX-LABEL: test9:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: vmovd %edi, %xmm1 ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vpermps %ymm0, %ymm1, %ymm0 ; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: ## kill: %XMM0<def> %XMM0<kill> %YMM0<kill> ; SKX-NEXT: andq $-32, %rsp
; SKX-NEXT: subq $64, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %ymm0, (%rsp)
; SKX-NEXT: andl $7, %edi
; SKX-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%e = extractelement <8 x float> %x, i32 %ind %e = extractelement <8 x float> %x, i32 %ind
ret float %e ret float %e
} }
define i32 @test10(<16 x i32> %x, i32 %ind) nounwind { define i32 @test10(<16 x i32> %x, i32 %ind) nounwind {
; KNL-LABEL: test10: ; KNL-LABEL: test10:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: vmovd %edi, %xmm1 ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vpermd %zmm0, %zmm1, %zmm0 ; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: vmovd %xmm0, %eax ; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %zmm0, (%rsp)
; KNL-NEXT: andl $15, %edi
; KNL-NEXT: movl (%rsp,%rdi,4), %eax
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test10: ; SKX-LABEL: test10:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: vmovd %edi, %xmm1 ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vpermd %zmm0, %zmm1, %zmm0 ; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: vmovd %xmm0, %eax ; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %zmm0, (%rsp)
; SKX-NEXT: andl $15, %edi
; SKX-NEXT: movl (%rsp,%rdi,4), %eax
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%e = extractelement <16 x i32> %x, i32 %ind %e = extractelement <16 x i32> %x, i32 %ind
ret i32 %e ret i32 %e
} }
define <16 x i32> @test11(<16 x i32>%a, <16 x i32>%b) { define <16 x i32> @test11(<16 x i32>%a, <16 x i32>%b) {
; KNL-LABEL: test11: ; KNL-LABEL: test11:
; KNL: ## BB#0: ; KNL: ## BB#0:
▲ Show 20 Lines • Show All 1,308 Lines • ▼ Show 20 Lines
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <4 x i64> %t1, i32 %index %t2 = extractelement <4 x i64> %t1, i32 %index
ret i64 %t2 ret i64 %t2
} }
define i64 @test_extractelement_variable_v8i64(<8 x i64> %t1, i32 %index) { define i64 @test_extractelement_variable_v8i64(<8 x i64> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v8i64: ; KNL-LABEL: test_extractelement_variable_v8i64:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: movslq %edi, %rax ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vmovq %rax, %xmm1 ; KNL-NEXT: Lcfi6:
; KNL-NEXT: vpermq %zmm0, %zmm1, %zmm0 ; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: vmovq %xmm0, %rax ; KNL-NEXT: Lcfi7:
; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi8:
; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %zmm0, (%rsp)
; KNL-NEXT: andl $7, %edi
; KNL-NEXT: movq (%rsp,%rdi,8), %rax
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test_extractelement_variable_v8i64: ; SKX-LABEL: test_extractelement_variable_v8i64:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: movslq %edi, %rax ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vmovq %rax, %xmm1 ; SKX-NEXT: Lcfi3:
; SKX-NEXT: vpermq %zmm0, %zmm1, %zmm0 ; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: vmovq %xmm0, %rax ; SKX-NEXT: Lcfi4:
; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi5:
; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %zmm0, (%rsp)
; SKX-NEXT: andl $7, %edi
; SKX-NEXT: movq (%rsp,%rdi,8), %rax
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <8 x i64> %t1, i32 %index %t2 = extractelement <8 x i64> %t1, i32 %index
ret i64 %t2 ret i64 %t2
} }
define double @test_extractelement_variable_v2f64(<2 x double> %t1, i32 %index) { define double @test_extractelement_variable_v2f64(<2 x double> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v2f64: ; KNL-LABEL: test_extractelement_variable_v2f64:
; KNL: ## BB#0: ; KNL: ## BB#0:
Show All 13 Lines; SKX-NEXT: retq
%t2 = extractelement <2 x double> %t1, i32 %index %t2 = extractelement <2 x double> %t1, i32 %index
ret double %t2 ret double %t2
} }
define double @test_extractelement_variable_v4f64(<4 x double> %t1, i32 %index) { define double @test_extractelement_variable_v4f64(<4 x double> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v4f64: ; KNL-LABEL: test_extractelement_variable_v4f64:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: pushq %rbp ; KNL-NEXT: pushq %rbp
; KNL-NEXT: Lcfi6: ; KNL-NEXT: Lcfi9:
; KNL-NEXT: .cfi_def_cfa_offset 16 ; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi7: ; KNL-NEXT: Lcfi10:
; KNL-NEXT: .cfi_offset %rbp, -16 ; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp ; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi8: ; KNL-NEXT: Lcfi11:
; KNL-NEXT: .cfi_def_cfa_register %rbp ; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-32, %rsp ; KNL-NEXT: andq $-32, %rsp
; KNL-NEXT: subq $64, %rsp ; KNL-NEXT: subq $64, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def> ; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %ymm0, (%rsp) ; KNL-NEXT: vmovaps %ymm0, (%rsp)
; KNL-NEXT: andl $3, %edi ; KNL-NEXT: andl $3, %edi
; KNL-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero ; KNL-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
; KNL-NEXT: movq %rbp, %rsp ; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp ; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test_extractelement_variable_v4f64: ; SKX-LABEL: test_extractelement_variable_v4f64:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: pushq %rbp ; SKX-NEXT: pushq %rbp
; SKX-NEXT: Lcfi3: ; SKX-NEXT: Lcfi6:
; SKX-NEXT: .cfi_def_cfa_offset 16 ; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi4: ; SKX-NEXT: Lcfi7:
; SKX-NEXT: .cfi_offset %rbp, -16 ; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp ; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi5: ; SKX-NEXT: Lcfi8:
; SKX-NEXT: .cfi_def_cfa_register %rbp ; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-32, %rsp ; SKX-NEXT: andq $-32, %rsp
; SKX-NEXT: subq $64, %rsp ; SKX-NEXT: subq $64, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def> ; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %ymm0, (%rsp) ; SKX-NEXT: vmovaps %ymm0, (%rsp)
; SKX-NEXT: andl $3, %edi ; SKX-NEXT: andl $3, %edi
; SKX-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero ; SKX-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
; SKX-NEXT: movq %rbp, %rsp ; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp ; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <4 x double> %t1, i32 %index %t2 = extractelement <4 x double> %t1, i32 %index
ret double %t2 ret double %t2
} }
define double @test_extractelement_variable_v8f64(<8 x double> %t1, i32 %index) { define double @test_extractelement_variable_v8f64(<8 x double> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v8f64: ; KNL-LABEL: test_extractelement_variable_v8f64:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: movslq %edi, %rax ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vmovq %rax, %xmm1 ; KNL-NEXT: Lcfi12:
; KNL-NEXT: vpermpd %zmm0, %zmm1, %zmm0 ; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: ## kill: %XMM0<def> %XMM0<kill> %ZMM0<kill> ; KNL-NEXT: Lcfi13:
; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi14:
; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %zmm0, (%rsp)
; KNL-NEXT: andl $7, %edi
; KNL-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test_extractelement_variable_v8f64: ; SKX-LABEL: test_extractelement_variable_v8f64:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: movslq %edi, %rax ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vmovq %rax, %xmm1 ; SKX-NEXT: Lcfi9:
; SKX-NEXT: vpermpd %zmm0, %zmm1, %zmm0 ; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: ## kill: %XMM0<def> %XMM0<kill> %ZMM0<kill> ; SKX-NEXT: Lcfi10:
; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi11:
; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %zmm0, (%rsp)
; SKX-NEXT: andl $7, %edi
; SKX-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <8 x double> %t1, i32 %index %t2 = extractelement <8 x double> %t1, i32 %index
ret double %t2 ret double %t2
} }
define i32 @test_extractelement_variable_v4i32(<4 x i32> %t1, i32 %index) { define i32 @test_extractelement_variable_v4i32(<4 x i32> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v4i32: ; KNL-LABEL: test_extractelement_variable_v4i32:
; KNL: ## BB#0: ; KNL: ## BB#0:
Show All 12 Lines
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <4 x i32> %t1, i32 %index %t2 = extractelement <4 x i32> %t1, i32 %index
ret i32 %t2 ret i32 %t2
} }
define i32 @test_extractelement_variable_v8i32(<8 x i32> %t1, i32 %index) { define i32 @test_extractelement_variable_v8i32(<8 x i32> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v8i32: ; KNL-LABEL: test_extractelement_variable_v8i32:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: vmovd %edi, %xmm1 ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vpermd %ymm0, %ymm1, %ymm0 ; KNL-NEXT: Lcfi15:
; KNL-NEXT: vmovd %xmm0, %eax ; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi16:
; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi17:
; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-32, %rsp
; KNL-NEXT: subq $64, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %ymm0, (%rsp)
; KNL-NEXT: andl $7, %edi
; KNL-NEXT: movl (%rsp,%rdi,4), %eax
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test_extractelement_variable_v8i32: ; SKX-LABEL: test_extractelement_variable_v8i32:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: vmovd %edi, %xmm1 ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vpermd %ymm0, %ymm1, %ymm0 ; SKX-NEXT: Lcfi12:
; SKX-NEXT: vmovd %xmm0, %eax ; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi13:
; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi14:
; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-32, %rsp
; SKX-NEXT: subq $64, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %ymm0, (%rsp)
; SKX-NEXT: andl $7, %edi
; SKX-NEXT: movl (%rsp,%rdi,4), %eax
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <8 x i32> %t1, i32 %index %t2 = extractelement <8 x i32> %t1, i32 %index
ret i32 %t2 ret i32 %t2
} }
define i32 @test_extractelement_variable_v16i32(<16 x i32> %t1, i32 %index) { define i32 @test_extractelement_variable_v16i32(<16 x i32> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v16i32: ; KNL-LABEL: test_extractelement_variable_v16i32:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: vmovd %edi, %xmm1 ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vpermd %zmm0, %zmm1, %zmm0 ; KNL-NEXT: Lcfi18:
; KNL-NEXT: vmovd %xmm0, %eax ; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi19:
; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi20:
; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %zmm0, (%rsp)
; KNL-NEXT: andl $15, %edi
; KNL-NEXT: movl (%rsp,%rdi,4), %eax
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test_extractelement_variable_v16i32: ; SKX-LABEL: test_extractelement_variable_v16i32:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: vmovd %edi, %xmm1 ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vpermd %zmm0, %zmm1, %zmm0 ; SKX-NEXT: Lcfi15:
; SKX-NEXT: vmovd %xmm0, %eax ; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi16:
; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi17:
; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %zmm0, (%rsp)
; SKX-NEXT: andl $15, %edi
; SKX-NEXT: movl (%rsp,%rdi,4), %eax
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <16 x i32> %t1, i32 %index %t2 = extractelement <16 x i32> %t1, i32 %index
ret i32 %t2 ret i32 %t2
} }
define float @test_extractelement_variable_v4f32(<4 x float> %t1, i32 %index) { define float @test_extractelement_variable_v4f32(<4 x float> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v4f32: ; KNL-LABEL: test_extractelement_variable_v4f32:
; KNL: ## BB#0: ; KNL: ## BB#0:
Show All 12 Lines
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <4 x float> %t1, i32 %index %t2 = extractelement <4 x float> %t1, i32 %index
ret float %t2 ret float %t2
} }
define float @test_extractelement_variable_v8f32(<8 x float> %t1, i32 %index) { define float @test_extractelement_variable_v8f32(<8 x float> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v8f32: ; KNL-LABEL: test_extractelement_variable_v8f32:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: vmovd %edi, %xmm1 ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vpermps %ymm0, %ymm1, %ymm0 ; KNL-NEXT: Lcfi21:
; KNL-NEXT: ## kill: %XMM0<def> %XMM0<kill> %YMM0<kill> ; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi22:
; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi23:
; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-32, %rsp
; KNL-NEXT: subq $64, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %ymm0, (%rsp)
; KNL-NEXT: andl $7, %edi
; KNL-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test_extractelement_variable_v8f32: ; SKX-LABEL: test_extractelement_variable_v8f32:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: vmovd %edi, %xmm1 ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vpermps %ymm0, %ymm1, %ymm0 ; SKX-NEXT: Lcfi18:
; SKX-NEXT: ## kill: %XMM0<def> %XMM0<kill> %YMM0<kill> ; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi19:
; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi20:
; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-32, %rsp
; SKX-NEXT: subq $64, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %ymm0, (%rsp)
; SKX-NEXT: andl $7, %edi
; SKX-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <8 x float> %t1, i32 %index %t2 = extractelement <8 x float> %t1, i32 %index
ret float %t2 ret float %t2
} }
define float @test_extractelement_variable_v16f32(<16 x float> %t1, i32 %index) { define float @test_extractelement_variable_v16f32(<16 x float> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v16f32: ; KNL-LABEL: test_extractelement_variable_v16f32:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: vmovd %edi, %xmm1 ; KNL-NEXT: pushq %rbp
; KNL-NEXT: vpermps %zmm0, %zmm1, %zmm0 ; KNL-NEXT: Lcfi24:
; KNL-NEXT: ## kill: %XMM0<def> %XMM0<kill> %ZMM0<kill> ; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi25:
; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi26:
; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %zmm0, (%rsp)
; KNL-NEXT: andl $15, %edi
; KNL-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test_extractelement_variable_v16f32: ; SKX-LABEL: test_extractelement_variable_v16f32:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: vmovd %edi, %xmm1 ; SKX-NEXT: pushq %rbp
; SKX-NEXT: vpermps %zmm0, %zmm1, %zmm0 ; SKX-NEXT: Lcfi21:
; SKX-NEXT: ## kill: %XMM0<def> %XMM0<kill> %ZMM0<kill> ; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi22:
; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi23:
; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovaps %zmm0, (%rsp)
; SKX-NEXT: andl $15, %edi
; SKX-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <16 x float> %t1, i32 %index %t2 = extractelement <16 x float> %t1, i32 %index
ret float %t2 ret float %t2
} }
define i16 @test_extractelement_variable_v8i16(<8 x i16> %t1, i32 %index) { define i16 @test_extractelement_variable_v8i16(<8 x i16> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v8i16: ; KNL-LABEL: test_extractelement_variable_v8i16:
; KNL: ## BB#0: ; KNL: ## BB#0:
Show All 13 Lines; SKX-NEXT: retq
%t2 = extractelement <8 x i16> %t1, i32 %index %t2 = extractelement <8 x i16> %t1, i32 %index
ret i16 %t2 ret i16 %t2
} }
define i16 @test_extractelement_variable_v16i16(<16 x i16> %t1, i32 %index) { define i16 @test_extractelement_variable_v16i16(<16 x i16> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v16i16: ; KNL-LABEL: test_extractelement_variable_v16i16:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: pushq %rbp ; KNL-NEXT: pushq %rbp
; KNL-NEXT: Lcfi9: ; KNL-NEXT: Lcfi27:
; KNL-NEXT: .cfi_def_cfa_offset 16 ; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi10: ; KNL-NEXT: Lcfi28:
; KNL-NEXT: .cfi_offset %rbp, -16 ; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp ; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi11: ; KNL-NEXT: Lcfi29:
; KNL-NEXT: .cfi_def_cfa_register %rbp ; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-32, %rsp ; KNL-NEXT: andq $-32, %rsp
; KNL-NEXT: subq $64, %rsp ; KNL-NEXT: subq $64, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def> ; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %ymm0, (%rsp) ; KNL-NEXT: vmovaps %ymm0, (%rsp)
; KNL-NEXT: andl $15, %edi ; KNL-NEXT: andl $15, %edi
; KNL-NEXT: movzwl (%rsp,%rdi,2), %eax ; KNL-NEXT: movzwl (%rsp,%rdi,2), %eax
; KNL-NEXT: movq %rbp, %rsp ; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp ; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test_extractelement_variable_v16i16: ; SKX-LABEL: test_extractelement_variable_v16i16:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: pushq %rbp ; SKX-NEXT: pushq %rbp
; SKX-NEXT: Lcfi6: ; SKX-NEXT: Lcfi24:
; SKX-NEXT: .cfi_def_cfa_offset 16 ; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi7: ; SKX-NEXT: Lcfi25:
; SKX-NEXT: .cfi_offset %rbp, -16 ; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp ; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi8: ; SKX-NEXT: Lcfi26:
; SKX-NEXT: .cfi_def_cfa_register %rbp ; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-32, %rsp ; SKX-NEXT: andq $-32, %rsp
; SKX-NEXT: subq $64, %rsp ; SKX-NEXT: subq $64, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def> ; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovdqu %ymm0, (%rsp) ; SKX-NEXT: vmovdqu %ymm0, (%rsp)
; SKX-NEXT: andl $15, %edi ; SKX-NEXT: andl $15, %edi
; SKX-NEXT: movzwl (%rsp,%rdi,2), %eax ; SKX-NEXT: movzwl (%rsp,%rdi,2), %eax
; SKX-NEXT: movq %rbp, %rsp ; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp ; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <16 x i16> %t1, i32 %index %t2 = extractelement <16 x i16> %t1, i32 %index
ret i16 %t2 ret i16 %t2
} }
; TODO - enable after fix define i16 @test_extractelement_variable_v32i16(<32 x i16> %t1, i32 %index) {
;define i16 @test_extractelement_variable_v32i16(<32 x i16> %t1, i32 %index) { ; KNL-LABEL: test_extractelement_variable_v32i16:
; %t2 = extractelement <32 x i16> %t1, i32 %index ; KNL: ## BB#0:
; ret i16 %t2 ; KNL-NEXT: pushq %rbp
;} ; KNL-NEXT: Lcfi30:
; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi31:
; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi32:
; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %ymm1, {{[0-9]+}}(%rsp)
; KNL-NEXT: vmovaps %ymm0, (%rsp)
; KNL-NEXT: andl $31, %edi
; KNL-NEXT: movzwl (%rsp,%rdi,2), %eax
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq
;
; SKX-LABEL: test_extractelement_variable_v32i16:
; SKX: ## BB#0:
; SKX-NEXT: pushq %rbp
; SKX-NEXT: Lcfi27:
; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi28:
; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi29:
; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovdqu16 %zmm0, (%rsp)
; SKX-NEXT: andl $31, %edi
; SKX-NEXT: movzwl (%rsp,%rdi,2), %eax
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq
%t2 = extractelement <32 x i16> %t1, i32 %index
ret i16 %t2
}
define i8 @test_extractelement_variable_v16i8(<16 x i8> %t1, i32 %index) { define i8 @test_extractelement_variable_v16i8(<16 x i8> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v16i8: ; KNL-LABEL: test_extractelement_variable_v16i8:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def> ; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %xmm0, -{{[0-9]+}}(%rsp) ; KNL-NEXT: vmovaps %xmm0, -{{[0-9]+}}(%rsp)
; KNL-NEXT: andl $15, %edi ; KNL-NEXT: andl $15, %edi
; KNL-NEXT: leaq -{{[0-9]+}}(%rsp), %rax ; KNL-NEXT: leaq -{{[0-9]+}}(%rsp), %rax
Show All 11 Lines; SKX-NEXT: retq
%t2 = extractelement <16 x i8> %t1, i32 %index %t2 = extractelement <16 x i8> %t1, i32 %index
ret i8 %t2 ret i8 %t2
} }
define i8 @test_extractelement_variable_v32i8(<32 x i8> %t1, i32 %index) { define i8 @test_extractelement_variable_v32i8(<32 x i8> %t1, i32 %index) {
; KNL-LABEL: test_extractelement_variable_v32i8: ; KNL-LABEL: test_extractelement_variable_v32i8:
; KNL: ## BB#0: ; KNL: ## BB#0:
; KNL-NEXT: pushq %rbp ; KNL-NEXT: pushq %rbp
; KNL-NEXT: Lcfi12: ; KNL-NEXT: Lcfi33:
; KNL-NEXT: .cfi_def_cfa_offset 16 ; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi13: ; KNL-NEXT: Lcfi34:
; KNL-NEXT: .cfi_offset %rbp, -16 ; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp ; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi14: ; KNL-NEXT: Lcfi35:
; KNL-NEXT: .cfi_def_cfa_register %rbp ; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-32, %rsp ; KNL-NEXT: andq $-32, %rsp
; KNL-NEXT: subq $64, %rsp ; KNL-NEXT: subq $64, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def> ; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %ymm0, (%rsp) ; KNL-NEXT: vmovaps %ymm0, (%rsp)
; KNL-NEXT: andl $31, %edi ; KNL-NEXT: andl $31, %edi
; KNL-NEXT: movq %rsp, %rax ; KNL-NEXT: movq %rsp, %rax
; KNL-NEXT: movb (%rdi,%rax), %al ; KNL-NEXT: movb (%rdi,%rax), %al
; KNL-NEXT: movq %rbp, %rsp ; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp ; KNL-NEXT: popq %rbp
; KNL-NEXT: retq ; KNL-NEXT: retq
; ;
; SKX-LABEL: test_extractelement_variable_v32i8: ; SKX-LABEL: test_extractelement_variable_v32i8:
; SKX: ## BB#0: ; SKX: ## BB#0:
; SKX-NEXT: pushq %rbp ; SKX-NEXT: pushq %rbp
; SKX-NEXT: Lcfi9: ; SKX-NEXT: Lcfi30:
; SKX-NEXT: .cfi_def_cfa_offset 16 ; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi10: ; SKX-NEXT: Lcfi31:
; SKX-NEXT: .cfi_offset %rbp, -16 ; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp ; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi11: ; SKX-NEXT: Lcfi32:
; SKX-NEXT: .cfi_def_cfa_register %rbp ; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-32, %rsp ; SKX-NEXT: andq $-32, %rsp
; SKX-NEXT: subq $64, %rsp ; SKX-NEXT: subq $64, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def> ; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovdqu %ymm0, (%rsp) ; SKX-NEXT: vmovdqu %ymm0, (%rsp)
; SKX-NEXT: andl $31, %edi ; SKX-NEXT: andl $31, %edi
; SKX-NEXT: movq %rsp, %rax ; SKX-NEXT: movq %rsp, %rax
; SKX-NEXT: movb (%rdi,%rax), %al ; SKX-NEXT: movb (%rdi,%rax), %al
; SKX-NEXT: movq %rbp, %rsp ; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp ; SKX-NEXT: popq %rbp
; SKX-NEXT: retq ; SKX-NEXT: retq
%t2 = extractelement <32 x i8> %t1, i32 %index %t2 = extractelement <32 x i8> %t1, i32 %index
ret i8 %t2 ret i8 %t2
} }
; TODO - enable after fix define i8 @test_extractelement_variable_v64i8(<64 x i8> %t1, i32 %index) {
;define i8 @test_extractelement_variable_v64i8(<64 x i8> %t1, i32 %index) { ; KNL-LABEL: test_extractelement_variable_v64i8:
; %t2 = extractelement <64 x i8> %t1, i32 %index ; KNL: ## BB#0:
; ret i8 %t2 ; KNL-NEXT: pushq %rbp
;} ; KNL-NEXT: Lcfi36:
; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi37:
; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi38:
; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; KNL-NEXT: vmovaps %ymm1, {{[0-9]+}}(%rsp)
; KNL-NEXT: vmovaps %ymm0, (%rsp)
; KNL-NEXT: andl $63, %edi
; KNL-NEXT: movq %rsp, %rax
; KNL-NEXT: movb (%rdi,%rax), %al
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq
;
; SKX-LABEL: test_extractelement_variable_v64i8:
; SKX: ## BB#0:
; SKX-NEXT: pushq %rbp
; SKX-NEXT: Lcfi33:
; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi34:
; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi35:
; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: ## kill: %EDI<def> %EDI<kill> %RDI<def>
; SKX-NEXT: vmovdqu8 %zmm0, (%rsp)
; SKX-NEXT: andl $63, %edi
; SKX-NEXT: movq %rsp, %rax
; SKX-NEXT: movb (%rdi,%rax), %al
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq
%t2 = extractelement <64 x i8> %t1, i32 %index
ret i8 %t2
}
define i8 @test_extractelement_variable_v64i8_indexi8(<64 x i8> %t1, i8 %index) {
; KNL-LABEL: test_extractelement_variable_v64i8_indexi8:
; KNL: ## BB#0:
; KNL-NEXT: pushq %rbp
; KNL-NEXT: Lcfi39:
; KNL-NEXT: .cfi_def_cfa_offset 16
; KNL-NEXT: Lcfi40:
; KNL-NEXT: .cfi_offset %rbp, -16
; KNL-NEXT: movq %rsp, %rbp
; KNL-NEXT: Lcfi41:
; KNL-NEXT: .cfi_def_cfa_register %rbp
; KNL-NEXT: andq $-64, %rsp
; KNL-NEXT: subq $128, %rsp
; KNL-NEXT: addb %dil, %dil
; KNL-NEXT: vmovaps %ymm1, {{[0-9]+}}(%rsp)
; KNL-NEXT: vmovaps %ymm0, (%rsp)
; KNL-NEXT: movzbl %dil, %eax
; KNL-NEXT: andl $63, %eax
; KNL-NEXT: movq %rsp, %rcx
; KNL-NEXT: movb (%rax,%rcx), %al
; KNL-NEXT: movq %rbp, %rsp
; KNL-NEXT: popq %rbp
; KNL-NEXT: retq
;
; SKX-LABEL: test_extractelement_variable_v64i8_indexi8:
; SKX: ## BB#0:
; SKX-NEXT: pushq %rbp
; SKX-NEXT: Lcfi36:
; SKX-NEXT: .cfi_def_cfa_offset 16
; SKX-NEXT: Lcfi37:
; SKX-NEXT: .cfi_offset %rbp, -16
; SKX-NEXT: movq %rsp, %rbp
; SKX-NEXT: Lcfi38:
; SKX-NEXT: .cfi_def_cfa_register %rbp
; SKX-NEXT: andq $-64, %rsp
; SKX-NEXT: subq $128, %rsp
; SKX-NEXT: addb %dil, %dil
; SKX-NEXT: vmovdqu8 %zmm0, (%rsp)
; SKX-NEXT: movzbl %dil, %eax
; SKX-NEXT: andl $63, %eax
; SKX-NEXT: movq %rsp, %rcx
; SKX-NEXT: movb (%rax,%rcx), %al
; SKX-NEXT: movq %rbp, %rsp
; SKX-NEXT: popq %rbp
; SKX-NEXT: retq
%i = add i8 %index, %index
%t2 = extractelement <64 x i8> %t1, i8 %i
ret i8 %t2
}

llvm/trunk/test/CodeGen/X86/extractelement-index.ll

Show First 20 Lines • Show All 532 Lines • ▼ Show 20 Lines
; SSE-NEXT: andl $7, %edi ; SSE-NEXT: andl $7, %edi
; SSE-NEXT: movaps %xmm1, {{[0-9]+}}(%rsp) ; SSE-NEXT: movaps %xmm1, {{[0-9]+}}(%rsp)
; SSE-NEXT: movaps %xmm0, (%rsp) ; SSE-NEXT: movaps %xmm0, (%rsp)
; SSE-NEXT: movl (%rsp,%rdi,4), %eax ; SSE-NEXT: movl (%rsp,%rdi,4), %eax
; SSE-NEXT: movq %rbp, %rsp ; SSE-NEXT: movq %rbp, %rsp
; SSE-NEXT: popq %rbp ; SSE-NEXT: popq %rbp
; SSE-NEXT: retq ; SSE-NEXT: retq
; ;
; AVX1-LABEL: extractelement_v8i32_var: ; AVX-LABEL: extractelement_v8i32_var:
; AVX1: # BB#0: ; AVX: # BB#0:
; AVX1-NEXT: pushq %rbp ; AVX-NEXT: pushq %rbp
; AVX1-NEXT: movq %rsp, %rbp ; AVX-NEXT: movq %rsp, %rbp
; AVX1-NEXT: andq $-32, %rsp ; AVX-NEXT: andq $-32, %rsp
; AVX1-NEXT: subq $64, %rsp ; AVX-NEXT: subq $64, %rsp
; AVX1-NEXT: andl $7, %edi ; AVX-NEXT: andl $7, %edi
; AVX1-NEXT: vmovaps %ymm0, (%rsp) ; AVX-NEXT: vmovaps %ymm0, (%rsp)
; AVX1-NEXT: movl (%rsp,%rdi,4), %eax ; AVX-NEXT: movl (%rsp,%rdi,4), %eax
; AVX1-NEXT: movq %rbp, %rsp ; AVX-NEXT: movq %rbp, %rsp
; AVX1-NEXT: popq %rbp ; AVX-NEXT: popq %rbp
; AVX1-NEXT: vzeroupper ; AVX-NEXT: vzeroupper
; AVX1-NEXT: retq ; AVX-NEXT: retq
;
; AVX2-LABEL: extractelement_v8i32_var:
; AVX2: # BB#0:
; AVX2-NEXT: vmovd %edi, %xmm1
; AVX2-NEXT: vpermd %ymm0, %ymm1, %ymm0
; AVX2-NEXT: vmovd %xmm0, %eax
; AVX2-NEXT: vzeroupper
; AVX2-NEXT: retq
%b = extractelement <8 x i32> %a, i256 %i %b = extractelement <8 x i32> %a, i256 %i
ret i32 %b ret i32 %b
} }
define i64 @extractelement_v2i64_var(<2 x i64> %a, i256 %i) nounwind { define i64 @extractelement_v2i64_var(<2 x i64> %a, i256 %i) nounwind {
; SSE-LABEL: extractelement_v2i64_var: ; SSE-LABEL: extractelement_v2i64_var:
; SSE: # BB#0: ; SSE: # BB#0:
; SSE-NEXT: andl $1, %edi ; SSE-NEXT: andl $1, %edi
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/vector-shuffle-variable-256.ll

Show First 20 Lines • Show All 230 Lines • ▼ Show 20 Lines; AVX2-NEXT: retq
%r0 = insertelement <4 x i64> undef, i64 %x0, i32 0 %r0 = insertelement <4 x i64> undef, i64 %x0, i32 0
%r1 = insertelement <4 x i64> %r0, i64 %x1, i32 1 %r1 = insertelement <4 x i64> %r0, i64 %x1, i32 1
%r2 = insertelement <4 x i64> %r1, i64 %x2, i32 2 %r2 = insertelement <4 x i64> %r1, i64 %x2, i32 2
%r3 = insertelement <4 x i64> %r2, i64 %x3, i32 3 %r3 = insertelement <4 x i64> %r2, i64 %x3, i32 3
ret <4 x i64> %r3 ret <4 x i64> %r3
} }
define <8 x float> @var_shuffle_v8f32_v8f32_xxxxxxxx_i32(<8 x float> %x, i32 %i0, i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7) nounwind { define <8 x float> @var_shuffle_v8f32_v8f32_xxxxxxxx_i32(<8 x float> %x, i32 %i0, i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7) nounwind {
; AVX1-LABEL: var_shuffle_v8f32_v8f32_xxxxxxxx_i32: ; ALL-LABEL: var_shuffle_v8f32_v8f32_xxxxxxxx_i32:
; AVX1: # BB#0: ; ALL: # BB#0:
; AVX1-NEXT: pushq %rbp ; ALL-NEXT: pushq %rbp
; AVX1-NEXT: movq %rsp, %rbp ; ALL-NEXT: movq %rsp, %rbp
; AVX1-NEXT: andq $-32, %rsp ; ALL-NEXT: andq $-32, %rsp
; AVX1-NEXT: subq $64, %rsp ; ALL-NEXT: subq $64, %rsp
; AVX1-NEXT: # kill: %R9D<def> %R9D<kill> %R9<def> ; ALL-NEXT: # kill: %R9D<def> %R9D<kill> %R9<def>
; AVX1-NEXT: # kill: %R8D<def> %R8D<kill> %R8<def> ; ALL-NEXT: # kill: %R8D<def> %R8D<kill> %R8<def>
; AVX1-NEXT: # kill: %ECX<def> %ECX<kill> %RCX<def> ; ALL-NEXT: # kill: %ECX<def> %ECX<kill> %RCX<def>
; AVX1-NEXT: # kill: %EDX<def> %EDX<kill> %RDX<def> ; ALL-NEXT: # kill: %EDX<def> %EDX<kill> %RDX<def>
; AVX1-NEXT: # kill: %ESI<def> %ESI<kill> %RSI<def> ; ALL-NEXT: # kill: %ESI<def> %ESI<kill> %RSI<def>
; AVX1-NEXT: # kill: %EDI<def> %EDI<kill> %RDI<def> ; ALL-NEXT: # kill: %EDI<def> %EDI<kill> %RDI<def>
; AVX1-NEXT: andl $7, %edi ; ALL-NEXT: andl $7, %edi
; AVX1-NEXT: andl $7, %esi ; ALL-NEXT: andl $7, %esi
; AVX1-NEXT: andl $7, %edx ; ALL-NEXT: andl $7, %edx
; AVX1-NEXT: andl $7, %ecx ; ALL-NEXT: andl $7, %ecx
; AVX1-NEXT: andl $7, %r8d ; ALL-NEXT: andl $7, %r8d
; AVX1-NEXT: vmovaps %ymm0, (%rsp) ; ALL-NEXT: vmovaps %ymm0, (%rsp)
; AVX1-NEXT: andl $7, %r9d ; ALL-NEXT: andl $7, %r9d
; AVX1-NEXT: movl 16(%rbp), %r10d ; ALL-NEXT: movl 16(%rbp), %r10d
; AVX1-NEXT: andl $7, %r10d ; ALL-NEXT: andl $7, %r10d
; AVX1-NEXT: movl 24(%rbp), %eax ; ALL-NEXT: movl 24(%rbp), %eax
; AVX1-NEXT: andl $7, %eax ; ALL-NEXT: andl $7, %eax
; AVX1-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero ; ALL-NEXT: vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; AVX1-NEXT: vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero ; ALL-NEXT: vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
; AVX1-NEXT: vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero ; ALL-NEXT: vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; AVX1-NEXT: vinsertps {{.*#+}} xmm2 = xmm2[0],mem[0],xmm2[2,3] ; ALL-NEXT: vinsertps {{.*#+}} xmm2 = xmm2[0],mem[0],xmm2[2,3]
; AVX1-NEXT: vinsertps {{.*#+}} xmm2 = xmm2[0,1],mem[0],xmm2[3] ; ALL-NEXT: vinsertps {{.*#+}} xmm2 = xmm2[0,1],mem[0],xmm2[3]
; AVX1-NEXT: vinsertps {{.*#+}} xmm2 = xmm2[0,1,2],mem[0] ; ALL-NEXT: vinsertps {{.*#+}} xmm2 = xmm2[0,1,2],mem[0]
; AVX1-NEXT: vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero ; ALL-NEXT: vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; AVX1-NEXT: vinsertps {{.*#+}} xmm3 = xmm3[0],mem[0],xmm3[2,3] ; ALL-NEXT: vinsertps {{.*#+}} xmm3 = xmm3[0],mem[0],xmm3[2,3]
; AVX1-NEXT: vinsertps {{.*#+}} xmm0 = xmm3[0,1],xmm0[0],xmm3[3] ; ALL-NEXT: vinsertps {{.*#+}} xmm0 = xmm3[0,1],xmm0[0],xmm3[3]
; AVX1-NEXT: vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0] ; ALL-NEXT: vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm2, %ymm0 ; ALL-NEXT: vinsertf128 $1, %xmm0, %ymm2, %ymm0
; AVX1-NEXT: movq %rbp, %rsp ; ALL-NEXT: movq %rbp, %rsp
; AVX1-NEXT: popq %rbp ; ALL-NEXT: popq %rbp
; AVX1-NEXT: retq ; ALL-NEXT: retq
;
; AVX2-LABEL: var_shuffle_v8f32_v8f32_xxxxxxxx_i32:
; AVX2: # BB#0:
; AVX2-NEXT: vmovd %edi, %xmm1
; AVX2-NEXT: vpermps %ymm0, %ymm1, %ymm1
; AVX2-NEXT: vmovd %esi, %xmm2
; AVX2-NEXT: vpermps %ymm0, %ymm2, %ymm2
; AVX2-NEXT: vmovd %edx, %xmm3
; AVX2-NEXT: vpermps %ymm0, %ymm3, %ymm3
; AVX2-NEXT: vmovd %ecx, %xmm4
; AVX2-NEXT: vpermps %ymm0, %ymm4, %ymm4
; AVX2-NEXT: vmovd %r8d, %xmm5
; AVX2-NEXT: vpermps %ymm0, %ymm5, %ymm5
; AVX2-NEXT: vmovd %r9d, %xmm6
; AVX2-NEXT: vpermps %ymm0, %ymm6, %ymm6
; AVX2-NEXT: vmovss {{.*#+}} xmm7 = mem[0],zero,zero,zero
; AVX2-NEXT: vpermps %ymm0, %ymm7, %ymm7
; AVX2-NEXT: vmovss {{.*#+}} xmm8 = mem[0],zero,zero,zero
; AVX2-NEXT: vpermps %ymm0, %ymm8, %ymm0
; AVX2-NEXT: vinsertps {{.*#+}} xmm5 = xmm5[0],xmm6[0],xmm5[2,3]
; AVX2-NEXT: vinsertps {{.*#+}} xmm5 = xmm5[0,1],xmm7[0],xmm5[3]
; AVX2-NEXT: vinsertps {{.*#+}} xmm0 = xmm5[0,1,2],xmm0[0]
; AVX2-NEXT: vinsertps {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[2,3]
; AVX2-NEXT: vinsertps {{.*#+}} xmm1 = xmm1[0,1],xmm3[0],xmm1[3]
; AVX2-NEXT: vinsertps {{.*#+}} xmm1 = xmm1[0,1,2],xmm4[0]
; AVX2-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; AVX2-NEXT: retq
%x0 = extractelement <8 x float> %x, i32 %i0 %x0 = extractelement <8 x float> %x, i32 %i0
%x1 = extractelement <8 x float> %x, i32 %i1 %x1 = extractelement <8 x float> %x, i32 %i1
%x2 = extractelement <8 x float> %x, i32 %i2 %x2 = extractelement <8 x float> %x, i32 %i2
%x3 = extractelement <8 x float> %x, i32 %i3 %x3 = extractelement <8 x float> %x, i32 %i3
%x4 = extractelement <8 x float> %x, i32 %i4 %x4 = extractelement <8 x float> %x, i32 %i4
%x5 = extractelement <8 x float> %x, i32 %i5 %x5 = extractelement <8 x float> %x, i32 %i5
%x6 = extractelement <8 x float> %x, i32 %i6 %x6 = extractelement <8 x float> %x, i32 %i6
%x7 = extractelement <8 x float> %x, i32 %i7 %x7 = extractelement <8 x float> %x, i32 %i7
▲ Show 20 Lines • Show All 537 Lines • Show Last 20 Lines