This is an archive of the discontinued LLVM Phabricator instance.

Differential D116191

[X86] isVectorShiftByScalarCheap - vXi8 select(shift(x,splat0),shift(x,splat1)) is better than shift(x,select(splat0,splat1))
ClosedPublic

Authored by RKSimon on Dec 22 2021, 2:34 PM.

Details

Reviewers
spatel
pengfei
craig.topper
lebedev.ri
Commits
rGa3f50fb06dd4: [X86] isVectorShiftByScalarCheap - vXi8 select(shift(x,splat0),shift(x,splat1))…
Summary

Even though we don't have vXi8 vector shifts (apart from XOP), it is still better to prefer shift (or funnel-shift/rotate) by scalar where possible.

https://llvm.godbolt.org/z/6ss6ffTxv

Diff Detail

Event Timeline

RKSimon created this revision.Dec 22 2021, 2:34 PM
Herald added subscribers: hiraditya, tschuett. · View Herald TranscriptDec 22 2021, 2:34 PM
RKSimon requested review of this revision.Dec 22 2021, 2:34 PM
Herald added a project: Restricted Project. · View Herald TranscriptDec 22 2021, 2:34 PM
Harbormaster completed remote builds in B140445: Diff 395933.Dec 22 2021, 3:15 PM
pengfei added inline comments.Dec 22 2021, 6:25 PM
llvm/lib/Target/X86/X86ISelLowering.cpp
32997–33001

I'm confused by the summary and the code here. Do we prefer to shifts on both has XOP and not? Why do we always return false for XOP?

RKSimon added inline comments.Dec 23 2021, 2:39 AM
llvm/lib/Target/X86/X86ISelLowering.cpp
32997–33001

XOP has awesome per-element shift ops for all 128-bit vector types - so there's no need to prefer vector shifts by scalar amounts over per-element amounts.

pengfei accepted this revision.Dec 23 2021, 4:10 AM

LGTM.

llvm/lib/Target/X86/X86ISelLowering.cpp
32997–33001

Got it. Thanks!

This revision is now accepted and ready to land.Dec 23 2021, 4:10 AM
This revision was landed with ongoing or failed builds.Dec 23 2021, 6:30 AM
Closed by commit rGa3f50fb06dd4: [X86] isVectorShiftByScalarCheap - vXi8 select(shift(x,splat0),shift(x,splat1))… (authored by RKSimon). · Explain Why
This revision was automatically updated to reflect the committed changes.
RKSimon added a commit: rGa3f50fb06dd4: [X86] isVectorShiftByScalarCheap - vXi8 select(shift(x,splat0),shift(x,splat1))….

Revision Contents

PathSize
llvm/
lib/
Target/
X86/
5 lines
test/
Transforms/
CodeGenPrepare/
X86/
54 lines
49 lines
36 lines
36 lines

Diff 396019

llvm/lib/Target/X86/X86ISelLowering.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 32,988 Lines • ▼ Show 20 Linesbool X86TargetLowering::isLegalAddressingMode(const DataLayout &DL,
} }
return true; return true;
} }
bool X86TargetLowering::isVectorShiftByScalarCheap(Type *Ty) const { bool X86TargetLowering::isVectorShiftByScalarCheap(Type *Ty) const {
unsigned Bits = Ty->getScalarSizeInBits(); unsigned Bits = Ty->getScalarSizeInBits();
// 8-bit shifts are always expensive, but versions with a scalar amount aren't
// particularly cheaper than those without.
if (Bits == 8)
return false;
// XOP has v16i8/v8i16/v4i32/v2i64 variable vector shifts. // XOP has v16i8/v8i16/v4i32/v2i64 variable vector shifts.
// Splitting for v32i8/v16i16 on XOP+AVX2 targets is still preferred. // Splitting for v32i8/v16i16 on XOP+AVX2 targets is still preferred.
if (Subtarget.hasXOP() && if (Subtarget.hasXOP() &&
(Bits == 8 || Bits == 16 || Bits == 32 || Bits == 64)) (Bits == 8 || Bits == 16 || Bits == 32 || Bits == 64))
return false; return false;
pengfeiUnsubmitted
Not Done
ReplyInline Actions

I'm confused by the summary and the code here. Do we prefer to shifts on both has XOP and not? Why do we always return false for XOP?

pengfei: I'm confused by the summary and the code here. Do we prefer to shifts on both has XOP and not?
RKSimonAuthorUnsubmitted
Not Done
ReplyInline Actions

XOP has awesome per-element shift ops for all 128-bit vector types - so there's no need to prefer vector shifts by scalar amounts over per-element amounts.

RKSimon: XOP has awesome per-element shift ops for all 128-bit vector types - so there's no need to…
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Got it. Thanks!

pengfei: Got it. Thanks!
// AVX2 has vpsllv[dq] instructions (and other shifts) that make variable // AVX2 has vpsllv[dq] instructions (and other shifts) that make variable
// shifts just as cheap as scalar ones. // shifts just as cheap as scalar ones.
if (Subtarget.hasAVX2() && (Bits == 32 || Bits == 64)) if (Subtarget.hasAVX2() && (Bits == 32 || Bits == 64))
return false; return false;
// AVX512BW has shifts such as vpsllvw. // AVX512BW has shifts such as vpsllvw.
if (Subtarget.hasBWI() && Bits == 16) if (Subtarget.hasBWI() && Bits == 16)
▲ Show 20 Lines • Show All 21,656 Lines • Show Last 20 Lines

llvm/test/Transforms/CodeGenPrepare/X86/vec-shift-inseltpoison.ll

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx -S < %s | FileCheck %s --check-prefixes=ALL,AVX1 ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx -S < %s | FileCheck %s --check-prefixes=AVX1
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx2 -S < %s | FileCheck %s --check-prefixes=ALL,AVX2 ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx2 -S < %s | FileCheck %s --check-prefixes=AVX2
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx512bw -S < %s | FileCheck %s --check-prefixes=ALL,AVX512BW ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx512bw -S < %s | FileCheck %s --check-prefixes=AVX512BW
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx,+xop -S < %s | FileCheck %s --check-prefixes=ALL,XOP ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx,+xop -S < %s | FileCheck %s --check-prefixes=XOP
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx2,+xop -S < %s | FileCheck %s --check-prefixes=ALL,XOP ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx2,+xop -S < %s | FileCheck %s --check-prefixes=XOP
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx -S -enable-debugify < %s 2>&1 | FileCheck %s -check-prefix=DEBUG ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx -S -enable-debugify < %s 2>&1 | FileCheck %s -check-prefix=DEBUG
define <4 x i32> @vector_variable_shift_right_v4i32(<4 x i1> %cond, <4 x i32> %x, <4 x i32> %y, <4 x i32> %z) { define <4 x i32> @vector_variable_shift_right_v4i32(<4 x i1> %cond, <4 x i32> %x, <4 x i32> %y, <4 x i32> %z) {
; AVX1-LABEL: @vector_variable_shift_right_v4i32( ; AVX1-LABEL: @vector_variable_shift_right_v4i32(
; AVX1-NEXT: [[SPLAT1:%.*]] = shufflevector <4 x i32> [[X:%.*]], <4 x i32> poison, <4 x i32> zeroinitializer ; AVX1-NEXT: [[SPLAT1:%.*]] = shufflevector <4 x i32> [[X:%.*]], <4 x i32> poison, <4 x i32> zeroinitializer
; AVX1-NEXT: [[SPLAT2:%.*]] = shufflevector <4 x i32> [[Y:%.*]], <4 x i32> poison, <4 x i32> zeroinitializer ; AVX1-NEXT: [[SPLAT2:%.*]] = shufflevector <4 x i32> [[Y:%.*]], <4 x i32> poison, <4 x i32> zeroinitializer
; AVX1-NEXT: [[SEL:%.*]] = select <4 x i1> [[COND:%.*]], <4 x i32> [[SPLAT1]], <4 x i32> [[SPLAT2]] ; AVX1-NEXT: [[SEL:%.*]] = select <4 x i1> [[COND:%.*]], <4 x i32> [[SPLAT1]], <4 x i32> [[SPLAT2]]
; AVX1-NEXT: [[TMP1:%.*]] = lshr <4 x i32> [[Z:%.*]], [[SPLAT1]] ; AVX1-NEXT: [[TMP1:%.*]] = lshr <4 x i32> [[Z:%.*]], [[SPLAT1]]
Show All 17 Lines
; ;
; XOP-LABEL: @vector_variable_shift_right_v4i32( ; XOP-LABEL: @vector_variable_shift_right_v4i32(
; XOP-NEXT: [[SPLAT1:%.*]] = shufflevector <4 x i32> [[X:%.*]], <4 x i32> poison, <4 x i32> zeroinitializer ; XOP-NEXT: [[SPLAT1:%.*]] = shufflevector <4 x i32> [[X:%.*]], <4 x i32> poison, <4 x i32> zeroinitializer
; XOP-NEXT: [[SPLAT2:%.*]] = shufflevector <4 x i32> [[Y:%.*]], <4 x i32> poison, <4 x i32> zeroinitializer ; XOP-NEXT: [[SPLAT2:%.*]] = shufflevector <4 x i32> [[Y:%.*]], <4 x i32> poison, <4 x i32> zeroinitializer
; XOP-NEXT: [[SEL:%.*]] = select <4 x i1> [[COND:%.*]], <4 x i32> [[SPLAT1]], <4 x i32> [[SPLAT2]] ; XOP-NEXT: [[SEL:%.*]] = select <4 x i1> [[COND:%.*]], <4 x i32> [[SPLAT1]], <4 x i32> [[SPLAT2]]
; XOP-NEXT: [[SH:%.*]] = lshr <4 x i32> [[Z:%.*]], [[SEL]] ; XOP-NEXT: [[SH:%.*]] = lshr <4 x i32> [[Z:%.*]], [[SEL]]
; XOP-NEXT: ret <4 x i32> [[SH]] ; XOP-NEXT: ret <4 x i32> [[SH]]
; ;
%splat1 = shufflevector <4 x i32> %x, <4 x i32> poison, <4 x i32> zeroinitializer %splat1 = shufflevector <4 x i32> %x, <4 x i32> poison, <4 x i32> zeroinitializer
%splat2 = shufflevector <4 x i32> %y, <4 x i32> poison, <4 x i32> zeroinitializer %splat2 = shufflevector <4 x i32> %y, <4 x i32> poison, <4 x i32> zeroinitializer
%sel = select <4 x i1> %cond, <4 x i32> %splat1, <4 x i32> %splat2 %sel = select <4 x i1> %cond, <4 x i32> %splat1, <4 x i32> %splat2
%sh = lshr <4 x i32> %z, %sel %sh = lshr <4 x i32> %z, %sel
ret <4 x i32> %sh ret <4 x i32> %sh
} }
define <16 x i16> @vector_variable_shift_right_v16i16(<16 x i1> %cond, <16 x i16> %x, <16 x i16> %y, <16 x i16> %z) { define <16 x i16> @vector_variable_shift_right_v16i16(<16 x i1> %cond, <16 x i16> %x, <16 x i16> %y, <16 x i16> %z) {
Show All 24 Lines
; ;
; XOP-LABEL: @vector_variable_shift_right_v16i16( ; XOP-LABEL: @vector_variable_shift_right_v16i16(
; XOP-NEXT: [[SPLAT1:%.*]] = shufflevector <16 x i16> [[X:%.*]], <16 x i16> poison, <16 x i32> zeroinitializer ; XOP-NEXT: [[SPLAT1:%.*]] = shufflevector <16 x i16> [[X:%.*]], <16 x i16> poison, <16 x i32> zeroinitializer
; XOP-NEXT: [[SPLAT2:%.*]] = shufflevector <16 x i16> [[Y:%.*]], <16 x i16> poison, <16 x i32> zeroinitializer ; XOP-NEXT: [[SPLAT2:%.*]] = shufflevector <16 x i16> [[Y:%.*]], <16 x i16> poison, <16 x i32> zeroinitializer
; XOP-NEXT: [[SEL:%.*]] = select <16 x i1> [[COND:%.*]], <16 x i16> [[SPLAT1]], <16 x i16> [[SPLAT2]] ; XOP-NEXT: [[SEL:%.*]] = select <16 x i1> [[COND:%.*]], <16 x i16> [[SPLAT1]], <16 x i16> [[SPLAT2]]
; XOP-NEXT: [[SH:%.*]] = lshr <16 x i16> [[Z:%.*]], [[SEL]] ; XOP-NEXT: [[SH:%.*]] = lshr <16 x i16> [[Z:%.*]], [[SEL]]
; XOP-NEXT: ret <16 x i16> [[SH]] ; XOP-NEXT: ret <16 x i16> [[SH]]
; ;
%splat1 = shufflevector <16 x i16> %x, <16 x i16> poison, <16 x i32> zeroinitializer %splat1 = shufflevector <16 x i16> %x, <16 x i16> poison, <16 x i32> zeroinitializer
%splat2 = shufflevector <16 x i16> %y, <16 x i16> poison, <16 x i32> zeroinitializer %splat2 = shufflevector <16 x i16> %y, <16 x i16> poison, <16 x i32> zeroinitializer
%sel = select <16 x i1> %cond, <16 x i16> %splat1, <16 x i16> %splat2 %sel = select <16 x i1> %cond, <16 x i16> %splat1, <16 x i16> %splat2
%sh = lshr <16 x i16> %z, %sel %sh = lshr <16 x i16> %z, %sel
ret <16 x i16> %sh ret <16 x i16> %sh
} }
define <32 x i8> @vector_variable_shift_right_v32i8(<32 x i1> %cond, <32 x i8> %x, <32 x i8> %y, <32 x i8> %z) { define <32 x i8> @vector_variable_shift_right_v32i8(<32 x i1> %cond, <32 x i8> %x, <32 x i8> %y, <32 x i8> %z) {
; ALL-LABEL: @vector_variable_shift_right_v32i8( ; AVX1-LABEL: @vector_variable_shift_right_v32i8(
; ALL-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer ; AVX1-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer
; ALL-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer ; AVX1-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer
; ALL-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]] ; AVX1-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]]
; ALL-NEXT: [[SH:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SEL]] ; AVX1-NEXT: [[TMP1:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SPLAT1]]
; ALL-NEXT: ret <32 x i8> [[SH]] ; AVX1-NEXT: [[TMP2:%.*]] = lshr <32 x i8> [[Z]], [[SPLAT2]]
; AVX1-NEXT: [[TMP3:%.*]] = select <32 x i1> [[COND]], <32 x i8> [[TMP1]], <32 x i8> [[TMP2]]
; AVX1-NEXT: ret <32 x i8> [[TMP3]]
;
; AVX2-LABEL: @vector_variable_shift_right_v32i8(
; AVX2-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer
; AVX2-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer
; AVX2-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]]
; AVX2-NEXT: [[TMP1:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SPLAT1]]
; AVX2-NEXT: [[TMP2:%.*]] = lshr <32 x i8> [[Z]], [[SPLAT2]]
; AVX2-NEXT: [[TMP3:%.*]] = select <32 x i1> [[COND]], <32 x i8> [[TMP1]], <32 x i8> [[TMP2]]
; AVX2-NEXT: ret <32 x i8> [[TMP3]]
;
; AVX512BW-LABEL: @vector_variable_shift_right_v32i8(
; AVX512BW-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer
; AVX512BW-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer
; AVX512BW-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]]
; AVX512BW-NEXT: [[TMP1:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SPLAT1]]
; AVX512BW-NEXT: [[TMP2:%.*]] = lshr <32 x i8> [[Z]], [[SPLAT2]]
; AVX512BW-NEXT: [[TMP3:%.*]] = select <32 x i1> [[COND]], <32 x i8> [[TMP1]], <32 x i8> [[TMP2]]
; AVX512BW-NEXT: ret <32 x i8> [[TMP3]]
;
; XOP-LABEL: @vector_variable_shift_right_v32i8(
; XOP-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer
; XOP-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> poison, <32 x i32> zeroinitializer
; XOP-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]]
; XOP-NEXT: [[SH:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SEL]]
; XOP-NEXT: ret <32 x i8> [[SH]]
; ;
%splat1 = shufflevector <32 x i8> %x, <32 x i8> poison, <32 x i32> zeroinitializer %splat1 = shufflevector <32 x i8> %x, <32 x i8> poison, <32 x i32> zeroinitializer
%splat2 = shufflevector <32 x i8> %y, <32 x i8> poison, <32 x i32> zeroinitializer %splat2 = shufflevector <32 x i8> %y, <32 x i8> poison, <32 x i32> zeroinitializer
%sel = select <32 x i1> %cond, <32 x i8> %splat1, <32 x i8> %splat2 %sel = select <32 x i1> %cond, <32 x i8> %splat1, <32 x i8> %splat2
%sh = lshr <32 x i8> %z, %sel %sh = lshr <32 x i8> %z, %sel
ret <32 x i8> %sh ret <32 x i8> %sh
} }
; PR37428 - https://bugs.llvm.org/show_bug.cgi?id=37428 ; PR37428 - https://bugs.llvm.org/show_bug.cgi?id=37428
▲ Show 20 Lines • Show All 122 Lines • ▼ Show 20 Lines
; XOP-NEXT: [[TMP6:%.*]] = bitcast i32* [[TMP5]] to <4 x i32>* ; XOP-NEXT: [[TMP6:%.*]] = bitcast i32* [[TMP5]] to <4 x i32>*
; XOP-NEXT: store <4 x i32> [[TMP4]], <4 x i32>* [[TMP6]], align 4 ; XOP-NEXT: store <4 x i32> [[TMP4]], <4 x i32>* [[TMP6]], align 4
; XOP-NEXT: [[INDEX_NEXT]] = add i64 [[INDEX]], 4 ; XOP-NEXT: [[INDEX_NEXT]] = add i64 [[INDEX]], 4
; XOP-NEXT: [[TMP7:%.*]] = icmp eq i64 [[INDEX_NEXT]], [[N_VEC]] ; XOP-NEXT: [[TMP7:%.*]] = icmp eq i64 [[INDEX_NEXT]], [[N_VEC]]
; XOP-NEXT: br i1 [[TMP7]], label [[EXIT]], label [[VECTOR_BODY]] ; XOP-NEXT: br i1 [[TMP7]], label [[EXIT]], label [[VECTOR_BODY]]
; XOP: exit: ; XOP: exit:
; XOP-NEXT: ret void ; XOP-NEXT: ret void
; ;
entry: entry:
%cmp16 = icmp sgt i32 %count, 0 %cmp16 = icmp sgt i32 %count, 0
%wide.trip.count = zext i32 %count to i64 %wide.trip.count = zext i32 %count to i64
br i1 %cmp16, label %vector.ph, label %exit br i1 %cmp16, label %vector.ph, label %exit
vector.ph: vector.ph:
%n.vec = and i64 %wide.trip.count, 4294967292 %n.vec = and i64 %wide.trip.count, 4294967292
%splatinsert18 = insertelement <4 x i32> poison, i32 %amt0, i32 0 %splatinsert18 = insertelement <4 x i32> poison, i32 %amt0, i32 0
▲ Show 20 Lines • Show All 127 Lines • ▼ Show 20 Lines
; XOP-NEXT: [[ROT:%.*]] = call <8 x i32> @llvm.fshl.v8i32(<8 x i32> [[WIDE_LOAD21]], <8 x i32> [[WIDE_LOAD21]], <8 x i32> [[SHAMT]]) ; XOP-NEXT: [[ROT:%.*]] = call <8 x i32> @llvm.fshl.v8i32(<8 x i32> [[WIDE_LOAD21]], <8 x i32> [[WIDE_LOAD21]], <8 x i32> [[SHAMT]])
; XOP-NEXT: store <8 x i32> [[ROT]], <8 x i32>* [[T5]], align 4 ; XOP-NEXT: store <8 x i32> [[ROT]], <8 x i32>* [[T5]], align 4
; XOP-NEXT: [[INDEX_NEXT]] = add i64 [[INDEX]], 8 ; XOP-NEXT: [[INDEX_NEXT]] = add i64 [[INDEX]], 8
; XOP-NEXT: [[T7:%.*]] = icmp eq i64 [[INDEX_NEXT]], 1024 ; XOP-NEXT: [[T7:%.*]] = icmp eq i64 [[INDEX_NEXT]], 1024
; XOP-NEXT: br i1 [[T7]], label [[EXIT:%.*]], label [[LOOP]] ; XOP-NEXT: br i1 [[T7]], label [[EXIT:%.*]], label [[LOOP]]
; XOP: exit: ; XOP: exit:
; XOP-NEXT: ret void ; XOP-NEXT: ret void
; ;
entry: entry:
%i0 = insertelement <8 x i32> poison, i32 %rot0, i32 0 %i0 = insertelement <8 x i32> poison, i32 %rot0, i32 0
%s0 = shufflevector <8 x i32> %i0, <8 x i32> poison, <8 x i32> zeroinitializer %s0 = shufflevector <8 x i32> %i0, <8 x i32> poison, <8 x i32> zeroinitializer
%i1 = insertelement <8 x i32> poison, i32 %rot1, i32 0 %i1 = insertelement <8 x i32> poison, i32 %rot1, i32 0
%s1 = shufflevector <8 x i32> %i1, <8 x i32> poison, <8 x i32> zeroinitializer %s1 = shufflevector <8 x i32> %i1, <8 x i32> poison, <8 x i32> zeroinitializer
br label %loop br label %loop
loop: loop:
Show All 23 Lines

llvm/test/Transforms/CodeGenPrepare/X86/vec-shift.ll

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx -S < %s | FileCheck %s --check-prefixes=ALL,AVX1 ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx -S < %s | FileCheck %s --check-prefixes=AVX1
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx2 -S < %s | FileCheck %s --check-prefixes=ALL,AVX2 ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx2 -S < %s | FileCheck %s --check-prefixes=AVX2
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx512bw -S < %s | FileCheck %s --check-prefixes=ALL,AVX512BW ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx512bw -S < %s | FileCheck %s --check-prefixes=AVX512BW
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx,+xop -S < %s | FileCheck %s --check-prefixes=ALL,XOP ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx,+xop -S < %s | FileCheck %s --check-prefixes=XOP
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx2,+xop -S < %s | FileCheck %s --check-prefixes=ALL,XOP ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx2,+xop -S < %s | FileCheck %s --check-prefixes=XOP
; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx -S -enable-debugify < %s 2>&1 | FileCheck %s -check-prefix=DEBUG ; RUN: opt -codegenprepare -mtriple=x86_64-- -mattr=+avx -S -enable-debugify < %s 2>&1 | FileCheck %s -check-prefix=DEBUG
define <4 x i32> @vector_variable_shift_right_v4i32(<4 x i1> %cond, <4 x i32> %x, <4 x i32> %y, <4 x i32> %z) { define <4 x i32> @vector_variable_shift_right_v4i32(<4 x i1> %cond, <4 x i32> %x, <4 x i32> %y, <4 x i32> %z) {
; AVX1-LABEL: @vector_variable_shift_right_v4i32( ; AVX1-LABEL: @vector_variable_shift_right_v4i32(
; AVX1-NEXT: [[SPLAT1:%.*]] = shufflevector <4 x i32> [[X:%.*]], <4 x i32> undef, <4 x i32> zeroinitializer ; AVX1-NEXT: [[SPLAT1:%.*]] = shufflevector <4 x i32> [[X:%.*]], <4 x i32> undef, <4 x i32> zeroinitializer
; AVX1-NEXT: [[SPLAT2:%.*]] = shufflevector <4 x i32> [[Y:%.*]], <4 x i32> undef, <4 x i32> zeroinitializer ; AVX1-NEXT: [[SPLAT2:%.*]] = shufflevector <4 x i32> [[Y:%.*]], <4 x i32> undef, <4 x i32> zeroinitializer
; AVX1-NEXT: [[SEL:%.*]] = select <4 x i1> [[COND:%.*]], <4 x i32> [[SPLAT1]], <4 x i32> [[SPLAT2]] ; AVX1-NEXT: [[SEL:%.*]] = select <4 x i1> [[COND:%.*]], <4 x i32> [[SPLAT1]], <4 x i32> [[SPLAT2]]
; AVX1-NEXT: [[TMP1:%.*]] = lshr <4 x i32> [[Z:%.*]], [[SPLAT1]] ; AVX1-NEXT: [[TMP1:%.*]] = lshr <4 x i32> [[Z:%.*]], [[SPLAT1]]
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines;
%splat1 = shufflevector <16 x i16> %x, <16 x i16> undef, <16 x i32> zeroinitializer %splat1 = shufflevector <16 x i16> %x, <16 x i16> undef, <16 x i32> zeroinitializer
%splat2 = shufflevector <16 x i16> %y, <16 x i16> undef, <16 x i32> zeroinitializer %splat2 = shufflevector <16 x i16> %y, <16 x i16> undef, <16 x i32> zeroinitializer
%sel = select <16 x i1> %cond, <16 x i16> %splat1, <16 x i16> %splat2 %sel = select <16 x i1> %cond, <16 x i16> %splat1, <16 x i16> %splat2
%sh = lshr <16 x i16> %z, %sel %sh = lshr <16 x i16> %z, %sel
ret <16 x i16> %sh ret <16 x i16> %sh
} }
define <32 x i8> @vector_variable_shift_right_v32i8(<32 x i1> %cond, <32 x i8> %x, <32 x i8> %y, <32 x i8> %z) { define <32 x i8> @vector_variable_shift_right_v32i8(<32 x i1> %cond, <32 x i8> %x, <32 x i8> %y, <32 x i8> %z) {
; ALL-LABEL: @vector_variable_shift_right_v32i8( ; AVX1-LABEL: @vector_variable_shift_right_v32i8(
; ALL-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer ; AVX1-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer
; ALL-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer ; AVX1-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer
; ALL-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]] ; AVX1-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]]
; ALL-NEXT: [[SH:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SEL]] ; AVX1-NEXT: [[TMP1:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SPLAT1]]
; ALL-NEXT: ret <32 x i8> [[SH]] ; AVX1-NEXT: [[TMP2:%.*]] = lshr <32 x i8> [[Z]], [[SPLAT2]]
; AVX1-NEXT: [[TMP3:%.*]] = select <32 x i1> [[COND]], <32 x i8> [[TMP1]], <32 x i8> [[TMP2]]
; AVX1-NEXT: ret <32 x i8> [[TMP3]]
;
; AVX2-LABEL: @vector_variable_shift_right_v32i8(
; AVX2-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer
; AVX2-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer
; AVX2-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]]
; AVX2-NEXT: [[TMP1:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SPLAT1]]
; AVX2-NEXT: [[TMP2:%.*]] = lshr <32 x i8> [[Z]], [[SPLAT2]]
; AVX2-NEXT: [[TMP3:%.*]] = select <32 x i1> [[COND]], <32 x i8> [[TMP1]], <32 x i8> [[TMP2]]
; AVX2-NEXT: ret <32 x i8> [[TMP3]]
;
; AVX512BW-LABEL: @vector_variable_shift_right_v32i8(
; AVX512BW-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer
; AVX512BW-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer
; AVX512BW-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]]
; AVX512BW-NEXT: [[TMP1:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SPLAT1]]
; AVX512BW-NEXT: [[TMP2:%.*]] = lshr <32 x i8> [[Z]], [[SPLAT2]]
; AVX512BW-NEXT: [[TMP3:%.*]] = select <32 x i1> [[COND]], <32 x i8> [[TMP1]], <32 x i8> [[TMP2]]
; AVX512BW-NEXT: ret <32 x i8> [[TMP3]]
;
; XOP-LABEL: @vector_variable_shift_right_v32i8(
; XOP-NEXT: [[SPLAT1:%.*]] = shufflevector <32 x i8> [[X:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer
; XOP-NEXT: [[SPLAT2:%.*]] = shufflevector <32 x i8> [[Y:%.*]], <32 x i8> undef, <32 x i32> zeroinitializer
; XOP-NEXT: [[SEL:%.*]] = select <32 x i1> [[COND:%.*]], <32 x i8> [[SPLAT1]], <32 x i8> [[SPLAT2]]
; XOP-NEXT: [[SH:%.*]] = lshr <32 x i8> [[Z:%.*]], [[SEL]]
; XOP-NEXT: ret <32 x i8> [[SH]]
; ;
%splat1 = shufflevector <32 x i8> %x, <32 x i8> undef, <32 x i32> zeroinitializer %splat1 = shufflevector <32 x i8> %x, <32 x i8> undef, <32 x i32> zeroinitializer
%splat2 = shufflevector <32 x i8> %y, <32 x i8> undef, <32 x i32> zeroinitializer %splat2 = shufflevector <32 x i8> %y, <32 x i8> undef, <32 x i32> zeroinitializer
%sel = select <32 x i1> %cond, <32 x i8> %splat1, <32 x i8> %splat2 %sel = select <32 x i1> %cond, <32 x i8> %splat1, <32 x i8> %splat2
%sh = lshr <32 x i8> %z, %sel %sh = lshr <32 x i8> %z, %sel
ret <32 x i8> %sh ret <32 x i8> %sh
} }
▲ Show 20 Lines • Show All 305 Lines • Show Last 20 Lines

llvm/test/Transforms/CodeGenPrepare/X86/x86-shuffle-sink-inseltpoison.ll

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt -S -codegenprepare -mcpu=corei7 %s | FileCheck %s --check-prefixes=CHECK,CHECK-SSE2 ; RUN: opt -S -codegenprepare -mcpu=corei7 %s | FileCheck %s --check-prefixes=CHECK,CHECK-SSE2
; RUN: opt -S -codegenprepare -mcpu=bdver2 %s | FileCheck %s --check-prefixes=CHECK,CHECK-XOP ; RUN: opt -S -codegenprepare -mcpu=bdver2 %s | FileCheck %s --check-prefixes=CHECK,CHECK-XOP
; RUN: opt -S -codegenprepare -mcpu=core-avx2 %s | FileCheck %s --check-prefixes=CHECK,CHECK-AVX,CHECK-AVX2 ; RUN: opt -S -codegenprepare -mcpu=core-avx2 %s | FileCheck %s --check-prefixes=CHECK,CHECK-AVX,CHECK-AVX2
; RUN: opt -S -codegenprepare -mcpu=skylake-avx512 %s | FileCheck %s --check-prefixes=CHECK,CHECK-AVX,CHECK-AVX512BW ; RUN: opt -S -codegenprepare -mcpu=skylake-avx512 %s | FileCheck %s --check-prefixes=CHECK,CHECK-AVX,CHECK-AVX512BW
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128" target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-apple-darwin10.9.0" target triple = "x86_64-apple-darwin10.9.0"
define <16 x i8> @test_8bit(<16 x i8> %lhs, <16 x i8> %tmp, i1 %tst) { define <16 x i8> @test_8bit(<16 x i8> %lhs, <16 x i8> %tmp, i1 %tst) {
; CHECK-LABEL: @test_8bit( ; CHECK-SSE2-LABEL: @test_8bit(
; CHECK-NEXT: [[MASK:%.*]] = shufflevector <16 x i8> [[TMP:%.*]], <16 x i8> poison, <16 x i32> zeroinitializer ; CHECK-SSE2-NEXT: [[MASK:%.*]] = shufflevector <16 x i8> [[TMP:%.*]], <16 x i8> poison, <16 x i32> zeroinitializer
; CHECK-NEXT: br i1 [[TST:%.*]], label [[IF_TRUE:%.*]], label [[IF_FALSE:%.*]] ; CHECK-SSE2-NEXT: br i1 [[TST:%.*]], label [[IF_TRUE:%.*]], label [[IF_FALSE:%.*]]
; CHECK: if_true: ; CHECK-SSE2: if_true:
; CHECK-NEXT: ret <16 x i8> [[MASK]] ; CHECK-SSE2-NEXT: ret <16 x i8> [[MASK]]
; CHECK: if_false: ; CHECK-SSE2: if_false:
; CHECK-NEXT: [[RES:%.*]] = shl <16 x i8> [[LHS:%.*]], [[MASK]] ; CHECK-SSE2-NEXT: [[TMP1:%.*]] = shufflevector <16 x i8> [[TMP]], <16 x i8> poison, <16 x i32> zeroinitializer
; CHECK-NEXT: ret <16 x i8> [[RES]] ; CHECK-SSE2-NEXT: [[RES:%.*]] = shl <16 x i8> [[LHS:%.*]], [[TMP1]]
; CHECK-SSE2-NEXT: ret <16 x i8> [[RES]]
;
; CHECK-XOP-LABEL: @test_8bit(
; CHECK-XOP-NEXT: [[MASK:%.*]] = shufflevector <16 x i8> [[TMP:%.*]], <16 x i8> poison, <16 x i32> zeroinitializer
; CHECK-XOP-NEXT: br i1 [[TST:%.*]], label [[IF_TRUE:%.*]], label [[IF_FALSE:%.*]]
; CHECK-XOP: if_true:
; CHECK-XOP-NEXT: ret <16 x i8> [[MASK]]
; CHECK-XOP: if_false:
; CHECK-XOP-NEXT: [[RES:%.*]] = shl <16 x i8> [[LHS:%.*]], [[MASK]]
; CHECK-XOP-NEXT: ret <16 x i8> [[RES]]
;
; CHECK-AVX-LABEL: @test_8bit(
; CHECK-AVX-NEXT: [[MASK:%.*]] = shufflevector <16 x i8> [[TMP:%.*]], <16 x i8> poison, <16 x i32> zeroinitializer
; CHECK-AVX-NEXT: br i1 [[TST:%.*]], label [[IF_TRUE:%.*]], label [[IF_FALSE:%.*]]
; CHECK-AVX: if_true:
; CHECK-AVX-NEXT: ret <16 x i8> [[MASK]]
; CHECK-AVX: if_false:
; CHECK-AVX-NEXT: [[TMP1:%.*]] = shufflevector <16 x i8> [[TMP]], <16 x i8> poison, <16 x i32> zeroinitializer
; CHECK-AVX-NEXT: [[RES:%.*]] = shl <16 x i8> [[LHS:%.*]], [[TMP1]]
; CHECK-AVX-NEXT: ret <16 x i8> [[RES]]
; ;
%mask = shufflevector <16 x i8> %tmp, <16 x i8> poison, <16 x i32> zeroinitializer %mask = shufflevector <16 x i8> %tmp, <16 x i8> poison, <16 x i32> zeroinitializer
br i1 %tst, label %if_true, label %if_false br i1 %tst, label %if_true, label %if_false
if_true: if_true:
ret <16 x i8> %mask ret <16 x i8> %mask
if_false: if_false:
▲ Show 20 Lines • Show All 231 Lines • Show Last 20 Lines

llvm/test/Transforms/CodeGenPrepare/X86/x86-shuffle-sink.ll

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt -S -codegenprepare -mcpu=corei7 %s | FileCheck %s --check-prefixes=CHECK,CHECK-SSE2 ; RUN: opt -S -codegenprepare -mcpu=corei7 %s | FileCheck %s --check-prefixes=CHECK,CHECK-SSE2
; RUN: opt -S -codegenprepare -mcpu=bdver2 %s | FileCheck %s --check-prefixes=CHECK,CHECK-XOP ; RUN: opt -S -codegenprepare -mcpu=bdver2 %s | FileCheck %s --check-prefixes=CHECK,CHECK-XOP
; RUN: opt -S -codegenprepare -mcpu=core-avx2 %s | FileCheck %s --check-prefixes=CHECK,CHECK-AVX,CHECK-AVX2 ; RUN: opt -S -codegenprepare -mcpu=core-avx2 %s | FileCheck %s --check-prefixes=CHECK,CHECK-AVX,CHECK-AVX2
; RUN: opt -S -codegenprepare -mcpu=skylake-avx512 %s | FileCheck %s --check-prefixes=CHECK,CHECK-AVX,CHECK-AVX512BW ; RUN: opt -S -codegenprepare -mcpu=skylake-avx512 %s | FileCheck %s --check-prefixes=CHECK,CHECK-AVX,CHECK-AVX512BW
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128" target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-apple-darwin10.9.0" target triple = "x86_64-apple-darwin10.9.0"
define <16 x i8> @test_8bit(<16 x i8> %lhs, <16 x i8> %tmp, i1 %tst) { define <16 x i8> @test_8bit(<16 x i8> %lhs, <16 x i8> %tmp, i1 %tst) {
; CHECK-LABEL: @test_8bit( ; CHECK-SSE2-LABEL: @test_8bit(
; CHECK-NEXT: [[MASK:%.*]] = shufflevector <16 x i8> [[TMP:%.*]], <16 x i8> undef, <16 x i32> zeroinitializer ; CHECK-SSE2-NEXT: [[MASK:%.*]] = shufflevector <16 x i8> [[TMP:%.*]], <16 x i8> undef, <16 x i32> zeroinitializer
; CHECK-NEXT: br i1 [[TST:%.*]], label [[IF_TRUE:%.*]], label [[IF_FALSE:%.*]] ; CHECK-SSE2-NEXT: br i1 [[TST:%.*]], label [[IF_TRUE:%.*]], label [[IF_FALSE:%.*]]
; CHECK: if_true: ; CHECK-SSE2: if_true:
; CHECK-NEXT: ret <16 x i8> [[MASK]] ; CHECK-SSE2-NEXT: ret <16 x i8> [[MASK]]
; CHECK: if_false: ; CHECK-SSE2: if_false:
; CHECK-NEXT: [[RES:%.*]] = shl <16 x i8> [[LHS:%.*]], [[MASK]] ; CHECK-SSE2-NEXT: [[TMP1:%.*]] = shufflevector <16 x i8> [[TMP]], <16 x i8> undef, <16 x i32> zeroinitializer
; CHECK-NEXT: ret <16 x i8> [[RES]] ; CHECK-SSE2-NEXT: [[RES:%.*]] = shl <16 x i8> [[LHS:%.*]], [[TMP1]]
; CHECK-SSE2-NEXT: ret <16 x i8> [[RES]]
;
; CHECK-XOP-LABEL: @test_8bit(
; CHECK-XOP-NEXT: [[MASK:%.*]] = shufflevector <16 x i8> [[TMP:%.*]], <16 x i8> undef, <16 x i32> zeroinitializer
; CHECK-XOP-NEXT: br i1 [[TST:%.*]], label [[IF_TRUE:%.*]], label [[IF_FALSE:%.*]]
; CHECK-XOP: if_true:
; CHECK-XOP-NEXT: ret <16 x i8> [[MASK]]
; CHECK-XOP: if_false:
; CHECK-XOP-NEXT: [[RES:%.*]] = shl <16 x i8> [[LHS:%.*]], [[MASK]]
; CHECK-XOP-NEXT: ret <16 x i8> [[RES]]
;
; CHECK-AVX-LABEL: @test_8bit(
; CHECK-AVX-NEXT: [[MASK:%.*]] = shufflevector <16 x i8> [[TMP:%.*]], <16 x i8> undef, <16 x i32> zeroinitializer
; CHECK-AVX-NEXT: br i1 [[TST:%.*]], label [[IF_TRUE:%.*]], label [[IF_FALSE:%.*]]
; CHECK-AVX: if_true:
; CHECK-AVX-NEXT: ret <16 x i8> [[MASK]]
; CHECK-AVX: if_false:
; CHECK-AVX-NEXT: [[TMP1:%.*]] = shufflevector <16 x i8> [[TMP]], <16 x i8> undef, <16 x i32> zeroinitializer
; CHECK-AVX-NEXT: [[RES:%.*]] = shl <16 x i8> [[LHS:%.*]], [[TMP1]]
; CHECK-AVX-NEXT: ret <16 x i8> [[RES]]
; ;
%mask = shufflevector <16 x i8> %tmp, <16 x i8> undef, <16 x i32> zeroinitializer %mask = shufflevector <16 x i8> %tmp, <16 x i8> undef, <16 x i32> zeroinitializer
br i1 %tst, label %if_true, label %if_false br i1 %tst, label %if_true, label %if_false
if_true: if_true:
ret <16 x i8> %mask ret <16 x i8> %mask
if_false: if_false:
▲ Show 20 Lines • Show All 231 Lines • Show Last 20 Lines