This is an archive of the discontinued LLVM Phabricator instance.

Differential D136529

[DAG] canCreateUndefOrPoison - add support for SRA/SRL shift opcode (WIP)
Changes PlannedPublic

Authored by RKSimon on Oct 22 2022, 8:16 AM.

Details

Reviewers
craig.topper
foad
spatel
Summary

Right shifts won't create undef/poison if they are inexact with inrange shift amounts.

Matches handling in Operator::hasPoisonGeneratingFlags()

WIP: There are still a few regression cases to handle (either other missing opcodes or we are freezing ops that we can't push the freeze through multiple operands), but I'm cleaning out old branches and wanted to get this up for future reference.

Diff Detail

Unit TestsFailed

TimeTest
60,030 msx64 debian > MLIR.Examples/standalone::test.toy

Event Timeline

RKSimon created this revision.Oct 22 2022, 8:16 AM
Herald added a project: Restricted Project. · View Herald TranscriptOct 22 2022, 8:16 AM
Herald added subscribers: kosarev, StephenFan, frasercrmck and 24 others. · View Herald Transcript
RKSimon requested review of this revision.Oct 22 2022, 8:16 AM
Herald added a project: Restricted Project. · View Herald TranscriptOct 22 2022, 8:16 AM
Herald added subscribers: • pcwang-thead, MaskRay. · View Herald Transcript
RKSimon planned changes to this revision.Oct 22 2022, 8:16 AM
Harbormaster completed remote builds in B193746: Diff 469902.Oct 22 2022, 9:04 AM
RKSimon updated this revision to Diff 486559.Jan 5 2023, 7:03 AM

rebase

RKSimon planned changes to this revision.Jan 5 2023, 7:04 AM
RKSimon mentioned this in D140509: [DAGCombiner][RISCV] Pre-promote (zext (abs X)) to (abs (sext X)) when X has an illegal type..
foad added a comment.Jan 5 2023, 7:07 AM

Right shifts won't create undef/poison if they are exact with inrange shift amounts.

If they are inexact, surely?

RKSimon edited the summary of this revision. (Show Details)Jan 5 2023, 7:23 AM
Harbormaster completed remote builds in B205898: Diff 486559.Jan 5 2023, 8:42 AM
RKSimon updated this revision to Diff 495859.Feb 8 2023, 7:59 AM

rebase

Herald added a subscriber: luke. · View Herald TranscriptFeb 8 2023, 7:59 AM
RKSimon planned changes to this revision.Feb 8 2023, 7:59 AM
Harbormaster completed remote builds in B212620: Diff 495859.Feb 8 2023, 8:54 AM
RKSimon mentioned this in rGb7deb71ef5e0: [DAG] Fold freeze(build_pair(x,y)) -> build_pair(freeze(x),freeze(y)).Feb 8 2023, 9:54 AM
RKSimon mentioned this in rGce63cd3bf1df: [DAG] Fold freeze(concat_vectors(x,y,...)) -> concat_vectors(freeze(x),freeze….Feb 8 2023, 12:27 PM
RKSimon updated this revision to Diff 495918.Feb 8 2023, 12:27 PM

rebase

RKSimon planned changes to this revision.Feb 8 2023, 12:28 PM
Harbormaster completed remote builds in B212668: Diff 495918.Feb 8 2023, 1:13 PM
RKSimon added inline comments.Feb 11 2023, 11:32 AM
llvm/test/CodeGen/RISCV/rv64zbb.ll
255

@craig.topper I'm familiar with the riscv shift code, but moving the freeze out of the way has changed:

t2: i64,ch = CopyFromReg t0, Register:i64 %0
    t127: i64 = and t2, Constant:i64<4294967295>
  t30: i64 = srl t127, Constant:i64<1>

to

  t2: i64,ch = CopyFromReg t0, Register:i64 %0
t4: i64 = AssertZext t2, ValueType:ch:i31
  t32: i64 = srl t4, Constant:i64<1>
t33: i64 = or t4, t32
craig.topper added inline comments.Feb 11 2023, 11:25 PM
llvm/test/CodeGen/RISCV/rv64zbb.ll
255

srliw is a 32-bit shift right in a 64-bit register. It's equivalent to (srl (and X, 0xffffffff), C).

Looks like SimplifyDemandedBits is not figuring out that bit 31 is zero due to the AssertZExt.

So we end up with (srl (and X, 0x7fffffff), C) instead.

We don't have an and with immediate for that constant so we use a shift left followed by shift right for the combination.

I tried adding a computeKnownBits call during isel to try to recover the mask, but it looks like it hit the depth limit before it got to the AssertZExt.

I think all the ANDs before every SRL after type legalization allowed us to propagate more information before some of them they got removed. Some weird visitation order I guess.

Revision Contents

PathSize
llvm/
lib/
CodeGen/
SelectionDAG/
9 lines
test/
CodeGen/
AMDGPU/
12 lines
26 lines
2 lines
RISCV/
12 lines
X86/
68 lines
700 lines

Diff 469902

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 4,647 Lines • ▼ Show 20 Lines case ISD::SHL:
// If the max shift amount isn't in range, then the shift can create poison. // If the max shift amount isn't in range, then the shift can create poison.
if (!getValidMaximumShiftAmountConstant(Op, DemandedElts)) if (!getValidMaximumShiftAmountConstant(Op, DemandedElts))
return true; return true;
// Matches hasPoisonGeneratingFlags(). // Matches hasPoisonGeneratingFlags().
return ConsiderFlags && (Op->getFlags().hasNoSignedWrap() || return ConsiderFlags && (Op->getFlags().hasNoSignedWrap() ||
Op->getFlags().hasNoUnsignedWrap()); Op->getFlags().hasNoUnsignedWrap());
case ISD::SRA:
case ISD::SRL:
// If the max shift amount isn't in range, then the shift can create poison.
if (!getValidMaximumShiftAmountConstant(Op, DemandedElts))
return true;
// Matches hasPoisonGeneratingFlags().
return ConsiderFlags && Op->getFlags().hasExact();
default: default:
// Allow the target to implement this method for its nodes. // Allow the target to implement this method for its nodes.
if (Opcode >= ISD::BUILTIN_OP_END || Opcode == ISD::INTRINSIC_WO_CHAIN || if (Opcode >= ISD::BUILTIN_OP_END || Opcode == ISD::INTRINSIC_WO_CHAIN ||
Opcode == ISD::INTRINSIC_W_CHAIN || Opcode == ISD::INTRINSIC_VOID) Opcode == ISD::INTRINSIC_W_CHAIN || Opcode == ISD::INTRINSIC_VOID)
return TLI->canCreateUndefOrPoisonForTargetNode( return TLI->canCreateUndefOrPoisonForTargetNode(
Op, DemandedElts, *this, PoisonOnly, ConsiderFlags, Depth); Op, DemandedElts, *this, PoisonOnly, ConsiderFlags, Depth);
break; break;
} }
▲ Show 20 Lines • Show All 7,418 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/sdiv64.ll

Show First 20 Lines • Show All 967 Lines • ▼ Show 20 Lines
; GCN-NEXT: buffer_store_short v1, off, s[0:3], 0 offset:4 ; GCN-NEXT: buffer_store_short v1, off, s[0:3], 0 offset:4
; GCN-NEXT: s_endpgm ; GCN-NEXT: s_endpgm
; ;
; GCN-IR-LABEL: s_test_sdiv24_48: ; GCN-IR-LABEL: s_test_sdiv24_48:
; GCN-IR: ; %bb.0: ; %_udiv-special-cases ; GCN-IR: ; %bb.0: ; %_udiv-special-cases
; GCN-IR-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0xb ; GCN-IR-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0xb
; GCN-IR-NEXT: s_mov_b32 s15, 0 ; GCN-IR-NEXT: s_mov_b32 s15, 0
; GCN-IR-NEXT: s_waitcnt lgkmcnt(0) ; GCN-IR-NEXT: s_waitcnt lgkmcnt(0)
; GCN-IR-NEXT: s_sext_i32_i16 s5, s5 ; GCN-IR-NEXT: s_lshl_b64 s[2:3], s[4:5], 16
; GCN-IR-NEXT: s_ashr_i64 s[2:3], s[4:5], 24 ; GCN-IR-NEXT: s_lshl_b64 s[4:5], s[6:7], 16
; GCN-IR-NEXT: s_sext_i32_i16 s7, s7 ; GCN-IR-NEXT: s_ashr_i64 s[6:7], s[2:3], 40
; GCN-IR-NEXT: s_lshl_b64 s[2:3], s[2:3], 16
; GCN-IR-NEXT: s_ashr_i64 s[4:5], s[6:7], 24
; GCN-IR-NEXT: s_ashr_i64 s[6:7], s[2:3], 16
; GCN-IR-NEXT: s_ashr_i32 s2, s3, 31 ; GCN-IR-NEXT: s_ashr_i32 s2, s3, 31
; GCN-IR-NEXT: s_lshl_b64 s[4:5], s[4:5], 16
; GCN-IR-NEXT: s_mov_b32 s3, s2 ; GCN-IR-NEXT: s_mov_b32 s3, s2
; GCN-IR-NEXT: s_ashr_i64 s[8:9], s[4:5], 16 ; GCN-IR-NEXT: s_ashr_i64 s[8:9], s[4:5], 40
; GCN-IR-NEXT: s_ashr_i32 s4, s5, 31 ; GCN-IR-NEXT: s_ashr_i32 s4, s5, 31
; GCN-IR-NEXT: s_xor_b64 s[6:7], s[2:3], s[6:7] ; GCN-IR-NEXT: s_xor_b64 s[6:7], s[2:3], s[6:7]
; GCN-IR-NEXT: s_mov_b32 s5, s4 ; GCN-IR-NEXT: s_mov_b32 s5, s4
; GCN-IR-NEXT: s_sub_u32 s12, s6, s2 ; GCN-IR-NEXT: s_sub_u32 s12, s6, s2
; GCN-IR-NEXT: s_subb_u32 s13, s7, s2 ; GCN-IR-NEXT: s_subb_u32 s13, s7, s2
; GCN-IR-NEXT: s_xor_b64 s[6:7], s[4:5], s[8:9] ; GCN-IR-NEXT: s_xor_b64 s[6:7], s[4:5], s[8:9]
; GCN-IR-NEXT: s_sub_u32 s6, s6, s4 ; GCN-IR-NEXT: s_sub_u32 s6, s6, s4
; GCN-IR-NEXT: s_subb_u32 s7, s7, s4 ; GCN-IR-NEXT: s_subb_u32 s7, s7, s4
▲ Show 20 Lines • Show All 1,014 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/srem64.ll

Show First 20 Lines • Show All 1,010 Lines • ▼ Show 20 Lines
; ;
; GCN-IR-LABEL: s_test_srem33_64: ; GCN-IR-LABEL: s_test_srem33_64:
; GCN-IR: ; %bb.0: ; %_udiv-special-cases ; GCN-IR: ; %bb.0: ; %_udiv-special-cases
; GCN-IR-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9 ; GCN-IR-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; GCN-IR-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd ; GCN-IR-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; GCN-IR-NEXT: s_mov_b32 s13, 0 ; GCN-IR-NEXT: s_mov_b32 s13, 0
; GCN-IR-NEXT: s_waitcnt lgkmcnt(0) ; GCN-IR-NEXT: s_waitcnt lgkmcnt(0)
; GCN-IR-NEXT: s_ashr_i64 s[2:3], s[6:7], 31 ; GCN-IR-NEXT: s_ashr_i64 s[2:3], s[6:7], 31
; GCN-IR-NEXT: s_ashr_i64 s[6:7], s[0:1], 31 ; GCN-IR-NEXT: s_ashr_i64 s[8:9], s[0:1], 31
; GCN-IR-NEXT: s_ashr_i32 s0, s3, 31 ; GCN-IR-NEXT: s_ashr_i32 s0, s7, 31
; GCN-IR-NEXT: s_ashr_i32 s10, s7, 31 ; GCN-IR-NEXT: s_ashr_i32 s6, s1, 31
; GCN-IR-NEXT: s_mov_b32 s1, s0 ; GCN-IR-NEXT: s_mov_b32 s1, s0
; GCN-IR-NEXT: s_mov_b32 s11, s10 ; GCN-IR-NEXT: s_mov_b32 s7, s6
; GCN-IR-NEXT: s_xor_b64 s[2:3], s[2:3], s[0:1] ; GCN-IR-NEXT: s_xor_b64 s[2:3], s[2:3], s[0:1]
; GCN-IR-NEXT: s_xor_b64 s[6:7], s[6:7], s[10:11] ; GCN-IR-NEXT: s_xor_b64 s[8:9], s[8:9], s[6:7]
; GCN-IR-NEXT: s_sub_u32 s2, s2, s0 ; GCN-IR-NEXT: s_sub_u32 s2, s2, s0
; GCN-IR-NEXT: s_subb_u32 s3, s3, s0 ; GCN-IR-NEXT: s_subb_u32 s3, s3, s0
; GCN-IR-NEXT: s_sub_u32 s8, s6, s10 ; GCN-IR-NEXT: s_sub_u32 s8, s8, s6
; GCN-IR-NEXT: s_subb_u32 s9, s7, s10 ; GCN-IR-NEXT: s_subb_u32 s9, s9, s6
; GCN-IR-NEXT: v_cmp_eq_u64_e64 s[6:7], s[8:9], 0 ; GCN-IR-NEXT: v_cmp_eq_u64_e64 s[6:7], s[8:9], 0
; GCN-IR-NEXT: v_cmp_eq_u64_e64 s[10:11], s[2:3], 0 ; GCN-IR-NEXT: v_cmp_eq_u64_e64 s[10:11], s[2:3], 0
; GCN-IR-NEXT: s_or_b64 s[10:11], s[6:7], s[10:11] ; GCN-IR-NEXT: s_or_b64 s[10:11], s[6:7], s[10:11]
; GCN-IR-NEXT: s_flbit_i32_b32 s6, s8 ; GCN-IR-NEXT: s_flbit_i32_b32 s6, s8
; GCN-IR-NEXT: s_add_i32 s6, s6, 32 ; GCN-IR-NEXT: s_add_i32 s6, s6, 32
; GCN-IR-NEXT: s_flbit_i32_b32 s7, s9 ; GCN-IR-NEXT: s_flbit_i32_b32 s7, s9
; GCN-IR-NEXT: s_min_u32 s12, s6, s7 ; GCN-IR-NEXT: s_min_u32 s12, s6, s7
; GCN-IR-NEXT: s_flbit_i32_b32 s6, s2 ; GCN-IR-NEXT: s_flbit_i32_b32 s6, s2
▲ Show 20 Lines • Show All 117 Lines • ▼ Show 20 Lines
; GCN-NEXT: buffer_store_short v1, off, s[0:3], 0 offset:4 ; GCN-NEXT: buffer_store_short v1, off, s[0:3], 0 offset:4
; GCN-NEXT: s_endpgm ; GCN-NEXT: s_endpgm
; ;
; GCN-IR-LABEL: s_test_srem24_48: ; GCN-IR-LABEL: s_test_srem24_48:
; GCN-IR: ; %bb.0: ; %_udiv-special-cases ; GCN-IR: ; %bb.0: ; %_udiv-special-cases
; GCN-IR-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0xb ; GCN-IR-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0xb
; GCN-IR-NEXT: s_mov_b32 s13, 0 ; GCN-IR-NEXT: s_mov_b32 s13, 0
; GCN-IR-NEXT: s_waitcnt lgkmcnt(0) ; GCN-IR-NEXT: s_waitcnt lgkmcnt(0)
; GCN-IR-NEXT: s_sext_i32_i16 s5, s5 ; GCN-IR-NEXT: s_lshl_b64 s[2:3], s[4:5], 16
; GCN-IR-NEXT: s_sext_i32_i16 s7, s7 ; GCN-IR-NEXT: s_lshl_b64 s[4:5], s[6:7], 16
; GCN-IR-NEXT: s_ashr_i64 s[2:3], s[4:5], 24 ; GCN-IR-NEXT: s_ashr_i64 s[6:7], s[2:3], 40
; GCN-IR-NEXT: s_ashr_i64 s[4:5], s[6:7], 24
; GCN-IR-NEXT: s_lshl_b64 s[2:3], s[2:3], 16
; GCN-IR-NEXT: s_lshl_b64 s[4:5], s[4:5], 16
; GCN-IR-NEXT: s_ashr_i64 s[6:7], s[2:3], 16
; GCN-IR-NEXT: s_ashr_i32 s2, s3, 31 ; GCN-IR-NEXT: s_ashr_i32 s2, s3, 31
; GCN-IR-NEXT: s_ashr_i32 s10, s5, 31 ; GCN-IR-NEXT: s_ashr_i32 s10, s5, 31
; GCN-IR-NEXT: s_ashr_i64 s[8:9], s[4:5], 16 ; GCN-IR-NEXT: s_ashr_i64 s[8:9], s[4:5], 40
; GCN-IR-NEXT: s_mov_b32 s3, s2 ; GCN-IR-NEXT: s_mov_b32 s3, s2
; GCN-IR-NEXT: s_mov_b32 s11, s10 ; GCN-IR-NEXT: s_mov_b32 s11, s10
; GCN-IR-NEXT: s_xor_b64 s[4:5], s[6:7], s[2:3] ; GCN-IR-NEXT: s_xor_b64 s[4:5], s[6:7], s[2:3]
; GCN-IR-NEXT: s_xor_b64 s[6:7], s[8:9], s[10:11] ; GCN-IR-NEXT: s_xor_b64 s[6:7], s[8:9], s[10:11]
; GCN-IR-NEXT: s_sub_u32 s4, s4, s2 ; GCN-IR-NEXT: s_sub_u32 s4, s4, s2
; GCN-IR-NEXT: s_subb_u32 s5, s5, s2 ; GCN-IR-NEXT: s_subb_u32 s5, s5, s2
; GCN-IR-NEXT: s_sub_u32 s6, s6, s10 ; GCN-IR-NEXT: s_sub_u32 s6, s6, s10
; GCN-IR-NEXT: s_subb_u32 s7, s7, s10 ; GCN-IR-NEXT: s_subb_u32 s7, s7, s10
▲ Show 20 Lines • Show All 1,054 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/udiv64.ll

Show First 20 Lines • Show All 786 Lines • ▼ Show 20 Lines
; GCN-IR-NEXT: s_mov_b32 s11, 0 ; GCN-IR-NEXT: s_mov_b32 s11, 0
; GCN-IR-NEXT: s_waitcnt lgkmcnt(0) ; GCN-IR-NEXT: s_waitcnt lgkmcnt(0)
; GCN-IR-NEXT: s_and_b32 s3, s5, 0xffff ; GCN-IR-NEXT: s_and_b32 s3, s5, 0xffff
; GCN-IR-NEXT: s_and_b32 s2, s4, 0xff000000 ; GCN-IR-NEXT: s_and_b32 s2, s4, 0xff000000
; GCN-IR-NEXT: s_and_b32 s5, s7, 0xffff ; GCN-IR-NEXT: s_and_b32 s5, s7, 0xffff
; GCN-IR-NEXT: s_and_b32 s4, s6, 0xff000000 ; GCN-IR-NEXT: s_and_b32 s4, s6, 0xff000000
; GCN-IR-NEXT: s_lshr_b64 s[8:9], s[2:3], 24 ; GCN-IR-NEXT: s_lshr_b64 s[8:9], s[2:3], 24
; GCN-IR-NEXT: s_lshr_b64 s[2:3], s[4:5], 24 ; GCN-IR-NEXT: s_lshr_b64 s[2:3], s[4:5], 24
; GCN-IR-NEXT: s_and_b32 s9, s9, 0xffff
; GCN-IR-NEXT: s_and_b32 s3, s3, 0xffff
; GCN-IR-NEXT: v_cmp_eq_u64_e64 s[4:5], s[2:3], 0 ; GCN-IR-NEXT: v_cmp_eq_u64_e64 s[4:5], s[2:3], 0
; GCN-IR-NEXT: v_cmp_eq_u64_e64 s[6:7], s[8:9], 0 ; GCN-IR-NEXT: v_cmp_eq_u64_e64 s[6:7], s[8:9], 0
; GCN-IR-NEXT: s_or_b64 s[6:7], s[4:5], s[6:7] ; GCN-IR-NEXT: s_or_b64 s[6:7], s[4:5], s[6:7]
; GCN-IR-NEXT: s_flbit_i32_b32 s4, s2 ; GCN-IR-NEXT: s_flbit_i32_b32 s4, s2
; GCN-IR-NEXT: s_add_i32 s4, s4, 32 ; GCN-IR-NEXT: s_add_i32 s4, s4, 32
; GCN-IR-NEXT: s_flbit_i32_b32 s5, s3 ; GCN-IR-NEXT: s_flbit_i32_b32 s5, s3
; GCN-IR-NEXT: s_min_u32 s10, s4, s5 ; GCN-IR-NEXT: s_min_u32 s10, s4, s5
; GCN-IR-NEXT: s_flbit_i32_b32 s4, s8 ; GCN-IR-NEXT: s_flbit_i32_b32 s4, s8
▲ Show 20 Lines • Show All 1,074 Lines • Show Last 20 Lines

llvm/test/CodeGen/RISCV/rv64zbb.ll

Show First 20 Lines • Show All 246 Lines • ▼ Show 20 Lines
; RV64I: # %bb.0: ; RV64I: # %bb.0:
; RV64I-NEXT: srliw a0, a0, 1 ; RV64I-NEXT: srliw a0, a0, 1
; RV64I-NEXT: beqz a0, .LBB4_2 ; RV64I-NEXT: beqz a0, .LBB4_2
; RV64I-NEXT: # %bb.1: # %cond.false ; RV64I-NEXT: # %bb.1: # %cond.false
; RV64I-NEXT: addi sp, sp, -16 ; RV64I-NEXT: addi sp, sp, -16
; RV64I-NEXT: .cfi_def_cfa_offset 16 ; RV64I-NEXT: .cfi_def_cfa_offset 16
; RV64I-NEXT: sd ra, 8(sp) # 8-byte Folded Spill ; RV64I-NEXT: sd ra, 8(sp) # 8-byte Folded Spill
; RV64I-NEXT: .cfi_offset ra, -8 ; RV64I-NEXT: .cfi_offset ra, -8
; RV64I-NEXT: srliw a1, a0, 1 ; RV64I-NEXT: srli a1, a0, 1
RKSimonAuthorUnsubmitted
Not Done
ReplyInline Actions

@craig.topper I'm familiar with the riscv shift code, but moving the freeze out of the way has changed:

t2: i64,ch = CopyFromReg t0, Register:i64 %0
    t127: i64 = and t2, Constant:i64<4294967295>
  t30: i64 = srl t127, Constant:i64<1>

to

  t2: i64,ch = CopyFromReg t0, Register:i64 %0
t4: i64 = AssertZext t2, ValueType:ch:i31
  t32: i64 = srl t4, Constant:i64<1>
t33: i64 = or t4, t32
RKSimon: @craig.topper I'm familiar with the riscv shift code, but moving the freeze out of the way has…
craig.topperUnsubmitted
Not Done
ReplyInline Actions

srliw is a 32-bit shift right in a 64-bit register. It's equivalent to (srl (and X, 0xffffffff), C).

Looks like SimplifyDemandedBits is not figuring out that bit 31 is zero due to the AssertZExt.

So we end up with (srl (and X, 0x7fffffff), C) instead.

We don't have an and with immediate for that constant so we use a shift left followed by shift right for the combination.

I tried adding a computeKnownBits call during isel to try to recover the mask, but it looks like it hit the depth limit before it got to the AssertZExt.

I think all the ANDs before every SRL after type legalization allowed us to propagate more information before some of them they got removed. Some weird visitation order I guess.

craig.topper: srliw is a 32-bit shift right in a 64-bit register. It's equivalent to (srl (and X, 0xffffffff)…
; RV64I-NEXT: or a0, a0, a1 ; RV64I-NEXT: or a0, a0, a1
; RV64I-NEXT: srliw a1, a0, 2 ; RV64I-NEXT: srli a1, a0, 2
; RV64I-NEXT: or a0, a0, a1 ; RV64I-NEXT: or a0, a0, a1
; RV64I-NEXT: srliw a1, a0, 4 ; RV64I-NEXT: srli a1, a0, 4
; RV64I-NEXT: or a0, a0, a1 ; RV64I-NEXT: or a0, a0, a1
; RV64I-NEXT: srliw a1, a0, 8 ; RV64I-NEXT: slli a1, a0, 33
; RV64I-NEXT: srli a1, a1, 41
; RV64I-NEXT: or a0, a0, a1 ; RV64I-NEXT: or a0, a0, a1
; RV64I-NEXT: srliw a1, a0, 16 ; RV64I-NEXT: slli a1, a0, 33
; RV64I-NEXT: srli a1, a1, 49
; RV64I-NEXT: or a0, a0, a1 ; RV64I-NEXT: or a0, a0, a1
; RV64I-NEXT: not a0, a0 ; RV64I-NEXT: not a0, a0
; RV64I-NEXT: srli a1, a0, 1 ; RV64I-NEXT: srli a1, a0, 1
; RV64I-NEXT: lui a2, 349525 ; RV64I-NEXT: lui a2, 349525
; RV64I-NEXT: addiw a2, a2, 1365 ; RV64I-NEXT: addiw a2, a2, 1365
; RV64I-NEXT: and a1, a1, a2 ; RV64I-NEXT: and a1, a1, a2
; RV64I-NEXT: sub a0, a0, a1 ; RV64I-NEXT: sub a0, a0, a1
; RV64I-NEXT: lui a1, 209715 ; RV64I-NEXT: lui a1, 209715
▲ Show 20 Lines • Show All 774 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/freeze-binary.ll

Show First 20 Lines • Show All 406 Lines • ▼ Show 20 Lines; X64-NEXT: retq
%z = shl <2 x i64> %y, <i64 2, i64 2> %z = shl <2 x i64> %y, <i64 2, i64 2>
ret <2 x i64> %z ret <2 x i64> %z
} }
define i32 @freeze_ashr(i32 %a0) nounwind { define i32 @freeze_ashr(i32 %a0) nounwind {
; X86-LABEL: freeze_ashr: ; X86-LABEL: freeze_ashr:
; X86: # %bb.0: ; X86: # %bb.0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: sarl $3, %eax ; X86-NEXT: sarl $6, %eax
; X86-NEXT: sarl $3, %eax
; X86-NEXT: retl ; X86-NEXT: retl
; ;
; X64-LABEL: freeze_ashr: ; X64-LABEL: freeze_ashr:
; X64: # %bb.0: ; X64: # %bb.0:
; X64-NEXT: movl %edi, %eax ; X64-NEXT: movl %edi, %eax
; X64-NEXT: sarl $3, %eax ; X64-NEXT: sarl $6, %eax
; X64-NEXT: sarl $3, %eax
; X64-NEXT: retq ; X64-NEXT: retq
%x = ashr i32 %a0, 3 %x = ashr i32 %a0, 3
%y = freeze i32 %x %y = freeze i32 %x
%z = ashr i32 %y, 3 %z = ashr i32 %y, 3
ret i32 %z ret i32 %z
} }
define i32 @freeze_ashr_exact(i32 %a0) nounwind { define i32 @freeze_ashr_exact(i32 %a0) nounwind {
; X86-LABEL: freeze_ashr_exact: ; X86-LABEL: freeze_ashr_exact:
; X86: # %bb.0: ; X86: # %bb.0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: sarl $3, %eax ; X86-NEXT: sarl $9, %eax
; X86-NEXT: sarl $6, %eax
; X86-NEXT: retl ; X86-NEXT: retl
; ;
; X64-LABEL: freeze_ashr_exact: ; X64-LABEL: freeze_ashr_exact:
; X64: # %bb.0: ; X64: # %bb.0:
; X64-NEXT: movl %edi, %eax ; X64-NEXT: movl %edi, %eax
; X64-NEXT: sarl $3, %eax ; X64-NEXT: sarl $9, %eax
; X64-NEXT: sarl $6, %eax
; X64-NEXT: retq ; X64-NEXT: retq
%x = ashr exact i32 %a0, 3 %x = ashr exact i32 %a0, 3
%y = freeze i32 %x %y = freeze i32 %x
%z = ashr i32 %y, 6 %z = ashr i32 %y, 6
ret i32 %z ret i32 %z
} }
define i32 @freeze_ashr_outofrange(i32 %a0) nounwind { define i32 @freeze_ashr_outofrange(i32 %a0) nounwind {
Show All 10 Lines; X64-NEXT: retq
%y = freeze i32 %x %y = freeze i32 %x
%z = ashr i32 %y, 3 %z = ashr i32 %y, 3
ret i32 %z ret i32 %z
} }
define <8 x i16> @freeze_ashr_vec(<8 x i16> %a0) nounwind { define <8 x i16> @freeze_ashr_vec(<8 x i16> %a0) nounwind {
; X86-LABEL: freeze_ashr_vec: ; X86-LABEL: freeze_ashr_vec:
; X86: # %bb.0: ; X86: # %bb.0:
; X86-NEXT: movdqa %xmm0, %xmm2 ; X86-NEXT: psraw $4, %xmm0
; X86-NEXT: psraw $1, %xmm2
; X86-NEXT: movdqa {{.*#+}} xmm1 = [65535,0,65535,0,65535,0,65535,0]
; X86-NEXT: movdqa %xmm1, %xmm3
; X86-NEXT: pandn %xmm2, %xmm3
; X86-NEXT: psraw $3, %xmm0
; X86-NEXT: pand %xmm1, %xmm0
; X86-NEXT: por %xmm3, %xmm0
; X86-NEXT: movdqa %xmm0, %xmm2
; X86-NEXT: psraw $3, %xmm2
; X86-NEXT: psraw $1, %xmm0
; X86-NEXT: pand %xmm1, %xmm0
; X86-NEXT: pandn %xmm2, %xmm1
; X86-NEXT: por %xmm1, %xmm0
; X86-NEXT: retl ; X86-NEXT: retl
; ;
; X64-LABEL: freeze_ashr_vec: ; X64-LABEL: freeze_ashr_vec:
; X64: # %bb.0: ; X64: # %bb.0:
; X64-NEXT: vpsraw $1, %xmm0, %xmm1 ; X64-NEXT: vpsraw $4, %xmm0, %xmm0
; X64-NEXT: vpsraw $3, %xmm0, %xmm0
; X64-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2],xmm1[3],xmm0[4],xmm1[5],xmm0[6],xmm1[7]
; X64-NEXT: vpsraw $3, %xmm0, %xmm1
; X64-NEXT: vpsraw $1, %xmm0, %xmm0
; X64-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2],xmm1[3],xmm0[4],xmm1[5],xmm0[6],xmm1[7]
; X64-NEXT: retq ; X64-NEXT: retq
%x = ashr <8 x i16> %a0, <i16 3, i16 1, i16 3, i16 1, i16 3, i16 1, i16 3, i16 1> %x = ashr <8 x i16> %a0, <i16 3, i16 1, i16 3, i16 1, i16 3, i16 1, i16 3, i16 1>
%y = freeze <8 x i16> %x %y = freeze <8 x i16> %x
%z = ashr <8 x i16> %y, <i16 1, i16 3, i16 1, i16 3, i16 1, i16 3, i16 1, i16 3> %z = ashr <8 x i16> %y, <i16 1, i16 3, i16 1, i16 3, i16 1, i16 3, i16 1, i16 3>
ret <8 x i16> %z ret <8 x i16> %z
} }
define <4 x i32> @freeze_ashr_vec_outofrange(<4 x i32> %a0) nounwind { define <4 x i32> @freeze_ashr_vec_outofrange(<4 x i32> %a0) nounwind {
Show All 14 Lines; X64-NEXT: retq
%z = ashr <4 x i32> %y, <i32 2, i32 2, i32 2, i32 2> %z = ashr <4 x i32> %y, <i32 2, i32 2, i32 2, i32 2>
ret <4 x i32> %z ret <4 x i32> %z
} }
define i32 @freeze_lshr(i32 %a0) nounwind { define i32 @freeze_lshr(i32 %a0) nounwind {
; X86-LABEL: freeze_lshr: ; X86-LABEL: freeze_lshr:
; X86: # %bb.0: ; X86: # %bb.0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: shrl $2, %eax ; X86-NEXT: shrl $3, %eax
; X86-NEXT: shrl %eax
; X86-NEXT: retl ; X86-NEXT: retl
; ;
; X64-LABEL: freeze_lshr: ; X64-LABEL: freeze_lshr:
; X64: # %bb.0: ; X64: # %bb.0:
; X64-NEXT: movl %edi, %eax ; X64-NEXT: movl %edi, %eax
; X64-NEXT: shrl $2, %eax ; X64-NEXT: shrl $3, %eax
; X64-NEXT: shrl %eax
; X64-NEXT: retq ; X64-NEXT: retq
%x = lshr i32 %a0, 2 %x = lshr i32 %a0, 2
%y = freeze i32 %x %y = freeze i32 %x
%z = lshr i32 %y, 1 %z = lshr i32 %y, 1
ret i32 %z ret i32 %z
} }
define i32 @freeze_lshr_exact(i32 %a0) nounwind { define i32 @freeze_lshr_exact(i32 %a0) nounwind {
; X86-LABEL: freeze_lshr_exact: ; X86-LABEL: freeze_lshr_exact:
; X86: # %bb.0: ; X86: # %bb.0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: shrl $3, %eax ; X86-NEXT: shrl $8, %eax
; X86-NEXT: shrl $5, %eax
; X86-NEXT: retl ; X86-NEXT: retl
; ;
; X64-LABEL: freeze_lshr_exact: ; X64-LABEL: freeze_lshr_exact:
; X64: # %bb.0: ; X64: # %bb.0:
; X64-NEXT: movl %edi, %eax ; X64-NEXT: movl %edi, %eax
; X64-NEXT: shrl $3, %eax ; X64-NEXT: shrl $8, %eax
; X64-NEXT: shrl $5, %eax
; X64-NEXT: retq ; X64-NEXT: retq
%x = lshr exact i32 %a0, 3 %x = lshr exact i32 %a0, 3
%y = freeze i32 %x %y = freeze i32 %x
%z = lshr i32 %y, 5 %z = lshr i32 %y, 5
ret i32 %z ret i32 %z
} }
define i32 @freeze_lshr_outofrange(i32 %a0) nounwind { define i32 @freeze_lshr_outofrange(i32 %a0) nounwind {
Show All 10 Lines; X64-NEXT: retq
%y = freeze i32 %x %y = freeze i32 %x
%z = lshr i32 %y, 1 %z = lshr i32 %y, 1
ret i32 %z ret i32 %z
} }
define <8 x i16> @freeze_lshr_vec(<8 x i16> %a0) nounwind { define <8 x i16> @freeze_lshr_vec(<8 x i16> %a0) nounwind {
; X86-LABEL: freeze_lshr_vec: ; X86-LABEL: freeze_lshr_vec:
; X86: # %bb.0: ; X86: # %bb.0:
; X86-NEXT: movdqa %xmm0, %xmm2 ; X86-NEXT: psrlw $3, %xmm0
; X86-NEXT: psrlw $1, %xmm2
; X86-NEXT: movdqa {{.*#+}} xmm1 = [65535,0,65535,0,65535,0,65535,0]
; X86-NEXT: movdqa %xmm1, %xmm3
; X86-NEXT: pandn %xmm2, %xmm3
; X86-NEXT: psrlw $2, %xmm0
; X86-NEXT: pand %xmm1, %xmm0
; X86-NEXT: por %xmm3, %xmm0
; X86-NEXT: movdqa %xmm0, %xmm2
; X86-NEXT: psrlw $2, %xmm2
; X86-NEXT: psrlw $1, %xmm0
; X86-NEXT: pand %xmm1, %xmm0
; X86-NEXT: pandn %xmm2, %xmm1
; X86-NEXT: por %xmm1, %xmm0
; X86-NEXT: retl ; X86-NEXT: retl
; ;
; X64-LABEL: freeze_lshr_vec: ; X64-LABEL: freeze_lshr_vec:
; X64: # %bb.0: ; X64: # %bb.0:
; X64-NEXT: vpsrlw $1, %xmm0, %xmm1 ; X64-NEXT: vpsrlw $3, %xmm0, %xmm0
; X64-NEXT: vpsrlw $2, %xmm0, %xmm0
; X64-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2],xmm1[3],xmm0[4],xmm1[5],xmm0[6],xmm1[7]
; X64-NEXT: vpsrlw $2, %xmm0, %xmm1
; X64-NEXT: vpsrlw $1, %xmm0, %xmm0
; X64-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2],xmm1[3],xmm0[4],xmm1[5],xmm0[6],xmm1[7]
; X64-NEXT: retq ; X64-NEXT: retq
%x = lshr <8 x i16> %a0, <i16 2, i16 1, i16 2, i16 1, i16 2, i16 1, i16 2, i16 1> %x = lshr <8 x i16> %a0, <i16 2, i16 1, i16 2, i16 1, i16 2, i16 1, i16 2, i16 1>
%y = freeze <8 x i16> %x %y = freeze <8 x i16> %x
%z = lshr <8 x i16> %y, <i16 1, i16 2, i16 1, i16 2, i16 1, i16 2, i16 1, i16 2> %z = lshr <8 x i16> %y, <i16 1, i16 2, i16 1, i16 2, i16 1, i16 2, i16 1, i16 2>
ret <8 x i16> %z ret <8 x i16> %z
} }
define <4 x i32> @freeze_lshr_vec_outofrange(<4 x i32> %a0) nounwind { define <4 x i32> @freeze_lshr_vec_outofrange(<4 x i32> %a0) nounwind {
Show All 17 Lines

llvm/test/CodeGen/X86/midpoint-int-vec-512.ll

Show First 20 Lines • Show All 275 Lines • ▼ Show 20 Lines
; AVX512F-NEXT: vpcmpgtw %ymm1, %ymm0, %ymm5 ; AVX512F-NEXT: vpcmpgtw %ymm1, %ymm0, %ymm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512F-NEXT: vpminsw %ymm1, %ymm0, %ymm5 ; AVX512F-NEXT: vpminsw %ymm1, %ymm0, %ymm5
; AVX512F-NEXT: vpminsw %ymm2, %ymm3, %ymm6 ; AVX512F-NEXT: vpminsw %ymm2, %ymm3, %ymm6
; AVX512F-NEXT: vpmaxsw %ymm1, %ymm0, %ymm1 ; AVX512F-NEXT: vpmaxsw %ymm1, %ymm0, %ymm1
; AVX512F-NEXT: vpsubw %ymm5, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubw %ymm5, %ymm1, %ymm1
; AVX512F-NEXT: vpmaxsw %ymm2, %ymm3, %ymm2 ; AVX512F-NEXT: vpmaxsw %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vpsubw %ymm6, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubw %ymm6, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm2, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm2
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm5
; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512F-NEXT: vpsubw %ymm2, %ymm6, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm1
; AVX512F-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm1
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512F-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512F-NEXT: vpaddw %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpsubw %ymm2, %ymm5, %ymm2 ; AVX512F-NEXT: vpaddw %ymm0, %ymm1, %ymm0
; AVX512F-NEXT: vpsubw %ymm1, %ymm5, %ymm5 ; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm5, %zmm2
; AVX512F-NEXT: vpternlogq $226, %zmm1, %zmm4, %zmm2
; AVX512F-NEXT: vextracti64x4 $1, %zmm2, %ymm1
; AVX512F-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512F-NEXT: vpaddw %ymm0, %ymm2, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i16_signed_reg_reg: ; AVX512VL-FALLBACK-LABEL: vec512_i16_signed_reg_reg:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3
; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm2, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm2, %ymm3, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm1, %ymm0, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm1, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512VL-FALLBACK-NEXT: vpminsw %ymm1, %ymm0, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpminsw %ymm1, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminsw %ymm2, %ymm3, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminsw %ymm2, %ymm3, %ymm6
; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm1, %ymm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm1, %ymm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm5, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm5, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm2, %ymm3, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm2, %ymm3, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm2, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm5
; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm2, %ymm6, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm1
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm1
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm2, %ymm5, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpaddw %ymm0, %ymm1, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm5, %ymm5 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm5, %zmm2
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm1, %zmm4, %zmm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm2, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm0, %ymm2, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i16_signed_reg_reg: ; AVX512BW-LABEL: vec512_i16_signed_reg_reg:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vpcmpgtw %zmm1, %zmm0, %k1 ; AVX512BW-NEXT: vpcmpgtw %zmm1, %zmm0, %k1
; AVX512BW-NEXT: vpminsw %zmm1, %zmm0, %zmm2 ; AVX512BW-NEXT: vpminsw %zmm1, %zmm0, %zmm2
; AVX512BW-NEXT: vpmaxsw %zmm1, %zmm0, %zmm1 ; AVX512BW-NEXT: vpmaxsw %zmm1, %zmm0, %zmm1
; AVX512BW-NEXT: vpsubw %zmm2, %zmm1, %zmm1 ; AVX512BW-NEXT: vpsubw %zmm2, %zmm1, %zmm1
Show All 22 Lines
; AVX512F-NEXT: vpcmpeqw %ymm4, %ymm3, %ymm5 ; AVX512F-NEXT: vpcmpeqw %ymm4, %ymm3, %ymm5
; AVX512F-NEXT: vpminuw %ymm1, %ymm0, %ymm6 ; AVX512F-NEXT: vpminuw %ymm1, %ymm0, %ymm6
; AVX512F-NEXT: vpcmpeqw %ymm6, %ymm0, %ymm7 ; AVX512F-NEXT: vpcmpeqw %ymm6, %ymm0, %ymm7
; AVX512F-NEXT: vinserti64x4 $1, %ymm5, %zmm7, %zmm5 ; AVX512F-NEXT: vinserti64x4 $1, %ymm5, %zmm7, %zmm5
; AVX512F-NEXT: vpmaxuw %ymm1, %ymm0, %ymm1 ; AVX512F-NEXT: vpmaxuw %ymm1, %ymm0, %ymm1
; AVX512F-NEXT: vpmaxuw %ymm2, %ymm3, %ymm2 ; AVX512F-NEXT: vpmaxuw %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vpsubw %ymm4, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubw %ymm4, %ymm2, %ymm2
; AVX512F-NEXT: vpsubw %ymm6, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubw %ymm6, %ymm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm2, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm2
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm4
; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512F-NEXT: vpsubw %ymm2, %ymm6, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm1
; AVX512F-NEXT: vpternlogq $184, %zmm4, %zmm5, %zmm1
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512F-NEXT: vpxor %xmm4, %xmm4, %xmm4 ; AVX512F-NEXT: vpaddw %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpsubw %ymm2, %ymm4, %ymm2 ; AVX512F-NEXT: vpaddw %ymm0, %ymm1, %ymm0
; AVX512F-NEXT: vpsubw %ymm1, %ymm4, %ymm4 ; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm4, %zmm2
; AVX512F-NEXT: vpternlogq $216, %zmm5, %zmm1, %zmm2
; AVX512F-NEXT: vextracti64x4 $1, %zmm2, %ymm1
; AVX512F-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512F-NEXT: vpaddw %ymm0, %ymm2, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i16_unsigned_reg_reg: ; AVX512VL-FALLBACK-LABEL: vec512_i16_unsigned_reg_reg:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3
; AVX512VL-FALLBACK-NEXT: vpminuw %ymm2, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpminuw %ymm2, %ymm3, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpeqw %ymm4, %ymm3, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpeqw %ymm4, %ymm3, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminuw %ymm1, %ymm0, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminuw %ymm1, %ymm0, %ymm6
; AVX512VL-FALLBACK-NEXT: vpcmpeqw %ymm6, %ymm0, %ymm7 ; AVX512VL-FALLBACK-NEXT: vpcmpeqw %ymm6, %ymm0, %ymm7
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm5, %zmm7, %zmm5 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm5, %zmm7, %zmm5
; AVX512VL-FALLBACK-NEXT: vpmaxuw %ymm1, %ymm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxuw %ymm1, %ymm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpmaxuw %ymm2, %ymm3, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpmaxuw %ymm2, %ymm3, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm4, %ymm2, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm4, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm4
; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm2, %ymm6, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm1
; AVX512VL-FALLBACK-NEXT: vpternlogq $184, %zmm4, %zmm5, %zmm1
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vpxor %xmm4, %xmm4, %xmm4 ; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm2, %ymm4, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpaddw %ymm0, %ymm1, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm4, %ymm4 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm4, %zmm2
; AVX512VL-FALLBACK-NEXT: vpternlogq $216, %zmm5, %zmm1, %zmm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm2, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm0, %ymm2, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i16_unsigned_reg_reg: ; AVX512BW-LABEL: vec512_i16_unsigned_reg_reg:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vpcmpnleuw %zmm1, %zmm0, %k1 ; AVX512BW-NEXT: vpcmpnleuw %zmm1, %zmm0, %k1
; AVX512BW-NEXT: vpminuw %zmm1, %zmm0, %zmm2 ; AVX512BW-NEXT: vpminuw %zmm1, %zmm0, %zmm2
; AVX512BW-NEXT: vpmaxuw %zmm1, %zmm0, %zmm1 ; AVX512BW-NEXT: vpmaxuw %zmm1, %zmm0, %zmm1
; AVX512BW-NEXT: vpsubw %zmm2, %zmm1, %zmm1 ; AVX512BW-NEXT: vpsubw %zmm2, %zmm1, %zmm1
Show All 25 Lines
; AVX512F-NEXT: vpcmpgtw %ymm0, %ymm2, %ymm5 ; AVX512F-NEXT: vpcmpgtw %ymm0, %ymm2, %ymm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512F-NEXT: vpminsw %ymm0, %ymm2, %ymm5 ; AVX512F-NEXT: vpminsw %ymm0, %ymm2, %ymm5
; AVX512F-NEXT: vpminsw %ymm1, %ymm3, %ymm6 ; AVX512F-NEXT: vpminsw %ymm1, %ymm3, %ymm6
; AVX512F-NEXT: vpmaxsw %ymm0, %ymm2, %ymm0 ; AVX512F-NEXT: vpmaxsw %ymm0, %ymm2, %ymm0
; AVX512F-NEXT: vpsubw %ymm5, %ymm0, %ymm0 ; AVX512F-NEXT: vpsubw %ymm5, %ymm0, %ymm0
; AVX512F-NEXT: vpmaxsw %ymm1, %ymm3, %ymm1 ; AVX512F-NEXT: vpmaxsw %ymm1, %ymm3, %ymm1
; AVX512F-NEXT: vpsubw %ymm6, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubw %ymm6, %ymm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm1
; AVX512F-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm0
; AVX512F-NEXT: vpsubw %ymm1, %ymm5, %ymm1 ; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512F-NEXT: vpsubw %ymm0, %ymm5, %ymm5 ; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm5, %zmm1 ; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vpternlogq $226, %zmm0, %zmm4, %zmm1 ; AVX512F-NEXT: vpsubw %ymm0, %ymm6, %ymm0
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm0 ; AVX512F-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512F-NEXT: vpaddw %ymm3, %ymm0, %ymm0
; AVX512F-NEXT: vpaddw %ymm2, %ymm1, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0
; AVX512F-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm0
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512F-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512F-NEXT: vpaddw %ymm2, %ymm0, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i16_signed_mem_reg: ; AVX512VL-FALLBACK-LABEL: vec512_i16_signed_mem_reg:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm2 ; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm2
; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm3 ; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm3
; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm1, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm1, %ymm3, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm0, %ymm2, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm0, %ymm2, %ymm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512VL-FALLBACK-NEXT: vpminsw %ymm0, %ymm2, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpminsw %ymm0, %ymm2, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminsw %ymm1, %ymm3, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminsw %ymm1, %ymm3, %ymm6
; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm0, %ymm2, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm0, %ymm2, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm5, %ymm0, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm5, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm1, %ymm3, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm1, %ymm3, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm5, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm0, %ymm5, %ymm5 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm5, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm0, %zmm4, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm0, %ymm6, %ymm0
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm2, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm0
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm2, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i16_signed_mem_reg: ; AVX512BW-LABEL: vec512_i16_signed_mem_reg:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm1 ; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm1
; AVX512BW-NEXT: vpcmpgtw %zmm0, %zmm1, %k1 ; AVX512BW-NEXT: vpcmpgtw %zmm0, %zmm1, %k1
; AVX512BW-NEXT: vpminsw %zmm0, %zmm1, %zmm2 ; AVX512BW-NEXT: vpminsw %zmm0, %zmm1, %zmm2
; AVX512BW-NEXT: vpmaxsw %zmm0, %zmm1, %zmm0 ; AVX512BW-NEXT: vpmaxsw %zmm0, %zmm1, %zmm0
Show All 25 Lines
; AVX512F-NEXT: vpcmpgtw %ymm1, %ymm0, %ymm5 ; AVX512F-NEXT: vpcmpgtw %ymm1, %ymm0, %ymm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512F-NEXT: vpminsw %ymm1, %ymm0, %ymm5 ; AVX512F-NEXT: vpminsw %ymm1, %ymm0, %ymm5
; AVX512F-NEXT: vpminsw %ymm2, %ymm3, %ymm6 ; AVX512F-NEXT: vpminsw %ymm2, %ymm3, %ymm6
; AVX512F-NEXT: vpmaxsw %ymm1, %ymm0, %ymm1 ; AVX512F-NEXT: vpmaxsw %ymm1, %ymm0, %ymm1
; AVX512F-NEXT: vpsubw %ymm5, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubw %ymm5, %ymm1, %ymm1
; AVX512F-NEXT: vpmaxsw %ymm2, %ymm3, %ymm2 ; AVX512F-NEXT: vpmaxsw %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vpsubw %ymm6, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubw %ymm6, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm2, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm2
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm5
; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512F-NEXT: vpsubw %ymm2, %ymm6, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm1
; AVX512F-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm1
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512F-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512F-NEXT: vpaddw %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpsubw %ymm2, %ymm5, %ymm2 ; AVX512F-NEXT: vpaddw %ymm0, %ymm1, %ymm0
; AVX512F-NEXT: vpsubw %ymm1, %ymm5, %ymm5 ; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm5, %zmm2
; AVX512F-NEXT: vpternlogq $226, %zmm1, %zmm4, %zmm2
; AVX512F-NEXT: vextracti64x4 $1, %zmm2, %ymm1
; AVX512F-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512F-NEXT: vpaddw %ymm0, %ymm2, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i16_signed_reg_mem: ; AVX512VL-FALLBACK-LABEL: vec512_i16_signed_reg_mem:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm1 ; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm1
; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm2 ; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3
; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm2, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm2, %ymm3, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm1, %ymm0, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm1, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512VL-FALLBACK-NEXT: vpminsw %ymm1, %ymm0, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpminsw %ymm1, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminsw %ymm2, %ymm3, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminsw %ymm2, %ymm3, %ymm6
; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm1, %ymm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm1, %ymm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm5, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm5, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm2, %ymm3, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm2, %ymm3, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm2, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm5
; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm2, %ymm6, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm1
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm1
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm2, %ymm5, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpaddw %ymm0, %ymm1, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm5, %ymm5 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm5, %zmm2
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm1, %zmm4, %zmm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm2, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm0, %ymm2, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i16_signed_reg_mem: ; AVX512BW-LABEL: vec512_i16_signed_reg_mem:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm1 ; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm1
; AVX512BW-NEXT: vpcmpgtw %zmm1, %zmm0, %k1 ; AVX512BW-NEXT: vpcmpgtw %zmm1, %zmm0, %k1
; AVX512BW-NEXT: vpminsw %zmm1, %zmm0, %zmm2 ; AVX512BW-NEXT: vpminsw %zmm1, %zmm0, %zmm2
; AVX512BW-NEXT: vpmaxsw %zmm1, %zmm0, %zmm1 ; AVX512BW-NEXT: vpmaxsw %zmm1, %zmm0, %zmm1
Show All 26 Lines
; AVX512F-NEXT: vpcmpgtw %ymm0, %ymm2, %ymm5 ; AVX512F-NEXT: vpcmpgtw %ymm0, %ymm2, %ymm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512F-NEXT: vpminsw %ymm0, %ymm2, %ymm5 ; AVX512F-NEXT: vpminsw %ymm0, %ymm2, %ymm5
; AVX512F-NEXT: vpminsw %ymm1, %ymm3, %ymm6 ; AVX512F-NEXT: vpminsw %ymm1, %ymm3, %ymm6
; AVX512F-NEXT: vpmaxsw %ymm0, %ymm2, %ymm0 ; AVX512F-NEXT: vpmaxsw %ymm0, %ymm2, %ymm0
; AVX512F-NEXT: vpsubw %ymm5, %ymm0, %ymm0 ; AVX512F-NEXT: vpsubw %ymm5, %ymm0, %ymm0
; AVX512F-NEXT: vpmaxsw %ymm1, %ymm3, %ymm1 ; AVX512F-NEXT: vpmaxsw %ymm1, %ymm3, %ymm1
; AVX512F-NEXT: vpsubw %ymm6, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubw %ymm6, %ymm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm1
; AVX512F-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm0
; AVX512F-NEXT: vpsubw %ymm1, %ymm5, %ymm1 ; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512F-NEXT: vpsubw %ymm0, %ymm5, %ymm5 ; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm5, %zmm1 ; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vpternlogq $226, %zmm0, %zmm4, %zmm1 ; AVX512F-NEXT: vpsubw %ymm0, %ymm6, %ymm0
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm0 ; AVX512F-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512F-NEXT: vpaddw %ymm3, %ymm0, %ymm0
; AVX512F-NEXT: vpaddw %ymm2, %ymm1, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0
; AVX512F-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm0
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512F-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512F-NEXT: vpaddw %ymm2, %ymm0, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i16_signed_mem_mem: ; AVX512VL-FALLBACK-LABEL: vec512_i16_signed_mem_mem:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vmovdqa (%rsi), %ymm0 ; AVX512VL-FALLBACK-NEXT: vmovdqa (%rsi), %ymm0
; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rsi), %ymm1 ; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rsi), %ymm1
; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm2 ; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm2
; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm3 ; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm3
; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm1, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm1, %ymm3, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm0, %ymm2, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpgtw %ymm0, %ymm2, %ymm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512VL-FALLBACK-NEXT: vpminsw %ymm0, %ymm2, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpminsw %ymm0, %ymm2, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminsw %ymm1, %ymm3, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminsw %ymm1, %ymm3, %ymm6
; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm0, %ymm2, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm0, %ymm2, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm5, %ymm0, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm5, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm1, %ymm3, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxsw %ymm1, %ymm3, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm6, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm5, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubw %ymm0, %ymm5, %ymm5 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm5, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm0, %zmm4, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm0, %ymm6, %ymm0
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpsubw %ymm1, %ymm6, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm2, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm0
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm3, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddw %ymm2, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i16_signed_mem_mem: ; AVX512BW-LABEL: vec512_i16_signed_mem_mem:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm0 ; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm0
; AVX512BW-NEXT: vmovdqa64 (%rsi), %zmm1 ; AVX512BW-NEXT: vmovdqa64 (%rsi), %zmm1
; AVX512BW-NEXT: vpcmpgtw %zmm1, %zmm0, %k1 ; AVX512BW-NEXT: vpcmpgtw %zmm1, %zmm0, %k1
; AVX512BW-NEXT: vpminsw %zmm1, %zmm0, %zmm2 ; AVX512BW-NEXT: vpminsw %zmm1, %zmm0, %zmm2
Show All 21 Lines
; 8-bit width. 512 / 8 = 64 elts. ; 8-bit width. 512 / 8 = 64 elts.
; ---------------------------------------------------------------------------- ; ; ---------------------------------------------------------------------------- ;
; Values come from regs ; Values come from regs
define <64 x i8> @vec512_i8_signed_reg_reg(<64 x i8> %a1, <64 x i8> %a2) nounwind { define <64 x i8> @vec512_i8_signed_reg_reg(<64 x i8> %a1, <64 x i8> %a2) nounwind {
; AVX512F-LABEL: vec512_i8_signed_reg_reg: ; AVX512F-LABEL: vec512_i8_signed_reg_reg:
; AVX512F: # %bb.0: ; AVX512F: # %bb.0:
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm3
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm2
; AVX512F-NEXT: vpcmpgtb %ymm2, %ymm3, %ymm4 ; AVX512F-NEXT: vpcmpgtb %ymm3, %ymm2, %ymm4
; AVX512F-NEXT: vpcmpgtb %ymm1, %ymm0, %ymm5 ; AVX512F-NEXT: vpcmpgtb %ymm1, %ymm0, %ymm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512F-NEXT: vpminsb %ymm1, %ymm0, %ymm5 ; AVX512F-NEXT: vpminsb %ymm1, %ymm0, %ymm5
; AVX512F-NEXT: vpminsb %ymm2, %ymm3, %ymm6 ; AVX512F-NEXT: vpminsb %ymm3, %ymm2, %ymm6
; AVX512F-NEXT: vpmaxsb %ymm1, %ymm0, %ymm1 ; AVX512F-NEXT: vpmaxsb %ymm1, %ymm0, %ymm1
; AVX512F-NEXT: vpsubb %ymm5, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubb %ymm5, %ymm1, %ymm1
; AVX512F-NEXT: vpmaxsb %ymm2, %ymm3, %ymm2 ; AVX512F-NEXT: vpmaxsb %ymm3, %ymm2, %ymm3
; AVX512F-NEXT: vpsubb %ymm6, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubb %ymm6, %ymm3, %ymm3
; AVX512F-NEXT: vinserti64x4 $1, %ymm3, %zmm1, %zmm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm3
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm2, %ymm2 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm3, %zmm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512F-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm5, %zmm5
; AVX512F-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm1, %zmm1 ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512F-NEXT: vpand %ymm6, %ymm3, %ymm3
; AVX512F-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512F-NEXT: vpand %ymm6, %ymm1, %ymm1
; AVX512F-NEXT: vpsubb %ymm2, %ymm5, %ymm2 ; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vpsubb %ymm1, %ymm5, %ymm5 ; AVX512F-NEXT: vpsubb %ymm1, %ymm6, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm5, %zmm2 ; AVX512F-NEXT: vpsubb %ymm3, %ymm6, %ymm3
; AVX512F-NEXT: vpternlogq $226, %zmm1, %zmm4, %zmm2 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm3, %zmm1
; AVX512F-NEXT: vextracti64x4 $1, %zmm2, %ymm1 ; AVX512F-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm1
; AVX512F-NEXT: vpaddb %ymm3, %ymm1, %ymm1 ; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm3
; AVX512F-NEXT: vpaddb %ymm0, %ymm2, %ymm0 ; AVX512F-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512F-NEXT: vpaddb %ymm0, %ymm1, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i8_signed_reg_reg: ; AVX512VL-FALLBACK-LABEL: vec512_i8_signed_reg_reg:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm3
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm2
; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm2, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm3, %ymm2, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm1, %ymm0, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm1, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512VL-FALLBACK-NEXT: vpminsb %ymm1, %ymm0, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpminsb %ymm1, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminsb %ymm2, %ymm3, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminsb %ymm3, %ymm2, %ymm6
; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm1, %ymm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm1, %ymm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm5, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm5, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm2, %ymm3, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm3, %ymm2, %ymm3
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm2, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm3, %ymm3
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm3, %zmm1, %zmm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm3
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm2 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm3, %zmm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm5, %zmm5
; AVX512VL-FALLBACK-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm1, %zmm1 ; AVX512VL-FALLBACK-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm3, %ymm3
; AVX512VL-FALLBACK-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm2, %ymm5, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm5, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm6, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm5, %zmm2 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm3, %ymm6, %ymm3
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm1, %zmm4, %zmm2 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm3, %zmm1
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm2, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm1
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm3, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm3
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm0, %ymm2, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512VL-FALLBACK-NEXT: vpaddb %ymm0, %ymm1, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i8_signed_reg_reg: ; AVX512BW-LABEL: vec512_i8_signed_reg_reg:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vpcmpgtb %zmm1, %zmm0, %k1 ; AVX512BW-NEXT: vpcmpgtb %zmm1, %zmm0, %k1
; AVX512BW-NEXT: vpminsb %zmm1, %zmm0, %zmm2 ; AVX512BW-NEXT: vpminsb %zmm1, %zmm0, %zmm2
; AVX512BW-NEXT: vpmaxsb %zmm1, %zmm0, %zmm1 ; AVX512BW-NEXT: vpmaxsb %zmm1, %zmm0, %zmm1
; AVX512BW-NEXT: vpsubb %zmm2, %zmm1, %zmm1 ; AVX512BW-NEXT: vpsubb %zmm2, %zmm1, %zmm1
Show All 12 Lines; AVX512BW-NEXT: retq
%t9 = mul nsw <64 x i8> %t8, %t4 ; signed %t9 = mul nsw <64 x i8> %t8, %t4 ; signed
%a10 = add nsw <64 x i8> %t9, %a1 ; signed %a10 = add nsw <64 x i8> %t9, %a1 ; signed
ret <64 x i8> %a10 ret <64 x i8> %a10
} }
define <64 x i8> @vec512_i8_unsigned_reg_reg(<64 x i8> %a1, <64 x i8> %a2) nounwind { define <64 x i8> @vec512_i8_unsigned_reg_reg(<64 x i8> %a1, <64 x i8> %a2) nounwind {
; AVX512F-LABEL: vec512_i8_unsigned_reg_reg: ; AVX512F-LABEL: vec512_i8_unsigned_reg_reg:
; AVX512F: # %bb.0: ; AVX512F: # %bb.0:
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm3
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm2
; AVX512F-NEXT: vpminub %ymm2, %ymm3, %ymm4 ; AVX512F-NEXT: vpminub %ymm3, %ymm2, %ymm4
; AVX512F-NEXT: vpcmpeqb %ymm4, %ymm3, %ymm5 ; AVX512F-NEXT: vpcmpeqb %ymm4, %ymm2, %ymm5
; AVX512F-NEXT: vpminub %ymm1, %ymm0, %ymm6 ; AVX512F-NEXT: vpminub %ymm1, %ymm0, %ymm6
; AVX512F-NEXT: vpcmpeqb %ymm6, %ymm0, %ymm7 ; AVX512F-NEXT: vpcmpeqb %ymm6, %ymm0, %ymm7
; AVX512F-NEXT: vinserti64x4 $1, %ymm5, %zmm7, %zmm5 ; AVX512F-NEXT: vinserti64x4 $1, %ymm5, %zmm7, %zmm5
; AVX512F-NEXT: vpmaxub %ymm1, %ymm0, %ymm1 ; AVX512F-NEXT: vpmaxub %ymm1, %ymm0, %ymm1
; AVX512F-NEXT: vpmaxub %ymm2, %ymm3, %ymm2 ; AVX512F-NEXT: vpmaxub %ymm3, %ymm2, %ymm3
; AVX512F-NEXT: vpsubb %ymm4, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubb %ymm4, %ymm3, %ymm3
; AVX512F-NEXT: vpsubb %ymm6, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubb %ymm6, %ymm1, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm3, %zmm1, %zmm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm3
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm2, %ymm2 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm3, %zmm4
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512F-NEXT: vpandq %zmm5, %zmm4, %zmm4
; AVX512F-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm1, %zmm1 ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512F-NEXT: vpand %ymm6, %ymm3, %ymm3
; AVX512F-NEXT: vpxor %xmm4, %xmm4, %xmm4 ; AVX512F-NEXT: vpand %ymm6, %ymm1, %ymm1
; AVX512F-NEXT: vpsubb %ymm2, %ymm4, %ymm2 ; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vpsubb %ymm1, %ymm4, %ymm4 ; AVX512F-NEXT: vpsubb %ymm1, %ymm6, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm4, %zmm2 ; AVX512F-NEXT: vpsubb %ymm3, %ymm6, %ymm3
; AVX512F-NEXT: vpternlogq $216, %zmm5, %zmm1, %zmm2 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm3, %zmm1
; AVX512F-NEXT: vextracti64x4 $1, %zmm2, %ymm1 ; AVX512F-NEXT: vpandnq %zmm1, %zmm5, %zmm1
; AVX512F-NEXT: vpaddb %ymm3, %ymm1, %ymm1 ; AVX512F-NEXT: vpternlogq $248, {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm4, %zmm1
; AVX512F-NEXT: vpaddb %ymm0, %ymm2, %ymm0 ; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm3
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512F-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vpaddb %ymm0, %ymm1, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i8_unsigned_reg_reg: ; AVX512VL-FALLBACK-LABEL: vec512_i8_unsigned_reg_reg:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm3
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm2
; AVX512VL-FALLBACK-NEXT: vpminub %ymm2, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpminub %ymm3, %ymm2, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpeqb %ymm4, %ymm3, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpeqb %ymm4, %ymm2, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminub %ymm1, %ymm0, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminub %ymm1, %ymm0, %ymm6
; AVX512VL-FALLBACK-NEXT: vpcmpeqb %ymm6, %ymm0, %ymm7 ; AVX512VL-FALLBACK-NEXT: vpcmpeqb %ymm6, %ymm0, %ymm7
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm5, %zmm7, %zmm5 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm5, %zmm7, %zmm5
; AVX512VL-FALLBACK-NEXT: vpmaxub %ymm1, %ymm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxub %ymm1, %ymm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpmaxub %ymm2, %ymm3, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpmaxub %ymm3, %ymm2, %ymm3
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm4, %ymm2, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm4, %ymm3, %ymm3
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm3, %zmm1, %zmm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm3
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm2 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm3, %zmm4
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpandq %zmm5, %zmm4, %zmm4
; AVX512VL-FALLBACK-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm1, %zmm1 ; AVX512VL-FALLBACK-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm3, %ymm3
; AVX512VL-FALLBACK-NEXT: vpxor %xmm4, %xmm4, %xmm4 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm2, %ymm4, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm4, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm6, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm4, %zmm2 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm3, %ymm6, %ymm3
; AVX512VL-FALLBACK-NEXT: vpternlogq $216, %zmm5, %zmm1, %zmm2 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm3, %zmm1
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm2, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpandnq %zmm1, %zmm5, %zmm1
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm3, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpternlogq $248, {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm4, %zmm1
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm0, %ymm2, %ymm0 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm3
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512VL-FALLBACK-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm0, %ymm1, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i8_unsigned_reg_reg: ; AVX512BW-LABEL: vec512_i8_unsigned_reg_reg:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vpcmpnleub %zmm1, %zmm0, %k1 ; AVX512BW-NEXT: vpcmpnleub %zmm1, %zmm0, %k1
; AVX512BW-NEXT: vpminub %zmm1, %zmm0, %zmm2 ; AVX512BW-NEXT: vpminub %zmm1, %zmm0, %zmm2
; AVX512BW-NEXT: vpmaxub %zmm1, %zmm0, %zmm1 ; AVX512BW-NEXT: vpmaxub %zmm1, %zmm0, %zmm1
; AVX512BW-NEXT: vpsubb %zmm2, %zmm1, %zmm1 ; AVX512BW-NEXT: vpsubb %zmm2, %zmm1, %zmm1
Show All 14 Lines; AVX512BW-NEXT: retq
ret <64 x i8> %a10 ret <64 x i8> %a10
} }
; Values are loaded. Only check signed case. ; Values are loaded. Only check signed case.
define <64 x i8> @vec512_i8_signed_mem_reg(ptr %a1_addr, <64 x i8> %a2) nounwind { define <64 x i8> @vec512_i8_signed_mem_reg(ptr %a1_addr, <64 x i8> %a2) nounwind {
; AVX512F-LABEL: vec512_i8_signed_mem_reg: ; AVX512F-LABEL: vec512_i8_signed_mem_reg:
; AVX512F: # %bb.0: ; AVX512F: # %bb.0:
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm3
; AVX512F-NEXT: vmovdqa (%rdi), %ymm2 ; AVX512F-NEXT: vmovdqa (%rdi), %ymm1
; AVX512F-NEXT: vmovdqa 32(%rdi), %ymm3 ; AVX512F-NEXT: vmovdqa 32(%rdi), %ymm2
; AVX512F-NEXT: vpcmpgtb %ymm1, %ymm3, %ymm4 ; AVX512F-NEXT: vpcmpgtb %ymm3, %ymm2, %ymm4
; AVX512F-NEXT: vpcmpgtb %ymm0, %ymm2, %ymm5 ; AVX512F-NEXT: vpcmpgtb %ymm0, %ymm1, %ymm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512F-NEXT: vpminsb %ymm0, %ymm2, %ymm5 ; AVX512F-NEXT: vpminsb %ymm0, %ymm1, %ymm5
; AVX512F-NEXT: vpminsb %ymm1, %ymm3, %ymm6 ; AVX512F-NEXT: vpminsb %ymm3, %ymm2, %ymm6
; AVX512F-NEXT: vpmaxsb %ymm0, %ymm2, %ymm0 ; AVX512F-NEXT: vpmaxsb %ymm0, %ymm1, %ymm0
; AVX512F-NEXT: vpsubb %ymm5, %ymm0, %ymm0 ; AVX512F-NEXT: vpsubb %ymm5, %ymm0, %ymm0
; AVX512F-NEXT: vpmaxsb %ymm1, %ymm3, %ymm1 ; AVX512F-NEXT: vpmaxsb %ymm3, %ymm2, %ymm3
; AVX512F-NEXT: vpsubb %ymm6, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubb %ymm6, %ymm3, %ymm3
; AVX512F-NEXT: vinserti64x4 $1, %ymm3, %zmm0, %zmm0
; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm3
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm0
; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm0 ; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm3, %zmm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512F-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm5, %zmm5
; AVX512F-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm0, %zmm0 ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512F-NEXT: vpand %ymm6, %ymm3, %ymm3
; AVX512F-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512F-NEXT: vpand %ymm6, %ymm0, %ymm0
; AVX512F-NEXT: vpsubb %ymm1, %ymm5, %ymm1 ; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vpsubb %ymm0, %ymm5, %ymm5 ; AVX512F-NEXT: vpsubb %ymm0, %ymm6, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm5, %zmm1 ; AVX512F-NEXT: vpsubb %ymm3, %ymm6, %ymm3
; AVX512F-NEXT: vpternlogq $226, %zmm0, %zmm4, %zmm1 ; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm3, %zmm0
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm0 ; AVX512F-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm0
; AVX512F-NEXT: vpaddb %ymm3, %ymm0, %ymm0 ; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm3
; AVX512F-NEXT: vpaddb %ymm2, %ymm1, %ymm1 ; AVX512F-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0 ; AVX512F-NEXT: vpaddb %ymm1, %ymm0, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i8_signed_mem_reg: ; AVX512VL-FALLBACK-LABEL: vec512_i8_signed_mem_reg:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3
; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm2 ; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm1
; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm3 ; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm2
; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm1, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm3, %ymm2, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm0, %ymm2, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm0, %ymm1, %ymm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512VL-FALLBACK-NEXT: vpminsb %ymm0, %ymm2, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpminsb %ymm0, %ymm1, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminsb %ymm1, %ymm3, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminsb %ymm3, %ymm2, %ymm6
; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm0, %ymm2, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm0, %ymm1, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm5, %ymm0, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm5, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm1, %ymm3, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm3, %ymm2, %ymm3
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm3, %ymm3
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm3, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm3
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm3, %zmm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512VL-FALLBACK-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm5, %zmm5
; AVX512VL-FALLBACK-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm0, %zmm0 ; AVX512VL-FALLBACK-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm3, %ymm3
; AVX512VL-FALLBACK-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm5, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm0, %ymm5, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm0, %ymm6, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm5, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm3, %ymm6, %ymm3
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm0, %zmm4, %zmm1 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm3, %zmm0
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm0
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm3, %ymm0, %ymm0 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm2, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0 ; AVX512VL-FALLBACK-NEXT: vpaddb %ymm1, %ymm0, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i8_signed_mem_reg: ; AVX512BW-LABEL: vec512_i8_signed_mem_reg:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm1 ; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm1
; AVX512BW-NEXT: vpcmpgtb %zmm0, %zmm1, %k1 ; AVX512BW-NEXT: vpcmpgtb %zmm0, %zmm1, %k1
; AVX512BW-NEXT: vpminsb %zmm0, %zmm1, %zmm2 ; AVX512BW-NEXT: vpminsb %zmm0, %zmm1, %zmm2
; AVX512BW-NEXT: vpmaxsb %zmm0, %zmm1, %zmm0 ; AVX512BW-NEXT: vpmaxsb %zmm0, %zmm1, %zmm0
Show All 14 Lines; AVX512BW-NEXT: retq
%t9 = mul nsw <64 x i8> %t8, %t4 ; signed %t9 = mul nsw <64 x i8> %t8, %t4 ; signed
%a10 = add nsw <64 x i8> %t9, %a1 ; signed %a10 = add nsw <64 x i8> %t9, %a1 ; signed
ret <64 x i8> %a10 ret <64 x i8> %a10
} }
define <64 x i8> @vec512_i8_signed_reg_mem(<64 x i8> %a1, ptr %a2_addr) nounwind { define <64 x i8> @vec512_i8_signed_reg_mem(<64 x i8> %a1, ptr %a2_addr) nounwind {
; AVX512F-LABEL: vec512_i8_signed_reg_mem: ; AVX512F-LABEL: vec512_i8_signed_reg_mem:
; AVX512F: # %bb.0: ; AVX512F: # %bb.0:
; AVX512F-NEXT: vmovdqa (%rdi), %ymm1 ; AVX512F-NEXT: vmovdqa (%rdi), %ymm2
; AVX512F-NEXT: vmovdqa 32(%rdi), %ymm2 ; AVX512F-NEXT: vmovdqa 32(%rdi), %ymm3
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512F-NEXT: vpcmpgtb %ymm2, %ymm3, %ymm4 ; AVX512F-NEXT: vpcmpgtb %ymm3, %ymm1, %ymm4
; AVX512F-NEXT: vpcmpgtb %ymm1, %ymm0, %ymm5 ; AVX512F-NEXT: vpcmpgtb %ymm2, %ymm0, %ymm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512F-NEXT: vpminsb %ymm1, %ymm0, %ymm5 ; AVX512F-NEXT: vpminsb %ymm2, %ymm0, %ymm5
; AVX512F-NEXT: vpminsb %ymm2, %ymm3, %ymm6 ; AVX512F-NEXT: vpminsb %ymm3, %ymm1, %ymm6
; AVX512F-NEXT: vpmaxsb %ymm1, %ymm0, %ymm1 ; AVX512F-NEXT: vpmaxsb %ymm2, %ymm0, %ymm2
; AVX512F-NEXT: vpsubb %ymm5, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubb %ymm5, %ymm2, %ymm2
; AVX512F-NEXT: vpmaxsb %ymm2, %ymm3, %ymm2 ; AVX512F-NEXT: vpmaxsb %ymm3, %ymm1, %ymm3
; AVX512F-NEXT: vpsubb %ymm6, %ymm2, %ymm2 ; AVX512F-NEXT: vpsubb %ymm6, %ymm3, %ymm3
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512F-NEXT: vinserti64x4 $1, %ymm3, %zmm2, %zmm2
; AVX512F-NEXT: vpsrlw $1, %ymm2, %ymm3
; AVX512F-NEXT: vextracti64x4 $1, %zmm2, %ymm2
; AVX512F-NEXT: vpsrlw $1, %ymm2, %ymm2 ; AVX512F-NEXT: vpsrlw $1, %ymm2, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm3, %zmm5
; AVX512F-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm1, %zmm1 ; AVX512F-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm5, %zmm5
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512F-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512F-NEXT: vpand %ymm6, %ymm3, %ymm3
; AVX512F-NEXT: vpsubb %ymm2, %ymm5, %ymm2 ; AVX512F-NEXT: vpand %ymm6, %ymm2, %ymm2
; AVX512F-NEXT: vpsubb %ymm1, %ymm5, %ymm5 ; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm5, %zmm2 ; AVX512F-NEXT: vpsubb %ymm2, %ymm6, %ymm2
; AVX512F-NEXT: vpternlogq $226, %zmm1, %zmm4, %zmm2 ; AVX512F-NEXT: vpsubb %ymm3, %ymm6, %ymm3
; AVX512F-NEXT: vextracti64x4 $1, %zmm2, %ymm1 ; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm3, %zmm2
; AVX512F-NEXT: vpaddb %ymm3, %ymm1, %ymm1 ; AVX512F-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm2
; AVX512F-NEXT: vextracti64x4 $1, %zmm2, %ymm3
; AVX512F-NEXT: vpaddb %ymm1, %ymm3, %ymm1
; AVX512F-NEXT: vpaddb %ymm0, %ymm2, %ymm0 ; AVX512F-NEXT: vpaddb %ymm0, %ymm2, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i8_signed_reg_mem: ; AVX512VL-FALLBACK-LABEL: vec512_i8_signed_reg_mem:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm1 ; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm2
; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm2 ; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm3
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm3 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm2, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm3, %ymm1, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm1, %ymm0, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm2, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512VL-FALLBACK-NEXT: vpminsb %ymm1, %ymm0, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpminsb %ymm2, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminsb %ymm2, %ymm3, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminsb %ymm3, %ymm1, %ymm6
; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm1, %ymm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm2, %ymm0, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm5, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm5, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm2, %ymm3, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm3, %ymm1, %ymm3
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm2, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm3, %ymm3
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm3, %zmm2, %zmm2
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm3
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm3, %zmm5
; AVX512VL-FALLBACK-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm1, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm5, %zmm5
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2 ; AVX512VL-FALLBACK-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512VL-FALLBACK-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm3, %ymm3
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm2, %ymm5, %ymm2 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm5, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm5, %zmm2 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm2, %ymm6, %ymm2
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm1, %zmm4, %zmm2 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm3, %ymm6, %ymm3
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm2, %ymm1 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm3, %zmm2
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm3, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm2, %ymm3
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm1, %ymm3, %ymm1
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm0, %ymm2, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpaddb %ymm0, %ymm2, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i8_signed_reg_mem: ; AVX512BW-LABEL: vec512_i8_signed_reg_mem:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm1 ; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm1
; AVX512BW-NEXT: vpcmpgtb %zmm1, %zmm0, %k1 ; AVX512BW-NEXT: vpcmpgtb %zmm1, %zmm0, %k1
Show All 16 Lines; AVX512BW-NEXT: retq
%t9 = mul nsw <64 x i8> %t8, %t4 ; signed %t9 = mul nsw <64 x i8> %t8, %t4 ; signed
%a10 = add nsw <64 x i8> %t9, %a1 ; signed %a10 = add nsw <64 x i8> %t9, %a1 ; signed
ret <64 x i8> %a10 ret <64 x i8> %a10
} }
define <64 x i8> @vec512_i8_signed_mem_mem(ptr %a1_addr, ptr %a2_addr) nounwind { define <64 x i8> @vec512_i8_signed_mem_mem(ptr %a1_addr, ptr %a2_addr) nounwind {
; AVX512F-LABEL: vec512_i8_signed_mem_mem: ; AVX512F-LABEL: vec512_i8_signed_mem_mem:
; AVX512F: # %bb.0: ; AVX512F: # %bb.0:
; AVX512F-NEXT: vmovdqa (%rsi), %ymm0 ; AVX512F-NEXT: vmovdqa (%rsi), %ymm1
; AVX512F-NEXT: vmovdqa 32(%rsi), %ymm1 ; AVX512F-NEXT: vmovdqa 32(%rsi), %ymm2
; AVX512F-NEXT: vmovdqa (%rdi), %ymm2 ; AVX512F-NEXT: vmovdqa (%rdi), %ymm0
; AVX512F-NEXT: vmovdqa 32(%rdi), %ymm3 ; AVX512F-NEXT: vmovdqa 32(%rdi), %ymm3
; AVX512F-NEXT: vpcmpgtb %ymm1, %ymm3, %ymm4 ; AVX512F-NEXT: vpcmpgtb %ymm2, %ymm3, %ymm4
; AVX512F-NEXT: vpcmpgtb %ymm0, %ymm2, %ymm5 ; AVX512F-NEXT: vpcmpgtb %ymm1, %ymm0, %ymm5
; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512F-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512F-NEXT: vpminsb %ymm0, %ymm2, %ymm5 ; AVX512F-NEXT: vpminsb %ymm1, %ymm0, %ymm5
; AVX512F-NEXT: vpminsb %ymm1, %ymm3, %ymm6 ; AVX512F-NEXT: vpminsb %ymm2, %ymm3, %ymm6
; AVX512F-NEXT: vpmaxsb %ymm0, %ymm2, %ymm0 ; AVX512F-NEXT: vpmaxsb %ymm1, %ymm0, %ymm1
; AVX512F-NEXT: vpsubb %ymm5, %ymm0, %ymm0 ; AVX512F-NEXT: vpsubb %ymm5, %ymm1, %ymm1
; AVX512F-NEXT: vpmaxsb %ymm1, %ymm3, %ymm1 ; AVX512F-NEXT: vpmaxsb %ymm2, %ymm3, %ymm2
; AVX512F-NEXT: vpsubb %ymm6, %ymm1, %ymm1 ; AVX512F-NEXT: vpsubb %ymm6, %ymm2, %ymm2
; AVX512F-NEXT: vpsrlw $1, %ymm0, %ymm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm2
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512F-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm5
; AVX512F-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm0, %zmm0 ; AVX512F-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm5, %zmm5
; AVX512F-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512F-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512F-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512F-NEXT: vpand %ymm6, %ymm2, %ymm2
; AVX512F-NEXT: vpsubb %ymm1, %ymm5, %ymm1 ; AVX512F-NEXT: vpand %ymm6, %ymm1, %ymm1
; AVX512F-NEXT: vpsubb %ymm0, %ymm5, %ymm5 ; AVX512F-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm5, %zmm1 ; AVX512F-NEXT: vpsubb %ymm1, %ymm6, %ymm1
; AVX512F-NEXT: vpternlogq $226, %zmm0, %zmm4, %zmm1 ; AVX512F-NEXT: vpsubb %ymm2, %ymm6, %ymm2
; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm0 ; AVX512F-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm1
; AVX512F-NEXT: vpaddb %ymm3, %ymm0, %ymm0 ; AVX512F-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm1
; AVX512F-NEXT: vpaddb %ymm2, %ymm1, %ymm1 ; AVX512F-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512F-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0 ; AVX512F-NEXT: vpaddb %ymm3, %ymm2, %ymm2
; AVX512F-NEXT: vpaddb %ymm0, %ymm1, %ymm0
; AVX512F-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512F-NEXT: retq ; AVX512F-NEXT: retq
; ;
; AVX512VL-FALLBACK-LABEL: vec512_i8_signed_mem_mem: ; AVX512VL-FALLBACK-LABEL: vec512_i8_signed_mem_mem:
; AVX512VL-FALLBACK: # %bb.0: ; AVX512VL-FALLBACK: # %bb.0:
; AVX512VL-FALLBACK-NEXT: vmovdqa (%rsi), %ymm0 ; AVX512VL-FALLBACK-NEXT: vmovdqa (%rsi), %ymm1
; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rsi), %ymm1 ; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rsi), %ymm2
; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm2 ; AVX512VL-FALLBACK-NEXT: vmovdqa (%rdi), %ymm0
; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm3 ; AVX512VL-FALLBACK-NEXT: vmovdqa 32(%rdi), %ymm3
; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm1, %ymm3, %ymm4 ; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm2, %ymm3, %ymm4
; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm0, %ymm2, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpcmpgtb %ymm1, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm4, %zmm5, %zmm4
; AVX512VL-FALLBACK-NEXT: vpminsb %ymm0, %ymm2, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpminsb %ymm1, %ymm0, %ymm5
; AVX512VL-FALLBACK-NEXT: vpminsb %ymm1, %ymm3, %ymm6 ; AVX512VL-FALLBACK-NEXT: vpminsb %ymm2, %ymm3, %ymm6
; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm0, %ymm2, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm1, %ymm0, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm5, %ymm0, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm5, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm1, %ymm3, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpmaxsb %ymm2, %ymm3, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm6, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm0, %ymm0 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm1, %zmm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpsrlw $1, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm5
; AVX512VL-FALLBACK-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm0, %zmm0 ; AVX512VL-FALLBACK-NEXT: vpandq {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %zmm5, %zmm5
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm0, %ymm1 ; AVX512VL-FALLBACK-NEXT: vmovdqa {{.*#+}} ymm6 = [127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127,127]
; AVX512VL-FALLBACK-NEXT: vpxor %xmm5, %xmm5, %xmm5 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm5, %ymm1 ; AVX512VL-FALLBACK-NEXT: vpand %ymm6, %ymm1, %ymm1
; AVX512VL-FALLBACK-NEXT: vpsubb %ymm0, %ymm5, %ymm5 ; AVX512VL-FALLBACK-NEXT: vpxor %xmm6, %xmm6, %xmm6
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm5, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm1, %ymm6, %ymm1
; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm0, %zmm4, %zmm1 ; AVX512VL-FALLBACK-NEXT: vpsubb %ymm2, %ymm6, %ymm2
; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm0 ; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm1, %zmm2, %zmm1
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm3, %ymm0, %ymm0 ; AVX512VL-FALLBACK-NEXT: vpternlogq $226, %zmm5, %zmm4, %zmm1
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm2, %ymm1, %ymm1 ; AVX512VL-FALLBACK-NEXT: vextracti64x4 $1, %zmm1, %ymm2
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm0, %zmm1, %zmm0 ; AVX512VL-FALLBACK-NEXT: vpaddb %ymm3, %ymm2, %ymm2
; AVX512VL-FALLBACK-NEXT: vpaddb %ymm0, %ymm1, %ymm0
; AVX512VL-FALLBACK-NEXT: vinserti64x4 $1, %ymm2, %zmm0, %zmm0
; AVX512VL-FALLBACK-NEXT: retq ; AVX512VL-FALLBACK-NEXT: retq
; ;
; AVX512BW-LABEL: vec512_i8_signed_mem_mem: ; AVX512BW-LABEL: vec512_i8_signed_mem_mem:
; AVX512BW: # %bb.0: ; AVX512BW: # %bb.0:
; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm0 ; AVX512BW-NEXT: vmovdqa64 (%rdi), %zmm0
; AVX512BW-NEXT: vmovdqa64 (%rsi), %zmm1 ; AVX512BW-NEXT: vmovdqa64 (%rsi), %zmm1
; AVX512BW-NEXT: vpcmpgtb %zmm1, %zmm0, %k1 ; AVX512BW-NEXT: vpcmpgtb %zmm1, %zmm0, %k1
; AVX512BW-NEXT: vpminsb %zmm1, %zmm0, %zmm2 ; AVX512BW-NEXT: vpminsb %zmm1, %zmm0, %zmm2
Show All 20 Lines