This is an archive of the discontinued LLVM Phabricator instance.

Differential D60260

Match types of accumulator and result for llvm.experimental.vector.reduce.fadd/fmul
ClosedPublic

Authored by sdesmalen on Apr 4 2019, 5:24 AM.

Details

Reviewers
RKSimon
aemerson
efriedma
spatel
nikic
Commits
rZORG7cb5c154450d: Match types of accumulator and result for llvm.experimental.vector.reduce.
rG7cb5c154450d: Match types of accumulator and result for llvm.experimental.vector.reduce.
rGf83cccf917c1: Match types of accumulator and result for llvm.experimental.vector.reduce.
rL361133: Match types of accumulator and result for llvm.experimental.vector.reduce.
Summary

The scalar start/accumulator value of the fadd- and fmul reduction
should match the result type of the reduction, as well as the vector
element-type of the input vector. Although this was not explicitly
specified in the LangRef, it was taken for granted in code implementing
the reductions. The patch also fixes the LangRef by adding this
constraint.

Diff Detail

Repository
rL LLVM

Event Timeline

sdesmalen created this revision.Apr 4 2019, 5:24 AM
Herald added a subscriber: javed.absar. · View Herald TranscriptApr 4 2019, 5:24 AM
nikic added a subscriber: nikic.Apr 6 2019, 8:44 AM
dmgreen added a subscriber: dmgreen.Apr 12 2019, 11:10 AM
nikic requested changes to this revision.May 11 2019, 1:14 AM

This looks good, but please also update existing tests that specify three types in the intrinstic name. Here's ones I found:

  • test/CodeGen/X86/haddsub.ll
  • test/CodeGen/X86/vector-reduce-fadd-fast.ll
  • test/CodeGen/X86/vector-reduce-fadd.ll
  • test/CodeGen/X86/vector-reduce-fmul-fast.ll
  • test/CodeGen/X86/vector-reduce-fmul.ll

Looks like AArch64/Generic tests already use the (previously incorrect and now correct) two type form.

This revision now requires changes to proceed.May 11 2019, 1:14 AM
sdesmalen updated this revision to Diff 200009.May 17 2019, 3:19 AM
  • Updated more tests.
sdesmalen added a comment.May 17 2019, 3:20 AM
In D60260#1499005, @nikic wrote:

This looks good, but please also update existing tests that specify three types in the intrinstic name. Here's ones I found:

  • test/CodeGen/X86/haddsub.ll
  • test/CodeGen/X86/vector-reduce-fadd-fast.ll
  • test/CodeGen/X86/vector-reduce-fadd.ll
  • test/CodeGen/X86/vector-reduce-fmul-fast.ll
  • test/CodeGen/X86/vector-reduce-fmul.ll

Good spot! I've updated these tests now.

aemerson accepted this revision.May 17 2019, 9:05 AM

LGTM.

nikic accepted this revision.May 19 2019, 1:01 PM

LG

This revision is now accepted and ready to land.May 19 2019, 1:01 PM
Closed by commit rL361133: Match types of accumulator and result for llvm.experimental.vector.reduce. (authored by s.desmalen). · Explain WhyMay 20 2019, 2:51 AM
This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptMay 20 2019, 2:51 AM

Revision Contents

PathSize
llvm/
trunk/
docs/
6 lines
include/
llvm/
IR/
4 lines
lib/
IR/
6 lines
test/
Assembler/
34 lines
CodeGen/
AArch64/
32 lines
X86/
8 lines
66 lines
66 lines
64 lines
66 lines

Diff 200226

llvm/trunk/docs/LangRef.rst

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 13,574 Lines • ▼ Show 20 Lines
fast-math flags, then the reduction will be *ordered*, implying that the fast-math flags, then the reduction will be *ordered*, implying that the
operation respects the associativity of a scalarized reduction. operation respects the associativity of a scalarized reduction.
Arguments: Arguments:
"""""""""" """"""""""
The first argument to this intrinsic is a scalar accumulator value, which is The first argument to this intrinsic is a scalar accumulator value, which is
only used when there are no fast-math flags attached. This argument may be undef only used when there are no fast-math flags attached. This argument may be undef
when fast-math flags are used. when fast-math flags are used. The type of the accumulator matches the
element-type of the vector input.
The second argument must be a vector of floating-point values. The second argument must be a vector of floating-point values.
Examples: Examples:
""""""""" """""""""
.. code-block:: llvm .. code-block:: llvm
▲ Show 20 Lines • Show All 46 Lines • ▼ Show 20 Lines
fast-math flags, then the reduction will be *ordered*, implying that the fast-math flags, then the reduction will be *ordered*, implying that the
operation respects the associativity of a scalarized reduction. operation respects the associativity of a scalarized reduction.
Arguments: Arguments:
"""""""""" """"""""""
The first argument to this intrinsic is a scalar accumulator value, which is The first argument to this intrinsic is a scalar accumulator value, which is
only used when there are no fast-math flags attached. This argument may be undef only used when there are no fast-math flags attached. This argument may be undef
when fast-math flags are used. when fast-math flags are used. The type of the accumulator matches the
element-type of the vector input.
The second argument must be a vector of floating-point values. The second argument must be a vector of floating-point values.
Examples: Examples:
""""""""" """""""""
.. code-block:: llvm .. code-block:: llvm
▲ Show 20 Lines • Show All 3,383 Lines • Show Last 20 Lines

llvm/trunk/include/llvm/IR/Intrinsics.td

Show First 20 Lines • Show All 1,128 Lines • ▼ Show 20 Lines
// @llvm.memset.element.unordered.atomic.*(dest, value, length, elementsize) // @llvm.memset.element.unordered.atomic.*(dest, value, length, elementsize)
def int_memset_element_unordered_atomic def int_memset_element_unordered_atomic
: Intrinsic<[], [ llvm_anyptr_ty, llvm_i8_ty, llvm_anyint_ty, llvm_i32_ty ], : Intrinsic<[], [ llvm_anyptr_ty, llvm_i8_ty, llvm_anyint_ty, llvm_i32_ty ],
[ IntrArgMemOnly, NoCapture<0>, WriteOnly<0>, ImmArg<3> ]>; [ IntrArgMemOnly, NoCapture<0>, WriteOnly<0>, ImmArg<3> ]>;
//===------------------------ Reduction Intrinsics ------------------------===// //===------------------------ Reduction Intrinsics ------------------------===//
// //
def int_experimental_vector_reduce_fadd : Intrinsic<[llvm_anyfloat_ty], def int_experimental_vector_reduce_fadd : Intrinsic<[llvm_anyfloat_ty],
[llvm_anyfloat_ty, [LLVMMatchType<0>,
llvm_anyvector_ty], llvm_anyvector_ty],
[IntrNoMem]>; [IntrNoMem]>;
def int_experimental_vector_reduce_fmul : Intrinsic<[llvm_anyfloat_ty], def int_experimental_vector_reduce_fmul : Intrinsic<[llvm_anyfloat_ty],
[llvm_anyfloat_ty, [LLVMMatchType<0>,
llvm_anyvector_ty], llvm_anyvector_ty],
[IntrNoMem]>; [IntrNoMem]>;
def int_experimental_vector_reduce_add : Intrinsic<[llvm_anyint_ty], def int_experimental_vector_reduce_add : Intrinsic<[llvm_anyint_ty],
[llvm_anyvector_ty], [llvm_anyvector_ty],
[IntrNoMem]>; [IntrNoMem]>;
def int_experimental_vector_reduce_mul : Intrinsic<[llvm_anyint_ty], def int_experimental_vector_reduce_mul : Intrinsic<[llvm_anyint_ty],
[llvm_anyvector_ty], [llvm_anyvector_ty],
[IntrNoMem]>; [IntrNoMem]>;
▲ Show 20 Lines • Show All 49 Lines • Show Last 20 Lines

llvm/trunk/lib/IR/IRBuilder.cpp

Show First 20 Lines • Show All 315 Lines • ▼ Show 20 Linesstatic CallInst *getReductionIntrinsic(IRBuilderBase *Builder, Intrinsic::ID ID,
Type *Tys[] = { Src->getType()->getVectorElementType(), Src->getType() }; Type *Tys[] = { Src->getType()->getVectorElementType(), Src->getType() };
auto Decl = Intrinsic::getDeclaration(M, ID, Tys); auto Decl = Intrinsic::getDeclaration(M, ID, Tys);
return createCallHelper(Decl, Ops, Builder); return createCallHelper(Decl, Ops, Builder);
} }
CallInst *IRBuilderBase::CreateFAddReduce(Value *Acc, Value *Src) { CallInst *IRBuilderBase::CreateFAddReduce(Value *Acc, Value *Src) {
Module *M = GetInsertBlock()->getParent()->getParent(); Module *M = GetInsertBlock()->getParent()->getParent();
Value *Ops[] = {Acc, Src}; Value *Ops[] = {Acc, Src};
Type *Tys[] = {Src->getType()->getVectorElementType(), Acc->getType(), Type *Tys[] = {Acc->getType(), Src->getType()};
Src->getType()};
auto Decl = Intrinsic::getDeclaration( auto Decl = Intrinsic::getDeclaration(
M, Intrinsic::experimental_vector_reduce_fadd, Tys); M, Intrinsic::experimental_vector_reduce_fadd, Tys);
return createCallHelper(Decl, Ops, this); return createCallHelper(Decl, Ops, this);
} }
CallInst *IRBuilderBase::CreateFMulReduce(Value *Acc, Value *Src) { CallInst *IRBuilderBase::CreateFMulReduce(Value *Acc, Value *Src) {
Module *M = GetInsertBlock()->getParent()->getParent(); Module *M = GetInsertBlock()->getParent()->getParent();
Value *Ops[] = {Acc, Src}; Value *Ops[] = {Acc, Src};
Type *Tys[] = {Src->getType()->getVectorElementType(), Acc->getType(), Type *Tys[] = {Acc->getType(), Src->getType()};
Src->getType()};
auto Decl = Intrinsic::getDeclaration( auto Decl = Intrinsic::getDeclaration(
M, Intrinsic::experimental_vector_reduce_fmul, Tys); M, Intrinsic::experimental_vector_reduce_fmul, Tys);
return createCallHelper(Decl, Ops, this); return createCallHelper(Decl, Ops, this);
} }
CallInst *IRBuilderBase::CreateAddReduce(Value *Src) { CallInst *IRBuilderBase::CreateAddReduce(Value *Src) {
return getReductionIntrinsic(this, Intrinsic::experimental_vector_reduce_add, return getReductionIntrinsic(this, Intrinsic::experimental_vector_reduce_add,
Src); Src);
▲ Show 20 Lines • Show All 415 Lines • Show Last 20 Lines

llvm/trunk/test/Assembler/invalid-vecreduce.ll

; RUN: not opt -S < %s 2>&1 | FileCheck %s
; CHECK: Intrinsic has incorrect argument type!
; CHECK-NEXT: float (double, <2 x double>)* @llvm.experimental.vector.reduce.fadd.f32.f64.v2f64
define float @fadd_invalid_scalar_res(double %acc, <2 x double> %in) {
%res = call float @llvm.experimental.vector.reduce.fadd.f32.f64.v2f64(double %acc, <2 x double> %in)
ret float %res
}
; CHECK: Intrinsic has incorrect argument type!
; CHECK-NEXT: double (float, <2 x double>)* @llvm.experimental.vector.reduce.fadd.f64.f32.v2f64
define double @fadd_invalid_scalar_start(float %acc, <2 x double> %in) {
%res = call double @llvm.experimental.vector.reduce.fadd.f64.f32.v2f64(float %acc, <2 x double> %in)
ret double %res
}
; CHECK: Intrinsic has incorrect argument type!
; CHECK-NEXT: <2 x double> (double, <2 x double>)* @llvm.experimental.vector.reduce.fadd.v2f64.f64.v2f64
define <2 x double> @fadd_invalid_vector_res(double %acc, <2 x double> %in) {
%res = call <2 x double> @llvm.experimental.vector.reduce.fadd.v2f64.f64.v2f64(double %acc, <2 x double> %in)
ret <2 x double> %res
}
; CHECK: Intrinsic has incorrect argument type!
; CHECK-NEXT: double (<2 x double>, <2 x double>)* @llvm.experimental.vector.reduce.fadd.f64.v2f64.v2f64
define double @fadd_invalid_vector_start(<2 x double> %in, <2 x double> %acc) {
%res = call double @llvm.experimental.vector.reduce.fadd.f64.v2f64.v2f64(<2 x double> %acc, <2 x double> %in)
ret double %res
}
declare float @llvm.experimental.vector.reduce.fadd.f32.f64.v2f64(double %acc, <2 x double> %in)
declare double @llvm.experimental.vector.reduce.fadd.f64.f32.v2f64(float %acc, <2 x double> %in)
declare double @llvm.experimental.vector.reduce.fadd.f64.v2f64.v2f64(<2 x double> %acc, <2 x double> %in)
declare <2 x double> @llvm.experimental.vector.reduce.fadd.v2f64.f64.v2f64(double %acc, <2 x double> %in)

llvm/trunk/test/CodeGen/AArch64/vecreduce-fadd.ll

; RUN: llc < %s -mtriple=aarch64-eabi -aarch64-neon-syntax=generic -asm-verbose=0 -mattr=+fullfp16 | FileCheck %s ; RUN: llc < %s -mtriple=aarch64-eabi -aarch64-neon-syntax=generic -asm-verbose=0 -mattr=+fullfp16 | FileCheck %s
; RUN: llc < %s -mtriple=aarch64-eabi -aarch64-neon-syntax=generic -asm-verbose=0 | FileCheck %s --check-prefix=CHECKNOFP16 ; RUN: llc < %s -mtriple=aarch64-eabi -aarch64-neon-syntax=generic -asm-verbose=0 | FileCheck %s --check-prefix=CHECKNOFP16
define float @add_HalfS(<2 x float> %bin.rdx) { define float @add_HalfS(<2 x float> %bin.rdx) {
; CHECK-LABEL: add_HalfS: ; CHECK-LABEL: add_HalfS:
; CHECK: faddp s0, v0.2s ; CHECK: faddp s0, v0.2s
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%r = call fast float @llvm.experimental.vector.reduce.fadd.f32.v2f32(<2 x float> undef, <2 x float> %bin.rdx) %r = call fast float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float undef, <2 x float> %bin.rdx)
ret float %r ret float %r
} }
define half @add_HalfH(<4 x half> %bin.rdx) { define half @add_HalfH(<4 x half> %bin.rdx) {
; CHECK-LABEL: add_HalfH: ; CHECK-LABEL: add_HalfH:
; CHECK: mov h3, v0.h[1] ; CHECK: mov h3, v0.h[1]
; CHECK-NEXT: mov h1, v0.h[3] ; CHECK-NEXT: mov h1, v0.h[3]
; CHECK-NEXT: mov h2, v0.h[2] ; CHECK-NEXT: mov h2, v0.h[2]
; CHECK-NEXT: fadd h0, h0, h3 ; CHECK-NEXT: fadd h0, h0, h3
; CHECK-NEXT: fadd h0, h0, h2 ; CHECK-NEXT: fadd h0, h0, h2
; CHECK-NEXT: fadd h0, h0, h1 ; CHECK-NEXT: fadd h0, h0, h1
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECKNOFP16-LABEL: add_HalfH: ; CHECKNOFP16-LABEL: add_HalfH:
; CHECKNOFP16-NOT: faddp ; CHECKNOFP16-NOT: faddp
; CHECKNOFP16-NOT: fadd h{{[0-9]+}} ; CHECKNOFP16-NOT: fadd h{{[0-9]+}}
; CHECKNOFP16-NOT: fadd v{{[0-9]+}}.{{[0-9]}}h ; CHECKNOFP16-NOT: fadd v{{[0-9]+}}.{{[0-9]}}h
; CHECKNOFP16: ret ; CHECKNOFP16: ret
%r = call fast half @llvm.experimental.vector.reduce.fadd.f16.v4f16(<4 x half> undef, <4 x half> %bin.rdx) %r = call fast half @llvm.experimental.vector.reduce.fadd.f16.v4f16(half undef, <4 x half> %bin.rdx)
ret half %r ret half %r
} }
define half @add_H(<8 x half> %bin.rdx) { define half @add_H(<8 x half> %bin.rdx) {
; CHECK-LABEL: add_H: ; CHECK-LABEL: add_H:
; CHECK: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-NEXT: fadd v0.4h, v0.4h, v1.4h ; CHECK-NEXT: fadd v0.4h, v0.4h, v1.4h
; CHECK-NEXT: mov h1, v0.h[1] ; CHECK-NEXT: mov h1, v0.h[1]
; CHECK-NEXT: mov h2, v0.h[2] ; CHECK-NEXT: mov h2, v0.h[2]
; CHECK-NEXT: fadd h1, h0, h1 ; CHECK-NEXT: fadd h1, h0, h1
; CHECK-NEXT: fadd h1, h1, h2 ; CHECK-NEXT: fadd h1, h1, h2
; CHECK-NEXT: mov h0, v0.h[3] ; CHECK-NEXT: mov h0, v0.h[3]
; CHECK-NEXT: fadd h0, h1, h0 ; CHECK-NEXT: fadd h0, h1, h0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECKNOFP16-LABEL: add_H: ; CHECKNOFP16-LABEL: add_H:
; CHECKNOFP16-NOT: faddp ; CHECKNOFP16-NOT: faddp
; CHECKNOFP16-NOT: fadd h{{[0-9]+}} ; CHECKNOFP16-NOT: fadd h{{[0-9]+}}
; CHECKNOFP16-NOT: fadd v{{[0-9]+}}.{{[0-9]}}h ; CHECKNOFP16-NOT: fadd v{{[0-9]+}}.{{[0-9]}}h
; CHECKNOFP16: ret ; CHECKNOFP16: ret
%r = call fast half @llvm.experimental.vector.reduce.fadd.f16.v8f16(<8 x half> undef, <8 x half> %bin.rdx) %r = call fast half @llvm.experimental.vector.reduce.fadd.f16.v8f16(half undef, <8 x half> %bin.rdx)
ret half %r ret half %r
} }
define float @add_S(<4 x float> %bin.rdx) { define float @add_S(<4 x float> %bin.rdx) {
; CHECK-LABEL: add_S: ; CHECK-LABEL: add_S:
; CHECK: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-NEXT: fadd v0.2s, v0.2s, v1.2s ; CHECK-NEXT: fadd v0.2s, v0.2s, v1.2s
; CHECK-NEXT: faddp s0, v0.2s ; CHECK-NEXT: faddp s0, v0.2s
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%r = call fast float @llvm.experimental.vector.reduce.fadd.f32.v4f32(<4 x float> undef, <4 x float> %bin.rdx) %r = call fast float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float undef, <4 x float> %bin.rdx)
ret float %r ret float %r
} }
define double @add_D(<2 x double> %bin.rdx) { define double @add_D(<2 x double> %bin.rdx) {
; CHECK-LABEL: add_D: ; CHECK-LABEL: add_D:
; CHECK: faddp d0, v0.2d ; CHECK: faddp d0, v0.2d
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%r = call fast double @llvm.experimental.vector.reduce.fadd.f64.v2f64(<2 x double> undef, <2 x double> %bin.rdx) %r = call fast double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double undef, <2 x double> %bin.rdx)
ret double %r ret double %r
} }
define half @add_2H(<16 x half> %bin.rdx) { define half @add_2H(<16 x half> %bin.rdx) {
; CHECK-LABEL: add_2H: ; CHECK-LABEL: add_2H:
; CHECK: fadd v0.8h, v0.8h, v1.8h ; CHECK: fadd v0.8h, v0.8h, v1.8h
; CHECK-NEXT: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK-NEXT: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-NEXT: fadd v0.4h, v0.4h, v1.4h ; CHECK-NEXT: fadd v0.4h, v0.4h, v1.4h
; CHECK-NEXT: mov h1, v0.h[1] ; CHECK-NEXT: mov h1, v0.h[1]
; CHECK-NEXT: mov h2, v0.h[2] ; CHECK-NEXT: mov h2, v0.h[2]
; CHECK-NEXT: fadd h1, h0, h1 ; CHECK-NEXT: fadd h1, h0, h1
; CHECK-NEXT: fadd h1, h1, h2 ; CHECK-NEXT: fadd h1, h1, h2
; CHECK-NEXT: mov h0, v0.h[3] ; CHECK-NEXT: mov h0, v0.h[3]
; CHECK-NEXT: fadd h0, h1, h0 ; CHECK-NEXT: fadd h0, h1, h0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECKNOFP16-LABEL: add_2H: ; CHECKNOFP16-LABEL: add_2H:
; CHECKNOFP16-NOT: faddp ; CHECKNOFP16-NOT: faddp
; CHECKNOFP16-NOT: fadd h{{[0-9]+}} ; CHECKNOFP16-NOT: fadd h{{[0-9]+}}
; CHECKNOFP16-NOT: fadd v{{[0-9]+}}.{{[0-9]}}h ; CHECKNOFP16-NOT: fadd v{{[0-9]+}}.{{[0-9]}}h
; CHECKNOFP16: ret ; CHECKNOFP16: ret
%r = call fast half @llvm.experimental.vector.reduce.fadd.f16.v16f16(<16 x half> undef, <16 x half> %bin.rdx) %r = call fast half @llvm.experimental.vector.reduce.fadd.f16.v16f16(half undef, <16 x half> %bin.rdx)
ret half %r ret half %r
} }
define float @add_2S(<8 x float> %bin.rdx) { define float @add_2S(<8 x float> %bin.rdx) {
; CHECK-LABEL: add_2S: ; CHECK-LABEL: add_2S:
; CHECK: fadd v0.4s, v0.4s, v1.4s ; CHECK: fadd v0.4s, v0.4s, v1.4s
; CHECK-NEXT: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK-NEXT: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-NEXT: fadd v0.2s, v0.2s, v1.2s ; CHECK-NEXT: fadd v0.2s, v0.2s, v1.2s
; CHECK-NEXT: faddp s0, v0.2s ; CHECK-NEXT: faddp s0, v0.2s
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%r = call fast float @llvm.experimental.vector.reduce.fadd.f32.v8f32(<8 x float> undef, <8 x float> %bin.rdx) %r = call fast float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float undef, <8 x float> %bin.rdx)
ret float %r ret float %r
} }
define double @add_2D(<4 x double> %bin.rdx) { define double @add_2D(<4 x double> %bin.rdx) {
; CHECK-LABEL: add_2D: ; CHECK-LABEL: add_2D:
; CHECK: fadd v0.2d, v0.2d, v1.2d ; CHECK: fadd v0.2d, v0.2d, v1.2d
; CHECK-NEXT: faddp d0, v0.2d ; CHECK-NEXT: faddp d0, v0.2d
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%r = call fast double @llvm.experimental.vector.reduce.fadd.f64.v4f64(<4 x double> undef, <4 x double> %bin.rdx) %r = call fast double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double undef, <4 x double> %bin.rdx)
ret double %r ret double %r
} }
; Function Attrs: nounwind readnone ; Function Attrs: nounwind readnone
declare half @llvm.experimental.vector.reduce.fadd.f16.v4f16(<4 x half>, <4 x half>) declare half @llvm.experimental.vector.reduce.fadd.f16.v4f16(half, <4 x half>)
declare half @llvm.experimental.vector.reduce.fadd.f16.v8f16(<8 x half>, <8 x half>) declare half @llvm.experimental.vector.reduce.fadd.f16.v8f16(half, <8 x half>)
declare half @llvm.experimental.vector.reduce.fadd.f16.v16f16(<16 x half>, <16 x half>) declare half @llvm.experimental.vector.reduce.fadd.f16.v16f16(half, <16 x half>)
declare float @llvm.experimental.vector.reduce.fadd.f32.v2f32(<2 x float>, <2 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float, <2 x float>)
declare float @llvm.experimental.vector.reduce.fadd.f32.v4f32(<4 x float>, <4 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float, <4 x float>)
declare float @llvm.experimental.vector.reduce.fadd.f32.v8f32(<8 x float>, <8 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float, <8 x float>)
declare double @llvm.experimental.vector.reduce.fadd.f64.v2f64(<2 x double>, <2 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double, <2 x double>)
declare double @llvm.experimental.vector.reduce.fadd.f64.v4f64(<4 x double>, <4 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double, <4 x double>)

llvm/trunk/test/CodeGen/X86/haddsub.ll

Show First 20 Lines • Show All 1,622 Lines • ▼ Show 20 Lines; AVX-NEXT: retq
store float %x1, float* %p2 store float %x1, float* %p2
%x01 = fadd float %x0, %x1 %x01 = fadd float %x0, %x1
ret float %x01 ret float %x01
} }
; Repeat tests from general reductions to verify output for hoppy targets: ; Repeat tests from general reductions to verify output for hoppy targets:
; PR38971: https://bugs.llvm.org/show_bug.cgi?id=38971 ; PR38971: https://bugs.llvm.org/show_bug.cgi?id=38971
declare float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float, <8 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float, <8 x float>)
declare double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double, <4 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double, <4 x double>)
define float @fadd_reduce_v8f32(float %a0, <8 x float> %a1) { define float @fadd_reduce_v8f32(float %a0, <8 x float> %a1) {
; SSE3-SLOW-LABEL: fadd_reduce_v8f32: ; SSE3-SLOW-LABEL: fadd_reduce_v8f32:
; SSE3-SLOW: # %bb.0: ; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: addps %xmm2, %xmm1 ; SSE3-SLOW-NEXT: addps %xmm2, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm2 ; SSE3-SLOW-NEXT: movaps %xmm1, %xmm2
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm1[1] ; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm1[1]
; SSE3-SLOW-NEXT: addps %xmm1, %xmm2 ; SSE3-SLOW-NEXT: addps %xmm1, %xmm2
Show All 25 Lines
; AVX-FAST: # %bb.0: ; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vextractf128 $1, %ymm1, %xmm0 ; AVX-FAST-NEXT: vextractf128 $1, %ymm1, %xmm0
; AVX-FAST-NEXT: vaddps %xmm0, %xmm1, %xmm0 ; AVX-FAST-NEXT: vaddps %xmm0, %xmm1, %xmm0
; AVX-FAST-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX-FAST-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-FAST-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX-FAST-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0 ; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper ; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq ; AVX-FAST-NEXT: retq
%r = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float %a0, <8 x float> %a1) %r = call fast float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float %a0, <8 x float> %a1)
ret float %r ret float %r
} }
define double @fadd_reduce_v4f64(double %a0, <4 x double> %a1) { define double @fadd_reduce_v4f64(double %a0, <4 x double> %a1) {
; SSE3-SLOW-LABEL: fadd_reduce_v4f64: ; SSE3-SLOW-LABEL: fadd_reduce_v4f64:
; SSE3-SLOW: # %bb.0: ; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: addpd %xmm2, %xmm1 ; SSE3-SLOW-NEXT: addpd %xmm2, %xmm1
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0 ; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
Show All 19 Lines
; ;
; AVX-FAST-LABEL: fadd_reduce_v4f64: ; AVX-FAST-LABEL: fadd_reduce_v4f64:
; AVX-FAST: # %bb.0: ; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vextractf128 $1, %ymm1, %xmm0 ; AVX-FAST-NEXT: vextractf128 $1, %ymm1, %xmm0
; AVX-FAST-NEXT: vaddpd %xmm0, %xmm1, %xmm0 ; AVX-FAST-NEXT: vaddpd %xmm0, %xmm1, %xmm0
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0 ; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper ; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq ; AVX-FAST-NEXT: retq
%r = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double %a0, <4 x double> %a1) %r = call fast double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double %a0, <4 x double> %a1)
ret double %r ret double %r
} }
define float @PR39936_v8f32(<8 x float>) { define float @PR39936_v8f32(<8 x float>) {
; SSSE3-SLOW-LABEL: PR39936_v8f32: ; SSSE3-SLOW-LABEL: PR39936_v8f32:
; SSSE3-SLOW: # %bb.0: ; SSSE3-SLOW: # %bb.0:
; SSSE3-SLOW-NEXT: haddps %xmm1, %xmm0 ; SSSE3-SLOW-NEXT: haddps %xmm1, %xmm0
; SSSE3-SLOW-NEXT: movaps %xmm0, %xmm1 ; SSSE3-SLOW-NEXT: movaps %xmm0, %xmm1
▲ Show 20 Lines • Show All 58 Lines • Show Last 20 Lines

llvm/trunk/test/CodeGen/X86/vector-reduce-fadd-fast.ll

Show First 20 Lines • Show All 41 Lines • ▼ Show 20 Lines
; AVX2-NEXT: vaddss %xmm0, %xmm1, %xmm0 ; AVX2-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32: ; AVX512-LABEL: test_v2f32:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v2f32(float %a0, <2 x float> %a1) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float %a0, <2 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v4f32(float %a0, <4 x float> %a1) { define float @test_v4f32(float %a0, <4 x float> %a1) {
; SSE2-LABEL: test_v4f32: ; SSE2-LABEL: test_v4f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm1, %xmm2 ; SSE2-NEXT: movaps %xmm1, %xmm2
; SSE2-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm1[1] ; SSE2-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm1[1]
Show All 37 Lines
; ;
; AVX512-LABEL: test_v4f32: ; AVX512-LABEL: test_v4f32:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm1[1,0]
; AVX512-NEXT: vaddps %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddps %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v4f32(float %a0, <4 x float> %a1) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float %a0, <4 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v8f32(float %a0, <8 x float> %a1) { define float @test_v8f32(float %a0, <8 x float> %a1) {
; SSE2-LABEL: test_v8f32: ; SSE2-LABEL: test_v8f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: addps %xmm2, %xmm1 ; SSE2-NEXT: addps %xmm2, %xmm1
; SSE2-NEXT: movaps %xmm1, %xmm2 ; SSE2-NEXT: movaps %xmm1, %xmm2
▲ Show 20 Lines • Show All 51 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm0 ; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm0
; AVX512-NEXT: vaddps %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddps %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float %a0, <8 x float> %a1) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float %a0, <8 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v16f32(float %a0, <16 x float> %a1) { define float @test_v16f32(float %a0, <16 x float> %a1) {
; SSE2-LABEL: test_v16f32: ; SSE2-LABEL: test_v16f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: addps %xmm4, %xmm2 ; SSE2-NEXT: addps %xmm4, %xmm2
; SSE2-NEXT: addps %xmm3, %xmm1 ; SSE2-NEXT: addps %xmm3, %xmm1
▲ Show 20 Lines • Show All 60 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v16f32(float %a0, <16 x float> %a1) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v16f32(float %a0, <16 x float> %a1)
ret float %1 ret float %1
} }
; ;
; vXf32 (zero) ; vXf32 (zero)
; ;
define float @test_v2f32_zero(<2 x float> %a0) { define float @test_v2f32_zero(<2 x float> %a0) {
Show All 28 Lines
; AVX2-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32_zero: ; AVX512-LABEL: test_v2f32_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v2f32(float 0.0, <2 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float 0.0, <2 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v4f32_zero(<4 x float> %a0) { define float @test_v4f32_zero(<4 x float> %a0) {
; SSE2-LABEL: test_v4f32_zero: ; SSE2-LABEL: test_v4f32_zero:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
; SSE2-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1] ; SSE2-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
Show All 38 Lines
; ;
; AVX512-LABEL: test_v4f32_zero: ; AVX512-LABEL: test_v4f32_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v4f32(float 0.0, <4 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float 0.0, <4 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v8f32_zero(<8 x float> %a0) { define float @test_v8f32_zero(<8 x float> %a0) {
; SSE2-LABEL: test_v8f32_zero: ; SSE2-LABEL: test_v8f32_zero:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: addps %xmm1, %xmm0 ; SSE2-NEXT: addps %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float 0.0, <8 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float 0.0, <8 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v16f32_zero(<16 x float> %a0) { define float @test_v16f32_zero(<16 x float> %a0) {
; SSE2-LABEL: test_v16f32_zero: ; SSE2-LABEL: test_v16f32_zero:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: addps %xmm3, %xmm1 ; SSE2-NEXT: addps %xmm3, %xmm1
; SSE2-NEXT: addps %xmm2, %xmm0 ; SSE2-NEXT: addps %xmm2, %xmm0
▲ Show 20 Lines • Show All 61 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v16f32(float 0.0, <16 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v16f32(float 0.0, <16 x float> %a0)
ret float %1 ret float %1
} }
; ;
; vXf32 (undef) ; vXf32 (undef)
; ;
define float @test_v2f32_undef(<2 x float> %a0) { define float @test_v2f32_undef(<2 x float> %a0) {
Show All 28 Lines
; AVX2-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32_undef: ; AVX512-LABEL: test_v2f32_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v2f32(float undef, <2 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float undef, <2 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v4f32_undef(<4 x float> %a0) { define float @test_v4f32_undef(<4 x float> %a0) {
; SSE2-LABEL: test_v4f32_undef: ; SSE2-LABEL: test_v4f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
; SSE2-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1] ; SSE2-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
Show All 38 Lines
; ;
; AVX512-LABEL: test_v4f32_undef: ; AVX512-LABEL: test_v4f32_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v4f32(float undef, <4 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float undef, <4 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v8f32_undef(<8 x float> %a0) { define float @test_v8f32_undef(<8 x float> %a0) {
; SSE2-LABEL: test_v8f32_undef: ; SSE2-LABEL: test_v8f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: addps %xmm1, %xmm0 ; SSE2-NEXT: addps %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float undef, <8 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float undef, <8 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v16f32_undef(<16 x float> %a0) { define float @test_v16f32_undef(<16 x float> %a0) {
; SSE2-LABEL: test_v16f32_undef: ; SSE2-LABEL: test_v16f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: addps %xmm3, %xmm1 ; SSE2-NEXT: addps %xmm3, %xmm1
; SSE2-NEXT: addps %xmm2, %xmm0 ; SSE2-NEXT: addps %xmm2, %xmm0
▲ Show 20 Lines • Show All 61 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.f32.v16f32(float undef, <16 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fadd.f32.v16f32(float undef, <16 x float> %a0)
ret float %1 ret float %1
} }
; ;
; vXf64 (accum) ; vXf64 (accum)
; ;
define double @test_v2f64(double %a0, <2 x double> %a1) { define double @test_v2f64(double %a0, <2 x double> %a1) {
Show All 21 Lines
; AVX2-NEXT: vaddsd %xmm0, %xmm1, %xmm0 ; AVX2-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64: ; AVX512-LABEL: test_v2f64:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm1[1,0]
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v2f64(double %a0, <2 x double> %a1) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double %a0, <2 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v4f64(double %a0, <4 x double> %a1) { define double @test_v4f64(double %a0, <4 x double> %a1) {
; SSE-LABEL: test_v4f64: ; SSE-LABEL: test_v4f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addpd %xmm2, %xmm1 ; SSE-NEXT: addpd %xmm2, %xmm1
; SSE-NEXT: movapd %xmm1, %xmm0 ; SSE-NEXT: movapd %xmm1, %xmm0
Show All 30 Lines
; AVX512-LABEL: test_v4f64: ; AVX512-LABEL: test_v4f64:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm0 ; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm0
; AVX512-NEXT: vaddpd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddpd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double %a0, <4 x double> %a1) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double %a0, <4 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v8f64(double %a0, <8 x double> %a1) { define double @test_v8f64(double %a0, <8 x double> %a1) {
; SSE-LABEL: test_v8f64: ; SSE-LABEL: test_v8f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addpd %xmm4, %xmm2 ; SSE-NEXT: addpd %xmm4, %xmm2
; SSE-NEXT: addpd %xmm3, %xmm1 ; SSE-NEXT: addpd %xmm3, %xmm1
Show All 37 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm1, %ymm0 ; AVX512-NEXT: vextractf64x4 $1, %zmm1, %ymm0
; AVX512-NEXT: vaddpd %zmm0, %zmm1, %zmm0 ; AVX512-NEXT: vaddpd %zmm0, %zmm1, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v8f64(double %a0, <8 x double> %a1) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v8f64(double %a0, <8 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v16f64(double %a0, <16 x double> %a1) { define double @test_v16f64(double %a0, <16 x double> %a1) {
; SSE-LABEL: test_v16f64: ; SSE-LABEL: test_v16f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addpd %xmm6, %xmm2 ; SSE-NEXT: addpd %xmm6, %xmm2
; SSE-NEXT: addpd %xmm7, %xmm3 ; SSE-NEXT: addpd %xmm7, %xmm3
▲ Show 20 Lines • Show All 48 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v16f64(double %a0, <16 x double> %a1) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v16f64(double %a0, <16 x double> %a1)
ret double %1 ret double %1
} }
; ;
; vXf64 (zero) ; vXf64 (zero)
; ;
define double @test_v2f64_zero(<2 x double> %a0) { define double @test_v2f64_zero(<2 x double> %a0) {
Show All 22 Lines
; AVX2-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64_zero: ; AVX512-LABEL: test_v2f64_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v2f64(double 0.0, <2 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double 0.0, <2 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v4f64_zero(<4 x double> %a0) { define double @test_v4f64_zero(<4 x double> %a0) {
; SSE-LABEL: test_v4f64_zero: ; SSE-LABEL: test_v4f64_zero:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addpd %xmm1, %xmm0 ; SSE-NEXT: addpd %xmm1, %xmm0
; SSE-NEXT: movapd %xmm0, %xmm1 ; SSE-NEXT: movapd %xmm0, %xmm1
Show All 31 Lines
; AVX512-LABEL: test_v4f64_zero: ; AVX512-LABEL: test_v4f64_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double 0.0, <4 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double 0.0, <4 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v8f64_zero(<8 x double> %a0) { define double @test_v8f64_zero(<8 x double> %a0) {
; SSE-LABEL: test_v8f64_zero: ; SSE-LABEL: test_v8f64_zero:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addpd %xmm3, %xmm1 ; SSE-NEXT: addpd %xmm3, %xmm1
; SSE-NEXT: addpd %xmm2, %xmm0 ; SSE-NEXT: addpd %xmm2, %xmm0
Show All 38 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v8f64(double 0.0, <8 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v8f64(double 0.0, <8 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v16f64_zero(<16 x double> %a0) { define double @test_v16f64_zero(<16 x double> %a0) {
; SSE-LABEL: test_v16f64_zero: ; SSE-LABEL: test_v16f64_zero:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addpd %xmm6, %xmm2 ; SSE-NEXT: addpd %xmm6, %xmm2
; SSE-NEXT: addpd %xmm4, %xmm0 ; SSE-NEXT: addpd %xmm4, %xmm0
▲ Show 20 Lines • Show All 48 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v16f64(double 0.0, <16 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v16f64(double 0.0, <16 x double> %a0)
ret double %1 ret double %1
} }
; ;
; vXf64 (undef) ; vXf64 (undef)
; ;
define double @test_v2f64_undef(<2 x double> %a0) { define double @test_v2f64_undef(<2 x double> %a0) {
Show All 22 Lines
; AVX2-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX2-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: retq ; AVX2-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64_undef: ; AVX512-LABEL: test_v2f64_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v2f64(double undef, <2 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double undef, <2 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v4f64_undef(<4 x double> %a0) { define double @test_v4f64_undef(<4 x double> %a0) {
; SSE-LABEL: test_v4f64_undef: ; SSE-LABEL: test_v4f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addpd %xmm1, %xmm0 ; SSE-NEXT: addpd %xmm1, %xmm0
; SSE-NEXT: movapd %xmm0, %xmm1 ; SSE-NEXT: movapd %xmm0, %xmm1
Show All 31 Lines
; AVX512-LABEL: test_v4f64_undef: ; AVX512-LABEL: test_v4f64_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double undef, <4 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double undef, <4 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v8f64_undef(<8 x double> %a0) { define double @test_v8f64_undef(<8 x double> %a0) {
; SSE-LABEL: test_v8f64_undef: ; SSE-LABEL: test_v8f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addpd %xmm3, %xmm1 ; SSE-NEXT: addpd %xmm3, %xmm1
; SSE-NEXT: addpd %xmm2, %xmm0 ; SSE-NEXT: addpd %xmm2, %xmm0
Show All 38 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v8f64(double undef, <8 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v8f64(double undef, <8 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v16f64_undef(<16 x double> %a0) { define double @test_v16f64_undef(<16 x double> %a0) {
; SSE-LABEL: test_v16f64_undef: ; SSE-LABEL: test_v16f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addpd %xmm6, %xmm2 ; SSE-NEXT: addpd %xmm6, %xmm2
; SSE-NEXT: addpd %xmm4, %xmm0 ; SSE-NEXT: addpd %xmm4, %xmm0
▲ Show 20 Lines • Show All 48 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vaddpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.f64.v16f64(double undef, <16 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fadd.f64.v16f64(double undef, <16 x double> %a0)
ret double %1 ret double %1
} }
declare float @llvm.experimental.vector.reduce.fadd.f32.f32.v2f32(float, <2 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float, <2 x float>)
declare float @llvm.experimental.vector.reduce.fadd.f32.f32.v4f32(float, <4 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float, <4 x float>)
declare float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float, <8 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float, <8 x float>)
declare float @llvm.experimental.vector.reduce.fadd.f32.f32.v16f32(float, <16 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v16f32(float, <16 x float>)
declare double @llvm.experimental.vector.reduce.fadd.f64.f64.v2f64(double, <2 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double, <2 x double>)
declare double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double, <4 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double, <4 x double>)
declare double @llvm.experimental.vector.reduce.fadd.f64.f64.v8f64(double, <8 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v8f64(double, <8 x double>)
declare double @llvm.experimental.vector.reduce.fadd.f64.f64.v16f64(double, <16 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v16f64(double, <16 x double>)

llvm/trunk/test/CodeGen/X86/vector-reduce-fadd.ll

Show All 33 Lines
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32: ; AVX512-LABEL: test_v2f32:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm1[1,1,3,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v2f32(float %a0, <2 x float> %a1) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float %a0, <2 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v4f32(float %a0, <4 x float> %a1) { define float @test_v4f32(float %a0, <4 x float> %a1) {
; SSE2-LABEL: test_v4f32: ; SSE2-LABEL: test_v4f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: addss %xmm1, %xmm0 ; SSE2-NEXT: addss %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm1, %xmm2 ; SSE2-NEXT: movaps %xmm1, %xmm2
Show All 34 Lines
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3]
; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v4f32(float %a0, <4 x float> %a1) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float %a0, <4 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v8f32(float %a0, <8 x float> %a1) { define float @test_v8f32(float %a0, <8 x float> %a1) {
; SSE2-LABEL: test_v8f32: ; SSE2-LABEL: test_v8f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: addss %xmm1, %xmm0 ; SSE2-NEXT: addss %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm1, %xmm3 ; SSE2-NEXT: movaps %xmm1, %xmm3
▲ Show 20 Lines • Show All 69 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3]
; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float %a0, <8 x float> %a1) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float %a0, <8 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v16f32(float %a0, <16 x float> %a1) { define float @test_v16f32(float %a0, <16 x float> %a1) {
; SSE2-LABEL: test_v16f32: ; SSE2-LABEL: test_v16f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: addss %xmm1, %xmm0 ; SSE2-NEXT: addss %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm1, %xmm5 ; SSE2-NEXT: movaps %xmm1, %xmm5
▲ Show 20 Lines • Show All 134 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3]
; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3]
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v16f32(float %a0, <16 x float> %a1) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v16f32(float %a0, <16 x float> %a1)
ret float %1 ret float %1
} }
; ;
; vXf32 (zero) ; vXf32 (zero)
; ;
define float @test_v2f32_zero(<2 x float> %a0) { define float @test_v2f32_zero(<2 x float> %a0) {
Show All 23 Lines
; ;
; AVX512-LABEL: test_v2f32_zero: ; AVX512-LABEL: test_v2f32_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vxorps %xmm1, %xmm1, %xmm1 ; AVX512-NEXT: vxorps %xmm1, %xmm1, %xmm1
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm1 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm1
; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v2f32(float 0.0, <2 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float 0.0, <2 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v4f32_zero(<4 x float> %a0) { define float @test_v4f32_zero(<4 x float> %a0) {
; SSE2-LABEL: test_v4f32_zero: ; SSE2-LABEL: test_v4f32_zero:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: xorps %xmm1, %xmm1 ; SSE2-NEXT: xorps %xmm1, %xmm1
; SSE2-NEXT: addss %xmm0, %xmm1 ; SSE2-NEXT: addss %xmm0, %xmm1
Show All 38 Lines
; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm1 ; AVX512-NEXT: vaddss %xmm1, %xmm0, %xmm1
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v4f32(float 0.0, <4 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float 0.0, <4 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v8f32_zero(<8 x float> %a0) { define float @test_v8f32_zero(<8 x float> %a0) {
; SSE2-LABEL: test_v8f32_zero: ; SSE2-LABEL: test_v8f32_zero:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: xorps %xmm2, %xmm2 ; SSE2-NEXT: xorps %xmm2, %xmm2
; SSE2-NEXT: addss %xmm0, %xmm2 ; SSE2-NEXT: addss %xmm0, %xmm2
▲ Show 20 Lines • Show All 73 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float 0.0, <8 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float 0.0, <8 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v16f32_zero(<16 x float> %a0) { define float @test_v16f32_zero(<16 x float> %a0) {
; SSE2-LABEL: test_v16f32_zero: ; SSE2-LABEL: test_v16f32_zero:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: xorps %xmm4, %xmm4 ; SSE2-NEXT: xorps %xmm4, %xmm4
; SSE2-NEXT: addss %xmm0, %xmm4 ; SSE2-NEXT: addss %xmm0, %xmm4
▲ Show 20 Lines • Show All 138 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v16f32(float 0.0, <16 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v16f32(float 0.0, <16 x float> %a0)
ret float %1 ret float %1
} }
; ;
; vXf32 (undef) ; vXf32 (undef)
; ;
define float @test_v2f32_undef(<2 x float> %a0) { define float @test_v2f32_undef(<2 x float> %a0) {
Show All 15 Lines
; AVX-NEXT: vaddss {{.*}}(%rip), %xmm0, %xmm0 ; AVX-NEXT: vaddss {{.*}}(%rip), %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32_undef: ; AVX512-LABEL: test_v2f32_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss {{.*}}(%rip), %xmm0, %xmm0 ; AVX512-NEXT: vaddss {{.*}}(%rip), %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v2f32(float undef, <2 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float undef, <2 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v4f32_undef(<4 x float> %a0) { define float @test_v4f32_undef(<4 x float> %a0) {
; SSE2-LABEL: test_v4f32_undef: ; SSE2-LABEL: test_v4f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[2,3] ; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[2,3]
Show All 30 Lines
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss {{.*}}(%rip), %xmm1, %xmm1 ; AVX512-NEXT: vaddss {{.*}}(%rip), %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v4f32(float undef, <4 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float undef, <4 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v8f32_undef(<8 x float> %a0) { define float @test_v8f32_undef(<8 x float> %a0) {
; SSE2-LABEL: test_v8f32_undef: ; SSE2-LABEL: test_v8f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm2 ; SSE2-NEXT: movaps %xmm0, %xmm2
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[1,1],xmm0[2,3] ; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[1,1],xmm0[2,3]
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float undef, <8 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float undef, <8 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v16f32_undef(<16 x float> %a0) { define float @test_v16f32_undef(<16 x float> %a0) {
; SSE2-LABEL: test_v16f32_undef: ; SSE2-LABEL: test_v16f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm4 ; SSE2-NEXT: movaps %xmm0, %xmm4
; SSE2-NEXT: shufps {{.*#+}} xmm4 = xmm4[1,1],xmm0[2,3] ; SSE2-NEXT: shufps {{.*#+}} xmm4 = xmm4[1,1],xmm0[2,3]
▲ Show 20 Lines • Show All 130 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fadd.f32.f32.v16f32(float undef, <16 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fadd.f32.v16f32(float undef, <16 x float> %a0)
ret float %1 ret float %1
} }
; ;
; vXf64 (accum) ; vXf64 (accum)
; ;
define double @test_v2f64(double %a0, <2 x double> %a1) { define double @test_v2f64(double %a0, <2 x double> %a1) {
Show All 12 Lines
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64: ; AVX512-LABEL: test_v2f64:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v2f64(double %a0, <2 x double> %a1) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double %a0, <2 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v4f64(double %a0, <4 x double> %a1) { define double @test_v4f64(double %a0, <4 x double> %a1) {
; SSE-LABEL: test_v4f64: ; SSE-LABEL: test_v4f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addsd %xmm1, %xmm0 ; SSE-NEXT: addsd %xmm1, %xmm0
; SSE-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1,1]
Show All 21 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX512-NEXT: vaddsd %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm1
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double %a0, <4 x double> %a1) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double %a0, <4 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v8f64(double %a0, <8 x double> %a1) { define double @test_v8f64(double %a0, <8 x double> %a1) {
; SSE-LABEL: test_v8f64: ; SSE-LABEL: test_v8f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: addsd %xmm1, %xmm0 ; SSE-NEXT: addsd %xmm1, %xmm0
; SSE-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1,1]
▲ Show 20 Lines • Show All 42 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vaddsd %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1 ; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v8f64(double %a0, <8 x double> %a1) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v8f64(double %a0, <8 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v16f64(double %a0, <16 x double> %a1) { define double @test_v16f64(double %a0, <16 x double> %a1) {
; SSE-LABEL: test_v16f64: ; SSE-LABEL: test_v16f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: movapd {{[0-9]+}}(%rsp), %xmm8 ; SSE-NEXT: movapd {{[0-9]+}}(%rsp), %xmm8
; SSE-NEXT: addsd %xmm1, %xmm0 ; SSE-NEXT: addsd %xmm1, %xmm0
▲ Show 20 Lines • Show All 84 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vextractf32x4 $3, %zmm2, %xmm1 ; AVX512-NEXT: vextractf32x4 $3, %zmm2, %xmm1
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v16f64(double %a0, <16 x double> %a1) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v16f64(double %a0, <16 x double> %a1)
ret double %1 ret double %1
} }
; ;
; vXf64 (zero) ; vXf64 (zero)
; ;
define double @test_v2f64_zero(<2 x double> %a0) { define double @test_v2f64_zero(<2 x double> %a0) {
Show All 15 Lines
; ;
; AVX512-LABEL: test_v2f64_zero: ; AVX512-LABEL: test_v2f64_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vxorpd %xmm1, %xmm1, %xmm1 ; AVX512-NEXT: vxorpd %xmm1, %xmm1, %xmm1
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm1 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v2f64(double 0.0, <2 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double 0.0, <2 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v4f64_zero(<4 x double> %a0) { define double @test_v4f64_zero(<4 x double> %a0) {
; SSE-LABEL: test_v4f64_zero: ; SSE-LABEL: test_v4f64_zero:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: xorpd %xmm2, %xmm2 ; SSE-NEXT: xorpd %xmm2, %xmm2
; SSE-NEXT: addsd %xmm0, %xmm2 ; SSE-NEXT: addsd %xmm0, %xmm2
Show All 24 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddsd %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm0 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm1 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double 0.0, <4 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double 0.0, <4 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v8f64_zero(<8 x double> %a0) { define double @test_v8f64_zero(<8 x double> %a0) {
; SSE-LABEL: test_v8f64_zero: ; SSE-LABEL: test_v8f64_zero:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: xorpd %xmm4, %xmm4 ; SSE-NEXT: xorpd %xmm4, %xmm4
; SSE-NEXT: addsd %xmm0, %xmm4 ; SSE-NEXT: addsd %xmm0, %xmm4
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vaddsd %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddsd %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vextractf32x4 $3, %zmm0, %xmm0 ; AVX512-NEXT: vextractf32x4 $3, %zmm0, %xmm0
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm1 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v8f64(double 0.0, <8 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v8f64(double 0.0, <8 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v16f64_zero(<16 x double> %a0) { define double @test_v16f64_zero(<16 x double> %a0) {
; SSE-LABEL: test_v16f64_zero: ; SSE-LABEL: test_v16f64_zero:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: xorpd %xmm8, %xmm8 ; SSE-NEXT: xorpd %xmm8, %xmm8
; SSE-NEXT: addsd %xmm0, %xmm8 ; SSE-NEXT: addsd %xmm0, %xmm8
▲ Show 20 Lines • Show All 86 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vaddsd %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1 ; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v16f64(double 0.0, <16 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v16f64(double 0.0, <16 x double> %a0)
ret double %1 ret double %1
} }
; ;
; vXf64 (undef) ; vXf64 (undef)
; ;
define double @test_v2f64_undef(<2 x double> %a0) { define double @test_v2f64_undef(<2 x double> %a0) {
Show All 9 Lines
; AVX-NEXT: vaddsd {{.*}}(%rip), %xmm0, %xmm0 ; AVX-NEXT: vaddsd {{.*}}(%rip), %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64_undef: ; AVX512-LABEL: test_v2f64_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vaddsd {{.*}}(%rip), %xmm0, %xmm0 ; AVX512-NEXT: vaddsd {{.*}}(%rip), %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v2f64(double undef, <2 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double undef, <2 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v4f64_undef(<4 x double> %a0) { define double @test_v4f64_undef(<4 x double> %a0) {
; SSE-LABEL: test_v4f64_undef: ; SSE-LABEL: test_v4f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1]
; SSE-NEXT: addsd {{.*}}(%rip), %xmm0 ; SSE-NEXT: addsd {{.*}}(%rip), %xmm0
Show All 18 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vaddsd {{.*}}(%rip), %xmm1, %xmm1 ; AVX512-NEXT: vaddsd {{.*}}(%rip), %xmm1, %xmm1
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm0 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm1 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double undef, <4 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double undef, <4 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v8f64_undef(<8 x double> %a0) { define double @test_v8f64_undef(<8 x double> %a0) {
; SSE-LABEL: test_v8f64_undef: ; SSE-LABEL: test_v8f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1]
; SSE-NEXT: addsd {{.*}}(%rip), %xmm0 ; SSE-NEXT: addsd {{.*}}(%rip), %xmm0
Show All 39 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vaddsd %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vaddsd %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vextractf32x4 $3, %zmm0, %xmm0 ; AVX512-NEXT: vextractf32x4 $3, %zmm0, %xmm0
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm1 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v8f64(double undef, <8 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v8f64(double undef, <8 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v16f64_undef(<16 x double> %a0) { define double @test_v16f64_undef(<16 x double> %a0) {
; SSE-LABEL: test_v16f64_undef: ; SSE-LABEL: test_v16f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1]
; SSE-NEXT: addsd {{.*}}(%rip), %xmm0 ; SSE-NEXT: addsd {{.*}}(%rip), %xmm0
▲ Show 20 Lines • Show All 80 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vaddsd %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1 ; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fadd.f64.f64.v16f64(double undef, <16 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fadd.f64.v16f64(double undef, <16 x double> %a0)
ret double %1 ret double %1
} }
declare float @llvm.experimental.vector.reduce.fadd.f32.f32.v2f32(float, <2 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v2f32(float, <2 x float>)
declare float @llvm.experimental.vector.reduce.fadd.f32.f32.v4f32(float, <4 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v4f32(float, <4 x float>)
declare float @llvm.experimental.vector.reduce.fadd.f32.f32.v8f32(float, <8 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v8f32(float, <8 x float>)
declare float @llvm.experimental.vector.reduce.fadd.f32.f32.v16f32(float, <16 x float>) declare float @llvm.experimental.vector.reduce.fadd.f32.v16f32(float, <16 x float>)
declare double @llvm.experimental.vector.reduce.fadd.f64.f64.v2f64(double, <2 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v2f64(double, <2 x double>)
declare double @llvm.experimental.vector.reduce.fadd.f64.f64.v4f64(double, <4 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v4f64(double, <4 x double>)
declare double @llvm.experimental.vector.reduce.fadd.f64.f64.v8f64(double, <8 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v8f64(double, <8 x double>)
declare double @llvm.experimental.vector.reduce.fadd.f64.f64.v16f64(double, <16 x double>) declare double @llvm.experimental.vector.reduce.fadd.f64.v16f64(double, <16 x double>)

llvm/trunk/test/CodeGen/X86/vector-reduce-fmul-fast.ll

Show All 29 Lines
; AVX-NEXT: vmulss %xmm0, %xmm1, %xmm0 ; AVX-NEXT: vmulss %xmm0, %xmm1, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32: ; AVX512-LABEL: test_v2f32:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v2f32(float %a0, <2 x float> %a1) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v2f32(float %a0, <2 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v4f32(float %a0, <4 x float> %a1) { define float @test_v4f32(float %a0, <4 x float> %a1) {
; SSE2-LABEL: test_v4f32: ; SSE2-LABEL: test_v4f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm1, %xmm2 ; SSE2-NEXT: movaps %xmm1, %xmm2
; SSE2-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm1[1] ; SSE2-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm1[1]
Show All 22 Lines
; ;
; AVX512-LABEL: test_v4f32: ; AVX512-LABEL: test_v4f32:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm1[1,0]
; AVX512-NEXT: vmulps %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulps %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v4f32(float %a0, <4 x float> %a1) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v4f32(float %a0, <4 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v8f32(float %a0, <8 x float> %a1) { define float @test_v8f32(float %a0, <8 x float> %a1) {
; SSE2-LABEL: test_v8f32: ; SSE2-LABEL: test_v8f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: mulps %xmm2, %xmm1 ; SSE2-NEXT: mulps %xmm2, %xmm1
; SSE2-NEXT: movaps %xmm1, %xmm2 ; SSE2-NEXT: movaps %xmm1, %xmm2
Show All 30 Lines
; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm0 ; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm0
; AVX512-NEXT: vmulps %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulps %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v8f32(float %a0, <8 x float> %a1) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v8f32(float %a0, <8 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v16f32(float %a0, <16 x float> %a1) { define float @test_v16f32(float %a0, <16 x float> %a1) {
; SSE2-LABEL: test_v16f32: ; SSE2-LABEL: test_v16f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: mulps %xmm4, %xmm2 ; SSE2-NEXT: mulps %xmm4, %xmm2
; SSE2-NEXT: mulps %xmm3, %xmm1 ; SSE2-NEXT: mulps %xmm3, %xmm1
Show All 37 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v16f32(float %a0, <16 x float> %a1) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v16f32(float %a0, <16 x float> %a1)
ret float %1 ret float %1
} }
; ;
; vXf32 (one) ; vXf32 (one)
; ;
define float @test_v2f32_zero(<2 x float> %a0) { define float @test_v2f32_zero(<2 x float> %a0) {
Show All 17 Lines
; AVX-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32_zero: ; AVX512-LABEL: test_v2f32_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v2f32(float 1.0, <2 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v2f32(float 1.0, <2 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v4f32_zero(<4 x float> %a0) { define float @test_v4f32_zero(<4 x float> %a0) {
; SSE2-LABEL: test_v4f32_zero: ; SSE2-LABEL: test_v4f32_zero:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
; SSE2-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1] ; SSE2-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
Show All 23 Lines
; ;
; AVX512-LABEL: test_v4f32_zero: ; AVX512-LABEL: test_v4f32_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v4f32(float 1.0, <4 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v4f32(float 1.0, <4 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v8f32_zero(<8 x float> %a0) { define float @test_v8f32_zero(<8 x float> %a0) {
; SSE2-LABEL: test_v8f32_zero: ; SSE2-LABEL: test_v8f32_zero:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: mulps %xmm1, %xmm0 ; SSE2-NEXT: mulps %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
Show All 31 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v8f32(float 1.0, <8 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v8f32(float 1.0, <8 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v16f32_zero(<16 x float> %a0) { define float @test_v16f32_zero(<16 x float> %a0) {
; SSE2-LABEL: test_v16f32_zero: ; SSE2-LABEL: test_v16f32_zero:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: mulps %xmm3, %xmm1 ; SSE2-NEXT: mulps %xmm3, %xmm1
; SSE2-NEXT: mulps %xmm2, %xmm0 ; SSE2-NEXT: mulps %xmm2, %xmm0
Show All 38 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v16f32(float 1.0, <16 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v16f32(float 1.0, <16 x float> %a0)
ret float %1 ret float %1
} }
; ;
; vXf32 (undef) ; vXf32 (undef)
; ;
define float @test_v2f32_undef(<2 x float> %a0) { define float @test_v2f32_undef(<2 x float> %a0) {
Show All 17 Lines
; AVX-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32_undef: ; AVX512-LABEL: test_v2f32_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v2f32(float undef, <2 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v2f32(float undef, <2 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v4f32_undef(<4 x float> %a0) { define float @test_v4f32_undef(<4 x float> %a0) {
; SSE2-LABEL: test_v4f32_undef: ; SSE2-LABEL: test_v4f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
; SSE2-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1] ; SSE2-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
Show All 23 Lines
; ;
; AVX512-LABEL: test_v4f32_undef: ; AVX512-LABEL: test_v4f32_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v4f32(float undef, <4 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v4f32(float undef, <4 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v8f32_undef(<8 x float> %a0) { define float @test_v8f32_undef(<8 x float> %a0) {
; SSE2-LABEL: test_v8f32_undef: ; SSE2-LABEL: test_v8f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: mulps %xmm1, %xmm0 ; SSE2-NEXT: mulps %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
Show All 31 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v8f32(float undef, <8 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v8f32(float undef, <8 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v16f32_undef(<16 x float> %a0) { define float @test_v16f32_undef(<16 x float> %a0) {
; SSE2-LABEL: test_v16f32_undef: ; SSE2-LABEL: test_v16f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: mulps %xmm3, %xmm1 ; SSE2-NEXT: mulps %xmm3, %xmm1
; SSE2-NEXT: mulps %xmm2, %xmm0 ; SSE2-NEXT: mulps %xmm2, %xmm0
Show All 38 Lines
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulps %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.f32.v16f32(float undef, <16 x float> %a0) %1 = call fast float @llvm.experimental.vector.reduce.fmul.f32.v16f32(float undef, <16 x float> %a0)
ret float %1 ret float %1
} }
; ;
; vXf64 (accum) ; vXf64 (accum)
; ;
define double @test_v2f64(double %a0, <2 x double> %a1) { define double @test_v2f64(double %a0, <2 x double> %a1) {
Show All 10 Lines
; AVX-NEXT: vmulsd %xmm0, %xmm1, %xmm0 ; AVX-NEXT: vmulsd %xmm0, %xmm1, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64: ; AVX512-LABEL: test_v2f64:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm1[1,0]
; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v2f64(double %a0, <2 x double> %a1) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v2f64(double %a0, <2 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v4f64(double %a0, <4 x double> %a1) { define double @test_v4f64(double %a0, <4 x double> %a1) {
; SSE-LABEL: test_v4f64: ; SSE-LABEL: test_v4f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulpd %xmm2, %xmm1 ; SSE-NEXT: mulpd %xmm2, %xmm1
; SSE-NEXT: movapd %xmm1, %xmm0 ; SSE-NEXT: movapd %xmm1, %xmm0
Show All 13 Lines
; AVX512-LABEL: test_v4f64: ; AVX512-LABEL: test_v4f64:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm0 ; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm0
; AVX512-NEXT: vmulpd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulpd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v4f64(double %a0, <4 x double> %a1) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v4f64(double %a0, <4 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v8f64(double %a0, <8 x double> %a1) { define double @test_v8f64(double %a0, <8 x double> %a1) {
; SSE-LABEL: test_v8f64: ; SSE-LABEL: test_v8f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulpd %xmm4, %xmm2 ; SSE-NEXT: mulpd %xmm4, %xmm2
; SSE-NEXT: mulpd %xmm3, %xmm1 ; SSE-NEXT: mulpd %xmm3, %xmm1
Show All 18 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm1, %ymm0 ; AVX512-NEXT: vextractf64x4 $1, %zmm1, %ymm0
; AVX512-NEXT: vmulpd %zmm0, %zmm1, %zmm0 ; AVX512-NEXT: vmulpd %zmm0, %zmm1, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v8f64(double %a0, <8 x double> %a1) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v8f64(double %a0, <8 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v16f64(double %a0, <16 x double> %a1) { define double @test_v16f64(double %a0, <16 x double> %a1) {
; SSE-LABEL: test_v16f64: ; SSE-LABEL: test_v16f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulpd %xmm6, %xmm2 ; SSE-NEXT: mulpd %xmm6, %xmm2
; SSE-NEXT: mulpd %xmm7, %xmm3 ; SSE-NEXT: mulpd %xmm7, %xmm3
Show All 25 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v16f64(double %a0, <16 x double> %a1) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v16f64(double %a0, <16 x double> %a1)
ret double %1 ret double %1
} }
; ;
; vXf64 (one) ; vXf64 (one)
; ;
define double @test_v2f64_zero(<2 x double> %a0) { define double @test_v2f64_zero(<2 x double> %a0) {
Show All 11 Lines
; AVX-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64_zero: ; AVX512-LABEL: test_v2f64_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v2f64(double 1.0, <2 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v2f64(double 1.0, <2 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v4f64_zero(<4 x double> %a0) { define double @test_v4f64_zero(<4 x double> %a0) {
; SSE-LABEL: test_v4f64_zero: ; SSE-LABEL: test_v4f64_zero:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulpd %xmm1, %xmm0 ; SSE-NEXT: mulpd %xmm1, %xmm0
; SSE-NEXT: movapd %xmm0, %xmm1 ; SSE-NEXT: movapd %xmm0, %xmm1
Show All 14 Lines
; AVX512-LABEL: test_v4f64_zero: ; AVX512-LABEL: test_v4f64_zero:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v4f64(double 1.0, <4 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v4f64(double 1.0, <4 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v8f64_zero(<8 x double> %a0) { define double @test_v8f64_zero(<8 x double> %a0) {
; SSE-LABEL: test_v8f64_zero: ; SSE-LABEL: test_v8f64_zero:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulpd %xmm3, %xmm1 ; SSE-NEXT: mulpd %xmm3, %xmm1
; SSE-NEXT: mulpd %xmm2, %xmm0 ; SSE-NEXT: mulpd %xmm2, %xmm0
Show All 19 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v8f64(double 1.0, <8 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v8f64(double 1.0, <8 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v16f64_zero(<16 x double> %a0) { define double @test_v16f64_zero(<16 x double> %a0) {
; SSE-LABEL: test_v16f64_zero: ; SSE-LABEL: test_v16f64_zero:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulpd %xmm6, %xmm2 ; SSE-NEXT: mulpd %xmm6, %xmm2
; SSE-NEXT: mulpd %xmm4, %xmm0 ; SSE-NEXT: mulpd %xmm4, %xmm0
Show All 25 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v16f64(double 1.0, <16 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v16f64(double 1.0, <16 x double> %a0)
ret double %1 ret double %1
} }
; ;
; vXf64 (undef) ; vXf64 (undef)
; ;
define double @test_v2f64_undef(<2 x double> %a0) { define double @test_v2f64_undef(<2 x double> %a0) {
Show All 11 Lines
; AVX-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64_undef: ; AVX512-LABEL: test_v2f64_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v2f64(double undef, <2 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v2f64(double undef, <2 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v4f64_undef(<4 x double> %a0) { define double @test_v4f64_undef(<4 x double> %a0) {
; SSE-LABEL: test_v4f64_undef: ; SSE-LABEL: test_v4f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulpd %xmm1, %xmm0 ; SSE-NEXT: mulpd %xmm1, %xmm0
; SSE-NEXT: movapd %xmm0, %xmm1 ; SSE-NEXT: movapd %xmm0, %xmm1
Show All 14 Lines
; AVX512-LABEL: test_v4f64_undef: ; AVX512-LABEL: test_v4f64_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v4f64(double undef, <4 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v4f64(double undef, <4 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v8f64_undef(<8 x double> %a0) { define double @test_v8f64_undef(<8 x double> %a0) {
; SSE-LABEL: test_v8f64_undef: ; SSE-LABEL: test_v8f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulpd %xmm3, %xmm1 ; SSE-NEXT: mulpd %xmm3, %xmm1
; SSE-NEXT: mulpd %xmm2, %xmm0 ; SSE-NEXT: mulpd %xmm2, %xmm0
Show All 19 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v8f64(double undef, <8 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v8f64(double undef, <8 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v16f64_undef(<16 x double> %a0) { define double @test_v16f64_undef(<16 x double> %a0) {
; SSE-LABEL: test_v16f64_undef: ; SSE-LABEL: test_v16f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulpd %xmm6, %xmm2 ; SSE-NEXT: mulpd %xmm6, %xmm2
; SSE-NEXT: mulpd %xmm4, %xmm0 ; SSE-NEXT: mulpd %xmm4, %xmm0
Show All 25 Lines
; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1 ; AVX512-NEXT: vextractf64x4 $1, %zmm0, %ymm1
; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0 ; AVX512-NEXT: vmulpd %zmm1, %zmm0, %zmm0
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulpd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.f64.v16f64(double undef, <16 x double> %a0) %1 = call fast double @llvm.experimental.vector.reduce.fmul.f64.v16f64(double undef, <16 x double> %a0)
ret double %1 ret double %1
} }
declare float @llvm.experimental.vector.reduce.fmul.f32.f32.v2f32(float, <2 x float>) declare float @llvm.experimental.vector.reduce.fmul.f32.v2f32(float, <2 x float>)
declare float @llvm.experimental.vector.reduce.fmul.f32.f32.v4f32(float, <4 x float>) declare float @llvm.experimental.vector.reduce.fmul.f32.v4f32(float, <4 x float>)
declare float @llvm.experimental.vector.reduce.fmul.f32.f32.v8f32(float, <8 x float>) declare float @llvm.experimental.vector.reduce.fmul.f32.v8f32(float, <8 x float>)
declare float @llvm.experimental.vector.reduce.fmul.f32.f32.v16f32(float, <16 x float>) declare float @llvm.experimental.vector.reduce.fmul.f32.v16f32(float, <16 x float>)
declare double @llvm.experimental.vector.reduce.fmul.f64.f64.v2f64(double, <2 x double>) declare double @llvm.experimental.vector.reduce.fmul.f64.v2f64(double, <2 x double>)
declare double @llvm.experimental.vector.reduce.fmul.f64.f64.v4f64(double, <4 x double>) declare double @llvm.experimental.vector.reduce.fmul.f64.v4f64(double, <4 x double>)
declare double @llvm.experimental.vector.reduce.fmul.f64.f64.v8f64(double, <8 x double>) declare double @llvm.experimental.vector.reduce.fmul.f64.v8f64(double, <8 x double>)
declare double @llvm.experimental.vector.reduce.fmul.f64.f64.v16f64(double, <16 x double>) declare double @llvm.experimental.vector.reduce.fmul.f64.v16f64(double, <16 x double>)

llvm/trunk/test/CodeGen/X86/vector-reduce-fmul.ll

Show All 32 Lines
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32: ; AVX512-LABEL: test_v2f32:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm1[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v2f32(float %a0, <2 x float> %a1) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v2f32(float %a0, <2 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v4f32(float %a0, <4 x float> %a1) { define float @test_v4f32(float %a0, <4 x float> %a1) {
; SSE2-LABEL: test_v4f32: ; SSE2-LABEL: test_v4f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: mulss %xmm1, %xmm0 ; SSE2-NEXT: mulss %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm1, %xmm2 ; SSE2-NEXT: movaps %xmm1, %xmm2
Show All 34 Lines
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3]
; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v4f32(float %a0, <4 x float> %a1) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v4f32(float %a0, <4 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v8f32(float %a0, <8 x float> %a1) { define float @test_v8f32(float %a0, <8 x float> %a1) {
; SSE2-LABEL: test_v8f32: ; SSE2-LABEL: test_v8f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: mulss %xmm1, %xmm0 ; SSE2-NEXT: mulss %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm1, %xmm3 ; SSE2-NEXT: movaps %xmm1, %xmm3
▲ Show 20 Lines • Show All 69 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3]
; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v8f32(float %a0, <8 x float> %a1) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v8f32(float %a0, <8 x float> %a1)
ret float %1 ret float %1
} }
define float @test_v16f32(float %a0, <16 x float> %a1) { define float @test_v16f32(float %a0, <16 x float> %a1) {
; SSE2-LABEL: test_v16f32: ; SSE2-LABEL: test_v16f32:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: mulss %xmm1, %xmm0 ; SSE2-NEXT: mulss %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm1, %xmm5 ; SSE2-NEXT: movaps %xmm1, %xmm5
▲ Show 20 Lines • Show All 134 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3]
; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[3,1,2,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v16f32(float %a0, <16 x float> %a1) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v16f32(float %a0, <16 x float> %a1)
ret float %1 ret float %1
} }
; ;
; vXf32 (one) ; vXf32 (one)
; ;
define float @test_v2f32_one(<2 x float> %a0) { define float @test_v2f32_one(<2 x float> %a0) {
Show All 17 Lines
; AVX-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32_one: ; AVX512-LABEL: test_v2f32_one:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v2f32(float 1.0, <2 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v2f32(float 1.0, <2 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v4f32_one(<4 x float> %a0) { define float @test_v4f32_one(<4 x float> %a0) {
; SSE2-LABEL: test_v4f32_one: ; SSE2-LABEL: test_v4f32_one:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[2,3] ; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[2,3]
Show All 30 Lines
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm1 ; AVX512-NEXT: vmulss %xmm1, %xmm0, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v4f32(float 1.0, <4 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v4f32(float 1.0, <4 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v8f32_one(<8 x float> %a0) { define float @test_v8f32_one(<8 x float> %a0) {
; SSE2-LABEL: test_v8f32_one: ; SSE2-LABEL: test_v8f32_one:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm2 ; SSE2-NEXT: movaps %xmm0, %xmm2
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[1,1],xmm0[2,3] ; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[1,1],xmm0[2,3]
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v8f32(float 1.0, <8 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v8f32(float 1.0, <8 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v16f32_one(<16 x float> %a0) { define float @test_v16f32_one(<16 x float> %a0) {
; SSE2-LABEL: test_v16f32_one: ; SSE2-LABEL: test_v16f32_one:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm4 ; SSE2-NEXT: movaps %xmm0, %xmm4
; SSE2-NEXT: shufps {{.*#+}} xmm4 = xmm4[1,1],xmm0[2,3] ; SSE2-NEXT: shufps {{.*#+}} xmm4 = xmm4[1,1],xmm0[2,3]
▲ Show 20 Lines • Show All 130 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v16f32(float 1.0, <16 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v16f32(float 1.0, <16 x float> %a0)
ret float %1 ret float %1
} }
; ;
; vXf32 (undef) ; vXf32 (undef)
; ;
define float @test_v2f32_undef(<2 x float> %a0) { define float @test_v2f32_undef(<2 x float> %a0) {
Show All 15 Lines
; AVX-NEXT: vmulss {{.*}}(%rip), %xmm0, %xmm0 ; AVX-NEXT: vmulss {{.*}}(%rip), %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f32_undef: ; AVX512-LABEL: test_v2f32_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss {{.*}}(%rip), %xmm0, %xmm0 ; AVX512-NEXT: vmulss {{.*}}(%rip), %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v2f32(float undef, <2 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v2f32(float undef, <2 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v4f32_undef(<4 x float> %a0) { define float @test_v4f32_undef(<4 x float> %a0) {
; SSE2-LABEL: test_v4f32_undef: ; SSE2-LABEL: test_v4f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm1 ; SSE2-NEXT: movaps %xmm0, %xmm1
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[2,3] ; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[2,3]
Show All 30 Lines
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss {{.*}}(%rip), %xmm1, %xmm1 ; AVX512-NEXT: vmulss {{.*}}(%rip), %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v4f32(float undef, <4 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v4f32(float undef, <4 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v8f32_undef(<8 x float> %a0) { define float @test_v8f32_undef(<8 x float> %a0) {
; SSE2-LABEL: test_v8f32_undef: ; SSE2-LABEL: test_v8f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm2 ; SSE2-NEXT: movaps %xmm0, %xmm2
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[1,1],xmm0[2,3] ; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[1,1],xmm0[2,3]
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v8f32(float undef, <8 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v8f32(float undef, <8 x float> %a0)
ret float %1 ret float %1
} }
define float @test_v16f32_undef(<16 x float> %a0) { define float @test_v16f32_undef(<16 x float> %a0) {
; SSE2-LABEL: test_v16f32_undef: ; SSE2-LABEL: test_v16f32_undef:
; SSE2: # %bb.0: ; SSE2: # %bb.0:
; SSE2-NEXT: movaps %xmm0, %xmm4 ; SSE2-NEXT: movaps %xmm0, %xmm4
; SSE2-NEXT: shufps {{.*#+}} xmm4 = xmm4[1,1],xmm0[2,3] ; SSE2-NEXT: shufps {{.*#+}} xmm4 = xmm4[1,1],xmm0[2,3]
▲ Show 20 Lines • Show All 130 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3] ; AVX512-NEXT: vmovshdup {{.*#+}} xmm2 = xmm0[1,1,3,3]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm0[1,0]
; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulss %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3] ; AVX512-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulss %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call float @llvm.experimental.vector.reduce.fmul.f32.f32.v16f32(float undef, <16 x float> %a0) %1 = call float @llvm.experimental.vector.reduce.fmul.f32.v16f32(float undef, <16 x float> %a0)
ret float %1 ret float %1
} }
; ;
; vXf64 (accum) ; vXf64 (accum)
; ;
define double @test_v2f64(double %a0, <2 x double> %a1) { define double @test_v2f64(double %a0, <2 x double> %a1) {
Show All 12 Lines
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64: ; AVX512-LABEL: test_v2f64:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v2f64(double %a0, <2 x double> %a1) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v2f64(double %a0, <2 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v4f64(double %a0, <4 x double> %a1) { define double @test_v4f64(double %a0, <4 x double> %a1) {
; SSE-LABEL: test_v4f64: ; SSE-LABEL: test_v4f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulsd %xmm1, %xmm0 ; SSE-NEXT: mulsd %xmm1, %xmm0
; SSE-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1,1]
Show All 21 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX512-NEXT: vmulsd %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm1 ; AVX512-NEXT: vextractf128 $1, %ymm1, %xmm1
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v4f64(double %a0, <4 x double> %a1) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v4f64(double %a0, <4 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v8f64(double %a0, <8 x double> %a1) { define double @test_v8f64(double %a0, <8 x double> %a1) {
; SSE-LABEL: test_v8f64: ; SSE-LABEL: test_v8f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: mulsd %xmm1, %xmm0 ; SSE-NEXT: mulsd %xmm1, %xmm0
; SSE-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1,1]
▲ Show 20 Lines • Show All 42 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vmulsd %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1 ; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v8f64(double %a0, <8 x double> %a1) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v8f64(double %a0, <8 x double> %a1)
ret double %1 ret double %1
} }
define double @test_v16f64(double %a0, <16 x double> %a1) { define double @test_v16f64(double %a0, <16 x double> %a1) {
; SSE-LABEL: test_v16f64: ; SSE-LABEL: test_v16f64:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: movapd {{[0-9]+}}(%rsp), %xmm8 ; SSE-NEXT: movapd {{[0-9]+}}(%rsp), %xmm8
; SSE-NEXT: mulsd %xmm1, %xmm0 ; SSE-NEXT: mulsd %xmm1, %xmm0
▲ Show 20 Lines • Show All 84 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vextractf32x4 $3, %zmm2, %xmm1 ; AVX512-NEXT: vextractf32x4 $3, %zmm2, %xmm1
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v16f64(double %a0, <16 x double> %a1) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v16f64(double %a0, <16 x double> %a1)
ret double %1 ret double %1
} }
; ;
; vXf64 (one) ; vXf64 (one)
; ;
define double @test_v2f64_one(<2 x double> %a0) { define double @test_v2f64_one(<2 x double> %a0) {
Show All 11 Lines
; AVX-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64_one: ; AVX512-LABEL: test_v2f64_one:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v2f64(double 1.0, <2 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v2f64(double 1.0, <2 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v4f64_one(<4 x double> %a0) { define double @test_v4f64_one(<4 x double> %a0) {
; SSE-LABEL: test_v4f64_one: ; SSE-LABEL: test_v4f64_one:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: movapd %xmm0, %xmm2 ; SSE-NEXT: movapd %xmm0, %xmm2
; SSE-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm0[1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm0[1]
Show All 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm1 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm1
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm0 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm1 ; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v4f64(double 1.0, <4 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v4f64(double 1.0, <4 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v8f64_one(<8 x double> %a0) { define double @test_v8f64_one(<8 x double> %a0) {
; SSE-LABEL: test_v8f64_one: ; SSE-LABEL: test_v8f64_one:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: movapd %xmm0, %xmm4 ; SSE-NEXT: movapd %xmm0, %xmm4
; SSE-NEXT: unpckhpd {{.*#+}} xmm4 = xmm4[1],xmm0[1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm4 = xmm4[1],xmm0[1]
▲ Show 20 Lines • Show All 41 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vmulsd %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulsd %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vextractf32x4 $3, %zmm0, %xmm0 ; AVX512-NEXT: vextractf32x4 $3, %zmm0, %xmm0
; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm1 ; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v8f64(double 1.0, <8 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v8f64(double 1.0, <8 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v16f64_one(<16 x double> %a0) { define double @test_v16f64_one(<16 x double> %a0) {
; SSE-LABEL: test_v16f64_one: ; SSE-LABEL: test_v16f64_one:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: movapd %xmm0, %xmm8 ; SSE-NEXT: movapd %xmm0, %xmm8
; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1]
▲ Show 20 Lines • Show All 81 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vmulsd %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1 ; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v16f64(double 1.0, <16 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v16f64(double 1.0, <16 x double> %a0)
ret double %1 ret double %1
} }
; ;
; vXf64 (undef) ; vXf64 (undef)
; ;
define double @test_v2f64_undef(<2 x double> %a0) { define double @test_v2f64_undef(<2 x double> %a0) {
Show All 9 Lines
; AVX-NEXT: vmulsd {{.*}}(%rip), %xmm0, %xmm0 ; AVX-NEXT: vmulsd {{.*}}(%rip), %xmm0, %xmm0
; AVX-NEXT: retq ; AVX-NEXT: retq
; ;
; AVX512-LABEL: test_v2f64_undef: ; AVX512-LABEL: test_v2f64_undef:
; AVX512: # %bb.0: ; AVX512: # %bb.0:
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vmulsd {{.*}}(%rip), %xmm0, %xmm0 ; AVX512-NEXT: vmulsd {{.*}}(%rip), %xmm0, %xmm0
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v2f64(double undef, <2 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v2f64(double undef, <2 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v4f64_undef(<4 x double> %a0) { define double @test_v4f64_undef(<4 x double> %a0) {
; SSE-LABEL: test_v4f64_undef: ; SSE-LABEL: test_v4f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1]
; SSE-NEXT: mulsd {{.*}}(%rip), %xmm0 ; SSE-NEXT: mulsd {{.*}}(%rip), %xmm0
Show All 18 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX512-NEXT: vmulsd {{.*}}(%rip), %xmm1, %xmm1 ; AVX512-NEXT: vmulsd {{.*}}(%rip), %xmm1, %xmm1
; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm0 ; AVX512-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm1 ; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v4f64(double undef, <4 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v4f64(double undef, <4 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v8f64_undef(<8 x double> %a0) { define double @test_v8f64_undef(<8 x double> %a0) {
; SSE-LABEL: test_v8f64_undef: ; SSE-LABEL: test_v8f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1]
; SSE-NEXT: mulsd {{.*}}(%rip), %xmm0 ; SSE-NEXT: mulsd {{.*}}(%rip), %xmm0
Show All 39 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vmulsd %xmm2, %xmm1, %xmm1 ; AVX512-NEXT: vmulsd %xmm2, %xmm1, %xmm1
; AVX512-NEXT: vextractf32x4 $3, %zmm0, %xmm0 ; AVX512-NEXT: vextractf32x4 $3, %zmm0, %xmm0
; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm1 ; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm1
; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm0 = xmm0[1,0]
; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0 ; AVX512-NEXT: vmulsd %xmm0, %xmm1, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v8f64(double undef, <8 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v8f64(double undef, <8 x double> %a0)
ret double %1 ret double %1
} }
define double @test_v16f64_undef(<16 x double> %a0) { define double @test_v16f64_undef(<16 x double> %a0) {
; SSE-LABEL: test_v16f64_undef: ; SSE-LABEL: test_v16f64_undef:
; SSE: # %bb.0: ; SSE: # %bb.0:
; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1] ; SSE-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1,1]
; SSE-NEXT: mulsd {{.*}}(%rip), %xmm0 ; SSE-NEXT: mulsd {{.*}}(%rip), %xmm0
▲ Show 20 Lines • Show All 80 Lines • ▼ Show 20 Lines
; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm2 = xmm2[1,0]
; AVX512-NEXT: vmulsd %xmm2, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm2, %xmm0, %xmm0
; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1 ; AVX512-NEXT: vextractf32x4 $3, %zmm1, %xmm1
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0] ; AVX512-NEXT: vpermilpd {{.*#+}} xmm1 = xmm1[1,0]
; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0 ; AVX512-NEXT: vmulsd %xmm1, %xmm0, %xmm0
; AVX512-NEXT: vzeroupper ; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq ; AVX512-NEXT: retq
%1 = call double @llvm.experimental.vector.reduce.fmul.f64.f64.v16f64(double undef, <16 x double> %a0) %1 = call double @llvm.experimental.vector.reduce.fmul.f64.v16f64(double undef, <16 x double> %a0)
ret double %1 ret double %1
} }
declare float @llvm.experimental.vector.reduce.fmul.f32.f32.v2f32(float, <2 x float>) declare float @llvm.experimental.vector.reduce.fmul.f32.v2f32(float, <2 x float>)
declare float @llvm.experimental.vector.reduce.fmul.f32.f32.v4f32(float, <4 x float>) declare float @llvm.experimental.vector.reduce.fmul.f32.v4f32(float, <4 x float>)
declare float @llvm.experimental.vector.reduce.fmul.f32.f32.v8f32(float, <8 x float>) declare float @llvm.experimental.vector.reduce.fmul.f32.v8f32(float, <8 x float>)
declare float @llvm.experimental.vector.reduce.fmul.f32.f32.v16f32(float, <16 x float>) declare float @llvm.experimental.vector.reduce.fmul.f32.v16f32(float, <16 x float>)
declare double @llvm.experimental.vector.reduce.fmul.f64.f64.v2f64(double, <2 x double>) declare double @llvm.experimental.vector.reduce.fmul.f64.v2f64(double, <2 x double>)
declare double @llvm.experimental.vector.reduce.fmul.f64.f64.v4f64(double, <4 x double>) declare double @llvm.experimental.vector.reduce.fmul.f64.v4f64(double, <4 x double>)
declare double @llvm.experimental.vector.reduce.fmul.f64.f64.v8f64(double, <8 x double>) declare double @llvm.experimental.vector.reduce.fmul.f64.v8f64(double, <8 x double>)
declare double @llvm.experimental.vector.reduce.fmul.f64.f64.v16f64(double, <16 x double>) declare double @llvm.experimental.vector.reduce.fmul.f64.v16f64(double, <16 x double>)