This is an archive of the discontinued LLVM Phabricator instance.

Differential D45421

[X86] Emit native IR for pmuldq/pmuludq builtins.
ClosedPublic

Authored by craig.topper on Apr 8 2018, 6:28 PM.

Details

Reviewers
RKSimon
spatel
Commits
rG304edc1e75f7: [X86] Emit native IR for pmuldq/pmuludq builtins.
rC329605: [X86] Emit native IR for pmuldq/pmuludq builtins.
rL329605: [X86] Emit native IR for pmuldq/pmuludq builtins.
Summary

I believe all the pieces are now in place in the backend to make this correctly. We can truncate the vXi64 type to vXi32, extend it back up to the original width and multiply.

In the backend the truncate+extend will becomes sign_extend_inreg/zero_extend_inreg(really an and). Then those will be combined with the mul to PMULDQ/PMULUDQ. Then SimplifyDemandedBits will strip the sign_extend_inreg/zero_extend_inreg out.

The only question I have is whether its ok to emit the v2i32 intermediate type for the 128-bit version. I wasn't sure of any examples where we use an illegal type in our intrinsic/builtin handling. At least not a narrower type. I know pavg uses a wider type.

I think I could probably do this all in the header file using __builtin_convertvector if that's desired.

Diff Detail

Repository
rL LLVM

Event Timeline

craig.topper created this revision.Apr 8 2018, 6:28 PM
Harbormaster completed remote builds in B16870: Diff 141577.Apr 8 2018, 6:28 PM
Herald added a subscriber: cfe-commits. · View Herald TranscriptApr 8 2018, 6:28 PM
spatel added a comment.Apr 9 2018, 6:47 AM

The only question I have is whether its ok to emit the v2i32 intermediate type for the 128-bit version. I wasn't sure of any examples where we use an illegal type in our intrinsic/builtin handling. At least not a narrower type. I know pavg uses a wider type.

I don't know of any precedence at this level, but we created illegal scalar int types (i128/i256) as part of memcmp expansion knowing that we'd match and combine those specific patterns in the DAG for x86. I figured that as long as we take responsibility for handling the illegal types, it's ok to do that...nobody has complained so far. :)

RKSimon added a comment.Apr 9 2018, 9:49 AM

IIRC the SSE pmovsx/pmovzx generic implementations do this?

craig.topper added a comment.Apr 9 2018, 9:52 AM

Yes. @RKSimon is correct. pmovzx/pmovsx do use illegal types already.

spatel added inline comments.Apr 9 2018, 10:01 AM
test/CodeGen/sse2-builtins.c
7 ↗(On Diff #141577)

There should be matching codegen tests for the new IR patterns here or have they moved?

craig.topper added a comment.Apr 9 2018, 10:04 AM

Yes. I'll make the llvm changes before committing this. Just wanted to make sure this direction was ok first.

spatel added a comment.Apr 9 2018, 10:24 AM
In D45421#1061875, @craig.topper wrote:

Yes. I'll make the llvm changes before committing this. Just wanted to make sure this direction was ok first.

Ah, seems ok then. But instcombine is going to turn these casts into 'and' or 'shl+ashr', right? Shouldn't clang produce those patterns directly? More efficient and no need to toe the illegal type line.

craig.topper updated this revision to Diff 141691.Apr 9 2018, 11:10 AM

Use shifts or and to match what InstCombine will do. This sidesteps the illegal type question.

spatel accepted this revision.Apr 9 2018, 11:38 AM

LGTM (assuming the backend gets this right in all cases and we have tests for that).

This revision is now accepted and ready to land.Apr 9 2018, 11:38 AM
Closed by commit rL329605: [X86] Emit native IR for pmuldq/pmuludq builtins. (authored by ctopper). · Explain WhyApr 9 2018, 12:20 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathSize
cfe/
trunk/
lib/
CodeGen/
36 lines
test/
CodeGen/
10 lines
30 lines
40 lines
4 lines
6 lines

Diff 141707

cfe/trunk/lib/CodeGen/CGBuiltin.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 8,258 Lines • ▼ Show 20 Linesstatic Value *EmitX86MinMax(CodeGenFunction &CGF, ICmpInst::Predicate Pred,
if (Ops.size() == 2) if (Ops.size() == 2)
return Res; return Res;
assert(Ops.size() == 4); assert(Ops.size() == 4);
return EmitX86Select(CGF, Ops[3], Res, Ops[2]); return EmitX86Select(CGF, Ops[3], Res, Ops[2]);
} }
static Value *EmitX86Muldq(CodeGenFunction &CGF, bool IsSigned,
ArrayRef<Value *> Ops) {
llvm::Type *Ty = Ops[0]->getType();
// Arguments have a vXi32 type so cast to vXi64.
Ty = llvm::VectorType::get(CGF.Int64Ty,
Ty->getPrimitiveSizeInBits() / 64);
Value *LHS = CGF.Builder.CreateBitCast(Ops[0], Ty);
Value *RHS = CGF.Builder.CreateBitCast(Ops[1], Ty);
if (IsSigned) {
// Shift left then arithmetic shift right.
Constant *ShiftAmt = ConstantInt::get(Ty, 32);
LHS = CGF.Builder.CreateShl(LHS, ShiftAmt);
LHS = CGF.Builder.CreateAShr(LHS, ShiftAmt);
RHS = CGF.Builder.CreateShl(RHS, ShiftAmt);
RHS = CGF.Builder.CreateAShr(RHS, ShiftAmt);
} else {
// Clear the upper bits.
Constant *Mask = ConstantInt::get(Ty, 0xffffffff);
LHS = CGF.Builder.CreateAnd(LHS, Mask);
RHS = CGF.Builder.CreateAnd(RHS, Mask);
}
return CGF.Builder.CreateMul(LHS, RHS);
}
static Value *EmitX86SExtMask(CodeGenFunction &CGF, Value *Op, static Value *EmitX86SExtMask(CodeGenFunction &CGF, Value *Op,
llvm::Type *DstTy) { llvm::Type *DstTy) {
unsigned NumberOfElements = DstTy->getVectorNumElements(); unsigned NumberOfElements = DstTy->getVectorNumElements();
Value *Mask = getMaskVecValue(CGF, Op, NumberOfElements); Value *Mask = getMaskVecValue(CGF, Op, NumberOfElements);
return CGF.Builder.CreateSExt(Mask, DstTy, "vpmovm2"); return CGF.Builder.CreateSExt(Mask, DstTy, "vpmovm2");
} }
Value *CodeGenFunction::EmitX86CpuIs(const CallExpr *E) { Value *CodeGenFunction::EmitX86CpuIs(const CallExpr *E) {
▲ Show 20 Lines • Show All 688 Lines • ▼ Show 20 Lines#undef INTRINSIC_X86_XSAVE_ID
case X86::BI__builtin_ia32_pminud256: case X86::BI__builtin_ia32_pminud256:
case X86::BI__builtin_ia32_pminuq256_mask: case X86::BI__builtin_ia32_pminuq256_mask:
case X86::BI__builtin_ia32_pminub512_mask: case X86::BI__builtin_ia32_pminub512_mask:
case X86::BI__builtin_ia32_pminuw512_mask: case X86::BI__builtin_ia32_pminuw512_mask:
case X86::BI__builtin_ia32_pminud512_mask: case X86::BI__builtin_ia32_pminud512_mask:
case X86::BI__builtin_ia32_pminuq512_mask: case X86::BI__builtin_ia32_pminuq512_mask:
return EmitX86MinMax(*this, ICmpInst::ICMP_ULT, Ops); return EmitX86MinMax(*this, ICmpInst::ICMP_ULT, Ops);
case X86::BI__builtin_ia32_pmuludq128:
case X86::BI__builtin_ia32_pmuludq256:
case X86::BI__builtin_ia32_pmuludq512:
return EmitX86Muldq(*this, /*IsSigned*/false, Ops);
case X86::BI__builtin_ia32_pmuldq128:
case X86::BI__builtin_ia32_pmuldq256:
case X86::BI__builtin_ia32_pmuldq512:
return EmitX86Muldq(*this, /*IsSigned*/true, Ops);
// 3DNow! // 3DNow!
case X86::BI__builtin_ia32_pswapdsf: case X86::BI__builtin_ia32_pswapdsf:
case X86::BI__builtin_ia32_pswapdsi: { case X86::BI__builtin_ia32_pswapdsi: {
llvm::Type *MMXTy = llvm::Type::getX86_MMXTy(getLLVMContext()); llvm::Type *MMXTy = llvm::Type::getX86_MMXTy(getLLVMContext());
Ops[0] = Builder.CreateBitCast(Ops[0], MMXTy, "cast"); Ops[0] = Builder.CreateBitCast(Ops[0], MMXTy, "cast");
llvm::Function *F = CGM.getIntrinsic(Intrinsic::x86_3dnowa_pswapd); llvm::Function *F = CGM.getIntrinsic(Intrinsic::x86_3dnowa_pswapd);
return Builder.CreateCall(F, Ops, "pswapd"); return Builder.CreateCall(F, Ops, "pswapd");
} }
▲ Show 20 Lines • Show All 1,992 Lines • Show Last 20 Lines

cfe/trunk/test/CodeGen/avx2-builtins.c

Show First 20 Lines • Show All 829 Lines • ▼ Show 20 Lines
__m256i test_mm256_mpsadbw_epu8(__m256i x, __m256i y) { __m256i test_mm256_mpsadbw_epu8(__m256i x, __m256i y) {
// CHECK-LABEL: test_mm256_mpsadbw_epu8 // CHECK-LABEL: test_mm256_mpsadbw_epu8
// CHECK: call <16 x i16> @llvm.x86.avx2.mpsadbw(<32 x i8> %{{.*}}, <32 x i8> %{{.*}}, i8 3) // CHECK: call <16 x i16> @llvm.x86.avx2.mpsadbw(<32 x i8> %{{.*}}, <32 x i8> %{{.*}}, i8 3)
return _mm256_mpsadbw_epu8(x, y, 3); return _mm256_mpsadbw_epu8(x, y, 3);
} }
__m256i test_mm256_mul_epi32(__m256i a, __m256i b) { __m256i test_mm256_mul_epi32(__m256i a, __m256i b) {
// CHECK-LABEL: test_mm256_mul_epi32 // CHECK-LABEL: test_mm256_mul_epi32
// CHECK: call <4 x i64> @llvm.x86.avx2.pmul.dq(<8 x i32> %{{.*}}, <8 x i32> %{{.*}}) // CHECK: shl <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
// CHECK: ashr <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
// CHECK: shl <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
// CHECK: ashr <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
// CHECK: mul <4 x i64> %{{.*}}, %{{.*}}
return _mm256_mul_epi32(a, b); return _mm256_mul_epi32(a, b);
} }
__m256i test_mm256_mul_epu32(__m256i a, __m256i b) { __m256i test_mm256_mul_epu32(__m256i a, __m256i b) {
// CHECK-LABEL: test_mm256_mul_epu32 // CHECK-LABEL: test_mm256_mul_epu32
// CHECK: call <4 x i64> @llvm.x86.avx2.pmulu.dq(<8 x i32> %{{.*}}, <8 x i32> %{{.*}}) // CHECK: and <4 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
// CHECK: and <4 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
// CHECK: mul <4 x i64> %{{.*}}, %{{.*}}
return _mm256_mul_epu32(a, b); return _mm256_mul_epu32(a, b);
} }
__m256i test_mm256_mulhi_epu16(__m256i a, __m256i b) { __m256i test_mm256_mulhi_epu16(__m256i a, __m256i b) {
// CHECK-LABEL: test_mm256_mulhi_epu16 // CHECK-LABEL: test_mm256_mulhi_epu16
// CHECK: call <16 x i16> @llvm.x86.avx2.pmulhu.w(<16 x i16> %{{.*}}, <16 x i16> %{{.*}}) // CHECK: call <16 x i16> @llvm.x86.avx2.pmulhu.w(<16 x i16> %{{.*}}, <16 x i16> %{{.*}})
return _mm256_mulhi_epu16(a, b); return _mm256_mulhi_epu16(a, b);
} }
▲ Show 20 Lines • Show All 390 Lines • Show Last 20 Lines

cfe/trunk/test/CodeGen/avx512f-builtins.c

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 1,868 Lines • ▼ Show 20 Lines
__m512i test_mm512_add_epi64(__m512i __A, __m512i __B) { __m512i test_mm512_add_epi64(__m512i __A, __m512i __B) {
//CHECK-LABEL: @test_mm512_add_epi64 //CHECK-LABEL: @test_mm512_add_epi64
//CHECK: add <8 x i64> //CHECK: add <8 x i64>
return _mm512_add_epi64(__A,__B); return _mm512_add_epi64(__A,__B);
} }
__m512i test_mm512_mul_epi32(__m512i __A, __m512i __B) { __m512i test_mm512_mul_epi32(__m512i __A, __m512i __B) {
//CHECK-LABEL: @test_mm512_mul_epi32 //CHECK-LABEL: @test_mm512_mul_epi32
//CHECK: @llvm.x86.avx512.pmul.dq.512 //CHECK: shl <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: shl <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: mul <8 x i64> %{{.*}}, %{{.*}}
return _mm512_mul_epi32(__A,__B); return _mm512_mul_epi32(__A,__B);
} }
__m512i test_mm512_maskz_mul_epi32 (__mmask16 __k,__m512i __A, __m512i __B) { __m512i test_mm512_maskz_mul_epi32 (__mmask16 __k,__m512i __A, __m512i __B) {
//CHECK-LABEL: @test_mm512_maskz_mul_epi32 //CHECK-LABEL: @test_mm512_maskz_mul_epi32
//CHECK: @llvm.x86.avx512.pmul.dq.512 //CHECK: shl <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: shl <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: mul <8 x i64> %{{.*}}, %{{.*}}
//CHECK: select <8 x i1> %{{.*}}, <8 x i64> %{{.*}}, <8 x i64> %{{.*}} //CHECK: select <8 x i1> %{{.*}}, <8 x i64> %{{.*}}, <8 x i64> %{{.*}}
return _mm512_maskz_mul_epi32(__k,__A,__B); return _mm512_maskz_mul_epi32(__k,__A,__B);
} }
__m512i test_mm512_mask_mul_epi32 (__mmask16 __k,__m512i __A, __m512i __B, __m512i test_mm512_mask_mul_epi32 (__mmask16 __k,__m512i __A, __m512i __B,
__m512i __src) { __m512i __src) {
//CHECK-LABEL: @test_mm512_mask_mul_epi32 //CHECK-LABEL: @test_mm512_mask_mul_epi32
//CHECK: @llvm.x86.avx512.pmul.dq.512 //CHECK: shl <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: shl <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <8 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32, i64 32>
//CHECK: mul <8 x i64> %{{.*}}, %{{.*}}
//CHECK: select <8 x i1> %{{.*}}, <8 x i64> %{{.*}}, <8 x i64> %{{.*}} //CHECK: select <8 x i1> %{{.*}}, <8 x i64> %{{.*}}, <8 x i64> %{{.*}}
return _mm512_mask_mul_epi32(__src,__k,__A,__B); return _mm512_mask_mul_epi32(__src,__k,__A,__B);
} }
__m512i test_mm512_mul_epu32 (__m512i __A, __m512i __B) { __m512i test_mm512_mul_epu32 (__m512i __A, __m512i __B) {
//CHECK-LABEL: @test_mm512_mul_epu32 //CHECK-LABEL: @test_mm512_mul_epu32
//CHECK: @llvm.x86.avx512.pmulu.dq.512 //CHECK: and <8 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: and <8 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: mul <8 x i64> %{{.*}}, %{{.*}}
return _mm512_mul_epu32(__A,__B); return _mm512_mul_epu32(__A,__B);
} }
__m512i test_mm512_maskz_mul_epu32 (__mmask16 __k,__m512i __A, __m512i __B) { __m512i test_mm512_maskz_mul_epu32 (__mmask16 __k,__m512i __A, __m512i __B) {
//CHECK-LABEL: @test_mm512_maskz_mul_epu32 //CHECK-LABEL: @test_mm512_maskz_mul_epu32
//CHECK: @llvm.x86.avx512.pmulu.dq.512 //CHECK: and <8 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: and <8 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: mul <8 x i64> %{{.*}}, %{{.*}}
//CHECK: select <8 x i1> %{{.*}}, <8 x i64> %{{.*}}, <8 x i64> %{{.*}} //CHECK: select <8 x i1> %{{.*}}, <8 x i64> %{{.*}}, <8 x i64> %{{.*}}
return _mm512_maskz_mul_epu32(__k,__A,__B); return _mm512_maskz_mul_epu32(__k,__A,__B);
} }
__m512i test_mm512_mask_mul_epu32 (__mmask16 __k,__m512i __A, __m512i __B, __m512i test_mm512_mask_mul_epu32 (__mmask16 __k,__m512i __A, __m512i __B,
__m512i __src) { __m512i __src) {
//CHECK-LABEL: @test_mm512_mask_mul_epu32 //CHECK-LABEL: @test_mm512_mask_mul_epu32
//CHECK: @llvm.x86.avx512.pmulu.dq.512 //CHECK: and <8 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: and <8 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: mul <8 x i64> %{{.*}}, %{{.*}}
//CHECK: select <8 x i1> %{{.*}}, <8 x i64> %{{.*}}, <8 x i64> %{{.*}} //CHECK: select <8 x i1> %{{.*}}, <8 x i64> %{{.*}}, <8 x i64> %{{.*}}
return _mm512_mask_mul_epu32(__src,__k,__A,__B); return _mm512_mask_mul_epu32(__src,__k,__A,__B);
} }
__m512i test_mm512_maskz_mullo_epi32 (__mmask16 __k,__m512i __A, __m512i __B) { __m512i test_mm512_maskz_mullo_epi32 (__mmask16 __k,__m512i __A, __m512i __B) {
//CHECK-LABEL: @test_mm512_maskz_mullo_epi32 //CHECK-LABEL: @test_mm512_maskz_mullo_epi32
//CHECK: mul <16 x i32> %{{.*}}, %{{.*}} //CHECK: mul <16 x i32> %{{.*}}, %{{.*}}
//CHECK: select <16 x i1> %{{.*}}, <16 x i32> %{{.*}}, <16 x i32> %{{.*}} //CHECK: select <16 x i1> %{{.*}}, <16 x i32> %{{.*}}, <16 x i32> %{{.*}}
▲ Show 20 Lines • Show All 6,617 Lines • Show Last 20 Lines

cfe/trunk/test/CodeGen/avx512vl-builtins.c

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 721 Lines • ▼ Show 20 Lines__m128i test_mm_maskz_sub_epi64 (__mmask8 __U, __m128i __A, __m128i __B) {
//CHECK: sub <2 x i64> %{{.*}}, %{{.*}} //CHECK: sub <2 x i64> %{{.*}}, %{{.*}}
//CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}} //CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}}
return _mm_maskz_sub_epi64 (__U, __A, __B); return _mm_maskz_sub_epi64 (__U, __A, __B);
} }
__m256i test_mm256_mask_mul_epi32 (__m256i __W, __mmask8 __M, __m256i __X, __m256i test_mm256_mask_mul_epi32 (__m256i __W, __mmask8 __M, __m256i __X,
__m256i __Y) { __m256i __Y) {
//CHECK-LABEL: @test_mm256_mask_mul_epi32 //CHECK-LABEL: @test_mm256_mask_mul_epi32
//CHECK: @llvm.x86.avx2.pmul.dq //CHECK: shl <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
//CHECK: shl <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
//CHECK: mul <4 x i64> %{{.*}}, %{{.*}}
//CHECK: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}} //CHECK: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
return _mm256_mask_mul_epi32(__W, __M, __X, __Y); return _mm256_mask_mul_epi32(__W, __M, __X, __Y);
} }
__m256i test_mm256_maskz_mul_epi32 (__mmask8 __M, __m256i __X, __m256i __Y) { __m256i test_mm256_maskz_mul_epi32 (__mmask8 __M, __m256i __X, __m256i __Y) {
//CHECK-LABEL: @test_mm256_maskz_mul_epi32 //CHECK-LABEL: @test_mm256_maskz_mul_epi32
//CHECK: @llvm.x86.avx2.pmul.dq //CHECK: shl <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
//CHECK: shl <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
//CHECK: ashr <4 x i64> %{{.*}}, <i64 32, i64 32, i64 32, i64 32>
//CHECK: mul <4 x i64> %{{.*}}, %{{.*}}
//CHECK: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}} //CHECK: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
return _mm256_maskz_mul_epi32(__M, __X, __Y); return _mm256_maskz_mul_epi32(__M, __X, __Y);
} }
__m128i test_mm_mask_mul_epi32 (__m128i __W, __mmask8 __M, __m128i __X, __m128i test_mm_mask_mul_epi32 (__m128i __W, __mmask8 __M, __m128i __X,
__m128i __Y) { __m128i __Y) {
//CHECK-LABEL: @test_mm_mask_mul_epi32 //CHECK-LABEL: @test_mm_mask_mul_epi32
//CHECK: @llvm.x86.sse41.pmuldq //CHECK: shl <2 x i64> %{{.*}}, <i64 32, i64 32>
//CHECK: ashr <2 x i64> %{{.*}}, <i64 32, i64 32>
//CHECK: shl <2 x i64> %{{.*}}, <i64 32, i64 32>
//CHECK: ashr <2 x i64> %{{.*}}, <i64 32, i64 32>
//CHECK: mul <2 x i64> %{{.*}}, %{{.*}}
//CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}} //CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}}
return _mm_mask_mul_epi32(__W, __M, __X, __Y); return _mm_mask_mul_epi32(__W, __M, __X, __Y);
} }
__m128i test_mm_maskz_mul_epi32 (__mmask8 __M, __m128i __X, __m128i __Y) { __m128i test_mm_maskz_mul_epi32 (__mmask8 __M, __m128i __X, __m128i __Y) {
//CHECK-LABEL: @test_mm_maskz_mul_epi32 //CHECK-LABEL: @test_mm_maskz_mul_epi32
//CHECK: @llvm.x86.sse41.pmuldq //CHECK: shl <2 x i64> %{{.*}}, <i64 32, i64 32>
//CHECK: ashr <2 x i64> %{{.*}}, <i64 32, i64 32>
//CHECK: shl <2 x i64> %{{.*}}, <i64 32, i64 32>
//CHECK: ashr <2 x i64> %{{.*}}, <i64 32, i64 32>
//CHECK: mul <2 x i64> %{{.*}}, %{{.*}}
//CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}} //CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}}
return _mm_maskz_mul_epi32(__M, __X, __Y); return _mm_maskz_mul_epi32(__M, __X, __Y);
} }
__m256i test_mm256_mask_mul_epu32 (__m256i __W, __mmask8 __M, __m256i __X, __m256i test_mm256_mask_mul_epu32 (__m256i __W, __mmask8 __M, __m256i __X,
__m256i __Y) { __m256i __Y) {
//CHECK-LABEL: @test_mm256_mask_mul_epu32 //CHECK-LABEL: @test_mm256_mask_mul_epu32
//CHECK: @llvm.x86.avx2.pmulu.dq //CHECK: and <4 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: and <4 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: mul <4 x i64> %{{.*}}, %{{.*}}
//CHECK: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}} //CHECK: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
return _mm256_mask_mul_epu32(__W, __M, __X, __Y); return _mm256_mask_mul_epu32(__W, __M, __X, __Y);
} }
__m256i test_mm256_maskz_mul_epu32 (__mmask8 __M, __m256i __X, __m256i __Y) { __m256i test_mm256_maskz_mul_epu32 (__mmask8 __M, __m256i __X, __m256i __Y) {
//CHECK-LABEL: @test_mm256_maskz_mul_epu32 //CHECK-LABEL: @test_mm256_maskz_mul_epu32
//CHECK: @llvm.x86.avx2.pmulu.dq //CHECK: and <4 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: and <4 x i64> %{{.*}}, <i64 4294967295, i64 4294967295, i64 4294967295, i64 4294967295>
//CHECK: mul <4 x i64> %{{.*}}, %{{.*}}
//CHECK: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}} //CHECK: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
return _mm256_maskz_mul_epu32(__M, __X, __Y); return _mm256_maskz_mul_epu32(__M, __X, __Y);
} }
__m128i test_mm_mask_mul_epu32 (__m128i __W, __mmask8 __M, __m128i __X, __m128i test_mm_mask_mul_epu32 (__m128i __W, __mmask8 __M, __m128i __X,
__m128i __Y) { __m128i __Y) {
//CHECK-LABEL: @test_mm_mask_mul_epu32 //CHECK-LABEL: @test_mm_mask_mul_epu32
//CHECK: @llvm.x86.sse2.pmulu.dq //CHECK: and <2 x i64> %{{.*}}, <i64 4294967295, i64 4294967295>
//CHECK: and <2 x i64> %{{.*}}, <i64 4294967295, i64 4294967295>
//CHECK: mul <2 x i64> %{{.*}}, %{{.*}}
//CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}} //CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}}
return _mm_mask_mul_epu32(__W, __M, __X, __Y); return _mm_mask_mul_epu32(__W, __M, __X, __Y);
} }
__m128i test_mm_maskz_mul_epu32 (__mmask8 __M, __m128i __X, __m128i __Y) { __m128i test_mm_maskz_mul_epu32 (__mmask8 __M, __m128i __X, __m128i __Y) {
//CHECK-LABEL: @test_mm_maskz_mul_epu32 //CHECK-LABEL: @test_mm_maskz_mul_epu32
//CHECK: @llvm.x86.sse2.pmulu.dq //CHECK: and <2 x i64> %{{.*}}, <i64 4294967295, i64 4294967295>
//CHECK: and <2 x i64> %{{.*}}, <i64 4294967295, i64 4294967295>
//CHECK: mul <2 x i64> %{{.*}}, %{{.*}}
//CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}} //CHECK: select <2 x i1> %{{.*}}, <2 x i64> %{{.*}}, <2 x i64> %{{.*}}
return _mm_maskz_mul_epu32(__M, __X, __Y); return _mm_maskz_mul_epu32(__M, __X, __Y);
} }
__m128i test_mm_maskz_mullo_epi32 (__mmask8 __M, __m128i __A, __m128i __B) { __m128i test_mm_maskz_mullo_epi32 (__mmask8 __M, __m128i __A, __m128i __B) {
//CHECK-LABEL: @test_mm_maskz_mullo_epi32 //CHECK-LABEL: @test_mm_maskz_mullo_epi32
//CHECK: mul <4 x i32> %{{.*}}, %{{.*}} //CHECK: mul <4 x i32> %{{.*}}, %{{.*}}
//CHECK: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}} //CHECK: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}}
▲ Show 20 Lines • Show All 6,676 Lines • Show Last 20 Lines

cfe/trunk/test/CodeGen/sse2-builtins.c

Show First 20 Lines • Show All 810 Lines • ▼ Show 20 Lines
int test_mm_movemask_pd(__m128d A) { int test_mm_movemask_pd(__m128d A) {
// CHECK-LABEL: test_mm_movemask_pd // CHECK-LABEL: test_mm_movemask_pd
// CHECK: call i32 @llvm.x86.sse2.movmsk.pd(<2 x double> %{{.*}}) // CHECK: call i32 @llvm.x86.sse2.movmsk.pd(<2 x double> %{{.*}})
return _mm_movemask_pd(A); return _mm_movemask_pd(A);
} }
__m128i test_mm_mul_epu32(__m128i A, __m128i B) { __m128i test_mm_mul_epu32(__m128i A, __m128i B) {
// CHECK-LABEL: test_mm_mul_epu32 // CHECK-LABEL: test_mm_mul_epu32
// CHECK: call <2 x i64> @llvm.x86.sse2.pmulu.dq(<4 x i32> %{{.*}}, <4 x i32> %{{.*}}) // CHECK: and <2 x i64> %{{.*}}, <i64 4294967295, i64 4294967295>
// CHECK: and <2 x i64> %{{.*}}, <i64 4294967295, i64 4294967295>
// CHECK: mul <2 x i64> %{{.*}}, %{{.*}}
return _mm_mul_epu32(A, B); return _mm_mul_epu32(A, B);
} }
__m128d test_mm_mul_pd(__m128d A, __m128d B) { __m128d test_mm_mul_pd(__m128d A, __m128d B) {
// CHECK-LABEL: test_mm_mul_pd // CHECK-LABEL: test_mm_mul_pd
// CHECK: fmul <2 x double> %{{.*}}, %{{.*}} // CHECK: fmul <2 x double> %{{.*}}, %{{.*}}
return _mm_mul_pd(A, B); return _mm_mul_pd(A, B);
} }
▲ Show 20 Lines • Show All 730 Lines • Show Last 20 Lines

cfe/trunk/test/CodeGen/sse41-builtins.c

Show First 20 Lines • Show All 306 Lines • ▼ Show 20 Lines
__m128i test_mm_mpsadbw_epu8(__m128i x, __m128i y) { __m128i test_mm_mpsadbw_epu8(__m128i x, __m128i y) {
// CHECK-LABEL: test_mm_mpsadbw_epu8 // CHECK-LABEL: test_mm_mpsadbw_epu8
// CHECK: call <8 x i16> @llvm.x86.sse41.mpsadbw(<16 x i8> %{{.*}}, <16 x i8> %{{.*}}, i8 1) // CHECK: call <8 x i16> @llvm.x86.sse41.mpsadbw(<16 x i8> %{{.*}}, <16 x i8> %{{.*}}, i8 1)
return _mm_mpsadbw_epu8(x, y, 1); return _mm_mpsadbw_epu8(x, y, 1);
} }
__m128i test_mm_mul_epi32(__m128i x, __m128i y) { __m128i test_mm_mul_epi32(__m128i x, __m128i y) {
// CHECK-LABEL: test_mm_mul_epi32 // CHECK-LABEL: test_mm_mul_epi32
// CHECK: call <2 x i64> @llvm.x86.sse41.pmuldq(<4 x i32> %{{.*}}, <4 x i32> %{{.*}}) // CHECK: shl <2 x i64> %{{.*}}, <i64 32, i64 32>
// CHECK: ashr <2 x i64> %{{.*}}, <i64 32, i64 32>
// CHECK: shl <2 x i64> %{{.*}}, <i64 32, i64 32>
// CHECK: ashr <2 x i64> %{{.*}}, <i64 32, i64 32>
// CHECK: mul <2 x i64> %{{.*}}, %{{.*}}
return _mm_mul_epi32(x, y); return _mm_mul_epi32(x, y);
} }
__m128i test_mm_mullo_epi32(__m128i x, __m128i y) { __m128i test_mm_mullo_epi32(__m128i x, __m128i y) {
// CHECK-LABEL: test_mm_mullo_epi32 // CHECK-LABEL: test_mm_mullo_epi32
// CHECK: mul <4 x i32> // CHECK: mul <4 x i32>
return _mm_mullo_epi32(x, y); return _mm_mullo_epi32(x, y);
} }
▲ Show 20 Lines • Show All 72 Lines • Show Last 20 Lines