This is an archive of the discontinued LLVM Phabricator instance.

Differential D153034

[PowerPC][Future] Enable __builtin_mma_xxm[t|f]acc
ClosedPublic

Authored by kamaub on Jun 15 2023, 7:58 AM.

Details

Reviewers
stefanp
nemanjai
amyk
lei
Group Reviewers
Restricted Project
Commits
rG62c1cf7c63f0: [PowerPC][Future] Enable __builtin_mma_xxm[t|f]acc
Summary

Future cpu instructions dmxxinstdmr512 and dmxxextfdmr512 insert and extract
quad vectors from the new wide accumulator(wacc) register class.
The introduction of these new instructions renders the p10 instructions
xxmtacc and xxmfacc obsolete since the new wacc register class is a better
choice for handing quad vector operations. This patch ensures that, for
future cpu, instructions dmxxinstdmr512 and dmxxextfdmr512 are generated
by custom lowering the intrinsics for xxm[t|f]acc to produce no instructions.

Diff Detail

Event Timeline

kamaub created this revision.Jun 15 2023, 7:58 AM
Herald added a project: Restricted Project. · View Herald TranscriptJun 15 2023, 7:58 AM
Herald added subscribers: shchenz, kbarton, hiraditya. · View Herald Transcript
kamaub requested review of this revision.Jun 15 2023, 7:58 AM
Herald added a project: Restricted Project. · View Herald TranscriptJun 15 2023, 7:58 AM
Herald added a subscriber: llvm-commits. · View Herald Transcript
Harbormaster completed remote builds in B239137: Diff 531759.Jun 15 2023, 10:12 AM
kamaub updated this revision to Diff 532240.Jun 16 2023, 11:04 AM

Updating wording.

kamaub retitled this revision from [PowerPC][Future] Enable __builtin_mma_xxm[t|f]acc on PFuture to [PowerPC][Future] Enable __builtin_mma_xxm[t|f]acc on future isa.Jun 16 2023, 11:09 AM
kamaub edited the summary of this revision. (Show Details)
kamaub updated this revision to Diff 532247.Jun 16 2023, 11:19 AM

Spelling mistake

amyk added a comment.Jun 16 2023, 11:39 AM

Should it be "Future ISA" instead of "Future isa"?

llvm/lib/Target/PowerPC/PPCISelLowering.cpp
10751
10753

Avoiding the use of we here.

10754
10760

Just a question since I am curious, but is there any special reason on why 4 is picked for the SmallVector?

Harbormaster completed remote builds in B239492: Diff 532247.Jun 16 2023, 12:21 PM
stefanp added a comment.Jun 16 2023, 12:29 PM

The SelectionDAG object may have an easier way for you do to what you are trying to do here. I've added a comment. If you use the selection DAG my nit will probably just disappear.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp
10759

There is an easier way to do this.
Take a look at:

DAG.ReplaceAllUsesWith(From, To);
10762

nit:
Full sentence for comment please. :)

kamaub updated this revision to Diff 533944.Jun 23 2023, 6:44 AM

Addressing review comments.

kamaub retitled this revision from [PowerPC][Future] Enable __builtin_mma_xxm[t|f]acc on future isa to [PowerPC][Future] Enable __builtin_mma_xxm[t|f]acc.Jun 23 2023, 6:44 AM
kamaub edited the summary of this revision. (Show Details)
kamaub edited the summary of this revision. (Show Details)
kamaub marked 6 inline comments as done.Jun 23 2023, 6:50 AM
lei added a subscriber: lei.Jun 23 2023, 7:16 AM
lei added inline comments.
llvm/lib/Target/PowerPC/PPCISelLowering.cpp
10746–10756

I find it confusing to start talking about new instructions when the block starts with the p10 intrinsics. I would add this comment to after handling of non-future ISA code. So after the if stmt and simplify it a bit as well.

llvm/test/CodeGen/PowerPC/mmaplus-intrinsics.ll
152

I realized these -O0 run line was not added as part of this patch, but I am wondering why we need to be explicitly running -O0 for these intrinsic tests. Are they generated differently for -O0 vs opt?

Harbormaster completed remote builds in B240748: Diff 533944.Jun 23 2023, 7:20 AM
kamaub updated this revision to Diff 534006.Jun 23 2023, 10:26 AM

Addressing review comments.

kamaub edited the summary of this revision. (Show Details)Jun 23 2023, 10:27 AM
Harbormaster completed remote builds in B240798: Diff 534006.Jun 23 2023, 11:53 AM
lei added inline comments.Jun 29 2023, 6:20 PM
llvm/lib/Target/PowerPC/PPCISelLowering.cpp
10746–10747

nit:

kamaub added inline comments.Jun 30 2023, 2:29 PM
llvm/test/CodeGen/PowerPC/mmaplus-intrinsics.ll
152

I think the reason for adding O0 test cases was to make sure that FastISEL worked in these tested situations. For this patch specifically the xxm[t|f]acc intrinsics are not removed differently at O0, but having the check to make sure the the quad vector type is lowered to the expected dmr moves as a result of the intrinsic removal makes sense to me.

Even then, the amount of new O0 checks just for this change could be considered overkill.

Here I am adding a few new test cases but also moving some from some from mma-instrinsics.ll so i could move those new test to their own file where I do not need to test fastISEL, but I did not see a strong reasoning to do so at the time.

kamaub updated this revision to Diff 536468.Jun 30 2023, 3:45 PM
kamaub marked an inline comment as done.

Addressing a review comment.

kamaub marked an inline comment as done.Jun 30 2023, 3:45 PM
Harbormaster completed remote builds in B242585: Diff 536468.Jun 30 2023, 5:44 PM
lei accepted this revision as: lei.Jul 12 2023, 1:21 PM

LGTM
Thx

This revision is now accepted and ready to land.Jul 12 2023, 1:21 PM
amyk accepted this revision as: amyk.Jul 14 2023, 8:41 AM

Also LGTM, thanks for answering my questions/addressing comments.

Closed by commit rG62c1cf7c63f0: [PowerPC][Future] Enable __builtin_mma_xxm[t|f]acc (authored by kamaub). · Explain WhyJul 14 2023, 11:38 AM
This revision was automatically updated to reflect the committed changes.
kamaub added a commit: rG62c1cf7c63f0: [PowerPC][Future] Enable __builtin_mma_xxm[t|f]acc.

Revision Contents

PathSize
llvm/
lib/
Target/
PowerPC/
14 lines
test/
CodeGen/
PowerPC/
801 lines

Diff 540517

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 10,735 Lines • ▼ Show 20 Lines for (int VecNo = 0; VecNo < NumVecs; VecNo++) {
DAG.getConstant(Subtarget.isLittleEndian() ? NumVecs - 1 - VecNo DAG.getConstant(Subtarget.isLittleEndian() ? NumVecs - 1 - VecNo
: VecNo, : VecNo,
dl, getPointerTy(DAG.getDataLayout()))); dl, getPointerTy(DAG.getDataLayout())));
RetOps.push_back(Extract); RetOps.push_back(Extract);
} }
return DAG.getMergeValues(RetOps, dl); return DAG.getMergeValues(RetOps, dl);
} }
case Intrinsic::ppc_mma_xxmfacc:
case Intrinsic::ppc_mma_xxmtacc: {
// Allow pre-isa-future subtargets to lower as normal.
if (!Subtarget.isISAFuture())
leiUnsubmitted
Done
ReplyInline Actions
case Intrinsic::ppc_mma_xxmtacc: {
+ // Allow pre-isa-future subtargets to lower as normal.
if (!Subtarget.isISAFuture())
- return SDValue(); // Allow pre-isa-future subtargets to lower as normal.
+ return SDValue();
+
// The intrinsics for xxmtacc and xxmfacc take one argument of

nit:

lei: nit:
return SDValue();
// The intrinsics for xxmtacc and xxmfacc take one argument of
// type v512i1, for future cpu the corresponding wacc instruction
// dmxx[inst|extf]dmr512 is always generated for type v512i1, negating
amykUnsubmitted
Done
ReplyInline Actions
// the v512i1 (vector_quad in C) type. Since these two intrinsics take one
- // argument of type v512i1 a corresponding dmxxinstdmr512 or dmxxextfdmr512
+ // argument of type v512i1, a corresponding dmxxinstdmr512 or dmxxextfdmr512
// is always generated for the operand negating the need to produce the
amyk:
// the need to produce the xxm[t|f]acc.
SDValue WideVec = Op.getOperand(1);
amykUnsubmitted
Done
ReplyInline Actions
// is always generated for the operand negating the need to produce the
- // xxm[t|f]acc. Here we account for this by custom lowering these
+ // xxm[t|f]acc. This situation is accounted for by custom lowering these
// instrinsics to produce no instructions on future isa.

Avoiding the use of we here.

amyk: Avoiding the use of `we` here.
DAG.ReplaceAllUsesWith(Op, WideVec);
amykUnsubmitted
Done
ReplyInline Actions
// xxm[t|f]acc. Here we account for this by custom lowering these
- // instrinsics to produce no instructions on PFuture.
+ // intrinsics to produce no instructions on PFuture.
if (!Subtarget.isISAFuture())
amyk:
return SDValue();
}
leiUnsubmitted
Done
ReplyInline Actions
case Intrinsic::ppc_mma_xxmtacc: {
- // Future cpu instructions dmxxinstdmr512 and dmxxextfdmr512 insert and
- // extract quad vectors(v512i1) from the new wide accumulator(wacc)
- // register class. The introduction of these new instructions renders the
- // p10 instructions xxmtacc and xxmfacc obsolete since the new wacc
- // register class is a better choice for handling quad vector operations.
- // The intrinsics for xxmtacc and xxmfacc take one argument of
- // type v512i1, a corresponding dmxx[inst|extf]dmr512 instruction
- // is always generated for the operand negating the need to produce the
- // xxm[t|f]acc. This is accounted for here for future cpu, instructions
- // dmxxinstdmr512 and dmxxextfdmr512 are generated by custom lowering
- // intrinsics xxmtacc and xxmfacc to produce no instructions.
+ // For future cpu, insert and extract quad vector instructions dmxxinstdmr512
+ // and dmxxextfdmr512 that uses the new wide accumulator (wacc) register class
+ // is a better choice for handling quad vector operations.
if (!Subtarget.isISAFuture())

I find it confusing to start talking about new instructions when the block starts with the p10 intrinsics. I would add this comment to after handling of non-future ISA code. So after the if stmt and simplify it a bit as well.

lei: I find it confusing to start talking about new instructions when the block starts with the p10…
case Intrinsic::ppc_unpack_longdouble: { case Intrinsic::ppc_unpack_longdouble: {
auto *Idx = dyn_cast<ConstantSDNode>(Op.getOperand(2)); auto *Idx = dyn_cast<ConstantSDNode>(Op.getOperand(2));
stefanpUnsubmitted
Done
ReplyInline Actions

There is an easier way to do this.
Take a look at:

DAG.ReplaceAllUsesWith(From, To);
stefanp: There is an easier way to do this. Take a look at: ``` DAG.ReplaceAllUsesWith(From, To); ```
assert(Idx && (Idx->getSExtValue() == 0 || Idx->getSExtValue() == 1) && assert(Idx && (Idx->getSExtValue() == 0 || Idx->getSExtValue() == 1) &&
amykUnsubmitted
Done
ReplyInline Actions

Just a question since I am curious, but is there any special reason on why 4 is picked for the SmallVector?

amyk: Just a question since I am curious, but is there any special reason on why `4` is picked for…
"Argument of long double unpack must be 0 or 1!"); "Argument of long double unpack must be 0 or 1!");
return DAG.getNode(ISD::EXTRACT_ELEMENT, dl, MVT::f64, Op.getOperand(1), return DAG.getNode(ISD::EXTRACT_ELEMENT, dl, MVT::f64, Op.getOperand(1),
stefanpUnsubmitted
Done
ReplyInline Actions

nit:
Full sentence for comment please. :)

stefanp: nit: Full sentence for comment please. :)
DAG.getConstant(!!(Idx->getSExtValue()), dl, DAG.getConstant(!!(Idx->getSExtValue()), dl,
Idx->getValueType(0))); Idx->getValueType(0)));
} }
case Intrinsic::ppc_compare_exp_lt: case Intrinsic::ppc_compare_exp_lt:
case Intrinsic::ppc_compare_exp_gt: case Intrinsic::ppc_compare_exp_gt:
case Intrinsic::ppc_compare_exp_eq: case Intrinsic::ppc_compare_exp_eq:
case Intrinsic::ppc_compare_exp_uo: { case Intrinsic::ppc_compare_exp_uo: {
▲ Show 20 Lines • Show All 7,818 Lines • Show Last 20 Lines

llvm/test/CodeGen/PowerPC/mmaplus-intrinsics.ll

Show First 20 Lines • Show All 105 Lines • ▼ Show 20 Lines
; CHECK-AIX32-NEXT: stxv 2, 0(3) ; CHECK-AIX32-NEXT: stxv 2, 0(3)
; CHECK-AIX32-NEXT: blr ; CHECK-AIX32-NEXT: blr
entry: entry:
%0 = tail call <512 x i1> @llvm.ppc.mma.assemble.acc(<16 x i8> %vc, <16 x i8> %vc, <16 x i8> %vc, <16 x i8> %vc) %0 = tail call <512 x i1> @llvm.ppc.mma.assemble.acc(<16 x i8> %vc, <16 x i8> %vc, <16 x i8> %vc, <16 x i8> %vc)
store <512 x i1> %0, ptr %ptr, align 64 store <512 x i1> %0, ptr %ptr, align 64
ret void ret void
} }
; xxmtacc with a loaded and stored vector quad.
define void @ld_st_xxmtacc(ptr %vqp, ptr %vpp, <16 x i8> %vc, ptr %resp) {
; CHECK-LABEL: ld_st_xxmtacc:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: lxv v3, 0(r3)
; CHECK-NEXT: lxv v5, 32(r3)
; CHECK-NEXT: lxv v2, 16(r3)
; CHECK-NEXT: lxv v4, 48(r3)
; CHECK-NEXT: dmxxinstfdmr512 wacc0, vsp36, vsp34, 0
; CHECK-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-NEXT: stxv v4, 48(r7)
; CHECK-NEXT: stxv v5, 32(r7)
; CHECK-NEXT: stxv v2, 16(r7)
; CHECK-NEXT: stxv v3, 0(r7)
; CHECK-NEXT: blr
;
; CHECK-BE-LABEL: ld_st_xxmtacc:
; CHECK-BE: # %bb.0: # %entry
; CHECK-BE-NEXT: lxv v3, 48(r3)
; CHECK-BE-NEXT: lxv v5, 16(r3)
; CHECK-BE-NEXT: lxv v2, 32(r3)
; CHECK-BE-NEXT: lxv v4, 0(r3)
; CHECK-BE-NEXT: dmxxinstfdmr512 wacc0, vsp36, vsp34, 0
; CHECK-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-BE-NEXT: stxv v5, 48(r7)
; CHECK-BE-NEXT: stxv v4, 32(r7)
; CHECK-BE-NEXT: stxv v3, 16(r7)
; CHECK-BE-NEXT: stxv v2, 0(r7)
; CHECK-BE-NEXT: blr
;
; CHECK-O0-LABEL: ld_st_xxmtacc:
; CHECK-O0: # %bb.0: # %entry
; CHECK-O0-NEXT: lxv vs0, 0(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp18
; CHECK-O0-NEXT: xxlor v5, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 16(r3)
; CHECK-O0-NEXT: xxlor v4, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 32(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp17
leiUnsubmitted
Not Done
ReplyInline Actions

I realized these -O0 run line was not added as part of this patch, but I am wondering why we need to be explicitly running -O0 for these intrinsic tests. Are they generated differently for -O0 vs opt?

lei: I realized these -O0 run line was not added as part of this patch, but I am wondering why we…
kamaubAuthorUnsubmitted
Done
ReplyInline Actions

I think the reason for adding O0 test cases was to make sure that FastISEL worked in these tested situations. For this patch specifically the xxm[t|f]acc intrinsics are not removed differently at O0, but having the check to make sure the the quad vector type is lowered to the expected dmr moves as a result of the intrinsic removal makes sense to me.

Even then, the amount of new O0 checks just for this change could be considered overkill.

Here I am adding a few new test cases but also moving some from some from mma-instrinsics.ll so i could move those new test to their own file where I do not need to test fastISEL, but I did not see a strong reasoning to do so at the time.

kamaub: I think the reason for adding O0 test cases was to make sure that FastISEL worked in these…
; CHECK-O0-NEXT: xxlor v3, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 48(r3)
; CHECK-O0-NEXT: xxlor v2, vs0, vs0
; CHECK-O0-NEXT: dmxxinstfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-NEXT: xxlor vs0, v4, v4
; CHECK-O0-NEXT: stxv vs0, 48(r7)
; CHECK-O0-NEXT: xxlor vs0, v5, v5
; CHECK-O0-NEXT: stxv vs0, 32(r7)
; CHECK-O0-NEXT: xxlor vs0, v2, v2
; CHECK-O0-NEXT: stxv vs0, 16(r7)
; CHECK-O0-NEXT: xxlor vs0, v3, v3
; CHECK-O0-NEXT: stxv vs0, 0(r7)
; CHECK-O0-NEXT: blr
;
; CHECK-O0-BE-LABEL: ld_st_xxmtacc:
; CHECK-O0-BE: # %bb.0: # %entry
; CHECK-O0-BE-NEXT: lxv vs0, 48(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp18
; CHECK-O0-BE-NEXT: xxlor v5, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 32(r3)
; CHECK-O0-BE-NEXT: xxlor v4, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 16(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp17
; CHECK-O0-BE-NEXT: xxlor v3, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 0(r3)
; CHECK-O0-BE-NEXT: xxlor v2, vs0, vs0
; CHECK-O0-BE-NEXT: dmxxinstfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-BE-NEXT: xxlor vs0, v5, v5
; CHECK-O0-BE-NEXT: stxv vs0, 48(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v4, v4
; CHECK-O0-BE-NEXT: stxv vs0, 32(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v3, v3
; CHECK-O0-BE-NEXT: stxv vs0, 16(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v2, v2
; CHECK-O0-BE-NEXT: stxv vs0, 0(r7)
; CHECK-O0-BE-NEXT: blr
;
; CHECK-AIX64-LABEL: ld_st_xxmtacc:
; CHECK-AIX64: # %bb.0: # %entry
; CHECK-AIX64-NEXT: lxv 3, 48(3)
; CHECK-AIX64-NEXT: lxv 5, 16(3)
; CHECK-AIX64-NEXT: lxv 2, 32(3)
; CHECK-AIX64-NEXT: lxv 4, 0(3)
; CHECK-AIX64-NEXT: dmxxinstfdmr512 0, 36, 34, 0
; CHECK-AIX64-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX64-NEXT: stxv 5, 48(5)
; CHECK-AIX64-NEXT: stxv 4, 32(5)
; CHECK-AIX64-NEXT: stxv 3, 16(5)
; CHECK-AIX64-NEXT: stxv 2, 0(5)
; CHECK-AIX64-NEXT: blr
;
; CHECK-AIX32-LABEL: ld_st_xxmtacc:
; CHECK-AIX32: # %bb.0: # %entry
; CHECK-AIX32-NEXT: lxv 3, 48(3)
; CHECK-AIX32-NEXT: lxv 5, 16(3)
; CHECK-AIX32-NEXT: lxv 2, 32(3)
; CHECK-AIX32-NEXT: lxv 4, 0(3)
; CHECK-AIX32-NEXT: dmxxinstfdmr512 0, 36, 34, 0
; CHECK-AIX32-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX32-NEXT: stxv 5, 48(5)
; CHECK-AIX32-NEXT: stxv 4, 32(5)
; CHECK-AIX32-NEXT: stxv 3, 16(5)
; CHECK-AIX32-NEXT: stxv 2, 0(5)
; CHECK-AIX32-NEXT: blr
entry:
%0 = load <512 x i1>, ptr %vqp, align 64
%1 = tail call <512 x i1> @llvm.ppc.mma.xxmtacc(<512 x i1> %0)
store <512 x i1> %1, ptr %resp, align 64
ret void
}
declare <512 x i1> @llvm.ppc.mma.xxmtacc(<512 x i1>)
; xxmtacc used with an instruction that is not a load or store.
define void @ld_op_st_xxmtacc(ptr %vqp, ptr %vpp, <16 x i8> %vc, ptr %resp) {
; CHECK-LABEL: ld_op_st_xxmtacc:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: lxv v5, 0(r3)
; CHECK-NEXT: lxv v1, 32(r3)
; CHECK-NEXT: lxv v4, 16(r3)
; CHECK-NEXT: lxv v0, 48(r3)
; CHECK-NEXT: dmxxinstfdmr512 wacc0, vsp32, vsp36, 0
; CHECK-NEXT: xvi4ger8pp wacc0, v2, v2
; CHECK-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-NEXT: stxv v4, 48(r7)
; CHECK-NEXT: stxv v5, 32(r7)
; CHECK-NEXT: stxv v2, 16(r7)
; CHECK-NEXT: stxv v3, 0(r7)
; CHECK-NEXT: blr
;
; CHECK-BE-LABEL: ld_op_st_xxmtacc:
; CHECK-BE: # %bb.0: # %entry
; CHECK-BE-NEXT: lxv v5, 48(r3)
; CHECK-BE-NEXT: lxv v1, 16(r3)
; CHECK-BE-NEXT: lxv v4, 32(r3)
; CHECK-BE-NEXT: lxv v0, 0(r3)
; CHECK-BE-NEXT: dmxxinstfdmr512 wacc0, vsp32, vsp36, 0
; CHECK-BE-NEXT: xvi4ger8pp wacc0, v2, v2
; CHECK-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-BE-NEXT: stxv v5, 48(r7)
; CHECK-BE-NEXT: stxv v4, 32(r7)
; CHECK-BE-NEXT: stxv v3, 16(r7)
; CHECK-BE-NEXT: stxv v2, 0(r7)
; CHECK-BE-NEXT: blr
;
; CHECK-O0-LABEL: ld_op_st_xxmtacc:
; CHECK-O0: # %bb.0: # %entry
; CHECK-O0-NEXT: lxv vs0, 0(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp16
; CHECK-O0-NEXT: xxlor v1, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 16(r3)
; CHECK-O0-NEXT: xxlor v0, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 32(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp18
; CHECK-O0-NEXT: xxlor v5, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 48(r3)
; CHECK-O0-NEXT: xxlor v4, vs0, vs0
; CHECK-O0-NEXT: dmxxinstfdmr512 wacc0, vsp36, vsp32, 0
; CHECK-O0-NEXT: xvi4ger8pp wacc0, v2, v2
; CHECK-O0-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-NEXT: xxlor vs0, v4, v4
; CHECK-O0-NEXT: stxv vs0, 48(r7)
; CHECK-O0-NEXT: xxlor vs0, v5, v5
; CHECK-O0-NEXT: stxv vs0, 32(r7)
; CHECK-O0-NEXT: xxlor vs0, v2, v2
; CHECK-O0-NEXT: stxv vs0, 16(r7)
; CHECK-O0-NEXT: xxlor vs0, v3, v3
; CHECK-O0-NEXT: stxv vs0, 0(r7)
; CHECK-O0-NEXT: blr
;
; CHECK-O0-BE-LABEL: ld_op_st_xxmtacc:
; CHECK-O0-BE: # %bb.0: # %entry
; CHECK-O0-BE-NEXT: lxv vs0, 48(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp16
; CHECK-O0-BE-NEXT: xxlor v1, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 32(r3)
; CHECK-O0-BE-NEXT: xxlor v0, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 16(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp18
; CHECK-O0-BE-NEXT: xxlor v5, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 0(r3)
; CHECK-O0-BE-NEXT: xxlor v4, vs0, vs0
; CHECK-O0-BE-NEXT: dmxxinstfdmr512 wacc0, vsp36, vsp32, 0
; CHECK-O0-BE-NEXT: xvi4ger8pp wacc0, v2, v2
; CHECK-O0-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-BE-NEXT: xxlor vs0, v5, v5
; CHECK-O0-BE-NEXT: stxv vs0, 48(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v4, v4
; CHECK-O0-BE-NEXT: stxv vs0, 32(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v3, v3
; CHECK-O0-BE-NEXT: stxv vs0, 16(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v2, v2
; CHECK-O0-BE-NEXT: stxv vs0, 0(r7)
; CHECK-O0-BE-NEXT: blr
;
; CHECK-AIX64-LABEL: ld_op_st_xxmtacc:
; CHECK-AIX64: # %bb.0: # %entry
; CHECK-AIX64-NEXT: lxv 5, 48(3)
; CHECK-AIX64-NEXT: lxv 1, 16(3)
; CHECK-AIX64-NEXT: lxv 4, 32(3)
; CHECK-AIX64-NEXT: lxv 0, 0(3)
; CHECK-AIX64-NEXT: dmxxinstfdmr512 0, 32, 36, 0
; CHECK-AIX64-NEXT: xvi4ger8pp 0, 2, 2
; CHECK-AIX64-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX64-NEXT: stxv 5, 48(5)
; CHECK-AIX64-NEXT: stxv 4, 32(5)
; CHECK-AIX64-NEXT: stxv 3, 16(5)
; CHECK-AIX64-NEXT: stxv 2, 0(5)
; CHECK-AIX64-NEXT: blr
;
; CHECK-AIX32-LABEL: ld_op_st_xxmtacc:
; CHECK-AIX32: # %bb.0: # %entry
; CHECK-AIX32-NEXT: lxv 5, 48(3)
; CHECK-AIX32-NEXT: lxv 1, 16(3)
; CHECK-AIX32-NEXT: lxv 4, 32(3)
; CHECK-AIX32-NEXT: lxv 0, 0(3)
; CHECK-AIX32-NEXT: dmxxinstfdmr512 0, 32, 36, 0
; CHECK-AIX32-NEXT: xvi4ger8pp 0, 2, 2
; CHECK-AIX32-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX32-NEXT: stxv 5, 48(5)
; CHECK-AIX32-NEXT: stxv 4, 32(5)
; CHECK-AIX32-NEXT: stxv 3, 16(5)
; CHECK-AIX32-NEXT: stxv 2, 0(5)
; CHECK-AIX32-NEXT: blr
entry:
%0 = load <512 x i1>, ptr %vqp, align 64
%1 = tail call <512 x i1> @llvm.ppc.mma.xxmtacc(<512 x i1> %0)
%2 = tail call <512 x i1> @llvm.ppc.mma.xvi4ger8pp(<512 x i1> %1, <16 x i8> %vc, <16 x i8> %vc)
store <512 x i1> %2, ptr %resp, align 64
ret void
}
declare <512 x i1> @llvm.ppc.mma.xvi4ger8pp(<512 x i1>, <16 x i8>, <16 x i8>)
; xxmfacc with a loaded and stored vector quad.
define void @ld_st_xxmfacc(ptr %vqp, ptr %vpp, <16 x i8> %vc, ptr %resp) {
; CHECK-LABEL: ld_st_xxmfacc:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: lxv v3, 0(r3)
; CHECK-NEXT: lxv v5, 32(r3)
; CHECK-NEXT: lxv v2, 16(r3)
; CHECK-NEXT: lxv v4, 48(r3)
; CHECK-NEXT: dmxxinstfdmr512 wacc0, vsp36, vsp34, 0
; CHECK-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-NEXT: stxv v4, 48(r3)
; CHECK-NEXT: stxv v5, 32(r3)
; CHECK-NEXT: stxv v2, 16(r3)
; CHECK-NEXT: stxv v3, 0(r3)
; CHECK-NEXT: stxv v4, 48(r7)
; CHECK-NEXT: stxv v5, 32(r7)
; CHECK-NEXT: stxv v2, 16(r7)
; CHECK-NEXT: stxv v3, 0(r7)
; CHECK-NEXT: blr
;
; CHECK-BE-LABEL: ld_st_xxmfacc:
; CHECK-BE: # %bb.0: # %entry
; CHECK-BE-NEXT: lxv v3, 48(r3)
; CHECK-BE-NEXT: lxv v5, 16(r3)
; CHECK-BE-NEXT: lxv v2, 32(r3)
; CHECK-BE-NEXT: lxv v4, 0(r3)
; CHECK-BE-NEXT: dmxxinstfdmr512 wacc0, vsp36, vsp34, 0
; CHECK-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-BE-NEXT: stxv v5, 48(r3)
; CHECK-BE-NEXT: stxv v4, 32(r3)
; CHECK-BE-NEXT: stxv v3, 16(r3)
; CHECK-BE-NEXT: stxv v2, 0(r3)
; CHECK-BE-NEXT: stxv v5, 48(r7)
; CHECK-BE-NEXT: stxv v4, 32(r7)
; CHECK-BE-NEXT: stxv v3, 16(r7)
; CHECK-BE-NEXT: stxv v2, 0(r7)
; CHECK-BE-NEXT: blr
;
; CHECK-O0-LABEL: ld_st_xxmfacc:
; CHECK-O0: # %bb.0: # %entry
; CHECK-O0-NEXT: lxv vs0, 0(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp18
; CHECK-O0-NEXT: xxlor v5, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 16(r3)
; CHECK-O0-NEXT: xxlor v4, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 32(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp17
; CHECK-O0-NEXT: xxlor v3, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 48(r3)
; CHECK-O0-NEXT: xxlor v2, vs0, vs0
; CHECK-O0-NEXT: dmxxinstfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-NEXT: xxlor vs3, v4, v4
; CHECK-O0-NEXT: stxv vs3, 48(r3)
; CHECK-O0-NEXT: xxlor vs2, v5, v5
; CHECK-O0-NEXT: stxv vs2, 32(r3)
; CHECK-O0-NEXT: xxlor vs1, v2, v2
; CHECK-O0-NEXT: stxv vs1, 16(r3)
; CHECK-O0-NEXT: xxlor vs0, v3, v3
; CHECK-O0-NEXT: stxv vs0, 0(r3)
; CHECK-O0-NEXT: stxv vs3, 48(r7)
; CHECK-O0-NEXT: stxv vs2, 32(r7)
; CHECK-O0-NEXT: stxv vs1, 16(r7)
; CHECK-O0-NEXT: stxv vs0, 0(r7)
; CHECK-O0-NEXT: blr
;
; CHECK-O0-BE-LABEL: ld_st_xxmfacc:
; CHECK-O0-BE: # %bb.0: # %entry
; CHECK-O0-BE-NEXT: lxv vs0, 48(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp18
; CHECK-O0-BE-NEXT: xxlor v5, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 32(r3)
; CHECK-O0-BE-NEXT: xxlor v4, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 16(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp17
; CHECK-O0-BE-NEXT: xxlor v3, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 0(r3)
; CHECK-O0-BE-NEXT: xxlor v2, vs0, vs0
; CHECK-O0-BE-NEXT: dmxxinstfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-BE-NEXT: xxlor vs3, v5, v5
; CHECK-O0-BE-NEXT: stxv vs3, 48(r3)
; CHECK-O0-BE-NEXT: xxlor vs2, v4, v4
; CHECK-O0-BE-NEXT: stxv vs2, 32(r3)
; CHECK-O0-BE-NEXT: xxlor vs1, v3, v3
; CHECK-O0-BE-NEXT: stxv vs1, 16(r3)
; CHECK-O0-BE-NEXT: xxlor vs0, v2, v2
; CHECK-O0-BE-NEXT: stxv vs0, 0(r3)
; CHECK-O0-BE-NEXT: stxv vs3, 48(r7)
; CHECK-O0-BE-NEXT: stxv vs2, 32(r7)
; CHECK-O0-BE-NEXT: stxv vs1, 16(r7)
; CHECK-O0-BE-NEXT: stxv vs0, 0(r7)
; CHECK-O0-BE-NEXT: blr
;
; CHECK-AIX64-LABEL: ld_st_xxmfacc:
; CHECK-AIX64: # %bb.0: # %entry
; CHECK-AIX64-NEXT: lxv 3, 48(3)
; CHECK-AIX64-NEXT: lxv 5, 16(3)
; CHECK-AIX64-NEXT: lxv 2, 32(3)
; CHECK-AIX64-NEXT: lxv 4, 0(3)
; CHECK-AIX64-NEXT: dmxxinstfdmr512 0, 36, 34, 0
; CHECK-AIX64-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX64-NEXT: stxv 5, 48(3)
; CHECK-AIX64-NEXT: stxv 4, 32(3)
; CHECK-AIX64-NEXT: stxv 3, 16(3)
; CHECK-AIX64-NEXT: stxv 2, 0(3)
; CHECK-AIX64-NEXT: stxv 5, 48(5)
; CHECK-AIX64-NEXT: stxv 4, 32(5)
; CHECK-AIX64-NEXT: stxv 3, 16(5)
; CHECK-AIX64-NEXT: stxv 2, 0(5)
; CHECK-AIX64-NEXT: blr
;
; CHECK-AIX32-LABEL: ld_st_xxmfacc:
; CHECK-AIX32: # %bb.0: # %entry
; CHECK-AIX32-NEXT: lxv 3, 48(3)
; CHECK-AIX32-NEXT: lxv 5, 16(3)
; CHECK-AIX32-NEXT: lxv 2, 32(3)
; CHECK-AIX32-NEXT: lxv 4, 0(3)
; CHECK-AIX32-NEXT: dmxxinstfdmr512 0, 36, 34, 0
; CHECK-AIX32-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX32-NEXT: stxv 5, 48(3)
; CHECK-AIX32-NEXT: stxv 4, 32(3)
; CHECK-AIX32-NEXT: stxv 3, 16(3)
; CHECK-AIX32-NEXT: stxv 2, 0(3)
; CHECK-AIX32-NEXT: stxv 5, 48(5)
; CHECK-AIX32-NEXT: stxv 4, 32(5)
; CHECK-AIX32-NEXT: stxv 3, 16(5)
; CHECK-AIX32-NEXT: stxv 2, 0(5)
; CHECK-AIX32-NEXT: blr
entry:
%0 = load <512 x i1>, ptr %vqp, align 64
%1 = tail call <512 x i1> @llvm.ppc.mma.xxmfacc(<512 x i1> %0)
store <512 x i1> %1, ptr %vqp, align 64
store <512 x i1> %1, ptr %resp, align 64
ret void
}
declare <512 x i1> @llvm.ppc.mma.xxmfacc(<512 x i1>)
; xxmfacc used with an instruction that is not a load or store.
define void @ld_op_st_xxmfacc(ptr %vqp, ptr %vpp, <16 x i8> %vc, ptr %resp) {
; CHECK-LABEL: ld_op_st_xxmfacc:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: lxv v5, 0(r3)
; CHECK-NEXT: lxv v1, 32(r3)
; CHECK-NEXT: lxv v4, 16(r3)
; CHECK-NEXT: lxv v0, 48(r3)
; CHECK-NEXT: dmxxinstfdmr512 wacc0, vsp32, vsp36, 0
; CHECK-NEXT: xvi4ger8pp wacc0, v2, v2
; CHECK-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-NEXT: stxv v4, 48(r7)
; CHECK-NEXT: stxv v5, 32(r7)
; CHECK-NEXT: stxv v2, 16(r7)
; CHECK-NEXT: stxv v3, 0(r7)
; CHECK-NEXT: blr
;
; CHECK-BE-LABEL: ld_op_st_xxmfacc:
; CHECK-BE: # %bb.0: # %entry
; CHECK-BE-NEXT: lxv v5, 48(r3)
; CHECK-BE-NEXT: lxv v1, 16(r3)
; CHECK-BE-NEXT: lxv v4, 32(r3)
; CHECK-BE-NEXT: lxv v0, 0(r3)
; CHECK-BE-NEXT: dmxxinstfdmr512 wacc0, vsp32, vsp36, 0
; CHECK-BE-NEXT: xvi4ger8pp wacc0, v2, v2
; CHECK-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-BE-NEXT: stxv v5, 48(r7)
; CHECK-BE-NEXT: stxv v4, 32(r7)
; CHECK-BE-NEXT: stxv v3, 16(r7)
; CHECK-BE-NEXT: stxv v2, 0(r7)
; CHECK-BE-NEXT: blr
;
; CHECK-O0-LABEL: ld_op_st_xxmfacc:
; CHECK-O0: # %bb.0: # %entry
; CHECK-O0-NEXT: lxv vs0, 0(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp16
; CHECK-O0-NEXT: xxlor v1, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 16(r3)
; CHECK-O0-NEXT: xxlor v0, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 32(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp18
; CHECK-O0-NEXT: xxlor v5, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 48(r3)
; CHECK-O0-NEXT: xxlor v4, vs0, vs0
; CHECK-O0-NEXT: dmxxinstfdmr512 wacc0, vsp36, vsp32, 0
; CHECK-O0-NEXT: xvi4ger8pp wacc0, v2, v2
; CHECK-O0-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-NEXT: xxlor vs0, v4, v4
; CHECK-O0-NEXT: stxv vs0, 48(r7)
; CHECK-O0-NEXT: xxlor vs0, v5, v5
; CHECK-O0-NEXT: stxv vs0, 32(r7)
; CHECK-O0-NEXT: xxlor vs0, v2, v2
; CHECK-O0-NEXT: stxv vs0, 16(r7)
; CHECK-O0-NEXT: xxlor vs0, v3, v3
; CHECK-O0-NEXT: stxv vs0, 0(r7)
; CHECK-O0-NEXT: blr
;
; CHECK-O0-BE-LABEL: ld_op_st_xxmfacc:
; CHECK-O0-BE: # %bb.0: # %entry
; CHECK-O0-BE-NEXT: lxv vs0, 48(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp16
; CHECK-O0-BE-NEXT: xxlor v1, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 32(r3)
; CHECK-O0-BE-NEXT: xxlor v0, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 16(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp18
; CHECK-O0-BE-NEXT: xxlor v5, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 0(r3)
; CHECK-O0-BE-NEXT: xxlor v4, vs0, vs0
; CHECK-O0-BE-NEXT: dmxxinstfdmr512 wacc0, vsp36, vsp32, 0
; CHECK-O0-BE-NEXT: xvi4ger8pp wacc0, v2, v2
; CHECK-O0-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-BE-NEXT: xxlor vs0, v5, v5
; CHECK-O0-BE-NEXT: stxv vs0, 48(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v4, v4
; CHECK-O0-BE-NEXT: stxv vs0, 32(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v3, v3
; CHECK-O0-BE-NEXT: stxv vs0, 16(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v2, v2
; CHECK-O0-BE-NEXT: stxv vs0, 0(r7)
; CHECK-O0-BE-NEXT: blr
;
; CHECK-AIX64-LABEL: ld_op_st_xxmfacc:
; CHECK-AIX64: # %bb.0: # %entry
; CHECK-AIX64-NEXT: lxv 5, 48(3)
; CHECK-AIX64-NEXT: lxv 1, 16(3)
; CHECK-AIX64-NEXT: lxv 4, 32(3)
; CHECK-AIX64-NEXT: lxv 0, 0(3)
; CHECK-AIX64-NEXT: dmxxinstfdmr512 0, 32, 36, 0
; CHECK-AIX64-NEXT: xvi4ger8pp 0, 2, 2
; CHECK-AIX64-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX64-NEXT: stxv 5, 48(5)
; CHECK-AIX64-NEXT: stxv 4, 32(5)
; CHECK-AIX64-NEXT: stxv 3, 16(5)
; CHECK-AIX64-NEXT: stxv 2, 0(5)
; CHECK-AIX64-NEXT: blr
;
; CHECK-AIX32-LABEL: ld_op_st_xxmfacc:
; CHECK-AIX32: # %bb.0: # %entry
; CHECK-AIX32-NEXT: lxv 5, 48(3)
; CHECK-AIX32-NEXT: lxv 1, 16(3)
; CHECK-AIX32-NEXT: lxv 4, 32(3)
; CHECK-AIX32-NEXT: lxv 0, 0(3)
; CHECK-AIX32-NEXT: dmxxinstfdmr512 0, 32, 36, 0
; CHECK-AIX32-NEXT: xvi4ger8pp 0, 2, 2
; CHECK-AIX32-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX32-NEXT: stxv 5, 48(5)
; CHECK-AIX32-NEXT: stxv 4, 32(5)
; CHECK-AIX32-NEXT: stxv 3, 16(5)
; CHECK-AIX32-NEXT: stxv 2, 0(5)
; CHECK-AIX32-NEXT: blr
entry:
%0 = load <512 x i1>, ptr %vqp, align 64
%1 = tail call <512 x i1> @llvm.ppc.mma.xvi4ger8pp(<512 x i1> %0, <16 x i8> %vc, <16 x i8> %vc)
%2 = tail call <512 x i1> @llvm.ppc.mma.xxmfacc(<512 x i1> %1)
store <512 x i1> %2, ptr %resp, align 64
ret void
}
; xxmtacc and xxmfacc used interleaved in more complexed mma code.
define void @cmplx_xxmacc(ptr %ptr1, ptr %ptr2, <16 x i8> %vc1, <16 x i8> %vc2) {
; CHECK-LABEL: cmplx_xxmacc:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: lxv v1, 0(r3)
; CHECK-NEXT: lxv v7, 32(r3)
; CHECK-NEXT: lxv v0, 16(r3)
; CHECK-NEXT: lxv v6, 48(r3)
; CHECK-NEXT: vmr v4, v3
; CHECK-NEXT: vmr v5, v2
; CHECK-NEXT: xxlor v2, v4, v4
; CHECK-NEXT: vmr v2, v5
; CHECK-NEXT: xxlor v3, v5, v5
; CHECK-NEXT: dmxxinstfdmr512 wacc0, vsp38, vsp32, 0
; CHECK-NEXT: xvf64gerpp wacc0, vsp34, v5
; CHECK-NEXT: xvf64gerpp wacc0, vsp36, v4
; CHECK-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-NEXT: stxv v4, 48(r3)
; CHECK-NEXT: stxv v5, 32(r3)
; CHECK-NEXT: stxv v2, 16(r3)
; CHECK-NEXT: stxv v3, 0(r3)
; CHECK-NEXT: blr
;
; CHECK-BE-LABEL: cmplx_xxmacc:
; CHECK-BE: # %bb.0: # %entry
; CHECK-BE-NEXT: lxv v1, 48(r3)
; CHECK-BE-NEXT: lxv v7, 16(r3)
; CHECK-BE-NEXT: lxv v0, 32(r3)
; CHECK-BE-NEXT: lxv v6, 0(r3)
; CHECK-BE-NEXT: vmr v4, v3
; CHECK-BE-NEXT: vmr v5, v2
; CHECK-BE-NEXT: xxlor v2, v4, v4
; CHECK-BE-NEXT: vmr v2, v5
; CHECK-BE-NEXT: xxlor v3, v5, v5
; CHECK-BE-NEXT: dmxxinstfdmr512 wacc0, vsp38, vsp32, 0
; CHECK-BE-NEXT: xvf64gerpp wacc0, vsp34, v5
; CHECK-BE-NEXT: xvf64gerpp wacc0, vsp36, v4
; CHECK-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-BE-NEXT: stxv v5, 48(r3)
; CHECK-BE-NEXT: stxv v4, 32(r3)
; CHECK-BE-NEXT: stxv v3, 16(r3)
; CHECK-BE-NEXT: stxv v2, 0(r3)
; CHECK-BE-NEXT: blr
;
; CHECK-O0-LABEL: cmplx_xxmacc:
; CHECK-O0: # %bb.0: # %entry
; CHECK-O0-NEXT: vmr v4, v3
; CHECK-O0-NEXT: vmr v5, v2
; CHECK-O0-NEXT: # implicit-def: $vsrp16
; CHECK-O0-NEXT: vmr v1, v5
; CHECK-O0-NEXT: xxlor v2, v0, v0
; CHECK-O0-NEXT: xxlor v3, v1, v1
; CHECK-O0-NEXT: vmr v2, v4
; CHECK-O0-NEXT: vmr v0, v5
; CHECK-O0-NEXT: lxv vs0, 0(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp20
; CHECK-O0-NEXT: xxlor v9, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 16(r3)
; CHECK-O0-NEXT: xxlor v8, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 32(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp19
; CHECK-O0-NEXT: xxlor v7, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 48(r3)
; CHECK-O0-NEXT: xxlor v6, vs0, vs0
; CHECK-O0-NEXT: dmxxinstfdmr512 wacc0, vsp38, vsp40, 0
; CHECK-O0-NEXT: xxlor vs0, v5, v5
; CHECK-O0-NEXT: xvf64gerpp wacc0, vsp32, vs0
; CHECK-O0-NEXT: xxlor vs0, v4, v4
; CHECK-O0-NEXT: xvf64gerpp wacc0, vsp34, vs0
; CHECK-O0-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-NEXT: xxlor vs0, v4, v4
; CHECK-O0-NEXT: stxv vs0, 48(r3)
; CHECK-O0-NEXT: xxlor vs0, v5, v5
; CHECK-O0-NEXT: stxv vs0, 32(r3)
; CHECK-O0-NEXT: xxlor vs0, v2, v2
; CHECK-O0-NEXT: stxv vs0, 16(r3)
; CHECK-O0-NEXT: xxlor vs0, v3, v3
; CHECK-O0-NEXT: stxv vs0, 0(r3)
; CHECK-O0-NEXT: blr
;
; CHECK-O0-BE-LABEL: cmplx_xxmacc:
; CHECK-O0-BE: # %bb.0: # %entry
; CHECK-O0-BE-NEXT: vmr v4, v3
; CHECK-O0-BE-NEXT: vmr v5, v2
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp16
; CHECK-O0-BE-NEXT: vmr v1, v5
; CHECK-O0-BE-NEXT: xxlor v2, v0, v0
; CHECK-O0-BE-NEXT: xxlor v3, v1, v1
; CHECK-O0-BE-NEXT: vmr v2, v4
; CHECK-O0-BE-NEXT: vmr v0, v5
; CHECK-O0-BE-NEXT: lxv vs0, 48(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp20
; CHECK-O0-BE-NEXT: xxlor v9, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 32(r3)
; CHECK-O0-BE-NEXT: xxlor v8, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 16(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp19
; CHECK-O0-BE-NEXT: xxlor v7, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 0(r3)
; CHECK-O0-BE-NEXT: xxlor v6, vs0, vs0
; CHECK-O0-BE-NEXT: dmxxinstfdmr512 wacc0, vsp38, vsp40, 0
; CHECK-O0-BE-NEXT: xxlor vs0, v5, v5
; CHECK-O0-BE-NEXT: xvf64gerpp wacc0, vsp32, vs0
; CHECK-O0-BE-NEXT: xxlor vs0, v4, v4
; CHECK-O0-BE-NEXT: xvf64gerpp wacc0, vsp34, vs0
; CHECK-O0-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-BE-NEXT: xxlor vs0, v5, v5
; CHECK-O0-BE-NEXT: stxv vs0, 48(r3)
; CHECK-O0-BE-NEXT: xxlor vs0, v4, v4
; CHECK-O0-BE-NEXT: stxv vs0, 32(r3)
; CHECK-O0-BE-NEXT: xxlor vs0, v3, v3
; CHECK-O0-BE-NEXT: stxv vs0, 16(r3)
; CHECK-O0-BE-NEXT: xxlor vs0, v2, v2
; CHECK-O0-BE-NEXT: stxv vs0, 0(r3)
; CHECK-O0-BE-NEXT: blr
;
; CHECK-AIX64-LABEL: cmplx_xxmacc:
; CHECK-AIX64: # %bb.0: # %entry
; CHECK-AIX64-NEXT: lxv 1, 48(3)
; CHECK-AIX64-NEXT: lxv 7, 16(3)
; CHECK-AIX64-NEXT: lxv 0, 32(3)
; CHECK-AIX64-NEXT: lxv 6, 0(3)
; CHECK-AIX64-NEXT: vmr 4, 3
; CHECK-AIX64-NEXT: vmr 5, 2
; CHECK-AIX64-NEXT: xxlor 2, 4, 4
; CHECK-AIX64-NEXT: vmr 2, 5
; CHECK-AIX64-NEXT: xxlor 3, 5, 5
; CHECK-AIX64-NEXT: dmxxinstfdmr512 0, 38, 32, 0
; CHECK-AIX64-NEXT: xvf64gerpp 0, 34, 5
; CHECK-AIX64-NEXT: xvf64gerpp 0, 36, 4
; CHECK-AIX64-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX64-NEXT: stxv 5, 48(3)
; CHECK-AIX64-NEXT: stxv 4, 32(3)
; CHECK-AIX64-NEXT: stxv 3, 16(3)
; CHECK-AIX64-NEXT: stxv 2, 0(3)
; CHECK-AIX64-NEXT: blr
;
; CHECK-AIX32-LABEL: cmplx_xxmacc:
; CHECK-AIX32: # %bb.0: # %entry
; CHECK-AIX32-NEXT: lxv 1, 48(3)
; CHECK-AIX32-NEXT: lxv 7, 16(3)
; CHECK-AIX32-NEXT: lxv 0, 32(3)
; CHECK-AIX32-NEXT: lxv 6, 0(3)
; CHECK-AIX32-NEXT: vmr 4, 3
; CHECK-AIX32-NEXT: vmr 5, 2
; CHECK-AIX32-NEXT: xxlor 2, 4, 4
; CHECK-AIX32-NEXT: vmr 2, 5
; CHECK-AIX32-NEXT: xxlor 3, 5, 5
; CHECK-AIX32-NEXT: dmxxinstfdmr512 0, 38, 32, 0
; CHECK-AIX32-NEXT: xvf64gerpp 0, 34, 5
; CHECK-AIX32-NEXT: xvf64gerpp 0, 36, 4
; CHECK-AIX32-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX32-NEXT: stxv 5, 48(3)
; CHECK-AIX32-NEXT: stxv 4, 32(3)
; CHECK-AIX32-NEXT: stxv 3, 16(3)
; CHECK-AIX32-NEXT: stxv 2, 0(3)
; CHECK-AIX32-NEXT: blr
entry:
%0 = tail call <256 x i1> @llvm.ppc.vsx.assemble.pair(<16 x i8> %vc2, <16 x i8> %vc1)
%1 = tail call <256 x i1> @llvm.ppc.vsx.assemble.pair(<16 x i8> %vc1, <16 x i8> %vc1)
%2 = load <512 x i1>, ptr %ptr1, align 64
%3 = tail call <512 x i1> @llvm.ppc.mma.xxmtacc(<512 x i1> %2)
%4 = tail call <512 x i1> @llvm.ppc.mma.xvf64gerpp(<512 x i1> %3, <256 x i1> %1, <16 x i8> %vc1)
%5 = tail call <512 x i1> @llvm.ppc.mma.xxmfacc(<512 x i1> %4)
%6 = tail call <512 x i1> @llvm.ppc.mma.xvf64gerpp(<512 x i1> %5, <256 x i1> %0, <16 x i8> %vc2)
store <512 x i1> %6, ptr %ptr1, align 64
ret void
}
declare <256 x i1> @llvm.ppc.vsx.assemble.pair(<16 x i8>, <16 x i8>)
declare <512 x i1> @llvm.ppc.mma.xvf64gerpp(<512 x i1>, <256 x i1>, <16 x i8>)
; xxsetaccz ; xxsetaccz
declare <512 x i1> @llvm.ppc.mma.xxsetaccz() declare <512 x i1> @llvm.ppc.mma.xxsetaccz()
define void @int_xxsetaccz(ptr %ptr) { define void @int_xxsetaccz(ptr %ptr) {
; CHECK-LABEL: int_xxsetaccz: ; CHECK-LABEL: int_xxsetaccz:
; CHECK: # %bb.0: # %entry ; CHECK: # %bb.0: # %entry
; CHECK-NEXT: xxsetaccz wacc0 ; CHECK-NEXT: xxsetaccz wacc0
; CHECK-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0 ; CHECK-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-NEXT: stxv v4, 48(r3) ; CHECK-NEXT: stxv v4, 48(r3)
▲ Show 20 Lines • Show All 257 Lines • ▼ Show 20 Linesentry:
%2 = call <512 x i1> @llvm.ppc.mma.xvf32gerpp(<512 x i1> %0, <16 x i8> %vc, <16 x i8> %vc) %2 = call <512 x i1> @llvm.ppc.mma.xvf32gerpp(<512 x i1> %0, <16 x i8> %vc, <16 x i8> %vc)
%3 = call <512 x i1> @llvm.ppc.mma.xvf32gerpp(<512 x i1> %1, <16 x i8> %vc, <16 x i8> %vc) %3 = call <512 x i1> @llvm.ppc.mma.xvf32gerpp(<512 x i1> %1, <16 x i8> %vc, <16 x i8> %vc)
%4 = getelementptr inbounds <512 x i1>, ptr %res, i64 0 %4 = getelementptr inbounds <512 x i1>, ptr %res, i64 0
%5 = getelementptr inbounds <512 x i1>, ptr %res, i64 1 %5 = getelementptr inbounds <512 x i1>, ptr %res, i64 1
store <512 x i1> %2, ptr %4, align 64 store <512 x i1> %2, ptr %4, align 64
store <512 x i1> %3, ptr %5, align 64 store <512 x i1> %3, ptr %5, align 64
ret void ret void
} }
declare <256 x i1> @llvm.ppc.vsx.lxvp(ptr)
declare void @llvm.ppc.vsx.stxvp(<256 x i1>, ptr)
; Function Attrs: nofree nounwind
define void @test_ldst_1(ptr nocapture readonly %vqp, ptr %vpp, <16 x i8> %vc, ptr nocapture %resp) {
; CHECK-LABEL: test_ldst_1:
; CHECK: # %bb.0: # %entry
; CHECK-NEXT: lxv v5, 0(r3)
; CHECK-NEXT: lxv v1, 32(r3)
; CHECK-NEXT: lxv v4, 16(r3)
; CHECK-NEXT: lxv v0, 48(r3)
; CHECK-NEXT: dmxxinstfdmr512 wacc0, vsp32, vsp36, 0
; CHECK-NEXT: plxvp vsp36, 8(r4), 0
; CHECK-NEXT: pmxvf64gernn wacc0, vsp36, v2, 0, 0
; CHECK-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-NEXT: stxv v4, 48(r7)
; CHECK-NEXT: stxv v5, 32(r7)
; CHECK-NEXT: stxv v2, 16(r7)
; CHECK-NEXT: stxv v3, 0(r7)
; CHECK-NEXT: blr
;
; CHECK-BE-LABEL: test_ldst_1:
; CHECK-BE: # %bb.0: # %entry
; CHECK-BE-NEXT: lxv v5, 48(r3)
; CHECK-BE-NEXT: lxv v1, 16(r3)
; CHECK-BE-NEXT: lxv v4, 32(r3)
; CHECK-BE-NEXT: lxv v0, 0(r3)
; CHECK-BE-NEXT: dmxxinstfdmr512 wacc0, vsp32, vsp36, 0
; CHECK-BE-NEXT: plxvp vsp36, 8(r4), 0
; CHECK-BE-NEXT: pmxvf64gernn wacc0, vsp36, v2, 0, 0
; CHECK-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-BE-NEXT: stxv v5, 48(r7)
; CHECK-BE-NEXT: stxv v4, 32(r7)
; CHECK-BE-NEXT: stxv v3, 16(r7)
; CHECK-BE-NEXT: stxv v2, 0(r7)
; CHECK-BE-NEXT: blr
;
; CHECK-O0-LABEL: test_ldst_1:
; CHECK-O0: # %bb.0: # %entry
; CHECK-O0-NEXT: vmr v4, v2
; CHECK-O0-NEXT: lxv vs0, 0(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp16
; CHECK-O0-NEXT: xxlor v1, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 16(r3)
; CHECK-O0-NEXT: xxlor v0, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 32(r3)
; CHECK-O0-NEXT: # implicit-def: $vsrp17
; CHECK-O0-NEXT: xxlor v3, vs0, vs0
; CHECK-O0-NEXT: lxv vs0, 48(r3)
; CHECK-O0-NEXT: xxlor v2, vs0, vs0
; CHECK-O0-NEXT: dmxxinstfdmr512 wacc0, vsp34, vsp32, 0
; CHECK-O0-NEXT: plxvp vsp34, 8(r4), 0
; CHECK-O0-NEXT: xxlor vs0, v4, v4
; CHECK-O0-NEXT: pmxvf64gernn wacc0, vsp34, vs0, 0, 0
; CHECK-O0-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-NEXT: xxlor vs0, v4, v4
; CHECK-O0-NEXT: stxv vs0, 48(r7)
; CHECK-O0-NEXT: xxlor vs0, v5, v5
; CHECK-O0-NEXT: stxv vs0, 32(r7)
; CHECK-O0-NEXT: xxlor vs0, v2, v2
; CHECK-O0-NEXT: stxv vs0, 16(r7)
; CHECK-O0-NEXT: xxlor vs0, v3, v3
; CHECK-O0-NEXT: stxv vs0, 0(r7)
; CHECK-O0-NEXT: blr
;
; CHECK-O0-BE-LABEL: test_ldst_1:
; CHECK-O0-BE: # %bb.0: # %entry
; CHECK-O0-BE-NEXT: vmr v4, v2
; CHECK-O0-BE-NEXT: lxv vs0, 48(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp16
; CHECK-O0-BE-NEXT: xxlor v1, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 32(r3)
; CHECK-O0-BE-NEXT: xxlor v0, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 16(r3)
; CHECK-O0-BE-NEXT: # implicit-def: $vsrp17
; CHECK-O0-BE-NEXT: xxlor v3, vs0, vs0
; CHECK-O0-BE-NEXT: lxv vs0, 0(r3)
; CHECK-O0-BE-NEXT: xxlor v2, vs0, vs0
; CHECK-O0-BE-NEXT: dmxxinstfdmr512 wacc0, vsp34, vsp32, 0
; CHECK-O0-BE-NEXT: plxvp vsp34, 8(r4), 0
; CHECK-O0-BE-NEXT: xxlor vs0, v4, v4
; CHECK-O0-BE-NEXT: pmxvf64gernn wacc0, vsp34, vs0, 0, 0
; CHECK-O0-BE-NEXT: dmxxextfdmr512 wacc0, vsp34, vsp36, 0
; CHECK-O0-BE-NEXT: xxlor vs0, v5, v5
; CHECK-O0-BE-NEXT: stxv vs0, 48(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v4, v4
; CHECK-O0-BE-NEXT: stxv vs0, 32(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v3, v3
; CHECK-O0-BE-NEXT: stxv vs0, 16(r7)
; CHECK-O0-BE-NEXT: xxlor vs0, v2, v2
; CHECK-O0-BE-NEXT: stxv vs0, 0(r7)
; CHECK-O0-BE-NEXT: blr
;
; CHECK-AIX64-LABEL: test_ldst_1:
; CHECK-AIX64: # %bb.0: # %entry
; CHECK-AIX64-NEXT: lxv 5, 48(3)
; CHECK-AIX64-NEXT: lxv 1, 16(3)
; CHECK-AIX64-NEXT: lxv 4, 32(3)
; CHECK-AIX64-NEXT: lxv 0, 0(3)
; CHECK-AIX64-NEXT: dmxxinstfdmr512 0, 32, 36, 0
; CHECK-AIX64-NEXT: plxvp 36, 8(4), 0
; CHECK-AIX64-NEXT: pmxvf64gernn 0, 36, 2, 0, 0
; CHECK-AIX64-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX64-NEXT: stxv 5, 48(5)
; CHECK-AIX64-NEXT: stxv 4, 32(5)
; CHECK-AIX64-NEXT: stxv 3, 16(5)
; CHECK-AIX64-NEXT: stxv 2, 0(5)
; CHECK-AIX64-NEXT: blr
;
; CHECK-AIX32-LABEL: test_ldst_1:
; CHECK-AIX32: # %bb.0: # %entry
; CHECK-AIX32-NEXT: lxv 5, 48(3)
; CHECK-AIX32-NEXT: lxv 1, 16(3)
; CHECK-AIX32-NEXT: lxv 4, 32(3)
; CHECK-AIX32-NEXT: lxv 0, 0(3)
; CHECK-AIX32-NEXT: dmxxinstfdmr512 0, 32, 36, 0
; CHECK-AIX32-NEXT: plxvp 36, 8(4), 0
; CHECK-AIX32-NEXT: pmxvf64gernn 0, 36, 2, 0, 0
; CHECK-AIX32-NEXT: dmxxextfdmr512 0, 34, 36, 0
; CHECK-AIX32-NEXT: stxv 5, 48(5)
; CHECK-AIX32-NEXT: stxv 4, 32(5)
; CHECK-AIX32-NEXT: stxv 3, 16(5)
; CHECK-AIX32-NEXT: stxv 2, 0(5)
; CHECK-AIX32-NEXT: blr
entry:
%0 = load <512 x i1>, ptr %vqp, align 64
%1 = getelementptr i8, ptr %vpp, i64 8
%2 = tail call <256 x i1> @llvm.ppc.vsx.lxvp(ptr %1)
%3 = tail call <512 x i1> @llvm.ppc.mma.pmxvf64gernn(<512 x i1> %0, <256 x i1> %2, <16 x i8> %vc, i32 0, i32 0)
store <512 x i1> %3, ptr %resp, align 64
ret void
}
declare <512 x i1> @llvm.ppc.mma.pmxvf64gernn(<512 x i1>, <256 x i1>, <16 x i8>, i32, i32)
declare <512 x i1> @llvm.ppc.mma.xvf64gernp(<512 x i1>, <256 x i1>, <16 x i8>)