This is an archive of the discontinued LLVM Phabricator instance.

Differential D105092

[RISCV] (1/2) Add the tail policy argument to builtins/intrinsics.
ClosedPublic

Authored by HsiangKai on Jun 29 2021, 2:44 AM.

Details

Reviewers
craig.topper
frasercrmck
rogfer01
kito-cheng
khchen
arcbbb
evandro
Commits
rG7d39a8a92122: [RISCV] (1/2) Add the tail policy argument to builtins/intrinsics.
Summary

Add the tail policy argument to Clang builtins and IR intrinsics. There are two policies for tail elements. Tail agnostic means users do not care about the values in the tail elements and tail undisturbed means the values in the tail elements need to be kept after the operation. In order to let users control the tail policy, we add an additional argument at the end of the argument list.

For unmasked operations, we have no maskedoff and the tail policy is always tail agnostic. If users want to keep tail elements under unmasked operations, they could use all one mask in the masked operations to do it. So, we only add the additional argument for masked operations for most cases. There are exceptions listed below.

In this patch, we do not handle the following cases to reduce the complexity of the patch. There could be two separate patches for them.

  • Use dest argument to control tail policy

vmerge.vvm/vmerge.vxm/vmerge.vim (add _t builtins with additional dest argument)
vfmerge.vfm (add _t builtins with additional dest argument)
vmv.v.v (add _t builtins with additional dest argument)
vmv.v.x (add _t builtins with additional dest argument)
vmv.v.i (add _t builtins with additional dest argument)
vfmv.v.f (add _t builtins with additional dest argument)
vadc.vvm/vadc.vxm/vadc.vim (add _t builtins with additional dest argument)
vsbc.vvm/vsbc.vxm (add _t builtins with additional dest argument)

  • Always has tail argument for masked/unmasked intrinsics

Vector Single-Width Integer Multiply-Add Instructions (add _t and _mt builtins)
Vector Widening Integer Multiply-Add Instructions (add _t and _mt builtins)
Vector Single-Width Floating-Point Fused Multiply-Add Instructions (add _t and _mt builtins)
Vector Widening Floating-Point Fused Multiply-Add Instructions (add _t and _mt builtins)
Vector Reduction Operations (add _t and _mt builtins)
Vector Slideup Instructions (add _t and _mt builtins)
Vector Slidedown Instructions (add _t and _mt builtins)

Discussion: https://github.com/riscv/rvv-intrinsic-doc/pull/101

Diff Detail

Event Timeline

HsiangKai created this revision.Jun 29 2021, 2:44 AM
Herald added subscribers: StephenFan, vkmr, dexonsmith and 24 others. · View Herald TranscriptJun 29 2021, 2:44 AM
HsiangKai requested review of this revision.Jun 29 2021, 2:44 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptJun 29 2021, 2:44 AM
Herald added subscribers: llvm-commits, cfe-commits, MaskRay. · View Herald Transcript
HsiangKai edited the summary of this revision. (Show Details)Jun 29 2021, 2:54 AM
Harbormaster completed remote builds in B111468: Diff 355156.Jun 29 2021, 3:21 AM
khchen added a subscriber: khchen.Jun 29 2021, 6:59 AM
khchen added inline comments.
clang/utils/TableGen/RISCVVEmitter.cpp
1148 ↗(On Diff #355156)

maybe the policy argument should be a constant value ("Kz")?

craig.topper added inline comments.Jun 29 2021, 9:33 AM
clang/utils/TableGen/RISCVVEmitter.cpp
1148 ↗(On Diff #355156)

Agreed.

llvm/lib/Target/RISCV/MCTargetDesc/RISCVBaseInfo.h
81

This should be HasPolicyOpShift and HasPolicyOpMask to match SEWOp/VLOp naming.

150

hasPolicyOp

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp
435

Probably need to mask this to bit 0. TailAgnostic = Op.getImm() & 1. As written we'll set TailAgnostic if any bit in the immediate is non-zero.

llvm/lib/Target/RISCV/RISCVInstrFormats.td
182 ↗(On Diff #355156)

HasPolicyOp

HsiangKai updated this revision to Diff 355420.Jun 29 2021, 6:39 PM
  • Use constant value for tail policy argument.
  • Rename HasPolicy to HasPolicyOp.
Harbormaster completed remote builds in B111661: Diff 355420.Jun 29 2021, 7:22 PM
HsiangKai updated this revision to Diff 358205.Jul 13 2021, 2:31 AM

Add the TA argument to most of the intrinsics with mask.

HsiangKai edited the summary of this revision. (Show Details)Jul 13 2021, 2:32 AM
Harbormaster completed remote builds in B113694: Diff 358205.Jul 13 2021, 3:49 AM
HsiangKai updated this revision to Diff 358445.Jul 13 2021, 3:23 PM
Harbormaster completed remote builds in B113863: Diff 358445.Jul 13 2021, 3:24 PM
craig.topper added a comment.Jul 21 2021, 3:11 PM

Does lowerRISCVVMachineInstrToMCInst in RISCVMCInstLower.cpp need to know to skip the policy op?

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp
448–449

Why doesn't this need to be updated?

craig.topper added inline comments.Jul 21 2021, 5:13 PM
llvm/lib/Target/RISCV/RISCVInstrInfoVVLPatterns.td
25

Why are these the opposite polarity of what's in C?

HsiangKai updated this revision to Diff 361869.Jul 26 2021, 5:53 PM

Consider the policy operand in lowerRISCVVMachineInstrToMCInst and RISCVInsertVSETVLI.cpp.

HsiangKai marked 2 inline comments as done.Jul 26 2021, 7:09 PM
Harbormaster completed remote builds in B116320: Diff 361869.Jul 26 2021, 7:22 PM
HsiangKai retitled this revision from [PoC][RISCV] Add the tail policy argument to builtins/intrinsics. to [RISCV] Add the tail policy argument to builtins/intrinsics..Jul 26 2021, 8:25 PM
HsiangKai edited the summary of this revision. (Show Details)
HsiangKai added reviewers: khchen, arcbbb, evandro.
HsiangKai updated this revision to Diff 361926.Jul 27 2021, 12:32 AM

Update vadd-rv32.ll test case.

No update to other RVV test cases in this patch to make it reviewed easier.

Harbormaster completed remote builds in B116358: Diff 361926.Jul 27 2021, 1:23 AM
frasercrmck mentioned this in D106512: [RISCV] Remove the _COMMUTABLE and _TA versions of FMA and wide FMA vector instructions..Jul 27 2021, 4:52 AM
craig.topper mentioned this in rG643ce70a6466: [RISCV] Remove the _COMMUTABLE and _TA versions of FMA and wide FMA vector….Aug 4 2021, 10:40 AM
HsiangKai updated this revision to Diff 370285.Sep 2 2021, 8:44 AM

Update.

Harbormaster completed remote builds in B122316: Diff 370285.Sep 2 2021, 9:45 AM
craig.topper added inline comments.Sep 2 2021, 1:47 PM
llvm/include/llvm/IR/IntrinsicsRISCV.td
168–170

This needs ImmArg<ArgIndex<4>>. Similar for the rest. That will guarantee it is a constant and will make SelectionDAGBuilder create a target constant instead of a regular constant.

llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
248

Can we use

uint64_t Policy = Node->getConstantOperandVal(CurOp++);
llvm/lib/Target/RISCV/RISCVISelLowering.cpp
5212

TargetConstant?

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td
649

Why uimm5? I think VPseudoTernaryNoMaskWithPolicy is using ixlenimm like $sew

HsiangKai updated this revision to Diff 370440.Sep 2 2021, 4:10 PM

Address comments.

Harbormaster completed remote builds in B122421: Diff 370440.Sep 2 2021, 4:35 PM
craig.topper added inline comments.Sep 3 2021, 2:18 PM
llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td
2290

I think this should be timm:$policy

HsiangKai retitled this revision from [RISCV] Add the tail policy argument to builtins/intrinsics. to [RISCV] (1/2) Add the tail policy argument to builtins/intrinsics..Sep 6 2021, 6:44 AM

Clang part: https://reviews.llvm.org/D109322

HsiangKai added a child revision: D109322: [RISCV] (2/2) Add the tail policy argument to builtins/intrinsics..Sep 6 2021, 6:45 AM
HsiangKai updated this revision to Diff 370980.Sep 6 2021, 7:01 PM

Use timm for the $policy argument in the patterns.

Harbormaster completed remote builds in B122814: Diff 370980.Sep 6 2021, 7:41 PM
craig.topper added inline comments.Sep 7 2021, 9:14 PM
llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td
2294

Does timm:$policy work down here too?

HsiangKai updated this revision to Diff 371499.Sep 8 2021, 9:11 PM

Address comments.

Harbormaster completed remote builds in B123157: Diff 371499.Sep 8 2021, 9:41 PM
craig.topper accepted this revision.Sep 22 2021, 7:46 PM

LGTM

This revision is now accepted and ready to land.Sep 22 2021, 7:46 PM
craig.topper requested changes to this revision.Sep 22 2021, 7:48 PM
craig.topper added inline comments.
llvm/lib/Target/RISCV/RISCVInstrInfoVVLPatterns.td
1321

This should be TAIL_UNDISTURBED. It has a $merge operand

1333

Same here

1360

And here

1416

And here

1442

And here

This revision now requires changes to proceed.Sep 22 2021, 7:48 PM
craig.topper accepted this revision.Sep 22 2021, 7:52 PM

Nevermind.

LGTM

llvm/lib/Target/RISCV/RISCVInstrInfoVVLPatterns.td
1321

Err nevermind. Maybe we only cared about the mask. Probably should add the policy to riscv_vselect_vl, but that's for another patch.

This revision is now accepted and ready to land.Sep 22 2021, 7:52 PM
khchen added inline comments.Sep 23 2021, 12:05 AM
llvm/include/llvm/IR/IntrinsicsRISCV.td
162

maybe we could have another NFC patch to update those argument info comments.

HsiangKai added a comment.Sep 23 2021, 2:09 AM

I will wait for https://reviews.llvm.org/D109322 be accepted. These two patches need to get in together.

Herald added a subscriber: achieveartificialintelligence. · View Herald TranscriptSep 23 2021, 2:09 AM
HsiangKai added inline comments.Sep 23 2021, 2:09 AM
llvm/include/llvm/IR/IntrinsicsRISCV.td
162

I will update these comments. Thanks.

This revision was landed with ongoing or failed builds.Sep 24 2021, 2:11 AM
Closed by commit rG7d39a8a92122: [RISCV] (1/2) Add the tail policy argument to builtins/intrinsics. (authored by HsiangKai). · Explain Why
This revision was automatically updated to reflect the committed changes.
HsiangKai added a commit: rG7d39a8a92122: [RISCV] (1/2) Add the tail policy argument to builtins/intrinsics..

Large Diff

This large diff affects 232 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
llvm/
include/
llvm/
IR/
152 lines
lib/
Target/
RISCV/
MCTargetDesc/
5 lines
2 lines
30 lines
15 lines
6 lines
368 lines
20 lines
test/
CodeGen/
RISCV/
rvv/
2 lines
5 lines
2 lines
9 lines
28 lines
294 lines
8 lines
310 lines
4 lines
228 lines
220 lines
228 lines
220 lines
65 lines
360 lines
308 lines
316 lines
308 lines
228 lines
220 lines
228 lines
220 lines
228 lines
220 lines
228 lines
220 lines
150 lines
150 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
150 lines
150 lines
150 lines
150 lines
150 lines
150 lines
150 lines
150 lines
45 lines
45 lines
45 lines
45 lines
45 lines
45 lines
45 lines
45 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
75 lines
150 lines
150 lines
150 lines
150 lines
150 lines
150 lines
75 lines
75 lines
75 lines
75 lines
60 lines
75 lines
150 lines
150 lines
90 lines
90 lines
162 lines
162 lines
45 lines
45 lines
75 lines
75 lines
75 lines
75 lines
45 lines
45 lines
45 lines
45 lines
45 lines
45 lines
45 lines
45 lines
90 lines
90 lines
90 lines
90 lines
162 lines
162 lines
185 lines
185 lines
197 lines
197 lines
670 lines
670 lines
2316 lines
3318 lines
185 lines
185 lines
645 lines
700 lines
14 lines
774 lines
14 lines
840 lines
645 lines
700 lines
670 lines
670 lines
2316 lines
3318 lines
228 lines
220 lines
228 lines
220 lines
228 lines
220 lines
228 lines
220 lines
228 lines
220 lines
228 lines
220 lines
228 lines
220 lines
228 lines
220 lines
210 lines
210 lines
210 lines
210 lines
210 lines
210 lines
210 lines
210 lines
316 lines
308 lines
228 lines
220 lines
228 lines
220 lines
462 lines
518 lines
150 lines
150 lines
206 lines
198 lines
316 lines
308 lines
316 lines
308 lines
10 lines
120 lines
140 lines
102 lines
110 lines
102 lines
110 lines
308 lines
308 lines
228 lines
220 lines
308 lines
308 lines
308 lines
308 lines
272 lines
308 lines
272 lines
308 lines
228 lines
220 lines
228 lines
220 lines
316 lines
308 lines
150 lines
150 lines
270 lines
270 lines
150 lines
150 lines
270 lines
270 lines
150 lines
150 lines
150 lines
150 lines
150 lines
150 lines
150 lines
150 lines
270 lines
270 lines
150 lines
150 lines
270 lines
270 lines
316 lines
308 lines
140 lines
140 lines
26 lines

Diff 374764

llvm/include/llvm/IR/IntrinsicsRISCV.td

Show First 20 Lines • Show All 153 Lines • ▼ Show 20 Lineslet TargetPrefix = "riscv" in {
// NOTE: We model this with default memory properties since we model writing // NOTE: We model this with default memory properties since we model writing
// VL as a side effect. IntrReadMem, IntrHasSideEffects does not work. // VL as a side effect. IntrReadMem, IntrHasSideEffects does not work.
class RISCVUSLoadFF class RISCVUSLoadFF
: Intrinsic<[llvm_anyvector_ty, llvm_anyint_ty], : Intrinsic<[llvm_anyvector_ty, llvm_anyint_ty],
[LLVMPointerType<LLVMMatchType<0>>, LLVMMatchType<1>], [LLVMPointerType<LLVMMatchType<0>>, LLVMMatchType<1>],
[NoCapture<ArgIndex<0>>]>, [NoCapture<ArgIndex<0>>]>,
RISCVVIntrinsic; RISCVVIntrinsic;
// For unit stride load with mask // For unit stride load with mask
// Input: (maskedoff, pointer, mask, vl) // Input: (maskedoff, pointer, mask, vl, ta)
khchenUnsubmitted
Not Done
ReplyInline Actions

maybe we could have another NFC patch to update those argument info comments.

khchen: maybe we could have another NFC patch to update those `argument info` comments.
HsiangKaiAuthorUnsubmitted
Done
ReplyInline Actions

I will update these comments. Thanks.

HsiangKai: I will update these comments. Thanks.
class RISCVUSLoadMask class RISCVUSLoadMask
: Intrinsic<[llvm_anyvector_ty ], : Intrinsic<[llvm_anyvector_ty ],
[LLVMMatchType<0>, [LLVMMatchType<0>,
LLVMPointerType<LLVMMatchType<0>>, LLVMPointerType<LLVMMatchType<0>>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
llvm_anyint_ty], llvm_anyint_ty, LLVMMatchType<1>],
[NoCapture<ArgIndex<1>>, IntrReadMem]>, RISCVVIntrinsic; [NoCapture<ArgIndex<1>>, ImmArg<ArgIndex<4>>, IntrReadMem]>,
RISCVVIntrinsic;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

This needs ImmArg<ArgIndex<4>>. Similar for the rest. That will guarantee it is a constant and will make SelectionDAGBuilder create a target constant instead of a regular constant.

craig.topper: This needs ImmArg<ArgIndex<4>>. Similar for the rest. That will guarantee it is a constant and…
// For unit stride fault-only-first load with mask // For unit stride fault-only-first load with mask
// Input: (maskedoff, pointer, mask, vl) // Input: (maskedoff, pointer, mask, vl, ta)
// Output: (data, vl) // Output: (data, vl)
// NOTE: We model this with default memory properties since we model writing // NOTE: We model this with default memory properties since we model writing
// VL as a side effect. IntrReadMem, IntrHasSideEffects does not work. // VL as a side effect. IntrReadMem, IntrHasSideEffects does not work.
class RISCVUSLoadFFMask class RISCVUSLoadFFMask
: Intrinsic<[llvm_anyvector_ty, llvm_anyint_ty], : Intrinsic<[llvm_anyvector_ty, llvm_anyint_ty],
[LLVMMatchType<0>, [LLVMMatchType<0>,
LLVMPointerType<LLVMMatchType<0>>, LLVMPointerType<LLVMMatchType<0>>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
LLVMMatchType<1>], LLVMMatchType<1>, LLVMMatchType<1>],
[NoCapture<ArgIndex<1>>]>, RISCVVIntrinsic; [NoCapture<ArgIndex<1>>, ImmArg<ArgIndex<4>>]>, RISCVVIntrinsic;
// For strided load // For strided load
// Input: (pointer, stride, vl) // Input: (pointer, stride, vl)
class RISCVSLoad class RISCVSLoad
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMPointerType<LLVMMatchType<0>>, [LLVMPointerType<LLVMMatchType<0>>,
llvm_anyint_ty, LLVMMatchType<1>], llvm_anyint_ty, LLVMMatchType<1>],
[NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic; [NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic;
// For strided load with mask // For strided load with mask
// Input: (maskedoff, pointer, stride, mask, vl) // Input: (maskedoff, pointer, stride, mask, vl, ta)
class RISCVSLoadMask class RISCVSLoadMask
: Intrinsic<[llvm_anyvector_ty ], : Intrinsic<[llvm_anyvector_ty ],
[LLVMMatchType<0>, [LLVMMatchType<0>,
LLVMPointerType<LLVMMatchType<0>>, llvm_anyint_ty, LLVMPointerType<LLVMMatchType<0>>, llvm_anyint_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMMatchType<1>], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMMatchType<1>,
[NoCapture<ArgIndex<1>>, IntrReadMem]>, RISCVVIntrinsic; LLVMMatchType<1>],
[NoCapture<ArgIndex<1>>, ImmArg<ArgIndex<5>>, IntrReadMem]>,
RISCVVIntrinsic;
// For indexed load // For indexed load
// Input: (pointer, index, vl) // Input: (pointer, index, vl)
class RISCVILoad class RISCVILoad
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMPointerType<LLVMMatchType<0>>, [LLVMPointerType<LLVMMatchType<0>>,
llvm_anyvector_ty, llvm_anyint_ty], llvm_anyvector_ty, llvm_anyint_ty],
[NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic; [NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic;
// For indexed load with mask // For indexed load with mask
// Input: (maskedoff, pointer, index, mask, vl) // Input: (maskedoff, pointer, index, mask, vl, ta)
class RISCVILoadMask class RISCVILoadMask
: Intrinsic<[llvm_anyvector_ty ], : Intrinsic<[llvm_anyvector_ty ],
[LLVMMatchType<0>, [LLVMMatchType<0>,
LLVMPointerType<LLVMMatchType<0>>, llvm_anyvector_ty, LLVMPointerType<LLVMMatchType<0>>, llvm_anyvector_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[NoCapture<ArgIndex<1>>, IntrReadMem]>, RISCVVIntrinsic; LLVMMatchType<2>],
[NoCapture<ArgIndex<1>>, ImmArg<ArgIndex<5>>, IntrReadMem]>,
RISCVVIntrinsic;
// For unit stride store // For unit stride store
// Input: (vector_in, pointer, vl) // Input: (vector_in, pointer, vl)
class RISCVUSStore class RISCVUSStore
: Intrinsic<[], : Intrinsic<[],
[llvm_anyvector_ty, [llvm_anyvector_ty,
LLVMPointerType<LLVMMatchType<0>>, LLVMPointerType<LLVMMatchType<0>>,
llvm_anyint_ty], llvm_anyint_ty],
[NoCapture<ArgIndex<1>>, IntrWriteMem]>, RISCVVIntrinsic; [NoCapture<ArgIndex<1>>, IntrWriteMem]>, RISCVVIntrinsic;
Show All 40 Lines class RISCVIStoreMask
[NoCapture<ArgIndex<1>>, IntrWriteMem]>, RISCVVIntrinsic; [NoCapture<ArgIndex<1>>, IntrWriteMem]>, RISCVVIntrinsic;
// For destination vector type is the same as source vector. // For destination vector type is the same as source vector.
// Input: (vector_in, vl) // Input: (vector_in, vl)
class RISCVUnaryAANoMask class RISCVUnaryAANoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_anyint_ty], [LLVMMatchType<0>, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is the same as first source vector (with mask). // For destination vector type is the same as first source vector (with mask).
// Input: (vector_in, mask, vl) // Input: (vector_in, mask, vl, ta)
class RISCVUnaryAAMask class RISCVUnaryAAMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, [LLVMMatchType<0>, LLVMMatchType<0>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
LLVMMatchType<1>],
[ImmArg<ArgIndex<4>>, IntrNoMem]>, RISCVVIntrinsic;
class RISCVUnaryAAMaskNoTA
: Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is the same as first and second source vector. // For destination vector type is the same as first and second source vector.
// Input: (vector_in, vector_in, vl) // Input: (vector_in, vector_in, vl)
class RISCVBinaryAAANoMask class RISCVBinaryAAANoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, llvm_anyint_ty], [LLVMMatchType<0>, LLVMMatchType<0>, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is the same as first and second source vector. // For destination vector type is the same as first and second source vector.
// Input: (vector_in, int_vector_in, vl) // Input: (vector_in, int_vector_in, vl)
class RISCVRGatherVVNoMask class RISCVRGatherVVNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMVectorOfBitcastsToInt<0>, llvm_anyint_ty], [LLVMMatchType<0>, LLVMVectorOfBitcastsToInt<0>, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is the same as first and second source vector. // For destination vector type is the same as first and second source vector.
// Input: (vector_in, vector_in, int_vector_in, vl) // Input: (vector_in, vector_in, int_vector_in, vl, ta)
class RISCVRGatherVVMask class RISCVRGatherVVMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, LLVMVectorOfBitcastsToInt<0>, [LLVMMatchType<0>, LLVMMatchType<0>, LLVMVectorOfBitcastsToInt<0>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem]>, RISCVVIntrinsic; LLVMMatchType<1>],
[ImmArg<ArgIndex<5>>, IntrNoMem]>, RISCVVIntrinsic;
// Input: (vector_in, int16_vector_in, vl) // Input: (vector_in, int16_vector_in, vl)
class RISCVRGatherEI16VVNoMask class RISCVRGatherEI16VVNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMScalarOrSameVectorWidth<0, llvm_i16_ty>, [LLVMMatchType<0>, LLVMScalarOrSameVectorWidth<0, llvm_i16_ty>,
llvm_anyint_ty], llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is the same as first and second source vector. // For destination vector type is the same as first and second source vector.
// Input: (vector_in, vector_in, int16_vector_in, vl) // Input: (vector_in, vector_in, int16_vector_in, vl, ta)
class RISCVRGatherEI16VVMask class RISCVRGatherEI16VVMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, [LLVMMatchType<0>, LLVMMatchType<0>,
LLVMScalarOrSameVectorWidth<0, llvm_i16_ty>, LLVMScalarOrSameVectorWidth<0, llvm_i16_ty>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem]>, RISCVVIntrinsic; LLVMMatchType<1>],
[ImmArg<ArgIndex<5>>, IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is the same as first source vector, and the // For destination vector type is the same as first source vector, and the
// second operand is XLen. // second operand is XLen.
// Input: (vector_in, xlen_in, vl) // Input: (vector_in, xlen_in, vl)
class RISCVGatherVXNoMask class RISCVGatherVXNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_anyint_ty, LLVMMatchType<1>], [LLVMMatchType<0>, llvm_anyint_ty, LLVMMatchType<1>],
[IntrNoMem]>, RISCVVIntrinsic { [IntrNoMem]>, RISCVVIntrinsic {
} }
// For destination vector type is the same as first source vector (with mask). // For destination vector type is the same as first source vector (with mask).
// Second operand is XLen. // Second operand is XLen.
// Input: (maskedoff, vector_in, xlen_in, mask, vl) // Input: (maskedoff, vector_in, xlen_in, mask, vl, ta)
class RISCVGatherVXMask class RISCVGatherVXMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, llvm_anyint_ty, [LLVMMatchType<0>, LLVMMatchType<0>, llvm_anyint_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMMatchType<1>], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMMatchType<1>,
[IntrNoMem]>, RISCVVIntrinsic { LLVMMatchType<1>],
[ImmArg<ArgIndex<5>>, IntrNoMem]>, RISCVVIntrinsic {
} }
// For destination vector type is the same as first source vector. // For destination vector type is the same as first source vector.
// Input: (vector_in, vector_in/scalar_in, vl) // Input: (vector_in, vector_in/scalar_in, vl)
class RISCVBinaryAAXNoMask class RISCVBinaryAAXNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_any_ty, llvm_anyint_ty], [LLVMMatchType<0>, llvm_any_ty, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic { [IntrNoMem]>, RISCVVIntrinsic {
let SplatOperand = 2; let SplatOperand = 2;
} }
// For destination vector type is the same as first source vector (with mask). // For destination vector type is the same as first source vector (with mask).
// Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl) // Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl, ta)
class RISCVBinaryAAXMask class RISCVBinaryAAXMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, llvm_any_ty, [LLVMMatchType<0>, LLVMMatchType<0>, llvm_any_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem]>, RISCVVIntrinsic { LLVMMatchType<2>],
[ImmArg<ArgIndex<5>>, IntrNoMem]>, RISCVVIntrinsic {
let SplatOperand = 3; let SplatOperand = 3;
} }
// For destination vector type is the same as first source vector. The // For destination vector type is the same as first source vector. The
// second source operand must match the destination type or be an XLen scalar. // second source operand must match the destination type or be an XLen scalar.
// Input: (vector_in, vector_in/scalar_in, vl) // Input: (vector_in, vector_in/scalar_in, vl)
class RISCVBinaryAAShiftNoMask class RISCVBinaryAAShiftNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_any_ty, llvm_anyint_ty], [LLVMMatchType<0>, llvm_any_ty, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is the same as first source vector (with mask). // For destination vector type is the same as first source vector (with mask).
// The second source operand must match the destination type or be an XLen scalar. // The second source operand must match the destination type or be an XLen scalar.
// Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl) // Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl, ta)
class RISCVBinaryAAShiftMask class RISCVBinaryAAShiftMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, llvm_any_ty, [LLVMMatchType<0>, LLVMMatchType<0>, llvm_any_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem]>, RISCVVIntrinsic; LLVMMatchType<2>],
[ImmArg<ArgIndex<5>>, IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is NOT the same as first source vector. // For destination vector type is NOT the same as first source vector.
// Input: (vector_in, vector_in/scalar_in, vl) // Input: (vector_in, vector_in/scalar_in, vl)
class RISCVBinaryABXNoMask class RISCVBinaryABXNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[llvm_anyvector_ty, llvm_any_ty, llvm_anyint_ty], [llvm_anyvector_ty, llvm_any_ty, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic { [IntrNoMem]>, RISCVVIntrinsic {
let SplatOperand = 2; let SplatOperand = 2;
} }
// For destination vector type is NOT the same as first source vector (with mask). // For destination vector type is NOT the same as first source vector (with mask).
// Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl) // Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl, ta)
class RISCVBinaryABXMask class RISCVBinaryABXMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_anyvector_ty, llvm_any_ty, [LLVMMatchType<0>, llvm_anyvector_ty, llvm_any_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem]>, RISCVVIntrinsic { LLVMMatchType<3>],
[ImmArg<ArgIndex<5>>, IntrNoMem]>, RISCVVIntrinsic {
let SplatOperand = 3; let SplatOperand = 3;
} }
// For destination vector type is NOT the same as first source vector. The // For destination vector type is NOT the same as first source vector. The
// second source operand must match the destination type or be an XLen scalar. // second source operand must match the destination type or be an XLen scalar.
// Input: (vector_in, vector_in/scalar_in, vl) // Input: (vector_in, vector_in/scalar_in, vl)
class RISCVBinaryABShiftNoMask class RISCVBinaryABShiftNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[llvm_anyvector_ty, llvm_any_ty, llvm_anyint_ty], [llvm_anyvector_ty, llvm_any_ty, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is NOT the same as first source vector (with mask). // For destination vector type is NOT the same as first source vector (with mask).
// The second source operand must match the destination type or be an XLen scalar. // The second source operand must match the destination type or be an XLen scalar.
// Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl) // Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl, ta)
class RISCVBinaryABShiftMask class RISCVBinaryABShiftMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_anyvector_ty, llvm_any_ty, [LLVMMatchType<0>, llvm_anyvector_ty, llvm_any_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem]>, RISCVVIntrinsic; LLVMMatchType<3>],
[ImmArg<ArgIndex<5>>, IntrNoMem]>, RISCVVIntrinsic;
// For binary operations with V0 as input. // For binary operations with V0 as input.
// Input: (vector_in, vector_in/scalar_in, V0, vl) // Input: (vector_in, vector_in/scalar_in, V0, vl)
class RISCVBinaryWithV0 class RISCVBinaryWithV0
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_any_ty, [LLVMMatchType<0>, llvm_any_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
llvm_anyint_ty], llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic { [IntrNoMem]>, RISCVVIntrinsic {
▲ Show 20 Lines • Show All 60 Lines • ▼ Show 20 Lineslet TargetPrefix = "riscv" in {
class RISCVSaturatingBinaryAAXNoMask class RISCVSaturatingBinaryAAXNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_any_ty, llvm_anyint_ty], [LLVMMatchType<0>, llvm_any_ty, llvm_anyint_ty],
[IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic { [IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic {
let SplatOperand = 2; let SplatOperand = 2;
} }
// For Saturating binary operations with mask. // For Saturating binary operations with mask.
// The destination vector type is the same as first source vector. // The destination vector type is the same as first source vector.
// Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl) // Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl, ta)
class RISCVSaturatingBinaryAAXMask class RISCVSaturatingBinaryAAXMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, llvm_any_ty, [LLVMMatchType<0>, LLVMMatchType<0>, llvm_any_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic { LLVMMatchType<2>],
[ImmArg<ArgIndex<5>>, IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic {
let SplatOperand = 3; let SplatOperand = 3;
} }
// For Saturating binary operations. // For Saturating binary operations.
// The destination vector type is the same as first source vector. // The destination vector type is the same as first source vector.
// The second source operand matches the destination type or is an XLen scalar. // The second source operand matches the destination type or is an XLen scalar.
// Input: (vector_in, vector_in/scalar_in, vl) // Input: (vector_in, vector_in/scalar_in, vl)
class RISCVSaturatingBinaryAAShiftNoMask class RISCVSaturatingBinaryAAShiftNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_any_ty, llvm_anyint_ty], [LLVMMatchType<0>, llvm_any_ty, llvm_anyint_ty],
[IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic; [IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic;
// For Saturating binary operations with mask. // For Saturating binary operations with mask.
// The destination vector type is the same as first source vector. // The destination vector type is the same as first source vector.
// The second source operand matches the destination type or is an XLen scalar. // The second source operand matches the destination type or is an XLen scalar.
// Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl) // Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl, ta)
class RISCVSaturatingBinaryAAShiftMask class RISCVSaturatingBinaryAAShiftMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, llvm_any_ty, [LLVMMatchType<0>, LLVMMatchType<0>, llvm_any_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic; LLVMMatchType<2>],
[ImmArg<ArgIndex<5>>, IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic;
// For Saturating binary operations. // For Saturating binary operations.
// The destination vector type is NOT the same as first source vector. // The destination vector type is NOT the same as first source vector.
// The second source operand matches the destination type or is an XLen scalar. // The second source operand matches the destination type or is an XLen scalar.
// Input: (vector_in, vector_in/scalar_in, vl) // Input: (vector_in, vector_in/scalar_in, vl)
class RISCVSaturatingBinaryABShiftNoMask class RISCVSaturatingBinaryABShiftNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[llvm_anyvector_ty, llvm_any_ty, llvm_anyint_ty], [llvm_anyvector_ty, llvm_any_ty, llvm_anyint_ty],
[IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic; [IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic;
// For Saturating binary operations with mask. // For Saturating binary operations with mask.
// The destination vector type is NOT the same as first source vector (with mask). // The destination vector type is NOT the same as first source vector (with mask).
// The second source operand matches the destination type or is an XLen scalar. // The second source operand matches the destination type or is an XLen scalar.
// Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl) // Input: (maskedoff, vector_in, vector_in/scalar_in, mask, vl, ta)
class RISCVSaturatingBinaryABShiftMask class RISCVSaturatingBinaryABShiftMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_anyvector_ty, llvm_any_ty, [LLVMMatchType<0>, llvm_anyvector_ty, llvm_any_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic; LLVMMatchType<3>],
[ImmArg<ArgIndex<5>>, IntrNoMem, IntrHasSideEffects]>, RISCVVIntrinsic;
class RISCVTernaryAAAXNoMask class RISCVTernaryAAAXNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, llvm_anyint_ty, [LLVMMatchType<0>, LLVMMatchType<0>, llvm_anyint_ty,
LLVMMatchType<1>], LLVMMatchType<1>],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
class RISCVTernaryAAAXMask class RISCVTernaryAAAXMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, LLVMMatchType<0>, llvm_anyint_ty, [LLVMMatchType<0>, LLVMMatchType<0>, llvm_anyint_ty,
▲ Show 20 Lines • Show All 60 Lines • ▼ Show 20 Lines class RISCVMaskUnarySOutMask
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is NOT the same as source vector. // For destination vector type is NOT the same as source vector.
// Input: (vector_in, vl) // Input: (vector_in, vl)
class RISCVUnaryABNoMask class RISCVUnaryABNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[llvm_anyvector_ty, llvm_anyint_ty], [llvm_anyvector_ty, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For destination vector type is NOT the same as source vector (with mask). // For destination vector type is NOT the same as source vector (with mask).
// Input: (maskedoff, vector_in, mask, vl) // Input: (maskedoff, vector_in, mask, vl, ta)
class RISCVUnaryABMask class RISCVUnaryABMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_anyvector_ty, [LLVMMatchType<0>, llvm_anyvector_ty,
LLVMScalarOrSameVectorWidth<1, llvm_i1_ty>, LLVMScalarOrSameVectorWidth<1, llvm_i1_ty>,
llvm_anyint_ty], llvm_anyint_ty, LLVMMatchType<2>],
[IntrNoMem]>, RISCVVIntrinsic; [ImmArg<ArgIndex<4>>, IntrNoMem]>, RISCVVIntrinsic;
// For unary operations with the same vector type in/out without mask // For unary operations with the same vector type in/out without mask
// Output: (vector) // Output: (vector)
// Input: (vector_in, vl) // Input: (vector_in, vl)
class RISCVUnaryNoMask class RISCVUnaryNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_anyint_ty], [LLVMMatchType<0>, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For mask unary operations with mask type in/out with mask // For mask unary operations with mask type in/out with mask
Show All 12 Lines class RISCVNullaryIntrinsic
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For Conversion unary operations. // For Conversion unary operations.
// Input: (vector_in, vl) // Input: (vector_in, vl)
class RISCVConversionNoMask class RISCVConversionNoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[llvm_anyvector_ty, llvm_anyint_ty], [llvm_anyvector_ty, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
// For Conversion unary operations with mask. // For Conversion unary operations with mask.
// Input: (maskedoff, vector_in, mask, vl) // Input: (maskedoff, vector_in, mask, vl, ta)
class RISCVConversionMask class RISCVConversionMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_anyvector_ty, [LLVMMatchType<0>, llvm_anyvector_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty,
[IntrNoMem]>, RISCVVIntrinsic; LLVMMatchType<2>],
[ImmArg<ArgIndex<4>>, IntrNoMem]>, RISCVVIntrinsic;
// For atomic operations without mask // For atomic operations without mask
// Input: (base, index, value, vl) // Input: (base, index, value, vl)
class RISCVAMONoMask class RISCVAMONoMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMPointerType<LLVMMatchType<0>>, llvm_anyvector_ty, LLVMMatchType<0>, [LLVMPointerType<LLVMMatchType<0>>, llvm_anyvector_ty, LLVMMatchType<0>,
llvm_anyint_ty], llvm_anyint_ty],
[NoCapture<ArgIndex<0>>]>, RISCVVIntrinsic; [NoCapture<ArgIndex<0>>]>, RISCVVIntrinsic;
// For atomic operations with mask // For atomic operations with mask
// Input: (base, index, value, mask, vl) // Input: (base, index, value, mask, vl)
class RISCVAMOMask class RISCVAMOMask
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMPointerType<LLVMMatchType<0>>, llvm_anyvector_ty, LLVMMatchType<0>, [LLVMPointerType<LLVMMatchType<0>>, llvm_anyvector_ty, LLVMMatchType<0>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty], LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, llvm_anyint_ty],
[NoCapture<ArgIndex<0>>]>, RISCVVIntrinsic; [NoCapture<ArgIndex<0>>]>, RISCVVIntrinsic;
// For unit stride segment load // For unit stride segment load
// Input: (pointer, vl) // Input: (pointer, vl)
class RISCVUSSegLoad<int nf> class RISCVUSSegLoad<int nf>
: Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>, : Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>,
!add(nf, -1))), !add(nf, -1))),
[LLVMPointerToElt<0>, llvm_anyint_ty], [LLVMPointerToElt<0>, llvm_anyint_ty],
[NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic; [NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic;
// For unit stride segment load with mask // For unit stride segment load with mask
// Input: (maskedoff, pointer, mask, vl) // Input: (maskedoff, pointer, mask, vl, ta)
class RISCVUSSegLoadMask<int nf> class RISCVUSSegLoadMask<int nf>
: Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>, : Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>,
!add(nf, -1))), !add(nf, -1))),
!listconcat(!listsplat(LLVMMatchType<0>, nf), !listconcat(!listsplat(LLVMMatchType<0>, nf),
[LLVMPointerToElt<0>, [LLVMPointerToElt<0>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
llvm_anyint_ty]), llvm_anyint_ty, LLVMMatchType<1>]),
[NoCapture<ArgIndex<nf>>, IntrReadMem]>, RISCVVIntrinsic; [ImmArg<ArgIndex<!add(nf, 3)>>, NoCapture<ArgIndex<nf>>, IntrReadMem]>,
RISCVVIntrinsic;
// For unit stride fault-only-first segment load // For unit stride fault-only-first segment load
// Input: (pointer, vl) // Input: (pointer, vl)
// Output: (data, vl) // Output: (data, vl)
// NOTE: We model this with default memory properties since we model writing // NOTE: We model this with default memory properties since we model writing
// VL as a side effect. IntrReadMem, IntrHasSideEffects does not work. // VL as a side effect. IntrReadMem, IntrHasSideEffects does not work.
class RISCVUSSegLoadFF<int nf> class RISCVUSSegLoadFF<int nf>
: Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>, : Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>,
!add(nf, -1)), [llvm_anyint_ty]), !add(nf, -1)), [llvm_anyint_ty]),
[LLVMPointerToElt<0>, LLVMMatchType<1>], [LLVMPointerToElt<0>, LLVMMatchType<1>],
[NoCapture<ArgIndex<0>>]>, RISCVVIntrinsic; [NoCapture<ArgIndex<0>>]>, RISCVVIntrinsic;
// For unit stride fault-only-first segment load with mask // For unit stride fault-only-first segment load with mask
// Input: (maskedoff, pointer, mask, vl) // Input: (maskedoff, pointer, mask, vl, ta)
// Output: (data, vl) // Output: (data, vl)
// NOTE: We model this with default memory properties since we model writing // NOTE: We model this with default memory properties since we model writing
// VL as a side effect. IntrReadMem, IntrHasSideEffects does not work. // VL as a side effect. IntrReadMem, IntrHasSideEffects does not work.
class RISCVUSSegLoadFFMask<int nf> class RISCVUSSegLoadFFMask<int nf>
: Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>, : Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>,
!add(nf, -1)), [llvm_anyint_ty]), !add(nf, -1)), [llvm_anyint_ty]),
!listconcat(!listsplat(LLVMMatchType<0>, nf), !listconcat(!listsplat(LLVMMatchType<0>, nf),
[LLVMPointerToElt<0>, [LLVMPointerToElt<0>,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
LLVMMatchType<1>]), LLVMMatchType<1>, LLVMMatchType<1>]),
[NoCapture<ArgIndex<nf>>]>, RISCVVIntrinsic; [ImmArg<ArgIndex<!add(nf, 3)>>, NoCapture<ArgIndex<nf>>]>,
RISCVVIntrinsic;
// For stride segment load // For stride segment load
// Input: (pointer, offset, vl) // Input: (pointer, offset, vl)
class RISCVSSegLoad<int nf> class RISCVSSegLoad<int nf>
: Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>, : Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>,
!add(nf, -1))), !add(nf, -1))),
[LLVMPointerToElt<0>, llvm_anyint_ty, LLVMMatchType<1>], [LLVMPointerToElt<0>, llvm_anyint_ty, LLVMMatchType<1>],
[NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic; [NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic;
// For stride segment load with mask // For stride segment load with mask
// Input: (maskedoff, pointer, offset, mask, vl) // Input: (maskedoff, pointer, offset, mask, vl, ta)
class RISCVSSegLoadMask<int nf> class RISCVSSegLoadMask<int nf>
: Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>, : Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>,
!add(nf, -1))), !add(nf, -1))),
!listconcat(!listsplat(LLVMMatchType<0>, nf), !listconcat(!listsplat(LLVMMatchType<0>, nf),
[LLVMPointerToElt<0>, [LLVMPointerToElt<0>,
llvm_anyint_ty, llvm_anyint_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
LLVMMatchType<1>]), LLVMMatchType<1>, LLVMMatchType<1>]),
[NoCapture<ArgIndex<nf>>, IntrReadMem]>, RISCVVIntrinsic; [ImmArg<ArgIndex<!add(nf, 4)>>, NoCapture<ArgIndex<nf>>, IntrReadMem]>,
RISCVVIntrinsic;
// For indexed segment load // For indexed segment load
// Input: (pointer, index, vl) // Input: (pointer, index, vl)
class RISCVISegLoad<int nf> class RISCVISegLoad<int nf>
: Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>, : Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>,
!add(nf, -1))), !add(nf, -1))),
[LLVMPointerToElt<0>, llvm_anyvector_ty, llvm_anyint_ty], [LLVMPointerToElt<0>, llvm_anyvector_ty, llvm_anyint_ty],
[NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic; [NoCapture<ArgIndex<0>>, IntrReadMem]>, RISCVVIntrinsic;
// For indexed segment load with mask // For indexed segment load with mask
// Input: (maskedoff, pointer, index, mask, vl) // Input: (maskedoff, pointer, index, mask, vl, ta)
class RISCVISegLoadMask<int nf> class RISCVISegLoadMask<int nf>
: Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>, : Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>,
!add(nf, -1))), !add(nf, -1))),
!listconcat(!listsplat(LLVMMatchType<0>, nf), !listconcat(!listsplat(LLVMMatchType<0>, nf),
[LLVMPointerToElt<0>, [LLVMPointerToElt<0>,
llvm_anyvector_ty, llvm_anyvector_ty,
LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
llvm_anyint_ty]), llvm_anyint_ty, LLVMMatchType<2>]),
[NoCapture<ArgIndex<nf>>, IntrReadMem]>, RISCVVIntrinsic; [ImmArg<ArgIndex<!add(nf, 4)>>, NoCapture<ArgIndex<nf>>, IntrReadMem]>,
RISCVVIntrinsic;
// For unit stride segment store // For unit stride segment store
// Input: (value, pointer, vl) // Input: (value, pointer, vl)
class RISCVUSSegStore<int nf> class RISCVUSSegStore<int nf>
: Intrinsic<[], : Intrinsic<[],
!listconcat([llvm_anyvector_ty], !listconcat([llvm_anyvector_ty],
!listsplat(LLVMMatchType<0>, !add(nf, -1)), !listsplat(LLVMMatchType<0>, !add(nf, -1)),
[LLVMPointerToElt<0>, llvm_anyint_ty]), [LLVMPointerToElt<0>, llvm_anyint_ty]),
▲ Show 20 Lines • Show All 319 Lines • ▼ Show 20 Lineslet TargetPrefix = "riscv" in {
defm vfwadd_w : RISCVBinaryAAX; defm vfwadd_w : RISCVBinaryAAX;
defm vfwsub_w : RISCVBinaryAAX; defm vfwsub_w : RISCVBinaryAAX;
defm vsaddu : RISCVSaturatingBinaryAAX; defm vsaddu : RISCVSaturatingBinaryAAX;
defm vsadd : RISCVSaturatingBinaryAAX; defm vsadd : RISCVSaturatingBinaryAAX;
defm vssubu : RISCVSaturatingBinaryAAX; defm vssubu : RISCVSaturatingBinaryAAX;
defm vssub : RISCVSaturatingBinaryAAX; defm vssub : RISCVSaturatingBinaryAAX;
def int_riscv_vmerge : RISCVBinaryWithV0; defm vmerge : RISCVBinaryWithV0;
def int_riscv_vmv_v_v : Intrinsic<[llvm_anyvector_ty], def int_riscv_vmv_v_v : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, llvm_anyint_ty], [LLVMMatchType<0>, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
def int_riscv_vmv_v_x : Intrinsic<[llvm_anyint_ty], def int_riscv_vmv_v_x : Intrinsic<[llvm_anyint_ty],
[LLVMVectorElementType<0>, llvm_anyint_ty], [LLVMVectorElementType<0>, llvm_anyint_ty],
[IntrNoMem]>, RISCVVIntrinsic; [IntrNoMem]>, RISCVVIntrinsic;
def int_riscv_vfmv_v_f : Intrinsic<[llvm_anyfloat_ty], def int_riscv_vfmv_v_f : Intrinsic<[llvm_anyfloat_ty],
▲ Show 20 Lines • Show All 58 Lines • ▼ Show 20 Lines
defm vslide1down : RISCVBinaryAAX; defm vslide1down : RISCVBinaryAAX;
defm vfslide1up : RISCVBinaryAAX; defm vfslide1up : RISCVBinaryAAX;
defm vfslide1down : RISCVBinaryAAX; defm vfslide1down : RISCVBinaryAAX;
defm vrgather_vv : RISCVRGatherVV; defm vrgather_vv : RISCVRGatherVV;
defm vrgather_vx : RISCVRGatherVX; defm vrgather_vx : RISCVRGatherVX;
defm vrgatherei16_vv : RISCVRGatherEI16VV; defm vrgatherei16_vv : RISCVRGatherEI16VV;
def "int_riscv_vcompress" : RISCVUnaryAAMask; def "int_riscv_vcompress" : RISCVUnaryAAMaskNoTA;
defm vaaddu : RISCVSaturatingBinaryAAX; defm vaaddu : RISCVSaturatingBinaryAAX;
defm vaadd : RISCVSaturatingBinaryAAX; defm vaadd : RISCVSaturatingBinaryAAX;
defm vasubu : RISCVSaturatingBinaryAAX; defm vasubu : RISCVSaturatingBinaryAAX;
defm vasub : RISCVSaturatingBinaryAAX; defm vasub : RISCVSaturatingBinaryAAX;
defm vsmul : RISCVSaturatingBinaryAAX; defm vsmul : RISCVSaturatingBinaryAAX;
▲ Show 20 Lines • Show All 124 Lines • Show Last 20 Lines

llvm/lib/Target/RISCV/MCTargetDesc/RISCVBaseInfo.h

Show First 20 Lines • Show All 72 Lines • ▼ Show 20 Linesenum {
HasSEWOpMask = 1 << HasSEWOpShift, HasSEWOpMask = 1 << HasSEWOpShift,
// Does this instruction have a VL operand. It will be the second to last // Does this instruction have a VL operand. It will be the second to last
// explicit operand unless there is a vector policy operand. Used by RVV // explicit operand unless there is a vector policy operand. Used by RVV
// Pseudos. // Pseudos.
HasVLOpShift = HasSEWOpShift + 1, HasVLOpShift = HasSEWOpShift + 1,
HasVLOpMask = 1 << HasVLOpShift, HasVLOpMask = 1 << HasVLOpShift,
// Does this instruction have a vector policy operand. It will be the last // Does this instruction have a vector policy operand. It will be the last
craig.topperUnsubmitted
Not Done
ReplyInline Actions

This should be HasPolicyOpShift and HasPolicyOpMask to match SEWOp/VLOp naming.

craig.topper: This should be HasPolicyOpShift and HasPolicyOpMask to match SEWOp/VLOp naming.
// explicit operand. Used by RVV Pseudos. // explicit operand. Used by RVV Pseudos.
HasVecPolicyOpShift = HasVLOpShift + 1, HasVecPolicyOpShift = HasVLOpShift + 1,
HasVecPolicyOpMask = 1 << HasVecPolicyOpShift, HasVecPolicyOpMask = 1 << HasVecPolicyOpShift,
}; };
// Match with the definitions in RISCVInstrFormatsV.td // Match with the definitions in RISCVInstrFormatsV.td
enum VConstraintType { enum VConstraintType {
NoConstraint = 0, NoConstraint = 0,
VS2Constraint = 0b001, VS2Constraint = 0b001,
VS1Constraint = 0b010, VS1Constraint = 0b010,
VMConstraint = 0b100, VMConstraint = 0b100,
}; };
enum VLMUL : uint8_t { enum VLMUL : uint8_t {
LMUL_1 = 0, LMUL_1 = 0,
LMUL_2, LMUL_2,
LMUL_4, LMUL_4,
LMUL_8, LMUL_8,
LMUL_RESERVED, LMUL_RESERVED,
LMUL_F8, LMUL_F8,
LMUL_F4, LMUL_F4,
LMUL_F2 LMUL_F2
}; };
enum {
TAIL_UNDISTURBED = 0,
TAIL_AGNOSTIC = 1,
};
// Helper functions to read TSFlags. // Helper functions to read TSFlags.
/// \returns the format of the instruction. /// \returns the format of the instruction.
static inline unsigned getFormat(uint64_t TSFlags) { static inline unsigned getFormat(uint64_t TSFlags) {
return (TSFlags & InstFormatMask) >> InstFormatShift; return (TSFlags & InstFormatMask) >> InstFormatShift;
} }
/// \returns the constraint for the instruction. /// \returns the constraint for the instruction.
static inline VConstraintType getConstraint(uint64_t TSFlags) { static inline VConstraintType getConstraint(uint64_t TSFlags) {
return static_cast<VConstraintType> return static_cast<VConstraintType>
Show All 23 Lines
static inline bool hasVLOp(uint64_t TSFlags) { static inline bool hasVLOp(uint64_t TSFlags) {
return TSFlags & HasVLOpMask; return TSFlags & HasVLOpMask;
} }
/// \returns true if there is a vector policy operand for this instruction. /// \returns true if there is a vector policy operand for this instruction.
static inline bool hasVecPolicyOp(uint64_t TSFlags) { static inline bool hasVecPolicyOp(uint64_t TSFlags) {
return TSFlags & HasVecPolicyOpMask; return TSFlags & HasVecPolicyOpMask;
} }
// RISC-V Specific Machine Operand Flags // RISC-V Specific Machine Operand Flags
craig.topperUnsubmitted
Not Done
ReplyInline Actions

hasPolicyOp

craig.topper: hasPolicyOp
enum { enum {
MO_None = 0, MO_None = 0,
MO_CALL = 1, MO_CALL = 1,
MO_PLT = 2, MO_PLT = 2,
MO_LO = 3, MO_LO = 3,
MO_HI = 4, MO_HI = 4,
MO_PCREL_LO = 5, MO_PCREL_LO = 5,
MO_PCREL_HI = 6, MO_PCREL_HI = 6,
▲ Show 20 Lines • Show All 213 Lines • Show Last 20 Lines

llvm/lib/Target/RISCV/RISCVISelDAGToDAG.h

llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp

Show First 20 Lines • Show All 207 Lines • ▼ Show 20 Linesstatic SDValue createTuple(SelectionDAG &CurDAG, ArrayRef<SDValue> Regs,
case RISCVII::VLMUL::LMUL_4: case RISCVII::VLMUL::LMUL_4:
return createM4Tuple(CurDAG, Regs, NF); return createM4Tuple(CurDAG, Regs, NF);
} }
} }
void RISCVDAGToDAGISel::addVectorLoadStoreOperands( void RISCVDAGToDAGISel::addVectorLoadStoreOperands(
SDNode *Node, unsigned Log2SEW, const SDLoc &DL, unsigned CurOp, SDNode *Node, unsigned Log2SEW, const SDLoc &DL, unsigned CurOp,
bool IsMasked, bool IsStridedOrIndexed, SmallVectorImpl<SDValue> &Operands, bool IsMasked, bool IsStridedOrIndexed, SmallVectorImpl<SDValue> &Operands,
MVT *IndexVT) { bool IsLoad, MVT *IndexVT) {
SDValue Chain = Node->getOperand(0); SDValue Chain = Node->getOperand(0);
SDValue Glue; SDValue Glue;
SDValue Base; SDValue Base;
SelectBaseAddr(Node->getOperand(CurOp++), Base); SelectBaseAddr(Node->getOperand(CurOp++), Base);
Operands.push_back(Base); // Base pointer. Operands.push_back(Base); // Base pointer.
if (IsStridedOrIndexed) { if (IsStridedOrIndexed) {
Show All 12 Linesvoid RISCVDAGToDAGISel::addVectorLoadStoreOperands(
SDValue VL; SDValue VL;
selectVLOp(Node->getOperand(CurOp++), VL); selectVLOp(Node->getOperand(CurOp++), VL);
Operands.push_back(VL); Operands.push_back(VL);
MVT XLenVT = Subtarget->getXLenVT(); MVT XLenVT = Subtarget->getXLenVT();
SDValue SEWOp = CurDAG->getTargetConstant(Log2SEW, DL, XLenVT); SDValue SEWOp = CurDAG->getTargetConstant(Log2SEW, DL, XLenVT);
Operands.push_back(SEWOp); Operands.push_back(SEWOp);
// Masked load has the tail policy argument.
if (IsMasked && IsLoad) {
// Policy must be a constant.
uint64_t Policy = Node->getConstantOperandVal(CurOp++);
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Can we use

uint64_t Policy = Node->getConstantOperandVal(CurOp++);
craig.topper: Can we use ``` uint64_t Policy = Node->getConstantOperandVal(CurOp++); ```
SDValue PolicyOp = CurDAG->getTargetConstant(Policy, DL, XLenVT);
Operands.push_back(PolicyOp);
}
Operands.push_back(Chain); // Chain. Operands.push_back(Chain); // Chain.
if (Glue) if (Glue)
Operands.push_back(Glue); Operands.push_back(Glue);
} }
void RISCVDAGToDAGISel::selectVLSEG(SDNode *Node, bool IsMasked, void RISCVDAGToDAGISel::selectVLSEG(SDNode *Node, bool IsMasked,
bool IsStrided) { bool IsStrided) {
SDLoc DL(Node); SDLoc DL(Node);
unsigned NF = Node->getNumValues() - 1; unsigned NF = Node->getNumValues() - 1;
MVT VT = Node->getSimpleValueType(0); MVT VT = Node->getSimpleValueType(0);
unsigned Log2SEW = Log2_32(VT.getScalarSizeInBits()); unsigned Log2SEW = Log2_32(VT.getScalarSizeInBits());
RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT); RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT);
unsigned CurOp = 2; unsigned CurOp = 2;
SmallVector<SDValue, 8> Operands; SmallVector<SDValue, 8> Operands;
if (IsMasked) { if (IsMasked) {
SmallVector<SDValue, 8> Regs(Node->op_begin() + CurOp, SmallVector<SDValue, 8> Regs(Node->op_begin() + CurOp,
Node->op_begin() + CurOp + NF); Node->op_begin() + CurOp + NF);
SDValue MaskedOff = createTuple(*CurDAG, Regs, NF, LMUL); SDValue MaskedOff = createTuple(*CurDAG, Regs, NF, LMUL);
Operands.push_back(MaskedOff); Operands.push_back(MaskedOff);
CurOp += NF; CurOp += NF;
} }
addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, IsStrided, addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, IsStrided,
Operands); Operands, /*IsLoad=*/true);
const RISCV::VLSEGPseudo *P = const RISCV::VLSEGPseudo *P =
RISCV::getVLSEGPseudo(NF, IsMasked, IsStrided, /*FF*/ false, Log2SEW, RISCV::getVLSEGPseudo(NF, IsMasked, IsStrided, /*FF*/ false, Log2SEW,
static_cast<unsigned>(LMUL)); static_cast<unsigned>(LMUL));
MachineSDNode *Load = MachineSDNode *Load =
CurDAG->getMachineNode(P->Pseudo, DL, MVT::Untyped, MVT::Other, Operands); CurDAG->getMachineNode(P->Pseudo, DL, MVT::Untyped, MVT::Other, Operands);
if (auto *MemOp = dyn_cast<MemSDNode>(Node)) if (auto *MemOp = dyn_cast<MemSDNode>(Node))
Show All 24 Lines if (IsMasked) {
SmallVector<SDValue, 8> Regs(Node->op_begin() + CurOp, SmallVector<SDValue, 8> Regs(Node->op_begin() + CurOp,
Node->op_begin() + CurOp + NF); Node->op_begin() + CurOp + NF);
SDValue MaskedOff = createTuple(*CurDAG, Regs, NF, LMUL); SDValue MaskedOff = createTuple(*CurDAG, Regs, NF, LMUL);
Operands.push_back(MaskedOff); Operands.push_back(MaskedOff);
CurOp += NF; CurOp += NF;
} }
addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked,
/*IsStridedOrIndexed*/ false, Operands); /*IsStridedOrIndexed*/ false, Operands,
/*IsLoad=*/true);
const RISCV::VLSEGPseudo *P = const RISCV::VLSEGPseudo *P =
RISCV::getVLSEGPseudo(NF, IsMasked, /*Strided*/ false, /*FF*/ true, RISCV::getVLSEGPseudo(NF, IsMasked, /*Strided*/ false, /*FF*/ true,
Log2SEW, static_cast<unsigned>(LMUL)); Log2SEW, static_cast<unsigned>(LMUL));
MachineSDNode *Load = CurDAG->getMachineNode(P->Pseudo, DL, MVT::Untyped, MachineSDNode *Load = CurDAG->getMachineNode(P->Pseudo, DL, MVT::Untyped,
MVT::Other, MVT::Glue, Operands); MVT::Other, MVT::Glue, Operands);
SDNode *ReadVL = CurDAG->getMachineNode(RISCV::PseudoReadVL, DL, XLenVT, SDNode *ReadVL = CurDAG->getMachineNode(RISCV::PseudoReadVL, DL, XLenVT,
/*Glue*/ SDValue(Load, 2)); /*Glue*/ SDValue(Load, 2));
Show All 28 Lines SmallVector<SDValue, 8> Regs(Node->op_begin() + CurOp,
Node->op_begin() + CurOp + NF); Node->op_begin() + CurOp + NF);
SDValue MaskedOff = createTuple(*CurDAG, Regs, NF, LMUL); SDValue MaskedOff = createTuple(*CurDAG, Regs, NF, LMUL);
Operands.push_back(MaskedOff); Operands.push_back(MaskedOff);
CurOp += NF; CurOp += NF;
} }
MVT IndexVT; MVT IndexVT;
addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked,
/*IsStridedOrIndexed*/ true, Operands, &IndexVT); /*IsStridedOrIndexed*/ true, Operands,
/*IsLoad=*/true, &IndexVT);
assert(VT.getVectorElementCount() == IndexVT.getVectorElementCount() && assert(VT.getVectorElementCount() == IndexVT.getVectorElementCount() &&
"Element count mismatch"); "Element count mismatch");
RISCVII::VLMUL IndexLMUL = RISCVTargetLowering::getLMUL(IndexVT); RISCVII::VLMUL IndexLMUL = RISCVTargetLowering::getLMUL(IndexVT);
unsigned IndexLog2EEW = Log2_32(IndexVT.getScalarSizeInBits()); unsigned IndexLog2EEW = Log2_32(IndexVT.getScalarSizeInBits());
const RISCV::VLXSEGPseudo *P = RISCV::getVLXSEGPseudo( const RISCV::VLXSEGPseudo *P = RISCV::getVLXSEGPseudo(
NF, IsMasked, IsOrdered, IndexLog2EEW, static_cast<unsigned>(LMUL), NF, IsMasked, IsOrdered, IndexLog2EEW, static_cast<unsigned>(LMUL),
▲ Show 20 Lines • Show All 60 Lines • ▼ Show 20 Linesvoid RISCVDAGToDAGISel::selectVSXSEG(SDNode *Node, bool IsMasked,
SDValue StoreVal = createTuple(*CurDAG, Regs, NF, LMUL); SDValue StoreVal = createTuple(*CurDAG, Regs, NF, LMUL);
SmallVector<SDValue, 8> Operands; SmallVector<SDValue, 8> Operands;
Operands.push_back(StoreVal); Operands.push_back(StoreVal);
unsigned CurOp = 2 + NF; unsigned CurOp = 2 + NF;
MVT IndexVT; MVT IndexVT;
addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked,
/*IsStridedOrIndexed*/ true, Operands, &IndexVT); /*IsStridedOrIndexed*/ true, Operands,
/*IsLoad=*/false, &IndexVT);
assert(VT.getVectorElementCount() == IndexVT.getVectorElementCount() && assert(VT.getVectorElementCount() == IndexVT.getVectorElementCount() &&
"Element count mismatch"); "Element count mismatch");
RISCVII::VLMUL IndexLMUL = RISCVTargetLowering::getLMUL(IndexVT); RISCVII::VLMUL IndexLMUL = RISCVTargetLowering::getLMUL(IndexVT);
unsigned IndexLog2EEW = Log2_32(IndexVT.getScalarSizeInBits()); unsigned IndexLog2EEW = Log2_32(IndexVT.getScalarSizeInBits());
const RISCV::VSXSEGPseudo *P = RISCV::getVSXSEGPseudo( const RISCV::VSXSEGPseudo *P = RISCV::getVSXSEGPseudo(
NF, IsMasked, IsOrdered, IndexLog2EEW, static_cast<unsigned>(LMUL), NF, IsMasked, IsOrdered, IndexLog2EEW, static_cast<unsigned>(LMUL),
▲ Show 20 Lines • Show All 625 Lines • ▼ Show 20 Lines case Intrinsic::riscv_vluxei_mask: {
unsigned CurOp = 2; unsigned CurOp = 2;
SmallVector<SDValue, 8> Operands; SmallVector<SDValue, 8> Operands;
if (IsMasked) if (IsMasked)
Operands.push_back(Node->getOperand(CurOp++)); Operands.push_back(Node->getOperand(CurOp++));
MVT IndexVT; MVT IndexVT;
addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked,
/*IsStridedOrIndexed*/ true, Operands, /*IsStridedOrIndexed*/ true, Operands,
&IndexVT); /*IsLoad=*/true, &IndexVT);
assert(VT.getVectorElementCount() == IndexVT.getVectorElementCount() && assert(VT.getVectorElementCount() == IndexVT.getVectorElementCount() &&
"Element count mismatch"); "Element count mismatch");
RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT); RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT);
RISCVII::VLMUL IndexLMUL = RISCVTargetLowering::getLMUL(IndexVT); RISCVII::VLMUL IndexLMUL = RISCVTargetLowering::getLMUL(IndexVT);
unsigned IndexLog2EEW = Log2_32(IndexVT.getScalarSizeInBits()); unsigned IndexLog2EEW = Log2_32(IndexVT.getScalarSizeInBits());
const RISCV::VLX_VSXPseudo *P = RISCV::getVLXPseudo( const RISCV::VLX_VSXPseudo *P = RISCV::getVLXPseudo(
Show All 22 Lines case Intrinsic::riscv_vlse_mask: {
unsigned Log2SEW = Log2_32(VT.getScalarSizeInBits()); unsigned Log2SEW = Log2_32(VT.getScalarSizeInBits());
unsigned CurOp = 2; unsigned CurOp = 2;
SmallVector<SDValue, 8> Operands; SmallVector<SDValue, 8> Operands;
if (IsMasked) if (IsMasked)
Operands.push_back(Node->getOperand(CurOp++)); Operands.push_back(Node->getOperand(CurOp++));
addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, IsStrided, addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, IsStrided,
Operands); Operands, /*IsLoad=*/true);
RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT); RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT);
const RISCV::VLEPseudo *P = const RISCV::VLEPseudo *P =
RISCV::getVLEPseudo(IsMasked, IsStrided, /*FF*/ false, Log2SEW, RISCV::getVLEPseudo(IsMasked, IsStrided, /*FF*/ false, Log2SEW,
static_cast<unsigned>(LMUL)); static_cast<unsigned>(LMUL));
MachineSDNode *Load = MachineSDNode *Load =
CurDAG->getMachineNode(P->Pseudo, DL, Node->getVTList(), Operands); CurDAG->getMachineNode(P->Pseudo, DL, Node->getVTList(), Operands);
Show All 11 Lines case Intrinsic::riscv_vleff_mask: {
unsigned Log2SEW = Log2_32(VT.getScalarSizeInBits()); unsigned Log2SEW = Log2_32(VT.getScalarSizeInBits());
unsigned CurOp = 2; unsigned CurOp = 2;
SmallVector<SDValue, 7> Operands; SmallVector<SDValue, 7> Operands;
if (IsMasked) if (IsMasked)
Operands.push_back(Node->getOperand(CurOp++)); Operands.push_back(Node->getOperand(CurOp++));
addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked,
/*IsStridedOrIndexed*/ false, Operands); /*IsStridedOrIndexed*/ false, Operands,
/*IsLoad=*/true);
RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT); RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT);
const RISCV::VLEPseudo *P = const RISCV::VLEPseudo *P =
RISCV::getVLEPseudo(IsMasked, /*Strided*/ false, /*FF*/ true, Log2SEW, RISCV::getVLEPseudo(IsMasked, /*Strided*/ false, /*FF*/ true, Log2SEW,
static_cast<unsigned>(LMUL)); static_cast<unsigned>(LMUL));
MachineSDNode *Load = MachineSDNode *Load =
CurDAG->getMachineNode(P->Pseudo, DL, Node->getValueType(0), CurDAG->getMachineNode(P->Pseudo, DL, Node->getValueType(0),
MVT::Other, MVT::Glue, Operands); MVT::Other, MVT::Glue, Operands);
▲ Show 20 Lines • Show All 105 Lines • ▼ Show 20 Lines case Intrinsic::riscv_vsuxei_mask: {
unsigned CurOp = 2; unsigned CurOp = 2;
SmallVector<SDValue, 8> Operands; SmallVector<SDValue, 8> Operands;
Operands.push_back(Node->getOperand(CurOp++)); // Store value. Operands.push_back(Node->getOperand(CurOp++)); // Store value.
MVT IndexVT; MVT IndexVT;
addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked, addVectorLoadStoreOperands(Node, Log2SEW, DL, CurOp, IsMasked,
/*IsStridedOrIndexed*/ true, Operands, /*IsStridedOrIndexed*/ true, Operands,
&IndexVT); /*IsLoad=*/false, &IndexVT);
assert(VT.getVectorElementCount() == IndexVT.getVectorElementCount() && assert(VT.getVectorElementCount() == IndexVT.getVectorElementCount() &&
"Element count mismatch"); "Element count mismatch");
RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT); RISCVII::VLMUL LMUL = RISCVTargetLowering::getLMUL(VT);
RISCVII::VLMUL IndexLMUL = RISCVTargetLowering::getLMUL(IndexVT); RISCVII::VLMUL IndexLMUL = RISCVTargetLowering::getLMUL(IndexVT);
unsigned IndexLog2EEW = Log2_32(IndexVT.getScalarSizeInBits()); unsigned IndexLog2EEW = Log2_32(IndexVT.getScalarSizeInBits());
const RISCV::VLX_VSXPseudo *P = RISCV::getVSXPseudo( const RISCV::VLX_VSXPseudo *P = RISCV::getVSXPseudo(
▲ Show 20 Lines • Show All 666 Lines • Show Last 20 Lines

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 3,912 Lines • ▼ Show 20 Lines return DAG.getNode(RISCVISD::VSELECT_VL, DL, VT, SelectCond, SplattedVal,
Vec, VL); Vec, VL);
} }
case Intrinsic::riscv_vslide1up: case Intrinsic::riscv_vslide1up:
case Intrinsic::riscv_vslide1down: case Intrinsic::riscv_vslide1down:
case Intrinsic::riscv_vslide1up_mask: case Intrinsic::riscv_vslide1up_mask:
case Intrinsic::riscv_vslide1down_mask: { case Intrinsic::riscv_vslide1down_mask: {
// We need to special case these when the scalar is larger than XLen. // We need to special case these when the scalar is larger than XLen.
unsigned NumOps = Op.getNumOperands(); unsigned NumOps = Op.getNumOperands();
bool IsMasked = NumOps == 6; bool IsMasked = NumOps == 7;
unsigned OpOffset = IsMasked ? 1 : 0; unsigned OpOffset = IsMasked ? 1 : 0;
SDValue Scalar = Op.getOperand(2 + OpOffset); SDValue Scalar = Op.getOperand(2 + OpOffset);
if (Scalar.getValueType().bitsLE(XLenVT)) if (Scalar.getValueType().bitsLE(XLenVT))
break; break;
// Splatting a sign extended constant is fine. // Splatting a sign extended constant is fine.
if (auto *CVal = dyn_cast<ConstantSDNode>(Scalar)) if (auto *CVal = dyn_cast<ConstantSDNode>(Scalar))
if (isInt<32>(CVal->getSExtValue())) if (isInt<32>(CVal->getSExtValue()))
break; break;
MVT VT = Op.getSimpleValueType(); MVT VT = Op.getSimpleValueType();
assert(VT.getVectorElementType() == MVT::i64 && assert(VT.getVectorElementType() == MVT::i64 &&
Scalar.getValueType() == MVT::i64 && "Unexpected VTs"); Scalar.getValueType() == MVT::i64 && "Unexpected VTs");
// Convert the vector source to the equivalent nxvXi32 vector. // Convert the vector source to the equivalent nxvXi32 vector.
MVT I32VT = MVT::getVectorVT(MVT::i32, VT.getVectorElementCount() * 2); MVT I32VT = MVT::getVectorVT(MVT::i32, VT.getVectorElementCount() * 2);
SDValue Vec = DAG.getBitcast(I32VT, Op.getOperand(1 + OpOffset)); SDValue Vec = DAG.getBitcast(I32VT, Op.getOperand(1 + OpOffset));
SDValue ScalarLo = DAG.getNode(ISD::EXTRACT_ELEMENT, DL, MVT::i32, Scalar, SDValue ScalarLo = DAG.getNode(ISD::EXTRACT_ELEMENT, DL, MVT::i32, Scalar,
DAG.getConstant(0, DL, XLenVT)); DAG.getConstant(0, DL, XLenVT));
SDValue ScalarHi = DAG.getNode(ISD::EXTRACT_ELEMENT, DL, MVT::i32, Scalar, SDValue ScalarHi = DAG.getNode(ISD::EXTRACT_ELEMENT, DL, MVT::i32, Scalar,
DAG.getConstant(1, DL, XLenVT)); DAG.getConstant(1, DL, XLenVT));
// Double the VL since we halved SEW. // Double the VL since we halved SEW.
SDValue VL = Op.getOperand(NumOps - 1); SDValue VL = Op.getOperand(NumOps - (1 + OpOffset));
SDValue I32VL = SDValue I32VL =
DAG.getNode(ISD::SHL, DL, XLenVT, VL, DAG.getConstant(1, DL, XLenVT)); DAG.getNode(ISD::SHL, DL, XLenVT, VL, DAG.getConstant(1, DL, XLenVT));
MVT I32MaskVT = MVT::getVectorVT(MVT::i1, I32VT.getVectorElementCount()); MVT I32MaskVT = MVT::getVectorVT(MVT::i1, I32VT.getVectorElementCount());
SDValue I32Mask = DAG.getNode(RISCVISD::VMSET_VL, DL, I32MaskVT, VL); SDValue I32Mask = DAG.getNode(RISCVISD::VMSET_VL, DL, I32MaskVT, VL);
// Shift the two scalar parts in using SEW=32 slide1up/slide1down // Shift the two scalar parts in using SEW=32 slide1up/slide1down
// instructions. // instructions.
Show All 12 Lines case Intrinsic::riscv_vslide1down_mask: {
// Convert back to nxvXi64. // Convert back to nxvXi64.
Vec = DAG.getBitcast(VT, Vec); Vec = DAG.getBitcast(VT, Vec);
if (!IsMasked) if (!IsMasked)
return Vec; return Vec;
// Apply mask after the operation. // Apply mask after the operation.
SDValue Mask = Op.getOperand(NumOps - 2); SDValue Mask = Op.getOperand(NumOps - 3);
SDValue MaskedOff = Op.getOperand(1); SDValue MaskedOff = Op.getOperand(1);
return DAG.getNode(RISCVISD::VSELECT_VL, DL, VT, Mask, Vec, MaskedOff, VL); return DAG.getNode(RISCVISD::VSELECT_VL, DL, VT, Mask, Vec, MaskedOff, VL);
} }
} }
return lowerVectorIntrinsicSplats(Op, DAG, Subtarget); return lowerVectorIntrinsicSplats(Op, DAG, Subtarget);
} }
Show All 33 Lines case Intrinsic::riscv_masked_strided_load: {
SmallVector<SDValue, 8> Ops{Load->getChain(), IntID}; SmallVector<SDValue, 8> Ops{Load->getChain(), IntID};
if (!IsUnmasked) if (!IsUnmasked)
Ops.push_back(PassThru); Ops.push_back(PassThru);
Ops.push_back(Op.getOperand(3)); // Ptr Ops.push_back(Op.getOperand(3)); // Ptr
Ops.push_back(Op.getOperand(4)); // Stride Ops.push_back(Op.getOperand(4)); // Stride
if (!IsUnmasked) if (!IsUnmasked)
Ops.push_back(Mask); Ops.push_back(Mask);
Ops.push_back(VL); Ops.push_back(VL);
if (!IsUnmasked) {
SDValue Policy = DAG.getTargetConstant(RISCVII::TAIL_AGNOSTIC, DL, XLenVT);
Ops.push_back(Policy);
}
SDVTList VTs = DAG.getVTList({ContainerVT, MVT::Other}); SDVTList VTs = DAG.getVTList({ContainerVT, MVT::Other});
SDValue Result = SDValue Result =
DAG.getMemIntrinsicNode(ISD::INTRINSIC_W_CHAIN, DL, VTs, Ops, DAG.getMemIntrinsicNode(ISD::INTRINSIC_W_CHAIN, DL, VTs, Ops,
Load->getMemoryVT(), Load->getMemOperand()); Load->getMemoryVT(), Load->getMemOperand());
SDValue Chain = Result.getValue(1); SDValue Chain = Result.getValue(1);
Result = convertFromScalableVector(VT, Result, DAG, Subtarget); Result = convertFromScalableVector(VT, Result, DAG, Subtarget);
return DAG.getMergeValues({Result, Chain}, DL); return DAG.getMergeValues({Result, Chain}, DL);
▲ Show 20 Lines • Show All 819 Lines • ▼ Show 20 Lines if (VT.isFixedLengthVector()) {
PassThru = convertToScalableVector(ContainerVT, PassThru, DAG, Subtarget); PassThru = convertToScalableVector(ContainerVT, PassThru, DAG, Subtarget);
} }
if (!VL) if (!VL)
VL = getDefaultVLOps(VT, ContainerVT, DL, DAG, Subtarget).second; VL = getDefaultVLOps(VT, ContainerVT, DL, DAG, Subtarget).second;
SDVTList VTs = DAG.getVTList({ContainerVT, MVT::Other}); SDVTList VTs = DAG.getVTList({ContainerVT, MVT::Other});
SDValue IntID = DAG.getTargetConstant(Intrinsic::riscv_vle_mask, DL, XLenVT); SDValue IntID = DAG.getTargetConstant(Intrinsic::riscv_vle_mask, DL, XLenVT);
SDValue Ops[] = {Chain, IntID, PassThru, BasePtr, Mask, VL}; SDValue Policy = DAG.getTargetConstant(RISCVII::TAIL_AGNOSTIC, DL, XLenVT);
SDValue Ops[] = {Chain, IntID, PassThru, BasePtr, Mask, VL, Policy};
SDValue Result = SDValue Result =
DAG.getMemIntrinsicNode(ISD::INTRINSIC_W_CHAIN, DL, VTs, Ops, MemVT, MMO); DAG.getMemIntrinsicNode(ISD::INTRINSIC_W_CHAIN, DL, VTs, Ops, MemVT, MMO);
Chain = Result.getValue(1); Chain = Result.getValue(1);
if (VT.isFixedLengthVector()) if (VT.isFixedLengthVector())
Result = convertFromScalableVector(VT, Result, DAG, Subtarget); Result = convertFromScalableVector(VT, Result, DAG, Subtarget);
return DAG.getMergeValues({Result, Chain}, DL); return DAG.getMergeValues({Result, Chain}, DL);
▲ Show 20 Lines • Show All 329 Lines • ▼ Show 20 LinesSDValue RISCVTargetLowering::lowerMaskedGather(SDValue Op,
SmallVector<SDValue, 8> Ops{Chain, DAG.getTargetConstant(IntID, DL, XLenVT)}; SmallVector<SDValue, 8> Ops{Chain, DAG.getTargetConstant(IntID, DL, XLenVT)};
if (!IsUnmasked) if (!IsUnmasked)
Ops.push_back(PassThru); Ops.push_back(PassThru);
Ops.push_back(BasePtr); Ops.push_back(BasePtr);
Ops.push_back(Index); Ops.push_back(Index);
if (!IsUnmasked) if (!IsUnmasked)
Ops.push_back(Mask); Ops.push_back(Mask);
Ops.push_back(VL); Ops.push_back(VL);
if (!IsUnmasked)
Ops.push_back(DAG.getTargetConstant(RISCVII::TAIL_AGNOSTIC, DL, XLenVT));
craig.topperUnsubmitted
Not Done
ReplyInline Actions

TargetConstant?

craig.topper: TargetConstant?
SDVTList VTs = DAG.getVTList({ContainerVT, MVT::Other}); SDVTList VTs = DAG.getVTList({ContainerVT, MVT::Other});
SDValue Result = SDValue Result =
DAG.getMemIntrinsicNode(ISD::INTRINSIC_W_CHAIN, DL, VTs, Ops, MemVT, MMO); DAG.getMemIntrinsicNode(ISD::INTRINSIC_W_CHAIN, DL, VTs, Ops, MemVT, MMO);
Chain = Result.getValue(1); Chain = Result.getValue(1);
if (VT.isFixedLengthVector()) if (VT.isFixedLengthVector())
Result = convertFromScalableVector(VT, Result, DAG, Subtarget); Result = convertFromScalableVector(VT, Result, DAG, Subtarget);
▲ Show 20 Lines • Show All 4,619 Lines • Show Last 20 Lines

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

Show First 20 Lines • Show All 426 Lines • ▼ Show 20 Lines if (UseMI) {
if (UseMI && UseMI->isImplicitDef()) if (UseMI && UseMI->isImplicitDef())
TailAgnostic = true; TailAgnostic = true;
} }
} }
// Remove the tail policy so we can find the SEW and VL. // Remove the tail policy so we can find the SEW and VL.
if (HasPolicy) if (HasPolicy)
--NumOperands; --NumOperands;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Probably need to mask this to bit 0. TailAgnostic = Op.getImm() & 1. As written we'll set TailAgnostic if any bit in the immediate is non-zero.

craig.topper: Probably need to mask this to bit 0. `TailAgnostic = Op.getImm() & 1`. As written we'll set…
RISCVII::VLMUL VLMul = RISCVII::getLMul(TSFlags); RISCVII::VLMUL VLMul = RISCVII::getLMul(TSFlags);
unsigned Log2SEW = MI.getOperand(NumOperands - 1).getImm(); unsigned Log2SEW = MI.getOperand(NumOperands - 1).getImm();
// A Log2SEW of 0 is an operation on mask registers only. // A Log2SEW of 0 is an operation on mask registers only.
bool MaskRegOp = Log2SEW == 0; bool MaskRegOp = Log2SEW == 0;
unsigned SEW = Log2SEW ? 1 << Log2SEW : 8; unsigned SEW = Log2SEW ? 1 << Log2SEW : 8;
assert(RISCVVType::isValidSEW(SEW) && "Unexpected SEW"); assert(RISCVVType::isValidSEW(SEW) && "Unexpected SEW");
// If there are no explicit defs, this is a store instruction which can // If there are no explicit defs, this is a store instruction which can
// ignore the tail and mask policies. // ignore the tail and mask policies.
bool StoreOp = MI.getNumExplicitDefs() == 0; bool StoreOp = MI.getNumExplicitDefs() == 0;
if (RISCVII::hasVLOp(TSFlags)) { if (RISCVII::hasVLOp(TSFlags)) {
const MachineOperand &VLOp = MI.getOperand(NumOperands - 2); const MachineOperand &VLOp = MI.getOperand(NumOperands - 2);
craig.topperUnsubmitted
Done
ReplyInline Actions

Why doesn't this need to be updated?

craig.topper: Why doesn't this need to be updated?
if (VLOp.isImm()) { if (VLOp.isImm()) {
int64_t Imm = VLOp.getImm(); int64_t Imm = VLOp.getImm();
// Conver the VLMax sentintel to X0 register. // Conver the VLMax sentintel to X0 register.
if (Imm == RISCV::VLMaxSentinel) if (Imm == RISCV::VLMaxSentinel)
InstrInfo.setAVLReg(RISCV::X0); InstrInfo.setAVLReg(RISCV::X0);
else else
InstrInfo.setAVLImm(Imm); InstrInfo.setAVLImm(Imm);
} else { } else {
▲ Show 20 Lines • Show All 468 Lines • ▼ Show 20 Lines if (MI.getOpcode() == RISCV::PseudoVSETVLI ||
PrevVSETVLIMI = &MI; PrevVSETVLIMI = &MI;
continue; continue;
} }
uint64_t TSFlags = MI.getDesc().TSFlags; uint64_t TSFlags = MI.getDesc().TSFlags;
if (RISCVII::hasSEWOp(TSFlags)) { if (RISCVII::hasSEWOp(TSFlags)) {
VSETVLIInfo NewInfo = computeInfoForInstr(MI, TSFlags, MRI); VSETVLIInfo NewInfo = computeInfoForInstr(MI, TSFlags, MRI);
if (RISCVII::hasVLOp(TSFlags)) { if (RISCVII::hasVLOp(TSFlags)) {
MachineOperand &VLOp = MI.getOperand(MI.getNumExplicitOperands() - 2); unsigned Offset = 2;
if (RISCVII::hasVecPolicyOp(TSFlags))
Offset = 3;
MachineOperand &VLOp =
MI.getOperand(MI.getNumExplicitOperands() - Offset);
if (VLOp.isReg()) { if (VLOp.isReg()) {
// Erase the AVL operand from the instruction. // Erase the AVL operand from the instruction.
VLOp.setReg(RISCV::NoRegister); VLOp.setReg(RISCV::NoRegister);
VLOp.setIsKill(false); VLOp.setIsKill(false);
} }
MI.addOperand(MachineOperand::CreateReg(RISCV::VL, /*isDef*/ false, MI.addOperand(MachineOperand::CreateReg(RISCV::VL, /*isDef*/ false,
/*isImp*/ true)); /*isImp*/ true));
} }
▲ Show 20 Lines • Show All 107 Lines • Show Last 20 Lines

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

Show First 20 Lines • Show All 640 Lines • ▼ Show 20 Linesclass VPseudoUSLoadNoMask<VReg RetClass, int EEW, bit isFF> :
let HasDummyMask = 1; let HasDummyMask = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoUSLoadMask<VReg RetClass, int EEW, bit isFF> : class VPseudoUSLoadMask<VReg RetClass, int EEW, bit isFF> :
Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd), Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
(ins GetVRegNoV0<RetClass>.R:$merge, (ins GetVRegNoV0<RetClass>.R:$merge,
GPR:$rs1, GPR:$rs1,
VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew, ixlenimm:$policy),[]>,
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Why uimm5? I think VPseudoTernaryNoMaskWithPolicy is using ixlenimm like $sew

craig.topper: Why uimm5? I think VPseudoTernaryNoMaskWithPolicy is using ixlenimm like $sew
RISCVVPseudo, RISCVVPseudo,
RISCVVLE</*Masked*/1, /*Strided*/0, /*FF*/isFF, log2<EEW>.val, VLMul> { RISCVVLE</*Masked*/1, /*Strided*/0, /*FF*/isFF, log2<EEW>.val, VLMul> {
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
let Constraints = "$rd = $merge"; let Constraints = "$rd = $merge";
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasMergeOp = 1; let HasMergeOp = 1;
let HasVecPolicyOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoSLoadNoMask<VReg RetClass, int EEW>: class VPseudoSLoadNoMask<VReg RetClass, int EEW>:
Pseudo<(outs RetClass:$rd), Pseudo<(outs RetClass:$rd),
(ins GPR:$rs1, GPR:$rs2, AVL:$vl, ixlenimm:$sew),[]>, (ins GPR:$rs1, GPR:$rs2, AVL:$vl, ixlenimm:$sew),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVLE</*Masked*/0, /*Strided*/1, /*FF*/0, log2<EEW>.val, VLMul> { RISCVVLE</*Masked*/0, /*Strided*/1, /*FF*/0, log2<EEW>.val, VLMul> {
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasDummyMask = 1; let HasDummyMask = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoSLoadMask<VReg RetClass, int EEW>: class VPseudoSLoadMask<VReg RetClass, int EEW>:
Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd), Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
(ins GetVRegNoV0<RetClass>.R:$merge, (ins GetVRegNoV0<RetClass>.R:$merge,
GPR:$rs1, GPR:$rs2, GPR:$rs1, GPR:$rs2,
VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew, ixlenimm:$policy),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVLE</*Masked*/1, /*Strided*/1, /*FF*/0, log2<EEW>.val, VLMul> { RISCVVLE</*Masked*/1, /*Strided*/1, /*FF*/0, log2<EEW>.val, VLMul> {
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
let Constraints = "$rd = $merge"; let Constraints = "$rd = $merge";
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasMergeOp = 1; let HasMergeOp = 1;
let HasVecPolicyOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoILoadNoMask<VReg RetClass, VReg IdxClass, int EEW, bits<3> LMUL, class VPseudoILoadNoMask<VReg RetClass, VReg IdxClass, int EEW, bits<3> LMUL,
bit Ordered, bit EarlyClobber>: bit Ordered, bit EarlyClobber>:
Pseudo<(outs RetClass:$rd), Pseudo<(outs RetClass:$rd),
(ins GPR:$rs1, IdxClass:$rs2, AVL:$vl, ixlenimm:$sew),[]>, (ins GPR:$rs1, IdxClass:$rs2, AVL:$vl, ixlenimm:$sew),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVLX</*Masked*/0, Ordered, log2<EEW>.val, VLMul, LMUL> { RISCVVLX</*Masked*/0, Ordered, log2<EEW>.val, VLMul, LMUL> {
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasDummyMask = 1; let HasDummyMask = 1;
let Constraints = !if(!eq(EarlyClobber, 1), "@earlyclobber $rd", ""); let Constraints = !if(!eq(EarlyClobber, 1), "@earlyclobber $rd", "");
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoILoadMask<VReg RetClass, VReg IdxClass, int EEW, bits<3> LMUL, class VPseudoILoadMask<VReg RetClass, VReg IdxClass, int EEW, bits<3> LMUL,
bit Ordered, bit EarlyClobber>: bit Ordered, bit EarlyClobber>:
Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd), Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
(ins GetVRegNoV0<RetClass>.R:$merge, (ins GetVRegNoV0<RetClass>.R:$merge,
GPR:$rs1, IdxClass:$rs2, GPR:$rs1, IdxClass:$rs2,
VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew, ixlenimm:$policy),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVLX</*Masked*/1, Ordered, log2<EEW>.val, VLMul, LMUL> { RISCVVLX</*Masked*/1, Ordered, log2<EEW>.val, VLMul, LMUL> {
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
let Constraints = !if(!eq(EarlyClobber, 1), "@earlyclobber $rd, $rd = $merge", "$rd = $merge"); let Constraints = !if(!eq(EarlyClobber, 1), "@earlyclobber $rd, $rd = $merge", "$rd = $merge");
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasMergeOp = 1; let HasMergeOp = 1;
let HasVecPolicyOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoUSStoreNoMask<VReg StClass, int EEW>: class VPseudoUSStoreNoMask<VReg StClass, int EEW>:
Pseudo<(outs), Pseudo<(outs),
(ins StClass:$rd, GPR:$rs1, AVL:$vl, ixlenimm:$sew),[]>, (ins StClass:$rd, GPR:$rs1, AVL:$vl, ixlenimm:$sew),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVSE</*Masked*/0, /*Strided*/0, log2<EEW>.val, VLMul> { RISCVVSE</*Masked*/0, /*Strided*/0, log2<EEW>.val, VLMul> {
▲ Show 20 Lines • Show All 127 Lines • ▼ Show 20 Linesclass VPseudoUnaryMask<VReg RetClass, VReg OpClass, string Constraint = ""> :
let hasSideEffects = 0; let hasSideEffects = 0;
let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret; let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret;
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasMergeOp = 1; let HasMergeOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoUnaryMaskTA<VReg RetClass, VReg OpClass, string Constraint = ""> :
Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
(ins GetVRegNoV0<RetClass>.R:$merge, OpClass:$rs2,
VMaskOp:$vm, AVL:$vl, ixlenimm:$sew, ixlenimm:$policy), []>,
RISCVVPseudo {
let mayLoad = 0;
let mayStore = 0;
let hasSideEffects = 0;
let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret;
let HasVLOp = 1;
let HasSEWOp = 1;
let HasMergeOp = 1;
let HasVecPolicyOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
}
// mask unary operation without maskedoff // mask unary operation without maskedoff
class VPseudoMaskUnarySOutMask: class VPseudoMaskUnarySOutMask:
Pseudo<(outs GPR:$rd), Pseudo<(outs GPR:$rd),
(ins VR:$rs1, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew), []>, (ins VR:$rs1, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew), []>,
RISCVVPseudo { RISCVVPseudo {
let mayLoad = 0; let mayLoad = 0;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
▲ Show 20 Lines • Show All 99 Lines • ▼ Show 20 Linesclass VPseudoBinaryMask<VReg RetClass,
let hasSideEffects = 0; let hasSideEffects = 0;
let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret; let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret;
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasMergeOp = 1; let HasMergeOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoBinaryMaskTA<VReg RetClass,
RegisterClass Op1Class,
DAGOperand Op2Class,
string Constraint> :
Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
(ins GetVRegNoV0<RetClass>.R:$merge,
Op1Class:$rs2, Op2Class:$rs1,
VMaskOp:$vm, AVL:$vl, ixlenimm:$sew, ixlenimm:$policy), []>,
RISCVVPseudo {
let mayLoad = 0;
let mayStore = 0;
let hasSideEffects = 0;
let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret;
let HasVLOp = 1;
let HasSEWOp = 1;
let HasMergeOp = 1;
let HasVecPolicyOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
}
// Like VPseudoBinaryMask, but output can be V0. // Like VPseudoBinaryMask, but output can be V0.
class VPseudoBinaryMOutMask<VReg RetClass, class VPseudoBinaryMOutMask<VReg RetClass,
RegisterClass Op1Class, RegisterClass Op1Class,
DAGOperand Op2Class, DAGOperand Op2Class,
string Constraint> : string Constraint> :
Pseudo<(outs RetClass:$rd), Pseudo<(outs RetClass:$rd),
(ins RetClass:$merge, (ins RetClass:$merge,
Op1Class:$rs2, Op2Class:$rs1, Op1Class:$rs2, Op2Class:$rs1,
Show All 13 Lines
// tied to the destination so we can workaround the earlyclobber constraint. // tied to the destination so we can workaround the earlyclobber constraint.
// This allows maskedoff and rs2 to be the same register. // This allows maskedoff and rs2 to be the same register.
class VPseudoTiedBinaryMask<VReg RetClass, class VPseudoTiedBinaryMask<VReg RetClass,
DAGOperand Op2Class, DAGOperand Op2Class,
string Constraint> : string Constraint> :
Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd), Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
(ins GetVRegNoV0<RetClass>.R:$merge, (ins GetVRegNoV0<RetClass>.R:$merge,
Op2Class:$rs1, Op2Class:$rs1,
VMaskOp:$vm, AVL:$vl, ixlenimm:$sew), []>, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew, ixlenimm:$policy), []>,
RISCVVPseudo { RISCVVPseudo {
let mayLoad = 0; let mayLoad = 0;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret; let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret;
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasMergeOp = 0; // Merge is also rs2. let HasMergeOp = 0; // Merge is also rs2.
let HasVecPolicyOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoBinaryCarryIn<VReg RetClass, class VPseudoBinaryCarryIn<VReg RetClass,
VReg Op1Class, VReg Op1Class,
DAGOperand Op2Class, DAGOperand Op2Class,
LMULInfo MInfo, LMULInfo MInfo,
bit CarryIn, bit CarryIn,
▲ Show 20 Lines • Show All 130 Lines • ▼ Show 20 Linesclass VPseudoUSSegLoadNoMask<VReg RetClass, int EEW, bits<4> NF, bit isFF>:
let HasSEWOp = 1; let HasSEWOp = 1;
let HasDummyMask = 1; let HasDummyMask = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoUSSegLoadMask<VReg RetClass, int EEW, bits<4> NF, bit isFF>: class VPseudoUSSegLoadMask<VReg RetClass, int EEW, bits<4> NF, bit isFF>:
Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd), Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
(ins GetVRegNoV0<RetClass>.R:$merge, GPR:$rs1, (ins GetVRegNoV0<RetClass>.R:$merge, GPR:$rs1,
VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew, ixlenimm:$policy),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVLSEG<NF, /*Masked*/1, /*Strided*/0, /*FF*/isFF, log2<EEW>.val, VLMul> { RISCVVLSEG<NF, /*Masked*/1, /*Strided*/0, /*FF*/isFF, log2<EEW>.val, VLMul> {
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
let Constraints = "$rd = $merge"; let Constraints = "$rd = $merge";
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasMergeOp = 1; let HasMergeOp = 1;
let HasVecPolicyOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoSSegLoadNoMask<VReg RetClass, int EEW, bits<4> NF>: class VPseudoSSegLoadNoMask<VReg RetClass, int EEW, bits<4> NF>:
Pseudo<(outs RetClass:$rd), Pseudo<(outs RetClass:$rd),
(ins GPR:$rs1, GPR:$offset, AVL:$vl, ixlenimm:$sew),[]>, (ins GPR:$rs1, GPR:$offset, AVL:$vl, ixlenimm:$sew),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVLSEG<NF, /*Masked*/0, /*Strided*/1, /*FF*/0, log2<EEW>.val, VLMul> { RISCVVLSEG<NF, /*Masked*/0, /*Strided*/1, /*FF*/0, log2<EEW>.val, VLMul> {
let mayLoad = 1; let mayLoad = 1;
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasDummyMask = 1; let HasDummyMask = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoSSegLoadMask<VReg RetClass, int EEW, bits<4> NF>: class VPseudoSSegLoadMask<VReg RetClass, int EEW, bits<4> NF>:
Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd), Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
(ins GetVRegNoV0<RetClass>.R:$merge, GPR:$rs1, (ins GetVRegNoV0<RetClass>.R:$merge, GPR:$rs1,
GPR:$offset, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>, GPR:$offset, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew,
ixlenimm:$policy),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVLSEG<NF, /*Masked*/1, /*Strided*/1, /*FF*/0, log2<EEW>.val, VLMul> { RISCVVLSEG<NF, /*Masked*/1, /*Strided*/1, /*FF*/0, log2<EEW>.val, VLMul> {
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
let Constraints = "$rd = $merge"; let Constraints = "$rd = $merge";
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasMergeOp = 1; let HasMergeOp = 1;
let HasVecPolicyOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoISegLoadNoMask<VReg RetClass, VReg IdxClass, int EEW, bits<3> LMUL, class VPseudoISegLoadNoMask<VReg RetClass, VReg IdxClass, int EEW, bits<3> LMUL,
bits<4> NF, bit Ordered>: bits<4> NF, bit Ordered>:
Pseudo<(outs RetClass:$rd), Pseudo<(outs RetClass:$rd),
(ins GPR:$rs1, IdxClass:$offset, AVL:$vl, ixlenimm:$sew),[]>, (ins GPR:$rs1, IdxClass:$offset, AVL:$vl, ixlenimm:$sew),[]>,
RISCVVPseudo, RISCVVPseudo,
Show All 9 Linesclass VPseudoISegLoadNoMask<VReg RetClass, VReg IdxClass, int EEW, bits<3> LMUL,
let HasDummyMask = 1; let HasDummyMask = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoISegLoadMask<VReg RetClass, VReg IdxClass, int EEW, bits<3> LMUL, class VPseudoISegLoadMask<VReg RetClass, VReg IdxClass, int EEW, bits<3> LMUL,
bits<4> NF, bit Ordered>: bits<4> NF, bit Ordered>:
Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd), Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
(ins GetVRegNoV0<RetClass>.R:$merge, GPR:$rs1, (ins GetVRegNoV0<RetClass>.R:$merge, GPR:$rs1,
IdxClass:$offset, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>, IdxClass:$offset, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew,
ixlenimm:$policy),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVLXSEG<NF, /*Masked*/1, Ordered, log2<EEW>.val, VLMul, LMUL> { RISCVVLXSEG<NF, /*Masked*/1, Ordered, log2<EEW>.val, VLMul, LMUL> {
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let hasSideEffects = 0; let hasSideEffects = 0;
// For vector indexed segment loads, the destination vector register groups // For vector indexed segment loads, the destination vector register groups
// cannot overlap the source vector register group // cannot overlap the source vector register group
let Constraints = "@earlyclobber $rd, $rd = $merge"; let Constraints = "@earlyclobber $rd, $rd = $merge";
let HasVLOp = 1; let HasVLOp = 1;
let HasSEWOp = 1; let HasSEWOp = 1;
let HasMergeOp = 1; let HasMergeOp = 1;
let HasVecPolicyOp = 1;
let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst); let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
} }
class VPseudoUSSegStoreNoMask<VReg ValClass, int EEW, bits<4> NF>: class VPseudoUSSegStoreNoMask<VReg ValClass, int EEW, bits<4> NF>:
Pseudo<(outs), Pseudo<(outs),
(ins ValClass:$rd, GPR:$rs1, AVL:$vl, ixlenimm:$sew),[]>, (ins ValClass:$rd, GPR:$rs1, AVL:$vl, ixlenimm:$sew),[]>,
RISCVVPseudo, RISCVVPseudo,
RISCVVSSEG<NF, /*Masked*/0, /*Strided*/0, log2<EEW>.val, VLMul> { RISCVVSSEG<NF, /*Masked*/0, /*Strided*/0, log2<EEW>.val, VLMul> {
▲ Show 20 Lines • Show All 259 Lines • ▼ Show 20 Lines
multiclass VPseudoBinary<VReg RetClass, multiclass VPseudoBinary<VReg RetClass,
VReg Op1Class, VReg Op1Class,
DAGOperand Op2Class, DAGOperand Op2Class,
LMULInfo MInfo, LMULInfo MInfo,
string Constraint = ""> { string Constraint = ""> {
let VLMul = MInfo.value in { let VLMul = MInfo.value in {
def "_" # MInfo.MX : VPseudoBinaryNoMask<RetClass, Op1Class, Op2Class, def "_" # MInfo.MX : VPseudoBinaryNoMask<RetClass, Op1Class, Op2Class,
Constraint>; Constraint>;
def "_" # MInfo.MX # "_MASK" : VPseudoBinaryMask<RetClass, Op1Class, Op2Class, def "_" # MInfo.MX # "_MASK" : VPseudoBinaryMaskTA<RetClass, Op1Class, Op2Class,
Constraint>; Constraint>;
} }
} }
multiclass VPseudoBinaryM<VReg RetClass, multiclass VPseudoBinaryM<VReg RetClass,
VReg Op1Class, VReg Op1Class,
DAGOperand Op2Class, DAGOperand Op2Class,
LMULInfo MInfo, LMULInfo MInfo,
string Constraint = ""> { string Constraint = ""> {
Show All 10 Linesmulticlass VPseudoBinaryEmul<VReg RetClass,
VReg Op1Class, VReg Op1Class,
DAGOperand Op2Class, DAGOperand Op2Class,
LMULInfo lmul, LMULInfo lmul,
LMULInfo emul, LMULInfo emul,
string Constraint = ""> { string Constraint = ""> {
let VLMul = lmul.value in { let VLMul = lmul.value in {
def "_" # lmul.MX # "_" # emul.MX : VPseudoBinaryNoMask<RetClass, Op1Class, Op2Class, def "_" # lmul.MX # "_" # emul.MX : VPseudoBinaryNoMask<RetClass, Op1Class, Op2Class,
Constraint>; Constraint>;
def "_" # lmul.MX # "_" # emul.MX # "_MASK" : VPseudoBinaryMask<RetClass, Op1Class, Op2Class, def "_" # lmul.MX # "_" # emul.MX # "_MASK" : VPseudoBinaryMaskTA<RetClass, Op1Class, Op2Class,
Constraint>; Constraint>;
} }
} }
multiclass VPseudoTiedBinary<VReg RetClass, multiclass VPseudoTiedBinary<VReg RetClass,
DAGOperand Op2Class, DAGOperand Op2Class,
LMULInfo MInfo, LMULInfo MInfo,
string Constraint = ""> { string Constraint = ""> {
let VLMul = MInfo.value in { let VLMul = MInfo.value in {
▲ Show 20 Lines • Show All 175 Lines • ▼ Show 20 Lines foreach m = MxList.m in {
foreach f = FPList.fpinfo in { foreach f = FPList.fpinfo in {
let VLMul = m.value in { let VLMul = m.value in {
def "_" # f.FX # "_" # m.MX : VPseudoUnaryNoDummyMask<m.vrclass, f.fprclass>; def "_" # f.FX # "_" # m.MX : VPseudoUnaryNoDummyMask<m.vrclass, f.fprclass>;
} }
} }
} }
} }
multiclass VPseudoUnaryTAV_V {
foreach m = MxList.m in {
let VLMul = m.value in {
def "_V_" # m.MX : VPseudoUnaryNoMask<m.vrclass, m.vrclass>;
def "_V_" # m.MX # "_MASK" : VPseudoUnaryMaskTA<m.vrclass, m.vrclass>;
}
}
}
multiclass VPseudoUnaryV_V { multiclass VPseudoUnaryV_V {
foreach m = MxList.m in { foreach m = MxList.m in {
let VLMul = m.value in { let VLMul = m.value in {
def "_V_" # m.MX : VPseudoUnaryNoMask<m.vrclass, m.vrclass>; def "_V_" # m.MX : VPseudoUnaryNoMask<m.vrclass, m.vrclass>;
def "_V_" # m.MX # "_MASK" : VPseudoUnaryMask<m.vrclass, m.vrclass>; def "_V_" # m.MX # "_MASK" : VPseudoUnaryMask<m.vrclass, m.vrclass>;
} }
} }
} }
multiclass PseudoUnaryV_VF2 { multiclass PseudoUnaryV_VF2 {
defvar constraints = "@earlyclobber $rd"; defvar constraints = "@earlyclobber $rd";
foreach m = MxListVF2.m in foreach m = MxListVF2.m in
{ {
let VLMul = m.value in { let VLMul = m.value in {
def "_" # m.MX : VPseudoUnaryNoMask<m.vrclass, m.f2vrclass, constraints>; def "_" # m.MX : VPseudoUnaryNoMask<m.vrclass, m.f2vrclass, constraints>;
def "_" # m.MX # "_MASK" : VPseudoUnaryMask<m.vrclass, m.f2vrclass, def "_" # m.MX # "_MASK" : VPseudoUnaryMaskTA<m.vrclass, m.f2vrclass,
constraints>; constraints>;
} }
} }
} }
multiclass PseudoUnaryV_VF4 { multiclass PseudoUnaryV_VF4 {
defvar constraints = "@earlyclobber $rd"; defvar constraints = "@earlyclobber $rd";
foreach m = MxListVF4.m in foreach m = MxListVF4.m in
{ {
let VLMul = m.value in { let VLMul = m.value in {
def "_" # m.MX : VPseudoUnaryNoMask<m.vrclass, m.f4vrclass, constraints>; def "_" # m.MX : VPseudoUnaryNoMask<m.vrclass, m.f4vrclass, constraints>;
def "_" # m.MX # "_MASK" : VPseudoUnaryMask<m.vrclass, m.f4vrclass, def "_" # m.MX # "_MASK" : VPseudoUnaryMaskTA<m.vrclass, m.f4vrclass,
constraints>; constraints>;
} }
} }
} }
multiclass PseudoUnaryV_VF8 { multiclass PseudoUnaryV_VF8 {
defvar constraints = "@earlyclobber $rd"; defvar constraints = "@earlyclobber $rd";
foreach m = MxListVF8.m in foreach m = MxListVF8.m in
{ {
let VLMul = m.value in { let VLMul = m.value in {
def "_" # m.MX : VPseudoUnaryNoMask<m.vrclass, m.f8vrclass, constraints>; def "_" # m.MX : VPseudoUnaryNoMask<m.vrclass, m.f8vrclass, constraints>;
def "_" # m.MX # "_MASK" : VPseudoUnaryMask<m.vrclass, m.f8vrclass, def "_" # m.MX # "_MASK" : VPseudoUnaryMaskTA<m.vrclass, m.f8vrclass,
constraints>; constraints>;
} }
} }
} }
// The destination EEW is 1 since "For the purposes of register group overlap // The destination EEW is 1 since "For the purposes of register group overlap
// constraints, mask elements have EEW=1." // constraints, mask elements have EEW=1."
// The source EEW is 8, 16, 32, or 64. // The source EEW is 8, 16, 32, or 64.
// When the destination EEW is different from source EEW, we need to use // When the destination EEW is different from source EEW, we need to use
▲ Show 20 Lines • Show All 241 Lines • ▼ Show 20 Lines
} }
multiclass VPseudoConversion<VReg RetClass, multiclass VPseudoConversion<VReg RetClass,
VReg Op1Class, VReg Op1Class,
LMULInfo MInfo, LMULInfo MInfo,
string Constraint = ""> { string Constraint = ""> {
let VLMul = MInfo.value in { let VLMul = MInfo.value in {
def "_" # MInfo.MX : VPseudoUnaryNoMask<RetClass, Op1Class, Constraint>; def "_" # MInfo.MX : VPseudoUnaryNoMask<RetClass, Op1Class, Constraint>;
def "_" # MInfo.MX # "_MASK" : VPseudoUnaryMask<RetClass, Op1Class, def "_" # MInfo.MX # "_MASK" : VPseudoUnaryMaskTA<RetClass, Op1Class,
Constraint>; Constraint>;
} }
} }
multiclass VPseudoConversionV_V { multiclass VPseudoConversionV_V {
foreach m = MxList.m in foreach m = MxList.m in
defm _V : VPseudoConversion<m.vrclass, m.vrclass, m>; defm _V : VPseudoConversion<m.vrclass, m.vrclass, m>;
} }
▲ Show 20 Lines • Show All 166 Lines • ▼ Show 20 Lines Pat<(result_type (!cast<Intrinsic>(intrinsic_name#"_mask")
(op2_type op2_reg_class:$rs2), (op2_type op2_reg_class:$rs2),
(mask_type V0), (mask_type V0),
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX#"_MASK") (!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX#"_MASK")
(result_type result_reg_class:$merge), (result_type result_reg_class:$merge),
(op2_type op2_reg_class:$rs2), (op2_type op2_reg_class:$rs2),
(mask_type V0), GPR:$vl, sew)>; (mask_type V0), GPR:$vl, sew)>;
class VPatUnaryMaskTA<string intrinsic_name,
string inst,
string kind,
ValueType result_type,
ValueType op2_type,
ValueType mask_type,
int sew,
LMULInfo vlmul,
VReg result_reg_class,
VReg op2_reg_class> :
Pat<(result_type (!cast<Intrinsic>(intrinsic_name#"_mask")
(result_type result_reg_class:$merge),
(op2_type op2_reg_class:$rs2),
(mask_type V0),
VLOpFrag, (XLenVT timm:$policy))),
craig.topperUnsubmitted
Not Done
ReplyInline Actions

I think this should be timm:$policy

craig.topper: I think this should be timm:$policy
(!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX#"_MASK")
(result_type result_reg_class:$merge),
(op2_type op2_reg_class:$rs2),
(mask_type V0), GPR:$vl, sew, (XLenVT timm:$policy))>;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Does timm:$policy work down here too?

craig.topper: Does timm:$policy work down here too?
class VPatMaskUnaryNoMask<string intrinsic_name, class VPatMaskUnaryNoMask<string intrinsic_name,
string inst, string inst,
MTypeInfo mti> : MTypeInfo mti> :
Pat<(mti.Mask (!cast<Intrinsic>(intrinsic_name) Pat<(mti.Mask (!cast<Intrinsic>(intrinsic_name)
(mti.Mask VR:$rs2), (mti.Mask VR:$rs2),
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>(inst#"_M_"#mti.BX) (!cast<Instruction>(inst#"_M_"#mti.BX)
(mti.Mask VR:$rs2), (mti.Mask VR:$rs2),
▲ Show 20 Lines • Show All 85 Lines • ▼ Show 20 Lines Pat<(result_type (!cast<Intrinsic>(intrinsic_name#"_mask")
(mask_type V0), (mask_type V0),
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>(inst#"_MASK") (!cast<Instruction>(inst#"_MASK")
(result_type result_reg_class:$merge), (result_type result_reg_class:$merge),
(op1_type op1_reg_class:$rs1), (op1_type op1_reg_class:$rs1),
(op2_type op2_kind:$rs2), (op2_type op2_kind:$rs2),
(mask_type V0), GPR:$vl, sew)>; (mask_type V0), GPR:$vl, sew)>;
class VPatBinaryMaskTA<string intrinsic_name,
string inst,
ValueType result_type,
ValueType op1_type,
ValueType op2_type,
ValueType mask_type,
int sew,
VReg result_reg_class,
VReg op1_reg_class,
DAGOperand op2_kind> :
Pat<(result_type (!cast<Intrinsic>(intrinsic_name#"_mask")
(result_type result_reg_class:$merge),
(op1_type op1_reg_class:$rs1),
(op2_type op2_kind:$rs2),
(mask_type V0),
VLOpFrag, (XLenVT timm:$policy))),
(!cast<Instruction>(inst#"_MASK")
(result_type result_reg_class:$merge),
(op1_type op1_reg_class:$rs1),
(op2_type op2_kind:$rs2),
(mask_type V0), GPR:$vl, sew, (XLenVT timm:$policy))>;
// Same as above but source operands are swapped. // Same as above but source operands are swapped.
class VPatBinaryMaskSwapped<string intrinsic_name, class VPatBinaryMaskSwapped<string intrinsic_name,
string inst, string inst,
ValueType result_type, ValueType result_type,
ValueType op1_type, ValueType op1_type,
ValueType op2_type, ValueType op2_type,
ValueType mask_type, ValueType mask_type,
int sew, int sew,
Show All 36 Linesclass VPatTiedBinaryMask<string intrinsic_name,
int sew, int sew,
VReg result_reg_class, VReg result_reg_class,
DAGOperand op2_kind> : DAGOperand op2_kind> :
Pat<(result_type (!cast<Intrinsic>(intrinsic_name#"_mask") Pat<(result_type (!cast<Intrinsic>(intrinsic_name#"_mask")
(result_type result_reg_class:$merge), (result_type result_reg_class:$merge),
(result_type result_reg_class:$merge), (result_type result_reg_class:$merge),
(op2_type op2_kind:$rs2), (op2_type op2_kind:$rs2),
(mask_type V0), (mask_type V0),
VLOpFrag)), VLOpFrag, (XLenVT timm:$policy))),
(!cast<Instruction>(inst#"_MASK_TIED") (!cast<Instruction>(inst#"_MASK_TIED")
(result_type result_reg_class:$merge), (result_type result_reg_class:$merge),
(op2_type op2_kind:$rs2), (op2_type op2_kind:$rs2),
(mask_type V0), GPR:$vl, sew)>; (mask_type V0), GPR:$vl, sew, (XLenVT timm:$policy))>;
class VPatTernaryNoMask<string intrinsic, class VPatTernaryNoMask<string intrinsic,
string inst, string inst,
string kind, string kind,
ValueType result_type, ValueType result_type,
ValueType op1_type, ValueType op1_type,
ValueType op2_type, ValueType op2_type,
int sew, int sew,
▲ Show 20 Lines • Show All 144 Lines • ▼ Show 20 Lines
{ {
foreach vtiTofti = fractionList in foreach vtiTofti = fractionList in
{ {
defvar vti = vtiTofti.Vti; defvar vti = vtiTofti.Vti;
defvar fti = vtiTofti.Fti; defvar fti = vtiTofti.Fti;
def : VPatUnaryNoMask<intrinsic, instruction, suffix, def : VPatUnaryNoMask<intrinsic, instruction, suffix,
vti.Vector, fti.Vector, vti.Vector, fti.Vector,
vti.Log2SEW, vti.LMul, fti.RegClass>; vti.Log2SEW, vti.LMul, fti.RegClass>;
def : VPatUnaryMask<intrinsic, instruction, suffix, def : VPatUnaryMaskTA<intrinsic, instruction, suffix,
vti.Vector, fti.Vector, vti.Mask, vti.Vector, fti.Vector, vti.Mask,
vti.Log2SEW, vti.LMul, vti.RegClass, fti.RegClass>; vti.Log2SEW, vti.LMul, vti.RegClass, fti.RegClass>;
} }
} }
multiclass VPatUnaryV_V<string intrinsic, string instruction, multiclass VPatUnaryV_V<string intrinsic, string instruction,
list<VTypeInfo> vtilist> { list<VTypeInfo> vtilist> {
foreach vti = vtilist in { foreach vti = vtilist in {
def : VPatUnaryNoMask<intrinsic, instruction, "V", def : VPatUnaryNoMask<intrinsic, instruction, "V",
vti.Vector, vti.Vector, vti.Vector, vti.Vector,
vti.Log2SEW, vti.LMul, vti.RegClass>; vti.Log2SEW, vti.LMul, vti.RegClass>;
def : VPatUnaryMask<intrinsic, instruction, "V", def : VPatUnaryMaskTA<intrinsic, instruction, "V",
vti.Vector, vti.Vector, vti.Mask, vti.Vector, vti.Vector, vti.Mask,
vti.Log2SEW, vti.LMul, vti.RegClass, vti.RegClass>; vti.Log2SEW, vti.LMul, vti.RegClass, vti.RegClass>;
} }
} }
multiclass VPatNullaryV<string intrinsic, string instruction> multiclass VPatNullaryV<string intrinsic, string instruction>
{ {
foreach vti = AllIntegerVectors in { foreach vti = AllIntegerVectors in {
def : Pat<(vti.Vector (!cast<Intrinsic>(intrinsic) def : Pat<(vti.Vector (!cast<Intrinsic>(intrinsic)
VLOpFrag)), VLOpFrag)),
Show All 29 Lines
{ {
def : VPatBinaryNoMask<intrinsic, inst, result_type, op1_type, op2_type, def : VPatBinaryNoMask<intrinsic, inst, result_type, op1_type, op2_type,
sew, op1_reg_class, op2_kind>; sew, op1_reg_class, op2_kind>;
def : VPatBinaryMask<intrinsic, inst, result_type, op1_type, op2_type, def : VPatBinaryMask<intrinsic, inst, result_type, op1_type, op2_type,
mask_type, sew, result_reg_class, op1_reg_class, mask_type, sew, result_reg_class, op1_reg_class,
op2_kind>; op2_kind>;
} }
multiclass VPatBinaryTA<string intrinsic,
string inst,
ValueType result_type,
ValueType op1_type,
ValueType op2_type,
ValueType mask_type,
int sew,
VReg result_reg_class,
VReg op1_reg_class,
DAGOperand op2_kind>
{
def : VPatBinaryNoMask<intrinsic, inst, result_type, op1_type, op2_type,
sew, op1_reg_class, op2_kind>;
def : VPatBinaryMaskTA<intrinsic, inst, result_type, op1_type, op2_type,
mask_type, sew, result_reg_class, op1_reg_class,
op2_kind>;
}
multiclass VPatBinarySwapped<string intrinsic, multiclass VPatBinarySwapped<string intrinsic,
string inst, string inst,
ValueType result_type, ValueType result_type,
ValueType op1_type, ValueType op1_type,
ValueType op2_type, ValueType op2_type,
ValueType mask_type, ValueType mask_type,
int sew, int sew,
VReg result_reg_class, VReg result_reg_class,
▲ Show 20 Lines • Show All 63 Lines • ▼ Show 20 Linesmulticlass VPatConversion<string intrinsic,
VReg op1_reg_class> VReg op1_reg_class>
{ {
def : VPatUnaryNoMask<intrinsic, inst, kind, result_type, op1_type, def : VPatUnaryNoMask<intrinsic, inst, kind, result_type, op1_type,
sew, vlmul, op1_reg_class>; sew, vlmul, op1_reg_class>;
def : VPatUnaryMask<intrinsic, inst, kind, result_type, op1_type, def : VPatUnaryMask<intrinsic, inst, kind, result_type, op1_type,
mask_type, sew, vlmul, result_reg_class, op1_reg_class>; mask_type, sew, vlmul, result_reg_class, op1_reg_class>;
} }
multiclass VPatConversionTA<string intrinsic,
string inst,
string kind,
ValueType result_type,
ValueType op1_type,
ValueType mask_type,
int sew,
LMULInfo vlmul,
VReg result_reg_class,
VReg op1_reg_class>
{
def : VPatUnaryNoMask<intrinsic, inst, kind, result_type, op1_type,
sew, vlmul, op1_reg_class>;
def : VPatUnaryMaskTA<intrinsic, inst, kind, result_type, op1_type,
mask_type, sew, vlmul, result_reg_class, op1_reg_class>;
}
multiclass VPatBinaryV_VV<string intrinsic, string instruction, multiclass VPatBinaryV_VV<string intrinsic, string instruction,
list<VTypeInfo> vtilist> { list<VTypeInfo> vtilist> {
foreach vti = vtilist in foreach vti = vtilist in
defm : VPatBinary<intrinsic, instruction # "_VV_" # vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction # "_VV_" # vti.LMul.MX,
vti.Vector, vti.Vector, vti.Vector,vti.Mask, vti.Vector, vti.Vector, vti.Vector,vti.Mask,
vti.Log2SEW, vti.RegClass, vti.Log2SEW, vti.RegClass,
vti.RegClass, vti.RegClass>; vti.RegClass, vti.RegClass>;
} }
multiclass VPatBinaryV_VV_INT<string intrinsic, string instruction, multiclass VPatBinaryV_VV_INT<string intrinsic, string instruction,
list<VTypeInfo> vtilist> { list<VTypeInfo> vtilist> {
foreach vti = vtilist in { foreach vti = vtilist in {
defvar ivti = GetIntVTypeInfo<vti>.Vti; defvar ivti = GetIntVTypeInfo<vti>.Vti;
defm : VPatBinary<intrinsic, instruction # "_VV_" # vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction # "_VV_" # vti.LMul.MX,
vti.Vector, vti.Vector, ivti.Vector, vti.Mask, vti.Vector, vti.Vector, ivti.Vector, vti.Mask,
vti.Log2SEW, vti.RegClass, vti.Log2SEW, vti.RegClass,
vti.RegClass, vti.RegClass>; vti.RegClass, vti.RegClass>;
} }
} }
multiclass VPatBinaryV_VV_INT_EEW<string intrinsic, string instruction, multiclass VPatBinaryV_VV_INT_EEW<string intrinsic, string instruction,
int eew, list<VTypeInfo> vtilist> { int eew, list<VTypeInfo> vtilist> {
foreach vti = vtilist in { foreach vti = vtilist in {
// emul = lmul * eew / sew // emul = lmul * eew / sew
defvar vlmul = vti.LMul; defvar vlmul = vti.LMul;
defvar octuple_lmul = vlmul.octuple; defvar octuple_lmul = vlmul.octuple;
defvar octuple_emul = !srl(!mul(octuple_lmul, eew), vti.Log2SEW); defvar octuple_emul = !srl(!mul(octuple_lmul, eew), vti.Log2SEW);
if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then { if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then {
defvar emul_str = octuple_to_str<octuple_emul>.ret; defvar emul_str = octuple_to_str<octuple_emul>.ret;
defvar ivti = !cast<VTypeInfo>("VI" # eew # emul_str); defvar ivti = !cast<VTypeInfo>("VI" # eew # emul_str);
defvar inst = instruction # "_VV_" # vti.LMul.MX # "_" # emul_str; defvar inst = instruction # "_VV_" # vti.LMul.MX # "_" # emul_str;
defm : VPatBinary<intrinsic, inst, defm : VPatBinaryTA<intrinsic, inst,
vti.Vector, vti.Vector, ivti.Vector, vti.Mask, vti.Vector, vti.Vector, ivti.Vector, vti.Mask,
vti.Log2SEW, vti.RegClass, vti.Log2SEW, vti.RegClass,
vti.RegClass, ivti.RegClass>; vti.RegClass, ivti.RegClass>;
} }
} }
} }
multiclass VPatBinaryV_VX<string intrinsic, string instruction, multiclass VPatBinaryV_VX<string intrinsic, string instruction,
list<VTypeInfo> vtilist> { list<VTypeInfo> vtilist> {
foreach vti = vtilist in { foreach vti = vtilist in {
defvar kind = "V"#vti.ScalarSuffix; defvar kind = "V"#vti.ScalarSuffix;
defm : VPatBinary<intrinsic, instruction#"_"#kind#"_"#vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction#"_"#kind#"_"#vti.LMul.MX,
vti.Vector, vti.Vector, vti.Scalar, vti.Mask, vti.Vector, vti.Vector, vti.Scalar, vti.Mask,
vti.Log2SEW, vti.RegClass, vti.Log2SEW, vti.RegClass,
vti.RegClass, vti.ScalarRegClass>; vti.RegClass, vti.ScalarRegClass>;
} }
} }
multiclass VPatBinaryV_VX_INT<string intrinsic, string instruction, multiclass VPatBinaryV_VX_INT<string intrinsic, string instruction,
list<VTypeInfo> vtilist> { list<VTypeInfo> vtilist> {
foreach vti = vtilist in foreach vti = vtilist in
defm : VPatBinary<intrinsic, instruction # "_VX_" # vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction # "_VX_" # vti.LMul.MX,
vti.Vector, vti.Vector, XLenVT, vti.Mask, vti.Vector, vti.Vector, XLenVT, vti.Mask,
vti.Log2SEW, vti.RegClass, vti.Log2SEW, vti.RegClass,
vti.RegClass, GPR>; vti.RegClass, GPR>;
} }
multiclass VPatBinaryV_VI<string intrinsic, string instruction, multiclass VPatBinaryV_VI<string intrinsic, string instruction,
list<VTypeInfo> vtilist, Operand imm_type> { list<VTypeInfo> vtilist, Operand imm_type> {
foreach vti = vtilist in foreach vti = vtilist in
defm : VPatBinary<intrinsic, instruction # "_VI_" # vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction # "_VI_" # vti.LMul.MX,
vti.Vector, vti.Vector, XLenVT, vti.Mask, vti.Vector, vti.Vector, XLenVT, vti.Mask,
vti.Log2SEW, vti.RegClass, vti.Log2SEW, vti.RegClass,
vti.RegClass, imm_type>; vti.RegClass, imm_type>;
} }
multiclass VPatBinaryM_MM<string intrinsic, string instruction> { multiclass VPatBinaryM_MM<string intrinsic, string instruction> {
foreach mti = AllMasks in foreach mti = AllMasks in
def : VPatBinaryNoMask<intrinsic, instruction # "_MM_" # mti.LMul.MX, def : VPatBinaryNoMask<intrinsic, instruction # "_MM_" # mti.LMul.MX,
mti.Mask, mti.Mask, mti.Mask, mti.Mask, mti.Mask, mti.Mask,
mti.Log2SEW, VR, VR>; mti.Log2SEW, VR, VR>;
} }
multiclass VPatBinaryW_VV<string intrinsic, string instruction, multiclass VPatBinaryW_VV<string intrinsic, string instruction,
list<VTypeInfoToWide> vtilist> { list<VTypeInfoToWide> vtilist> {
foreach VtiToWti = vtilist in { foreach VtiToWti = vtilist in {
defvar Vti = VtiToWti.Vti; defvar Vti = VtiToWti.Vti;
defvar Wti = VtiToWti.Wti; defvar Wti = VtiToWti.Wti;
defm : VPatBinary<intrinsic, instruction # "_VV_" # Vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction # "_VV_" # Vti.LMul.MX,
Wti.Vector, Vti.Vector, Vti.Vector, Vti.Mask, Wti.Vector, Vti.Vector, Vti.Vector, Vti.Mask,
Vti.Log2SEW, Wti.RegClass, Vti.Log2SEW, Wti.RegClass,
Vti.RegClass, Vti.RegClass>; Vti.RegClass, Vti.RegClass>;
} }
} }
multiclass VPatBinaryW_VX<string intrinsic, string instruction, multiclass VPatBinaryW_VX<string intrinsic, string instruction,
list<VTypeInfoToWide> vtilist> { list<VTypeInfoToWide> vtilist> {
foreach VtiToWti = vtilist in { foreach VtiToWti = vtilist in {
defvar Vti = VtiToWti.Vti; defvar Vti = VtiToWti.Vti;
defvar Wti = VtiToWti.Wti; defvar Wti = VtiToWti.Wti;
defvar kind = "V"#Vti.ScalarSuffix; defvar kind = "V"#Vti.ScalarSuffix;
defm : VPatBinary<intrinsic, instruction#"_"#kind#"_"#Vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction#"_"#kind#"_"#Vti.LMul.MX,
Wti.Vector, Vti.Vector, Vti.Scalar, Vti.Mask, Wti.Vector, Vti.Vector, Vti.Scalar, Vti.Mask,
Vti.Log2SEW, Wti.RegClass, Vti.Log2SEW, Wti.RegClass,
Vti.RegClass, Vti.ScalarRegClass>; Vti.RegClass, Vti.ScalarRegClass>;
} }
} }
multiclass VPatBinaryW_WV<string intrinsic, string instruction, multiclass VPatBinaryW_WV<string intrinsic, string instruction,
list<VTypeInfoToWide> vtilist> { list<VTypeInfoToWide> vtilist> {
foreach VtiToWti = vtilist in { foreach VtiToWti = vtilist in {
defvar Vti = VtiToWti.Vti; defvar Vti = VtiToWti.Vti;
defvar Wti = VtiToWti.Wti; defvar Wti = VtiToWti.Wti;
def : VPatTiedBinaryNoMask<intrinsic, instruction # "_WV_" # Vti.LMul.MX, def : VPatTiedBinaryNoMask<intrinsic, instruction # "_WV_" # Vti.LMul.MX,
Wti.Vector, Vti.Vector, Wti.Vector, Vti.Vector,
Vti.Log2SEW, Wti.RegClass, Vti.RegClass>; Vti.Log2SEW, Wti.RegClass, Vti.RegClass>;
let AddedComplexity = 1 in let AddedComplexity = 1 in
def : VPatTiedBinaryMask<intrinsic, instruction # "_WV_" # Vti.LMul.MX, def : VPatTiedBinaryMask<intrinsic, instruction # "_WV_" # Vti.LMul.MX,
Wti.Vector, Vti.Vector, Vti.Mask, Wti.Vector, Vti.Vector, Vti.Mask,
Vti.Log2SEW, Wti.RegClass, Vti.RegClass>; Vti.Log2SEW, Wti.RegClass, Vti.RegClass>;
def : VPatBinaryMask<intrinsic, instruction # "_WV_" # Vti.LMul.MX, def : VPatBinaryMaskTA<intrinsic, instruction # "_WV_" # Vti.LMul.MX,
Wti.Vector, Wti.Vector, Vti.Vector, Vti.Mask, Wti.Vector, Wti.Vector, Vti.Vector, Vti.Mask,
Vti.Log2SEW, Wti.RegClass, Vti.Log2SEW, Wti.RegClass,
Wti.RegClass, Vti.RegClass>; Wti.RegClass, Vti.RegClass>;
} }
} }
multiclass VPatBinaryW_WX<string intrinsic, string instruction, multiclass VPatBinaryW_WX<string intrinsic, string instruction,
list<VTypeInfoToWide> vtilist> { list<VTypeInfoToWide> vtilist> {
foreach VtiToWti = vtilist in { foreach VtiToWti = vtilist in {
defvar Vti = VtiToWti.Vti; defvar Vti = VtiToWti.Vti;
defvar Wti = VtiToWti.Wti; defvar Wti = VtiToWti.Wti;
defvar kind = "W"#Vti.ScalarSuffix; defvar kind = "W"#Vti.ScalarSuffix;
defm : VPatBinary<intrinsic, instruction#"_"#kind#"_"#Vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction#"_"#kind#"_"#Vti.LMul.MX,
Wti.Vector, Wti.Vector, Vti.Scalar, Vti.Mask, Wti.Vector, Wti.Vector, Vti.Scalar, Vti.Mask,
Vti.Log2SEW, Wti.RegClass, Vti.Log2SEW, Wti.RegClass,
Wti.RegClass, Vti.ScalarRegClass>; Wti.RegClass, Vti.ScalarRegClass>;
} }
} }
multiclass VPatBinaryV_WV<string intrinsic, string instruction, multiclass VPatBinaryV_WV<string intrinsic, string instruction,
list<VTypeInfoToWide> vtilist> { list<VTypeInfoToWide> vtilist> {
foreach VtiToWti = vtilist in { foreach VtiToWti = vtilist in {
defvar Vti = VtiToWti.Vti; defvar Vti = VtiToWti.Vti;
defvar Wti = VtiToWti.Wti; defvar Wti = VtiToWti.Wti;
defm : VPatBinary<intrinsic, instruction # "_WV_" # Vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction # "_WV_" # Vti.LMul.MX,
Vti.Vector, Wti.Vector, Vti.Vector, Vti.Mask, Vti.Vector, Wti.Vector, Vti.Vector, Vti.Mask,
Vti.Log2SEW, Vti.RegClass, Vti.Log2SEW, Vti.RegClass,
Wti.RegClass, Vti.RegClass>; Wti.RegClass, Vti.RegClass>;
} }
} }
multiclass VPatBinaryV_WX<string intrinsic, string instruction, multiclass VPatBinaryV_WX<string intrinsic, string instruction,
list<VTypeInfoToWide> vtilist> { list<VTypeInfoToWide> vtilist> {
foreach VtiToWti = vtilist in { foreach VtiToWti = vtilist in {
defvar Vti = VtiToWti.Vti; defvar Vti = VtiToWti.Vti;
defvar Wti = VtiToWti.Wti; defvar Wti = VtiToWti.Wti;
defvar kind = "W"#Vti.ScalarSuffix; defvar kind = "W"#Vti.ScalarSuffix;
defm : VPatBinary<intrinsic, instruction#"_"#kind#"_"#Vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction#"_"#kind#"_"#Vti.LMul.MX,
Vti.Vector, Wti.Vector, Vti.Scalar, Vti.Mask, Vti.Vector, Wti.Vector, Vti.Scalar, Vti.Mask,
Vti.Log2SEW, Vti.RegClass, Vti.Log2SEW, Vti.RegClass,
Wti.RegClass, Vti.ScalarRegClass>; Wti.RegClass, Vti.ScalarRegClass>;
} }
} }
multiclass VPatBinaryV_WI<string intrinsic, string instruction, multiclass VPatBinaryV_WI<string intrinsic, string instruction,
list<VTypeInfoToWide> vtilist> { list<VTypeInfoToWide> vtilist> {
foreach VtiToWti = vtilist in { foreach VtiToWti = vtilist in {
defvar Vti = VtiToWti.Vti; defvar Vti = VtiToWti.Vti;
defvar Wti = VtiToWti.Wti; defvar Wti = VtiToWti.Wti;
defm : VPatBinary<intrinsic, instruction # "_WI_" # Vti.LMul.MX, defm : VPatBinaryTA<intrinsic, instruction # "_WI_" # Vti.LMul.MX,
Vti.Vector, Wti.Vector, XLenVT, Vti.Mask, Vti.Vector, Wti.Vector, XLenVT, Vti.Mask,
Vti.Log2SEW, Vti.RegClass, Vti.Log2SEW, Vti.RegClass,
Wti.RegClass, uimm5>; Wti.RegClass, uimm5>;
} }
} }
multiclass VPatBinaryV_VM<string intrinsic, string instruction, multiclass VPatBinaryV_VM<string intrinsic, string instruction,
bit CarryOut = 0, bit CarryOut = 0,
list<VTypeInfo> vtilist = AllIntegerVectors> { list<VTypeInfo> vtilist = AllIntegerVectors> {
foreach vti = vtilist in foreach vti = vtilist in
defm : VPatBinaryCarryIn<intrinsic, instruction, "VVM", defm : VPatBinaryCarryIn<intrinsic, instruction, "VVM",
▲ Show 20 Lines • Show All 316 Lines • ▼ Show 20 Lines if !le(wtiSEW, 64) then {
wtiM1.Vector, vti.Mask, wtiM1.Vector, vti.Mask,
vti.Log2SEW, vti.LMul, vti.Log2SEW, vti.LMul,
wtiM1.RegClass, vti.RegClass, wtiM1.RegClass, vti.RegClass,
wtiM1.RegClass>; wtiM1.RegClass>;
} }
} }
} }
multiclass VPatConversionVI_VF<string intrinsic, multiclass VPatClassifyVI_VF<string intrinsic,
string instruction> string instruction>
{ {
foreach fvti = AllFloatVectors in foreach fvti = AllFloatVectors in
{ {
defvar ivti = GetIntVTypeInfo<fvti>.Vti; defvar ivti = GetIntVTypeInfo<fvti>.Vti;
defm : VPatConversion<intrinsic, instruction, "V", defm : VPatConversion<intrinsic, instruction, "V",
ivti.Vector, fvti.Vector, ivti.Mask, fvti.Log2SEW, ivti.Vector, fvti.Vector, ivti.Mask, fvti.Log2SEW,
fvti.LMul, ivti.RegClass, fvti.RegClass>; fvti.LMul, ivti.RegClass, fvti.RegClass>;
} }
} }
multiclass VPatConversionVI_VF<string intrinsic,
string instruction>
{
foreach fvti = AllFloatVectors in
{
defvar ivti = GetIntVTypeInfo<fvti>.Vti;
defm : VPatConversionTA<intrinsic, instruction, "V",
ivti.Vector, fvti.Vector, ivti.Mask, fvti.Log2SEW,
fvti.LMul, ivti.RegClass, fvti.RegClass>;
}
}
multiclass VPatConversionVF_VI<string intrinsic, multiclass VPatConversionVF_VI<string intrinsic,
string instruction> string instruction>
{ {
foreach fvti = AllFloatVectors in foreach fvti = AllFloatVectors in
{ {
defvar ivti = GetIntVTypeInfo<fvti>.Vti; defvar ivti = GetIntVTypeInfo<fvti>.Vti;
defm : VPatConversion<intrinsic, instruction, "V", defm : VPatConversionTA<intrinsic, instruction, "V",
fvti.Vector, ivti.Vector, fvti.Mask, ivti.Log2SEW, fvti.Vector, ivti.Vector, fvti.Mask, ivti.Log2SEW,
ivti.LMul, fvti.RegClass, ivti.RegClass>; ivti.LMul, fvti.RegClass, ivti.RegClass>;
} }
} }
multiclass VPatConversionWI_VF<string intrinsic, string instruction> { multiclass VPatConversionWI_VF<string intrinsic, string instruction> {
foreach fvtiToFWti = AllWidenableFloatVectors in foreach fvtiToFWti = AllWidenableFloatVectors in
{ {
defvar fvti = fvtiToFWti.Vti; defvar fvti = fvtiToFWti.Vti;
defvar iwti = GetIntVTypeInfo<fvtiToFWti.Wti>.Vti; defvar iwti = GetIntVTypeInfo<fvtiToFWti.Wti>.Vti;
defm : VPatConversion<intrinsic, instruction, "V", defm : VPatConversionTA<intrinsic, instruction, "V",
iwti.Vector, fvti.Vector, iwti.Mask, fvti.Log2SEW, iwti.Vector, fvti.Vector, iwti.Mask, fvti.Log2SEW,
fvti.LMul, iwti.RegClass, fvti.RegClass>; fvti.LMul, iwti.RegClass, fvti.RegClass>;
} }
} }
multiclass VPatConversionWF_VI<string intrinsic, string instruction> { multiclass VPatConversionWF_VI<string intrinsic, string instruction> {
foreach vtiToWti = AllWidenableIntToFloatVectors in foreach vtiToWti = AllWidenableIntToFloatVectors in
{ {
defvar vti = vtiToWti.Vti; defvar vti = vtiToWti.Vti;
defvar fwti = vtiToWti.Wti; defvar fwti = vtiToWti.Wti;
defm : VPatConversion<intrinsic, instruction, "V", defm : VPatConversionTA<intrinsic, instruction, "V",
fwti.Vector, vti.Vector, fwti.Mask, vti.Log2SEW, fwti.Vector, vti.Vector, fwti.Mask, vti.Log2SEW,
vti.LMul, fwti.RegClass, vti.RegClass>; vti.LMul, fwti.RegClass, vti.RegClass>;
} }
} }
multiclass VPatConversionWF_VF <string intrinsic, string instruction> { multiclass VPatConversionWF_VF <string intrinsic, string instruction> {
foreach fvtiToFWti = AllWidenableFloatVectors in foreach fvtiToFWti = AllWidenableFloatVectors in
{ {
defvar fvti = fvtiToFWti.Vti; defvar fvti = fvtiToFWti.Vti;
defvar fwti = fvtiToFWti.Wti; defvar fwti = fvtiToFWti.Wti;
defm : VPatConversion<intrinsic, instruction, "V", defm : VPatConversionTA<intrinsic, instruction, "V",
fwti.Vector, fvti.Vector, fwti.Mask, fvti.Log2SEW, fwti.Vector, fvti.Vector, fwti.Mask, fvti.Log2SEW,
fvti.LMul, fwti.RegClass, fvti.RegClass>; fvti.LMul, fwti.RegClass, fvti.RegClass>;
} }
} }
multiclass VPatConversionVI_WF <string intrinsic, string instruction> { multiclass VPatConversionVI_WF <string intrinsic, string instruction> {
foreach vtiToWti = AllWidenableIntToFloatVectors in foreach vtiToWti = AllWidenableIntToFloatVectors in
{ {
defvar vti = vtiToWti.Vti; defvar vti = vtiToWti.Vti;
defvar fwti = vtiToWti.Wti; defvar fwti = vtiToWti.Wti;
defm : VPatConversion<intrinsic, instruction, "W", defm : VPatConversionTA<intrinsic, instruction, "W",
vti.Vector, fwti.Vector, vti.Mask, vti.Log2SEW, vti.Vector, fwti.Vector, vti.Mask, vti.Log2SEW,
vti.LMul, vti.RegClass, fwti.RegClass>; vti.LMul, vti.RegClass, fwti.RegClass>;
} }
} }
multiclass VPatConversionVF_WI <string intrinsic, string instruction> { multiclass VPatConversionVF_WI <string intrinsic, string instruction> {
foreach fvtiToFWti = AllWidenableFloatVectors in foreach fvtiToFWti = AllWidenableFloatVectors in
{ {
defvar fvti = fvtiToFWti.Vti; defvar fvti = fvtiToFWti.Vti;
defvar iwti = GetIntVTypeInfo<fvtiToFWti.Wti>.Vti; defvar iwti = GetIntVTypeInfo<fvtiToFWti.Wti>.Vti;
defm : VPatConversion<intrinsic, instruction, "W", defm : VPatConversionTA<intrinsic, instruction, "W",
fvti.Vector, iwti.Vector, fvti.Mask, fvti.Log2SEW, fvti.Vector, iwti.Vector, fvti.Mask, fvti.Log2SEW,
fvti.LMul, fvti.RegClass, iwti.RegClass>; fvti.LMul, fvti.RegClass, iwti.RegClass>;
} }
} }
multiclass VPatConversionVF_WF <string intrinsic, string instruction> { multiclass VPatConversionVF_WF <string intrinsic, string instruction> {
foreach fvtiToFWti = AllWidenableFloatVectors in foreach fvtiToFWti = AllWidenableFloatVectors in
{ {
defvar fvti = fvtiToFWti.Vti; defvar fvti = fvtiToFWti.Vti;
defvar fwti = fvtiToFWti.Wti; defvar fwti = fvtiToFWti.Wti;
defm : VPatConversion<intrinsic, instruction, "W", defm : VPatConversionTA<intrinsic, instruction, "W",
fvti.Vector, fwti.Vector, fvti.Mask, fvti.Log2SEW, fvti.Vector, fwti.Vector, fvti.Mask, fvti.Log2SEW,
fvti.LMul, fvti.RegClass, fwti.RegClass>; fvti.LMul, fvti.RegClass, fwti.RegClass>;
} }
} }
multiclass VPatAMOWD<string intrinsic, multiclass VPatAMOWD<string intrinsic,
string inst, string inst,
ValueType result_type, ValueType result_type,
ValueType offset_type, ValueType offset_type,
ValueType mask_type, ValueType mask_type,
▲ Show 20 Lines • Show All 190 Lines • ▼ Show 20 Lines def : Pat<(vti.Vector (int_riscv_vrsub (vti.Vector vti.RegClass:$rs2),
(!cast<Instruction>("PseudoVSUB_VV_"#vti.LMul.MX) vti.RegClass:$rs1, (!cast<Instruction>("PseudoVSUB_VV_"#vti.LMul.MX) vti.RegClass:$rs1,
vti.RegClass:$rs2, vti.RegClass:$rs2,
GPR:$vl, GPR:$vl,
vti.Log2SEW)>; vti.Log2SEW)>;
def : Pat<(vti.Vector (int_riscv_vrsub_mask (vti.Vector vti.RegClass:$merge), def : Pat<(vti.Vector (int_riscv_vrsub_mask (vti.Vector vti.RegClass:$merge),
(vti.Vector vti.RegClass:$rs2), (vti.Vector vti.RegClass:$rs2),
(vti.Vector vti.RegClass:$rs1), (vti.Vector vti.RegClass:$rs1),
(vti.Mask V0), (vti.Mask V0),
VLOpFrag)), VLOpFrag,
(XLenVT timm:$policy))),
(!cast<Instruction>("PseudoVSUB_VV_"#vti.LMul.MX#"_MASK") (!cast<Instruction>("PseudoVSUB_VV_"#vti.LMul.MX#"_MASK")
vti.RegClass:$merge, vti.RegClass:$merge,
vti.RegClass:$rs1, vti.RegClass:$rs1,
vti.RegClass:$rs2, vti.RegClass:$rs2,
(vti.Mask V0), (vti.Mask V0),
GPR:$vl, GPR:$vl,
vti.Log2SEW)>; vti.Log2SEW,
(XLenVT timm:$policy))>;
// Match VSUB with a small immediate to vadd.vi by negating the immediate. // Match VSUB with a small immediate to vadd.vi by negating the immediate.
def : Pat<(vti.Vector (int_riscv_vsub (vti.Vector vti.RegClass:$rs1), def : Pat<(vti.Vector (int_riscv_vsub (vti.Vector vti.RegClass:$rs1),
(vti.Scalar simm5_plus1:$rs2), (vti.Scalar simm5_plus1:$rs2),
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>("PseudoVADD_VI_"#vti.LMul.MX) vti.RegClass:$rs1, (!cast<Instruction>("PseudoVADD_VI_"#vti.LMul.MX) vti.RegClass:$rs1,
(NegImm simm5_plus1:$rs2), (NegImm simm5_plus1:$rs2),
GPR:$vl, GPR:$vl,
vti.Log2SEW)>; vti.Log2SEW)>;
def : Pat<(vti.Vector (int_riscv_vsub_mask (vti.Vector vti.RegClass:$merge), def : Pat<(vti.Vector (int_riscv_vsub_mask (vti.Vector vti.RegClass:$merge),
(vti.Vector vti.RegClass:$rs1), (vti.Vector vti.RegClass:$rs1),
(vti.Scalar simm5_plus1:$rs2), (vti.Scalar simm5_plus1:$rs2),
(vti.Mask V0), (vti.Mask V0),
VLOpFrag)), VLOpFrag,
(XLenVT timm:$policy))),
(!cast<Instruction>("PseudoVADD_VI_"#vti.LMul.MX#"_MASK") (!cast<Instruction>("PseudoVADD_VI_"#vti.LMul.MX#"_MASK")
vti.RegClass:$merge, vti.RegClass:$merge,
vti.RegClass:$rs1, vti.RegClass:$rs1,
(NegImm simm5_plus1:$rs2), (NegImm simm5_plus1:$rs2),
(vti.Mask V0), (vti.Mask V0),
GPR:$vl, GPR:$vl,
vti.Log2SEW)>; vti.Log2SEW,
(XLenVT timm:$policy))>;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// 12.2. Vector Widening Integer Add/Subtract // 12.2. Vector Widening Integer Add/Subtract
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
defm PseudoVWADDU : VPseudoBinaryW_VV_VX; defm PseudoVWADDU : VPseudoBinaryW_VV_VX;
defm PseudoVWSUBU : VPseudoBinaryW_VV_VX; defm PseudoVWSUBU : VPseudoBinaryW_VV_VX;
defm PseudoVWADD : VPseudoBinaryW_VV_VX; defm PseudoVWADD : VPseudoBinaryW_VV_VX;
▲ Show 20 Lines • Show All 204 Lines • ▼ Show 20 Lines
defm PseudoVFWMACC : VPseudoTernaryW_VV_VF; defm PseudoVFWMACC : VPseudoTernaryW_VV_VF;
defm PseudoVFWNMACC : VPseudoTernaryW_VV_VF; defm PseudoVFWNMACC : VPseudoTernaryW_VV_VF;
defm PseudoVFWMSAC : VPseudoTernaryW_VV_VF; defm PseudoVFWMSAC : VPseudoTernaryW_VV_VF;
defm PseudoVFWNMSAC : VPseudoTernaryW_VV_VF; defm PseudoVFWNMSAC : VPseudoTernaryW_VV_VF;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// 14.8. Vector Floating-Point Square-Root Instruction // 14.8. Vector Floating-Point Square-Root Instruction
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
defm PseudoVFSQRT : VPseudoUnaryV_V; defm PseudoVFSQRT : VPseudoUnaryTAV_V;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// 14.9. Vector Floating-Point Reciprocal Square-Root Estimate Instruction // 14.9. Vector Floating-Point Reciprocal Square-Root Estimate Instruction
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
defm PseudoVFRSQRT7 : VPseudoUnaryV_V; defm PseudoVFRSQRT7 : VPseudoUnaryTAV_V;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// 14.10. Vector Floating-Point Reciprocal Estimate Instruction // 14.10. Vector Floating-Point Reciprocal Estimate Instruction
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
defm PseudoVFREC7 : VPseudoUnaryV_V; defm PseudoVFREC7 : VPseudoUnaryTAV_V;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// 14.11. Vector Floating-Point Min/Max Instructions // 14.11. Vector Floating-Point Min/Max Instructions
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
defm PseudoVFMIN : VPseudoBinaryV_VV_VF; defm PseudoVFMIN : VPseudoBinaryV_VV_VF;
defm PseudoVFMAX : VPseudoBinaryV_VV_VF; defm PseudoVFMAX : VPseudoBinaryV_VV_VF;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 669 Lines • ▼ Show 20 Lines
defm : VPatBinaryM_VX<"int_riscv_vmfgt", "PseudoVMFGT", AllFloatVectors>; defm : VPatBinaryM_VX<"int_riscv_vmfgt", "PseudoVMFGT", AllFloatVectors>;
defm : VPatBinaryM_VX<"int_riscv_vmfge", "PseudoVMFGE", AllFloatVectors>; defm : VPatBinaryM_VX<"int_riscv_vmfge", "PseudoVMFGE", AllFloatVectors>;
defm : VPatBinarySwappedM_VV<"int_riscv_vmfgt", "PseudoVMFLT", AllFloatVectors>; defm : VPatBinarySwappedM_VV<"int_riscv_vmfgt", "PseudoVMFLT", AllFloatVectors>;
defm : VPatBinarySwappedM_VV<"int_riscv_vmfge", "PseudoVMFLE", AllFloatVectors>; defm : VPatBinarySwappedM_VV<"int_riscv_vmfge", "PseudoVMFLE", AllFloatVectors>;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// 14.14. Vector Floating-Point Classify Instruction // 14.14. Vector Floating-Point Classify Instruction
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
defm : VPatConversionVI_VF<"int_riscv_vfclass", "PseudoVFCLASS">; defm : VPatClassifyVI_VF<"int_riscv_vfclass", "PseudoVFCLASS">;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// 14.15. Vector Floating-Point Merge Instruction // 14.15. Vector Floating-Point Merge Instruction
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// We can use vmerge.vvm to support vector-vector vfmerge. // We can use vmerge.vvm to support vector-vector vfmerge.
defm : VPatBinaryV_VM<"int_riscv_vfmerge", "PseudoVMERGE", defm : VPatBinaryV_VM<"int_riscv_vfmerge", "PseudoVMERGE",
/*CarryOut = */0, /*vtilist=*/AllFloatVectors>; /*CarryOut = */0, /*vtilist=*/AllFloatVectors>;
defm : VPatBinaryV_XM<"int_riscv_vfmerge", "PseudoVFMERGE", defm : VPatBinaryV_XM<"int_riscv_vfmerge", "PseudoVFMERGE",
▲ Show 20 Lines • Show All 224 Lines • Show Last 20 Lines

llvm/lib/Target/RISCV/RISCVInstrInfoVVLPatterns.td

Show All 16 Lines
/// RISCVInstrInfoVPseudos.td. /// RISCVInstrInfoVPseudos.td.
/// ///
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Helpers to define the VL patterns. // Helpers to define the VL patterns.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def SDT_RISCVVLE_VL : SDTypeProfile<1, 2, [SDTCisVec<0>, SDTCisPtrTy<1>, def SDT_RISCVVLE_VL : SDTypeProfile<1, 2, [SDTCisVec<0>, SDTCisPtrTy<1>,
craig.topperUnsubmitted
Done
ReplyInline Actions

Why are these the opposite polarity of what's in C?

craig.topper: Why are these the opposite polarity of what's in C?
SDTCisVT<2, XLenVT>]>; SDTCisVT<2, XLenVT>]>;
def SDT_RISCVVSE_VL : SDTypeProfile<0, 3, [SDTCisVec<0>, SDTCisPtrTy<1>, def SDT_RISCVVSE_VL : SDTypeProfile<0, 3, [SDTCisVec<0>, SDTCisPtrTy<1>,
SDTCisVT<2, XLenVT>]>; SDTCisVT<2, XLenVT>]>;
def SDT_RISCVIntBinOp_VL : SDTypeProfile<1, 4, [SDTCisSameAs<0, 1>, def SDT_RISCVIntBinOp_VL : SDTypeProfile<1, 4, [SDTCisSameAs<0, 1>,
SDTCisSameAs<0, 2>, SDTCisSameAs<0, 2>,
SDTCisVec<0>, SDTCisInt<0>, SDTCisVec<0>, SDTCisInt<0>,
SDTCVecEltisVT<3, i1>, SDTCVecEltisVT<3, i1>,
▲ Show 20 Lines • Show All 253 Lines • ▼ Show 20 Lines def : Pat<(result_type (vop
(op_type op_reg_class:$rs1), (op_type op_reg_class:$rs1),
(op_type op_reg_class:$rs2), (op_type op_reg_class:$rs2),
(mask_type V0), (mask_type V0),
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>(instruction_name#"_VV_"# vlmul.MX#"_MASK") (!cast<Instruction>(instruction_name#"_VV_"# vlmul.MX#"_MASK")
(result_type (IMPLICIT_DEF)), (result_type (IMPLICIT_DEF)),
op_reg_class:$rs1, op_reg_class:$rs1,
op_reg_class:$rs2, op_reg_class:$rs2,
(mask_type V0), GPR:$vl, sew)>; (mask_type V0), GPR:$vl, sew, TAIL_AGNOSTIC)>;
} }
multiclass VPatBinaryVL_XI<SDNode vop, multiclass VPatBinaryVL_XI<SDNode vop,
string instruction_name, string instruction_name,
string suffix, string suffix,
ValueType result_type, ValueType result_type,
ValueType vop_type, ValueType vop_type,
ValueType mask_type, ValueType mask_type,
Show All 15 Lines def : Pat<(result_type (vop
(vop_type vop_reg_class:$rs1), (vop_type vop_reg_class:$rs1),
(vop_type (SplatPatKind (XLenVT xop_kind:$rs2))), (vop_type (SplatPatKind (XLenVT xop_kind:$rs2))),
(mask_type V0), (mask_type V0),
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>(instruction_name#_#suffix#_# vlmul.MX#"_MASK") (!cast<Instruction>(instruction_name#_#suffix#_# vlmul.MX#"_MASK")
(result_type (IMPLICIT_DEF)), (result_type (IMPLICIT_DEF)),
vop_reg_class:$rs1, vop_reg_class:$rs1,
xop_kind:$rs2, xop_kind:$rs2,
(mask_type V0), GPR:$vl, sew)>; (mask_type V0), GPR:$vl, sew, TAIL_AGNOSTIC)>;
} }
multiclass VPatBinaryVL_VV_VX<SDNode vop, string instruction_name> { multiclass VPatBinaryVL_VV_VX<SDNode vop, string instruction_name> {
foreach vti = AllIntegerVectors in { foreach vti = AllIntegerVectors in {
defm : VPatBinaryVL_VV<vop, instruction_name, defm : VPatBinaryVL_VV<vop, instruction_name,
vti.Vector, vti.Vector, vti.Mask, vti.Log2SEW, vti.Vector, vti.Vector, vti.Mask, vti.Log2SEW,
vti.LMul, vti.RegClass>; vti.LMul, vti.RegClass>;
defm : VPatBinaryVL_XI<vop, instruction_name, "VX", defm : VPatBinaryVL_XI<vop, instruction_name, "VX",
▲ Show 20 Lines • Show All 306 Lines • ▼ Show 20 Lines def : Pat<(riscv_sub_vl (vti.Vector (SplatPat (XLenVT GPR:$rs2))),
VLOpFrag), VLOpFrag),
(!cast<Instruction>("PseudoVRSUB_VX_"# vti.LMul.MX) (!cast<Instruction>("PseudoVRSUB_VX_"# vti.LMul.MX)
vti.RegClass:$rs1, GPR:$rs2, GPR:$vl, vti.Log2SEW)>; vti.RegClass:$rs1, GPR:$rs2, GPR:$vl, vti.Log2SEW)>;
def : Pat<(riscv_sub_vl (vti.Vector (SplatPat (XLenVT GPR:$rs2))), def : Pat<(riscv_sub_vl (vti.Vector (SplatPat (XLenVT GPR:$rs2))),
(vti.Vector vti.RegClass:$rs1), (vti.Mask V0), (vti.Vector vti.RegClass:$rs1), (vti.Mask V0),
VLOpFrag), VLOpFrag),
(!cast<Instruction>("PseudoVRSUB_VX_"# vti.LMul.MX#"_MASK") (!cast<Instruction>("PseudoVRSUB_VX_"# vti.LMul.MX#"_MASK")
(vti.Vector (IMPLICIT_DEF)), vti.RegClass:$rs1, GPR:$rs2, (vti.Vector (IMPLICIT_DEF)), vti.RegClass:$rs1, GPR:$rs2,
(vti.Mask V0), GPR:$vl, vti.Log2SEW)>; (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
def : Pat<(riscv_sub_vl (vti.Vector (SplatPat_simm5 simm5:$rs2)), def : Pat<(riscv_sub_vl (vti.Vector (SplatPat_simm5 simm5:$rs2)),
(vti.Vector vti.RegClass:$rs1), (vti.Mask true_mask), (vti.Vector vti.RegClass:$rs1), (vti.Mask true_mask),
VLOpFrag), VLOpFrag),
(!cast<Instruction>("PseudoVRSUB_VI_"# vti.LMul.MX) (!cast<Instruction>("PseudoVRSUB_VI_"# vti.LMul.MX)
vti.RegClass:$rs1, simm5:$rs2, GPR:$vl, vti.Log2SEW)>; vti.RegClass:$rs1, simm5:$rs2, GPR:$vl, vti.Log2SEW)>;
def : Pat<(riscv_sub_vl (vti.Vector (SplatPat_simm5 simm5:$rs2)), def : Pat<(riscv_sub_vl (vti.Vector (SplatPat_simm5 simm5:$rs2)),
(vti.Vector vti.RegClass:$rs1), (vti.Mask V0), (vti.Vector vti.RegClass:$rs1), (vti.Mask V0),
VLOpFrag), VLOpFrag),
(!cast<Instruction>("PseudoVRSUB_VI_"# vti.LMul.MX#"_MASK") (!cast<Instruction>("PseudoVRSUB_VI_"# vti.LMul.MX#"_MASK")
(vti.Vector (IMPLICIT_DEF)), vti.RegClass:$rs1, simm5:$rs2, (vti.Vector (IMPLICIT_DEF)), vti.RegClass:$rs1, simm5:$rs2,
(vti.Mask V0), GPR:$vl, vti.Log2SEW)>; (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
} }
// 12.3. Vector Integer Extension // 12.3. Vector Integer Extension
defm : VPatExtendSDNode_V_VL<riscv_zext_vl, "PseudoVZEXT", "VF2", defm : VPatExtendSDNode_V_VL<riscv_zext_vl, "PseudoVZEXT", "VF2",
AllFractionableVF2IntVectors>; AllFractionableVF2IntVectors>;
defm : VPatExtendSDNode_V_VL<riscv_sext_vl, "PseudoVSEXT", "VF2", defm : VPatExtendSDNode_V_VL<riscv_sext_vl, "PseudoVSEXT", "VF2",
AllFractionableVF2IntVectors>; AllFractionableVF2IntVectors>;
defm : VPatExtendSDNode_V_VL<riscv_zext_vl, "PseudoVZEXT", "VF4", defm : VPatExtendSDNode_V_VL<riscv_zext_vl, "PseudoVZEXT", "VF4",
▲ Show 20 Lines • Show All 643 Lines • ▼ Show 20 Lines def : Pat<(vti.Vector (riscv_vselect_vl (vti.Mask V0),
vti.RegClass:$rs2, vti.RegClass:$rs2,
vti.RegClass:$rs1, vti.RegClass:$rs1,
(vti.Mask true_mask), (vti.Mask true_mask),
VLOpFrag), VLOpFrag),
vti.RegClass:$merge, vti.RegClass:$merge,
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX#"_MASK") (!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX#"_MASK")
vti.RegClass:$merge, vti.RegClass:$rs2, vti.RegClass:$rs1, vti.RegClass:$merge, vti.RegClass:$rs2, vti.RegClass:$rs1,
(vti.Mask V0), GPR:$vl, vti.Log2SEW)>; (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

This should be TAIL_UNDISTURBED. It has a $merge operand

craig.topper: This should be TAIL_UNDISTURBED. It has a $merge operand
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Err nevermind. Maybe we only cared about the mask. Probably should add the policy to riscv_vselect_vl, but that's for another patch.

craig.topper: Err nevermind. Maybe we only cared about the mask. Probably should add the policy to…
def : Pat<(vti.Vector (riscv_vselect_vl (vti.Mask V0), def : Pat<(vti.Vector (riscv_vselect_vl (vti.Mask V0),
(riscv_vrgather_vx_vl (riscv_vrgather_vx_vl
vti.RegClass:$rs2, vti.RegClass:$rs2,
uimm5:$imm, uimm5:$imm,
(vti.Mask true_mask), (vti.Mask true_mask),
VLOpFrag), VLOpFrag),
vti.RegClass:$merge, vti.RegClass:$merge,
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>("PseudoVRGATHER_VI_"# vti.LMul.MX#"_MASK") (!cast<Instruction>("PseudoVRGATHER_VI_"# vti.LMul.MX#"_MASK")
vti.RegClass:$merge, vti.RegClass:$rs2, uimm5:$imm, vti.RegClass:$merge, vti.RegClass:$rs2, uimm5:$imm,
(vti.Mask V0), GPR:$vl, vti.Log2SEW)>; (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Same here

craig.topper: Same here
// emul = lmul * 16 / sew // emul = lmul * 16 / sew
defvar vlmul = vti.LMul; defvar vlmul = vti.LMul;
defvar octuple_lmul = vlmul.octuple; defvar octuple_lmul = vlmul.octuple;
defvar octuple_emul = !srl(!mul(octuple_lmul, 16), vti.Log2SEW); defvar octuple_emul = !srl(!mul(octuple_lmul, 16), vti.Log2SEW);
if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then { if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then {
defvar emul_str = octuple_to_str<octuple_emul>.ret; defvar emul_str = octuple_to_str<octuple_emul>.ret;
defvar ivti = !cast<VTypeInfo>("VI16" # emul_str); defvar ivti = !cast<VTypeInfo>("VI16" # emul_str);
Show All 10 Lines def : Pat<(vti.Vector (riscv_vselect_vl (vti.Mask V0),
vti.RegClass:$rs2, vti.RegClass:$rs2,
(ivti.Vector ivti.RegClass:$rs1), (ivti.Vector ivti.RegClass:$rs1),
(vti.Mask true_mask), (vti.Mask true_mask),
VLOpFrag), VLOpFrag),
vti.RegClass:$merge, vti.RegClass:$merge,
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>(inst#"_MASK") (!cast<Instruction>(inst#"_MASK")
vti.RegClass:$merge, vti.RegClass:$rs2, ivti.RegClass:$rs1, vti.RegClass:$merge, vti.RegClass:$rs2, ivti.RegClass:$rs1,
(vti.Mask V0), GPR:$vl, vti.Log2SEW)>; (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

And here

craig.topper: And here
} }
} }
} // Predicates = [HasStdExtV] } // Predicates = [HasStdExtV]
let Predicates = [HasStdExtV, HasStdExtF] in { let Predicates = [HasStdExtV, HasStdExtF] in {
// 17.2. Floating-Point Scalar Move Instructions // 17.2. Floating-Point Scalar Move Instructions
Show All 27 Lines def : Pat<(vti.Vector (riscv_vselect_vl (vti.Mask V0),
vti.RegClass:$rs2, vti.RegClass:$rs2,
(ivti.Vector vti.RegClass:$rs1), (ivti.Vector vti.RegClass:$rs1),
(vti.Mask true_mask), (vti.Mask true_mask),
VLOpFrag), VLOpFrag),
vti.RegClass:$merge, vti.RegClass:$merge,
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX#"_MASK") (!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX#"_MASK")
vti.RegClass:$merge, vti.RegClass:$rs2, vti.RegClass:$rs1, vti.RegClass:$merge, vti.RegClass:$rs2, vti.RegClass:$rs1,
(vti.Mask V0), GPR:$vl, vti.Log2SEW)>; (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
def : Pat<(vti.Vector (riscv_vselect_vl (vti.Mask V0), def : Pat<(vti.Vector (riscv_vselect_vl (vti.Mask V0),
(riscv_vrgather_vx_vl (riscv_vrgather_vx_vl
vti.RegClass:$rs2, vti.RegClass:$rs2,
uimm5:$imm, uimm5:$imm,
(vti.Mask true_mask), (vti.Mask true_mask),
VLOpFrag), VLOpFrag),
vti.RegClass:$merge, vti.RegClass:$merge,
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>("PseudoVRGATHER_VI_"# vti.LMul.MX#"_MASK") (!cast<Instruction>("PseudoVRGATHER_VI_"# vti.LMul.MX#"_MASK")
vti.RegClass:$merge, vti.RegClass:$rs2, uimm5:$imm, vti.RegClass:$merge, vti.RegClass:$rs2, uimm5:$imm,
(vti.Mask V0), GPR:$vl, vti.Log2SEW)>; (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

And here

craig.topper: And here
defvar vlmul = vti.LMul; defvar vlmul = vti.LMul;
defvar octuple_lmul = vlmul.octuple; defvar octuple_lmul = vlmul.octuple;
defvar octuple_emul = !srl(!mul(octuple_lmul, 16), vti.Log2SEW); defvar octuple_emul = !srl(!mul(octuple_lmul, 16), vti.Log2SEW);
if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then { if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then {
defvar emul_str = octuple_to_str<octuple_emul>.ret; defvar emul_str = octuple_to_str<octuple_emul>.ret;
defvar ivti = !cast<VTypeInfo>("VI16" # emul_str); defvar ivti = !cast<VTypeInfo>("VI16" # emul_str);
defvar inst = "PseudoVRGATHEREI16_VV_" # vti.LMul.MX # "_" # emul_str; defvar inst = "PseudoVRGATHEREI16_VV_" # vti.LMul.MX # "_" # emul_str;
Show All 9 Lines def : Pat<(vti.Vector (riscv_vselect_vl (vti.Mask V0),
vti.RegClass:$rs2, vti.RegClass:$rs2,
(ivti.Vector ivti.RegClass:$rs1), (ivti.Vector ivti.RegClass:$rs1),
(vti.Mask true_mask), (vti.Mask true_mask),
VLOpFrag), VLOpFrag),
vti.RegClass:$merge, vti.RegClass:$merge,
VLOpFrag)), VLOpFrag)),
(!cast<Instruction>(inst#"_MASK") (!cast<Instruction>(inst#"_MASK")
vti.RegClass:$merge, vti.RegClass:$rs2, ivti.RegClass:$rs1, vti.RegClass:$merge, vti.RegClass:$rs2, ivti.RegClass:$rs1,
(vti.Mask V0), GPR:$vl, vti.Log2SEW)>; (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

And here

craig.topper: And here
} }
} }
} // Predicates = [HasStdExtV, HasStdExtF] } // Predicates = [HasStdExtV, HasStdExtF]
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Miscellaneous RISCVISD SDNodes // Miscellaneous RISCVISD SDNodes
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 73 Lines • Show Last 20 Lines

llvm/test/CodeGen/RISCV/rvv/common-shuffle-patterns.ll

llvm/test/CodeGen/RISCV/rvv/fixed-vector-strided-load-store.ll

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-buildvec.ll

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-shuffles.ll

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-shuffles.ll

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-masked-gather.ll

llvm/test/CodeGen/RISCV/rvv/interleave-crash.ll

llvm/test/CodeGen/RISCV/rvv/mgather-sdnode.ll

llvm/test/CodeGen/RISCV/rvv/tail-agnostic-impdef-copy.mir

llvm/test/CodeGen/RISCV/rvv/vaadd-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vaadd-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vaaddu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vaaddu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vadd-policy.ll

llvm/test/CodeGen/RISCV/rvv/vadd-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vadd-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vand-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vand-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vasub-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vasub-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vasubu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vasubu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vdiv-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vdiv-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vdivu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vdivu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfadd-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfadd-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-f-x-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-f-x-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-f-xu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-f-xu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-rtz-x-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-rtz-x-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-rtz-xu-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-rtz-xu-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-x-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-x-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-xu-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfcvt-xu-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfdiv-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfdiv-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfmax-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfmax-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfmin-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfmin-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfmul-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfmul-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-f-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-f-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-f-x-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-f-x-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-f-xu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-f-xu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-rod-f-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-rod-f-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-rtz-x-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-rtz-x-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-rtz-xu-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-rtz-xu-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-x-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-x-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-xu-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfncvt-xu-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfrdiv-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfrdiv-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfrec7-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfrec7-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfrsqrt7-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfrsqrt7-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfrsub-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfrsub-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfsgnj-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfsgnj-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfsgnjn-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfsgnjn-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfsgnjx-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfsgnjx-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfslide1down-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfslide1down-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfslide1up-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfslide1up-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfsqrt-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfsqrt-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfsub-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfsub-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwadd-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwadd-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwadd.w-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwadd.w-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-f-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-f-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-f-x-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-f-x-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-f-xu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-f-xu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-rtz-x-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-rtz-x-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-rtz-xu-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-rtz-xu-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-x-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-x-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-xu-f-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwcvt-xu-f-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwmul-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwmul-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwsub-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwsub-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vfwsub.w-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vfwsub.w-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vle-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vle-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vleff-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vleff-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vloxei-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vloxei-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vloxseg-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vloxseg-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vlse-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vlse-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vlseg-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vlseg-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vlsegff-rv32-dead.ll

llvm/test/CodeGen/RISCV/rvv/vlsegff-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vlsegff-rv64-dead.ll

llvm/test/CodeGen/RISCV/rvv/vlsegff-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vlsseg-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vlsseg-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vluxei-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vluxei-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vluxseg-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vluxseg-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vmax-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vmax-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vmaxu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vmaxu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vmin-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vmin-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vminu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vminu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vmul-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vmul-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vmulh-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vmulh-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vmulhsu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vmulhsu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vmulhu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vmulhu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vnclip-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vnclip-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vnclipu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vnclipu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vnsra-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vnsra-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vnsrl-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vnsrl-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vor-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vor-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vrem-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vrem-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vremu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vremu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vrgather-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vrgather-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vrgatherei16-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vrgatherei16-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vrsub-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vrsub-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vsadd-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vsadd-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vsaddu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vsaddu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vsetvli-insert.ll

llvm/test/CodeGen/RISCV/rvv/vsext-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vsext-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vslide1down-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vslide1down-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vslide1up-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vslide1up-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vsll-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vsll-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vsmul-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vsmul-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vsra-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vsra-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vsrl-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vsrl-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vssra-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vssra-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vssrl-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vssrl-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vssub-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vssub-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vssubu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vssubu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vsub-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vsub-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwadd-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwadd-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwadd.w-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwadd.w-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwaddu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwaddu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwaddu.w-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwaddu.w-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwmul-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwmul-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwmulsu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwmulsu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwmulu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwmulu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwsub-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwsub-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwsub.w-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwsub.w-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwsubu-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwsubu-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vwsubu.w-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vwsubu.w-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vxor-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vxor-rv64.ll

llvm/test/CodeGen/RISCV/rvv/vzext-rv32.ll

llvm/test/CodeGen/RISCV/rvv/vzext-rv64.ll

llvm/test/CodeGen/RISCV/rvv/zvlsseg-zero-vl.ll