This is an archive of the discontinued LLVM Phabricator instance.

Differential D119834

[RISCV] Add fixed-length vector instrinsics for segment load
ClosedPublic

Authored by luke957 on Feb 15 2022, 5:12 AM.

Details

Reviewers
craig.topper
frasercrmck
Commits
rG0803dba7dd99: [RISCV] Add fixed-length vector instrinsics for segment load
Summary

Add fixed-length vector instrinsics for segment load.
Inspired by reviews.llvm.org/D107790.
These fixed-length vector segNload instrinsics would be helpful to https://reviews.llvm.org/D107210.

Diff Detail

Event Timeline

luke957 created this revision.Feb 15 2022, 5:12 AM
Herald added subscribers: VincentWu, achieveartificialintelligence, vkmr and 25 others. · View Herald TranscriptFeb 15 2022, 5:12 AM
luke957 requested review of this revision.Feb 15 2022, 5:12 AM
Herald added a project: Restricted Project. · View Herald TranscriptFeb 15 2022, 5:12 AM
Herald added subscribers: llvm-commits, • pcwang-thead, eopXD, MaskRay. · View Herald Transcript
luke957 edited the summary of this revision. (Show Details)Feb 15 2022, 5:23 AM
Harbormaster completed remote builds in B149664: Diff 408826.Feb 15 2022, 5:54 AM
luke957 added a comment.Feb 22 2022, 11:08 PM

Hi, any comments? :)

craig.topper added inline comments.Feb 22 2022, 11:33 PM
llvm/lib/Target/RISCV/RISCVISelLowering.cpp
4913

Unexcepted -> Unexpected

4921

You don't need a SmallVector, the number of items is fixed. You can use a plain array.

4925

I think you can do

SmallVector<EVT, 8> ContainerVTs(NF, ContainerVT);

and avoid the loop.

4935

Drop the curly braces around a single line body.

luke957 updated this revision to Diff 411120.Feb 24 2022, 7:16 AM

Address comments

luke957 updated this revision to Diff 411126.Feb 24 2022, 7:28 AM
Harbormaster completed remote builds in B151276: Diff 411126.Feb 24 2022, 8:37 AM
luke957 added a comment.Mar 2 2022, 8:46 PM

Gently ping.

Herald added a project: Restricted Project. · View Herald TranscriptMar 2 2022, 8:46 PM
craig.topper accepted this revision.Mar 7 2022, 3:13 PM

LGTM to me other than those 2 comments to change 8 to 9.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp
4921

Use 9 instead of 8. Since NF is <= 8 and you push one additional item for MVT::Other.

4928

Use 9 instead of 8.

This revision is now accepted and ready to land.Mar 7 2022, 3:13 PM
This revision was landed with ongoing or failed builds.Mar 10 2022, 12:33 AM
Closed by commit rG0803dba7dd99: [RISCV] Add fixed-length vector instrinsics for segment load (authored by luke957). · Explain Why
This revision was automatically updated to reflect the committed changes.
luke957 added a commit: rG0803dba7dd99: [RISCV] Add fixed-length vector instrinsics for segment load.
luke957 added a comment.Mar 10 2022, 1:27 AM

Comments addressed before commit. Thanks.

Revision Contents

PathSize
llvm/
include/
llvm/
IR/
10 lines
lib/
Target/
RISCV/
54 lines
test/
CodeGen/
RISCV/
rvv/
129 lines

Diff 414288

llvm/include/llvm/IR/IntrinsicsRISCV.td

Show First 20 Lines • Show All 1,493 Lines • ▼ Show 20 Lines def int_riscv_masked_strided_load
[LLVMMatchType<0>, llvm_anyptr_ty, [LLVMMatchType<0>, llvm_anyptr_ty,
llvm_anyint_ty, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>], llvm_anyint_ty, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>],
[NoCapture<ArgIndex<1>>, IntrReadMem]>; [NoCapture<ArgIndex<1>>, IntrReadMem]>;
def int_riscv_masked_strided_store def int_riscv_masked_strided_store
: Intrinsic<[], : Intrinsic<[],
[llvm_anyvector_ty, llvm_anyptr_ty, [llvm_anyvector_ty, llvm_anyptr_ty,
llvm_anyint_ty, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>], llvm_anyint_ty, LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>],
[NoCapture<ArgIndex<1>>, IntrWriteMem]>; [NoCapture<ArgIndex<1>>, IntrWriteMem]>;
// Segment loads for fixed vectors.
foreach nf = [2, 3, 4, 5, 6, 7, 8] in {
def int_riscv_seg # nf # _load
: Intrinsic<!listconcat([llvm_anyvector_ty], !listsplat(LLVMMatchType<0>,
!add(nf, -1))),
[llvm_anyptr_ty, llvm_anyint_ty],
[NoCapture<ArgIndex<0>>, IntrReadMem]>;
}
} // TargetPrefix = "riscv" } // TargetPrefix = "riscv"
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Scalar Cryptography // Scalar Cryptography
// //
// These intrinsics will lower directly into the corresponding instructions // These intrinsics will lower directly into the corresponding instructions
// added by the scalar cyptography extension, if the extension is present. // added by the scalar cyptography extension, if the extension is present.
▲ Show 20 Lines • Show All 82 Lines • Show Last 20 Lines

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 1,120 Lines • ▼ Show 20 Lines case Intrinsic::riscv_masked_strided_store:
Info.memVT = Info.memVT =
getValueType(DL, I.getArgOperand(0)->getType()->getScalarType()); getValueType(DL, I.getArgOperand(0)->getType()->getScalarType());
Info.align = Align( Info.align = Align(
DL.getTypeSizeInBits(I.getArgOperand(0)->getType()->getScalarType()) / DL.getTypeSizeInBits(I.getArgOperand(0)->getType()->getScalarType()) /
8); 8);
Info.size = MemoryLocation::UnknownSize; Info.size = MemoryLocation::UnknownSize;
Info.flags |= MachineMemOperand::MOStore; Info.flags |= MachineMemOperand::MOStore;
return true; return true;
case Intrinsic::riscv_seg2_load:
case Intrinsic::riscv_seg3_load:
case Intrinsic::riscv_seg4_load:
case Intrinsic::riscv_seg5_load:
case Intrinsic::riscv_seg6_load:
case Intrinsic::riscv_seg7_load:
case Intrinsic::riscv_seg8_load:
Info.opc = ISD::INTRINSIC_W_CHAIN;
Info.ptrVal = I.getArgOperand(0);
Info.memVT =
getValueType(DL, I.getType()->getStructElementType(0)->getScalarType());
Info.align =
Align(DL.getTypeSizeInBits(
I.getType()->getStructElementType(0)->getScalarType()) /
8);
Info.size = MemoryLocation::UnknownSize;
Info.flags |= MachineMemOperand::MOLoad;
return true;
} }
} }
bool RISCVTargetLowering::isLegalAddressingMode(const DataLayout &DL, bool RISCVTargetLowering::isLegalAddressingMode(const DataLayout &DL,
const AddrMode &AM, Type *Ty, const AddrMode &AM, Type *Ty,
unsigned AS, unsigned AS,
Instruction *I) const { Instruction *I) const {
// No global is ever allowed as a base. // No global is ever allowed as a base.
▲ Show 20 Lines • Show All 3,736 Lines • ▼ Show 20 Lines case Intrinsic::riscv_masked_strided_load: {
SDVTList VTs = DAG.getVTList({ContainerVT, MVT::Other}); SDVTList VTs = DAG.getVTList({ContainerVT, MVT::Other});
SDValue Result = SDValue Result =
DAG.getMemIntrinsicNode(ISD::INTRINSIC_W_CHAIN, DL, VTs, Ops, DAG.getMemIntrinsicNode(ISD::INTRINSIC_W_CHAIN, DL, VTs, Ops,
Load->getMemoryVT(), Load->getMemOperand()); Load->getMemoryVT(), Load->getMemOperand());
SDValue Chain = Result.getValue(1); SDValue Chain = Result.getValue(1);
Result = convertFromScalableVector(VT, Result, DAG, Subtarget); Result = convertFromScalableVector(VT, Result, DAG, Subtarget);
return DAG.getMergeValues({Result, Chain}, DL); return DAG.getMergeValues({Result, Chain}, DL);
} }
case Intrinsic::riscv_seg2_load:
case Intrinsic::riscv_seg3_load:
case Intrinsic::riscv_seg4_load:
case Intrinsic::riscv_seg5_load:
case Intrinsic::riscv_seg6_load:
case Intrinsic::riscv_seg7_load:
case Intrinsic::riscv_seg8_load: {
SDLoc DL(Op);
static const Intrinsic::ID VlsegInts[7] = {
Intrinsic::riscv_vlseg2, Intrinsic::riscv_vlseg3,
Intrinsic::riscv_vlseg4, Intrinsic::riscv_vlseg5,
Intrinsic::riscv_vlseg6, Intrinsic::riscv_vlseg7,
Intrinsic::riscv_vlseg8};
unsigned NF = Op->getNumValues() - 1;
assert(NF >= 2 && NF <= 8 && "Unexpected seg number");
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Unexcepted -> Unexpected

craig.topper: Unexcepted -> Unexpected
MVT XLenVT = Subtarget.getXLenVT();
MVT VT = Op->getSimpleValueType(0);
MVT ContainerVT = getContainerForFixedLengthVector(VT);
SDValue VL = DAG.getConstant(VT.getVectorNumElements(), DL, XLenVT);
SDValue IntID = DAG.getTargetConstant(VlsegInts[NF - 2], DL, XLenVT);
auto *Load = cast<MemIntrinsicSDNode>(Op);
SmallVector<EVT, 9> ContainerVTs(NF, ContainerVT);
craig.topperUnsubmitted
Not Done
ReplyInline Actions

You don't need a SmallVector, the number of items is fixed. You can use a plain array.

craig.topper: You don't need a SmallVector, the number of items is fixed. You can use a plain array.
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Use 9 instead of 8. Since NF is <= 8 and you push one additional item for MVT::Other.

craig.topper: Use 9 instead of 8. Since NF is <= 8 and you push one additional item for MVT::Other.
ContainerVTs.push_back(MVT::Other);
SDVTList VTs = DAG.getVTList(ContainerVTs);
SDValue Result =
DAG.getMemIntrinsicNode(ISD::INTRINSIC_W_CHAIN, DL, VTs,
craig.topperUnsubmitted
Not Done
ReplyInline Actions

I think you can do

SmallVector<EVT, 8> ContainerVTs(NF, ContainerVT);

and avoid the loop.

craig.topper: I think you can do ``` SmallVector<EVT, 8> ContainerVTs(NF, ContainerVT); ``` and avoid the…
{Load->getChain(), IntID, Op.getOperand(2), VL},
Load->getMemoryVT(), Load->getMemOperand());
SmallVector<SDValue, 9> Results;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Use 9 instead of 8.

craig.topper: Use 9 instead of 8.
for (unsigned int RetIdx = 0; RetIdx < NF; RetIdx++)
Results.push_back(convertFromScalableVector(VT, Result.getValue(RetIdx),
DAG, Subtarget));
Results.push_back(Result.getValue(NF));
return DAG.getMergeValues(Results, DL);
}
} }
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Drop the curly braces around a single line body.

craig.topper: Drop the curly braces around a single line body.
return lowerVectorIntrinsicScalars(Op, DAG, Subtarget); return lowerVectorIntrinsicScalars(Op, DAG, Subtarget);
} }
SDValue RISCVTargetLowering::LowerINTRINSIC_VOID(SDValue Op, SDValue RISCVTargetLowering::LowerINTRINSIC_VOID(SDValue Op,
SelectionDAG &DAG) const { SelectionDAG &DAG) const {
unsigned IntNo = Op.getConstantOperandVal(1); unsigned IntNo = Op.getConstantOperandVal(1);
switch (IntNo) { switch (IntNo) {
▲ Show 20 Lines • Show All 6,748 Lines • Show Last 20 Lines

llvm/test/CodeGen/RISCV/rvv/fixed-vector-segN-load.ll

  • This file was added.
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -mtriple riscv64 -mattr=+zve64x -riscv-v-vector-bits-min=128 < %s \
; RUN: | FileCheck %s
define <8 x i8> @load_factor2(<16 x i8>* %ptr) {
; CHECK-LABEL: load_factor2:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetivli zero, 8, e8, mf2, ta, mu
; CHECK-NEXT: vlseg2e8.v v7, (a0)
; CHECK-NEXT: # kill: def $v8 killed $v8 killed $v7_v8
; CHECK-NEXT: ret
%1 = bitcast <16 x i8>* %ptr to i8*
%2 = call { <8 x i8>, <8 x i8> } @llvm.riscv.seg2.load.v8i8.p0i8.i64(i8* %1, i64 8)
%3 = extractvalue { <8 x i8>, <8 x i8> } %2, 0
%4 = extractvalue { <8 x i8>, <8 x i8> } %2, 1
ret <8 x i8> %4
}
define <8 x i8> @load_factor3(<24 x i8>* %ptr) {
; CHECK-LABEL: load_factor3:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetivli zero, 8, e8, mf2, ta, mu
; CHECK-NEXT: vlseg3e8.v v6, (a0)
; CHECK-NEXT: # kill: def $v8 killed $v8 killed $v6_v7_v8
; CHECK-NEXT: ret
%1 = bitcast <24 x i8>* %ptr to i8*
%2 = call { <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg3.load.v8i8.p0i8.i64(i8* %1, i64 8)
%3 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8> } %2, 0
%4 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8> } %2, 1
%5 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8> } %2, 2
ret <8 x i8> %5
}
define <8 x i8> @load_factor4(<32 x i8>* %ptr) {
; CHECK-LABEL: load_factor4:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetivli zero, 8, e8, mf2, ta, mu
; CHECK-NEXT: vlseg4e8.v v5, (a0)
; CHECK-NEXT: # kill: def $v8 killed $v8 killed $v5_v6_v7_v8
; CHECK-NEXT: ret
%1 = bitcast <32 x i8>* %ptr to i8*
%2 = call { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg4.load.v8i8.p0i8.i64(i8* %1, i64 8)
%3 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 0
%4 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 1
%5 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 2
%6 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 3
ret <8 x i8> %6
}
define <8 x i8> @load_factor5(<40 x i8>* %ptr) {
; CHECK-LABEL: load_factor5:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetivli zero, 8, e8, mf2, ta, mu
; CHECK-NEXT: vlseg5e8.v v4, (a0)
; CHECK-NEXT: # kill: def $v8 killed $v8 killed $v4_v5_v6_v7_v8
; CHECK-NEXT: ret
%1 = bitcast <40 x i8>* %ptr to i8*
%2 = call { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg5.load.v8i8.p0i8.i64(i8* %1, i64 8)
%3 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 0
%4 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 1
%5 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 2
%6 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 3
%7 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 4
ret <8 x i8> %7
}
define <8 x i8> @load_factor6(<48 x i8>* %ptr) {
; CHECK-LABEL: load_factor6:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetivli zero, 8, e8, mf2, ta, mu
; CHECK-NEXT: vlseg6e8.v v3, (a0)
; CHECK-NEXT: # kill: def $v8 killed $v8 killed $v3_v4_v5_v6_v7_v8
; CHECK-NEXT: ret
%1 = bitcast <48 x i8>* %ptr to i8*
%2 = call { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg6.load.v8i8.p0i8.i64(i8* %1, i64 8)
%3 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 0
%4 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 1
%5 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 2
%6 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 3
%7 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 4
%8 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 5
ret <8 x i8> %8
}
define <8 x i8> @load_factor7(<56 x i8>* %ptr) {
; CHECK-LABEL: load_factor7:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetivli zero, 8, e8, mf2, ta, mu
; CHECK-NEXT: vlseg7e8.v v2, (a0)
; CHECK-NEXT: # kill: def $v8 killed $v8 killed $v2_v3_v4_v5_v6_v7_v8
; CHECK-NEXT: ret
%1 = bitcast <56 x i8>* %ptr to i8*
%2 = call { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg7.load.v8i8.p0i8.i64(i8* %1, i64 8)
%3 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 0
%4 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 1
%5 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 2
%6 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 3
%7 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 4
%8 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 5
%9 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 6
ret <8 x i8> %9
}
define <8 x i8> @load_factor8(<64 x i8>* %ptr) {
; CHECK-LABEL: load_factor8:
; CHECK: # %bb.0:
; CHECK-NEXT: vsetivli zero, 8, e8, mf2, ta, mu
; CHECK-NEXT: vlseg8e8.v v1, (a0)
; CHECK-NEXT: # kill: def $v8 killed $v8 killed $v1_v2_v3_v4_v5_v6_v7_v8
; CHECK-NEXT: ret
%1 = bitcast <64 x i8>* %ptr to i8*
%2 = call { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg8.load.v8i8.p0i8.i64(i8* %1, i64 8)
%3 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 0
%4 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 1
%5 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 2
%6 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 3
%7 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 4
%8 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 5
%9 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 6
%10 = extractvalue { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %2, 7
ret <8 x i8> %10
}
declare { <8 x i8>, <8 x i8> } @llvm.riscv.seg2.load.v8i8.p0i8.i64(i8*, i64)
declare { <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg3.load.v8i8.p0i8.i64(i8*, i64)
declare { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg4.load.v8i8.p0i8.i64(i8*, i64)
declare { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg5.load.v8i8.p0i8.i64(i8*, i64)
declare { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg6.load.v8i8.p0i8.i64(i8*, i64)
declare { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg7.load.v8i8.p0i8.i64(i8*, i64)
declare { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } @llvm.riscv.seg8.load.v8i8.p0i8.i64(i8*, i64)