[RISCV] Initial infrastructure for code generation of the RISC-V V-extension

[craig.topper]

Summary

The companion RFC (http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html) gives lots of details on the overall strategy, but we summarize it here:

• LLVM IR involving vector types is going to be selected using pseudo instructions (only MachineInstr). These pseudo instructions contain dummy operands to represent the vector type being operated and the vector length for the operation.

• These two dummy operands, as set by instruction selection, will be used by the custom inserter to prepend every operation with an appropriate vsetvli instruction that ensures the vector architecture is properly configured for the operation. Not in this patch: later passes will remove the redundant vsetvli instructions.

• Register classes of tuples of vector registers are used to represent vector register groups (LMUL > 1).

• Those pseudos are eventually lowered into the actual instructions when emitting the MCInsts.

About the patch:

Because there is a bit of initial infrastructure required, this is the minimal patch that allows us to select instructions for 3 LLVM IR instructions: load, add and store vectors of integers. LLVM IR operations have "whole-vector" semantics (as in they generate values for all the elements).

Later patches will extend the information represented in TableGen.

Authored-by: Roger Ferrer Ibanez <[e-mail]>
Co-Authored-by: Evandro Menezes <[e-mail]>
Co-Authored-by: Craig Topper <[e-mail]>

[khchen]

FYI: LLVM-RVV CodeGen RFC slide.

[rogfer01]

RFC: http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html

[StephenFan]

llvm/lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

208

May be

assert(MI.getOpcode() == RISCV::PseudoVSETVLI && "Unexpected pseudo instruction");
MCInstr = &TII.get(RISCV::VSETVLI);

is better

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1995

may be

assert(SEWIndex >= 0 && "SEWIndex must be >= 0");

is better

llvm/utils/TableGen/GlobalISelEmitter.cpp

190–193

Is this if statement necessary?

[StephenFan]

llvm/lib/Target/RISCV/RISCVInstrInfoPseudoV.td

209 ↗

(On Diff #298303)

The PseudoVSETVLI has the hasSideEffects = 1, mayLoad = 0, mayStore = 0

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1979

Perhaps this logic should go into RISCVBaseInfo, as I would expect other parts of the compiler will need to manipulate this operand's data at some point. It would be nice to have getters/setters for that, rather than relying on the underlying "encoding".

llvm/test/CodeGen/RISCV/rvv/add-vsetvli-gpr.mir

17

Is this test function missing a body? I can't see how it would generate the expected MIR

[evandro]

llvm/utils/TableGen/GlobalISelEmitter.cpp

190–193

Yes, as the last test below could prove true.

[rogfer01]

llvm/test/CodeGen/RISCV/rvv/add-vsetvli-gpr.mir

17

This test checks the case when the vl is not RISCV::X0 by using the vreg %3.

We can't currently express this in LLVM but we still need some LLVM IR function. Perhaps we can add some comment explaining this.

[frasercrmck]

llvm/test/CodeGen/RISCV/rvv/add-vsetvli-gpr.mir

17

Okay yeah I see my original misunderstanding; sorry about that. Presumably there will eventually be intrinsics that can set vl; I've seen those in other proposals. Until then, a comment wouldn't hurt.

[NickHung]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

99

The reserved VL and VTYPE would immediately dead after implicit def. How do you support the calling convention? Both CSRs are caller-saved.

[NickHung]

Could you demonstrate that RVV intrinsic can share the same infrastructure?

[frasercrmck]

llvm/lib/Target/RISCV/RISCVInstrInfoPseudoV.td

145 ↗

(On Diff #298794)

I notice that the RFC mentions using early-clobber constraints but don't see it being used here. From the RFC:

early-clobber %2:vrm2 = PseudoVADD_VV_M2 %3:vrm2(tied-def 0), %0:vrm2, %1:vrm2,$noreg, $x0, 32, implicit $vl, implicit $vtype

(If you wonder about the early-clobber it is needed to fulfill some constraints between sources and destination registers under lmul>1)

I ask because I'm concerned about the use of tied and early-clobber on the same operand: it is a special-case in SlotIndexes (as once explained in the mailing lists) and I've seen issues with this on another target I was working on, where LLVM forgets about this special case in several places and generates wrong code (subregister lanes are incorrectly deemed to be undef). I worry we're going to see really hard-to-track bugs a few months down the line when trying to compile more complex programs.

Is early-clobber really needed? Perhaps you could explain which constraints under lmul>1 are fulfilled by using this?

[rogfer01]

llvm/lib/Target/RISCV/RISCVInstrInfoPseudoV.td

145 ↗

(On Diff #298794)

Hi, apologies I wasn't clear enough with this aspect of the proposal.

early-clobber is in practice only relevant for widenings and narrowings (and a few other instructions). That is the reason why it is not in this very first patch. In the particular case of widenings and narrowings, we cannot have a def operand and a use operand where their sew is different and their actual vector registers overlap (under some conditions).

For instance vwadd.vv v2, v1, v2 is not valid (the rule is a bit obscure as I understand vwadd.vv v2, v1, v3 might be valid, see https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc#52-vector-operands). The simplest way to avoid this issue was using early-clobber.

If we don't use early-clobber then I understand we need to amend somehow the instructions after RA. Perhaps it is possible to let RA know what registers are still feasible as it goes allocating them? (I have not looked into that, tbh)

There are some cases where early-clobber may be too strict. I asked the list whether there is a way to model something more than what early-clobber does (for operands that have the same EEW as the destination, consider vwadd.wv) in http://lists.llvm.org/pipermail/llvm-dev/2020-May/141383.html but apparently there is no straightforward solution at the moment.

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

99

To be honest we haven't outlined any calling convention at this stage yet. See for some ideas we're considering but nothing is set in stone https://github.com/riscv/rvv-intrinsic-doc/issues/38

However as you mention, even if both CSRs are caller-saved and given the current mechanism in which every instruction using them is prefixed with a vsetvl, vl is implicitly saved in a GPR whose value will be preserved through the call.

Until the calling convention is clarified, any pass that removes redundant vsetvl instructions must be aware that calls may have clobbered it.

From your question, though, now I realize that we want to extend the lowering of a call in selectiondag to assert in the regmask that vl and vtype are clobbered. I think this would the safe thing to do for the usual calling convention. Does this seem reasonable at this stage?

[rogfer01]

In D89449#2343798, @NickHung wrote:

Could you demonstrate that RVV intrinsic can share the same infrastructure?

I'll talk with @evandro and see if we can put together some patch showing it.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVInstrInfoPseudoV.td

145 ↗

(On Diff #298794)

Hi Roger, thanks for the explanation. I see where you're coming from and why early-clobber is needed. I think we should just go with early-clobber for now and fix the bugs that come our way, or improve LLVM in line with the question you asked on the list.

And on that note, it sounds like what we need is something like let Constraints = "$src != $dst": it sounds like a register allocation thing rather than a liveness thing, doesn't it?

[frasercrmck]

llvm/lib/Target/RISCV/RISCVRegisterInfo.td

277

Perhaps it would be good to explain in the code if/how an implementation with ELEN<32 or ELEN>64 could/would be supported, as there's bound to be one some day. For example, is it impossible, is it incompatible, is it awkward, or are there just performance implications? Basically, what are the tradeoffs to the imposed constraints? The RFC goes a little bit into that regarding i128 but that might get lost in time.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1919

Is VLMul here already in the proper encoding? The cases seem to align with the enum values.

1975

Seems weird to define a reserved register constant register. Are there are other examples of this in RISC-V or another target?

llvm/lib/Target/RISCV/RISCVInstrInfoPseudoV.td

1 ↗

(On Diff #298794)

nit 80 columns

llvm/lib/Target/RISCV/RISCVRegisterInfo.td

281

Not sure if we should be using these special fraction characters in source files.

282

Looks like some formatting of columns was lost here?

[craig.topper]

llvm/lib/Target/RISCV/CMakeLists.txt

16 ↗

(On Diff #298794)

I think these are in alphabetical order?

llvm/lib/Target/RISCV/RISCVInstrInfoPseudoV.td

174 ↗

(On Diff #298794)

Is this foreach just to make instruction a variable?

llvm/lib/Target/RISCV/RISCVMCInstLower.cpp

222

I think you can use explicit_operands() here to avoid the isImplicit check later.

234

Can this just be an llvm_unreachable?

llvm/lib/Target/RISCV/RISCVRegisterInfo.td

390–391

Is this list used?

llvm/lib/Target/RISCV/Utils/RISCVBaseInfo.cpp

17

This is arguably a layering violation since Utils is supposed to be usable by the MC layer. And MC layer tools don't use IR. But since its only a header it might not be an issue. Though it might break a modules build?

[jrtc27]

llvm/lib/Target/RISCV/Utils/RISCVBaseInfo.cpp

17

It's a TableGen'ed header that pull in who knows what else generated so that can cause issues in parallel builds due to not having dependencies in that direction. Not sure if that's a problem in this particular case but I have definitely seen that cause issues in our fork when we've made that mistake before in clang/.

[frasercrmck]

Would you be able to explain how spills & reloads of vector registers work with this method? Namely, LLVM can insert spills and reloads at any point in the instruction sequence (IIRC). I would imagine that this includes right between VSETVLI/PseudoInst pairs:

dead %25:gpr = PseudoVSETVLI %12:gpr, /*e32,mf4*/, implicit-def $vl, implicit-def $vtype
; insert spill of V4 here!
%17:vr = PseudoVFMUL_VF_M1 %18:vr(tied-def 0), killed %13:vr, %15:fpr64, $noreg, $noreg, -1, implicit $vl, implicit $vtype

I would imagine that the spill needs its own VSETVLI as it must spill the whole physical register: it must ensure a vtype of e.g. e8,m1. If it needs to spill V4M4 it would configure e8,m4. This is likely overwrite the previous configuration, so will we have to save/restore vtype around the spill/reload? I'm not sure that's possible in general. I don't even think storeRegToStackSlot and loadRegFromStackSlot allow the insertion of multiple MIs, and we'd need a reserved/scavenged register. Though perhaps there's another trick I'm not thinking of.

[HsiangKai]

In D89449#2361279, @frasercrmck wrote:

Would you be able to explain how spills & reloads of vector registers work with this method? Namely, LLVM can insert spills and reloads at any point in the instruction sequence (IIRC). I would imagine that this includes right between VSETVLI/PseudoInst pairs:

dead %25:gpr = PseudoVSETVLI %12:gpr, /*e32,mf4*/, implicit-def $vl, implicit-def $vtype
; insert spill of V4 here!
%17:vr = PseudoVFMUL_VF_M1 %18:vr(tied-def 0), killed %13:vr, %15:fpr64, $noreg, $noreg, -1, implicit $vl, implicit $vtype

I would imagine that the spill needs its own VSETVLI as it must spill the whole physical register: it must ensure a vtype of e.g. e8,m1. If it needs to spill V4M4 it would configure e8,m4. This is likely overwrite the previous configuration, so will we have to save/restore vtype around the spill/reload? I'm not sure that's possible in general. I don't even think storeRegToStackSlot and loadRegFromStackSlot allow the insertion of multiple MIs, and we'd need a reserved/scavenged register. Though perhaps there's another trick I'm not thinking of.

There is no need to specify the vtype for spilling code. In RVV specification, we have whole register load/store without setting vtype. To know LMUL values is enough to do spilling.

[craig.topper]

This patch doesn't apply cleanly to trunk. Especially RISCVRegisterInfo.td

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1961

else should be on the same line as the closing curly brace above

1975

Is this from clang-format? I would have expected ElementWidth to line up more with Multiplier on the previous line.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1964

I think this does need RegState::Define, but this path isn't exercised in this patch.

I found that AArch64 has a pass (AArch64DeadRegisterDefinitionsPass) that replaces defs of some instructions with WZR(their zero register). So I guess its not unprecedented.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1964

For what it's worth, I worked on a downstream target which would often define and use a reserved constant predicate register.

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/add-vsetvli-gpr.mir

32

I'm a little concerned that Machine IR immediately out of SelectionDAG doesn't reflect the real semantics. The Pseudos should implicit-def $vl and $type and not implicit use them. As long as their fused with vsetvli, they should have the semantics of the pair.

But I don't know how to get the right semantics without having 2 sets of pseudos. How many instructions would that come out to?

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

133

Should the types be qualified with hasStdExtD()? I have the same question for F but it looks like V extension currently implicitly enables the F extension.

[rogfer01]

llvm/test/CodeGen/RISCV/rvv/add-vsetvli-gpr.mir

32

One option could be to remove their Uses=[VL, VTYPE] but then they would come out without those implicit uses which is not ideal either but I wonder if this might lead to other issues elsewhere (e.g. in the machine inst verifier?)

Perhaps we could hook in the InstrEmitter (or whoever creates the actual MachineInstrs) somehow and add the implicit-vdefs. Then in the custom inserter remove those and put the implicit-uses after we have emitted vsetvli.

[HsiangKai]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

133

I am curious about V should imply F or not. It is vague in the V specification.

[jrtc27]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

133

Currently I read it as no, in that V+Zfinx is a valid combination and would have the FP vector-scalar instructions reading the FP scalar from the "integer' register file.

[evandro]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

133

V does not imply Zfh, F, D or Zfinx. However, V supports half, float and double types regardless of F or D or Zfinx. The V instructions which specify scalar half, float or double operands do require the respective extensions. At least as I read the spec.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

133

This doesn't sound like it is just talking about the scalar operands, but maybe I'm reading it wrong.

Vector floating-point instructions require the presence of base scalar floating-point extensions corresponding to the supported vector floating-point element widths.

Note
Profiles supporting 16-bit half-precision floating-point values will also have to implement scalar half-precision floating-point support in the f registers.

[evandro]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

133

Indeed. Now I wonder about the integer instructions if they have requirements on XLEN to support the corresponding integer SEWs.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

133

In section 3.3.1, we have this:

In the base vector "V" extension, only SEW up to ELEN = max(XLEN,FLEN) are required to be supported. Other platforms may impose different constraints on ELEN

So I wouldn't say it's required. We can have e64 on RV32, for instance. And at least for the integer scalar move instructions (17.1) it accounts for SEW > XLEN:

The vmv.x.s instruction copies a single SEW-wide element from index 0 of the source vector register to a destination integer register. If SEW > XLEN, the least-significant XLEN bits are transferred and the upper SEW-XLEN bits are ignored. If SEW < XLEN, the value is sign-extended to XLEN bits.

I haven't thought through the implications for SEW > XLEN on code generation but it might trip us up if we're not careful.

[craig.topper]

Commandeering so I can rebase the patch

[craig.topper]

Rebase. Remove RISCVGenSystemOperands.inc from CMakeLists.txt. It's the same as RISCVSearchableTables.inc now. I'll probably pre-commit that rename.

[craig.topper]

Rebase after pre-commiting rename of RISCVGenSearchableTables.inc

[craig.topper]

clang-format

[craig.topper]

-Simplify the code in RISCVExpandPseudoInsts by using MBBI directly instead of creating MI from it and using the TII pointer already in the class.
-Copy the dead flag from the dest register when expanding PseudoVSETVLI.

[craig.topper]

Make LowerRISCVVMachineInstrToMCInst local to RISCVMCCodeEmitter.cpp instead of exposing it in RISCV.h

[craig.topper]

Rename RISCVInstrInfoPseudoV.td to RISCVInstrInfoVPseudos.td so it alphabetizes next to RISCVInstrInfoV.td making their relationship more obvious.

[craig.topper]

Does anyone have any additional feedback on this patch or the direction? @evandro should be providing a patch to show how intrinsics will work soon.

[frasercrmck]

In D89449#2430127, @craig.topper wrote:

Does anyone have any additional feedback on this patch or the direction? @evandro should be providing a patch to show how intrinsics will work soon.

I'm generally happy with this approach; I've been using it downstream to support fixed-length vectors with success.

I think there was your outstanding question about the pseudos "using" VL and VTYPE immediately after ISel, and there's the idea you brought up in D92228 about duplicating the pseudos for masked and unmasked operations. I think it'd be good to come to some kind of agreement on those, but I'm also happy for them to be deferred until later. I don't think either affect the "correctness" of this patch.

[craig.topper]

In D89449#2430454, @frasercrmck wrote:

In D89449#2430127, @craig.topper wrote:

Does anyone have any additional feedback on this patch or the direction? @evandro should be providing a patch to show how intrinsics will work soon.

I'm generally happy with this approach; I've been using it downstream to support fixed-length vectors with success.

I think there was your outstanding question about the pseudos "using" VL and VTYPE immediately after ISel, and there's the idea you brought up in D92228 about duplicating the pseudos for masked and unmasked operations. I think it'd be good to come to some kind of agreement on those, but I'm also happy for them to be deferred until later. I don't think either affect the "correctness" of this patch.

I'm hoping the VL/VTYPE uses don't cause a problem as I don't know how to make InstrEmitter fix it. It looks like the VE target is also marking their instructions as using their VL register and don't insert LVL instructions until very late in the pipeline.

I don't know if we're going to have an answer for the masked/unmasked soon. I think we might just use earlyclobber in the initial patches and suffer the bad register allocation so we can make forward progress.

[rogfer01]

I don't know if we're going to have an answer for the masked/unmasked soon. I think we might just use earlyclobber in the initial patches and suffer the bad register allocation so we can make forward progress.

I also agree that at this point we have something correct, even if at times suboptimal, is better than nothing at all. We can always improve on top of that baseline.

[frasercrmck]

LGTM!

[evandro]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

133

I asked Andrew Waterman and his answer is, in English, that:

The data types fully supported in V, for both scalar and vector operations, mirror those supported by the base ISA. For example, RV32IMCFV, only SEW up to 32, for both integer and FP, both scalar and vector operations; RV32IMCFDV, only SEW up to 32, both scalar and vector, integer operations and SEW up to 64, both scalar and vector, FP operations.

[NickHung]

This comment was removed by NickHung.

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

99

At this stage, inserting vsetvli in front of every instruction is

llvm/lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

bool expandLoadAddress(MachineBasicBlock &MBB,
                          MachineBasicBlock::iterator MBBI,
                          MachineBasicBlock::iterator &NextMBBI);
   bool expandLoadTLSIEAddress(MachineBasicBlock &MBB,
                               MachineBasicBlock::iterator MBBI,
                               MachineBasicBlock::iterator &NextMBBI);
   bool expandLoadTLSGDAddress(MachineBasicBlock &MBB,
                               MachineBasicBlock::iterator MBBI,
                               MachineBasicBlock::iterator &NextMBBI);
+  bool expandVSetVL(MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI);
 };
 
 char RISCVExpandPseudo::ID = 0;
 
 bool RISCVExpandPseudo::runOnMachineFunction(MachineFunction &MF) {
   TII = static_cast<const RISCVInstrInfo *>(MF.getSubtarget().getInstrInfo());
   bool Modified = false;
   for (auto &MBB : MF)
bool RISCVExpandPseudo::expandMI(MachineBasicBlock &MBB,
   case RISCV::PseudoLLA:
     return expandLoadLocalAddress(MBB, MBBI, NextMBBI);
   case RISCV::PseudoLA:
     return expandLoadAddress(MBB, MBBI, NextMBBI);
   case RISCV::PseudoLA_TLS_IE:
     return expandLoadTLSIEAddress(MBB, MBBI, NextMBBI);
   case RISCV::PseudoLA_TLS_GD:
     return expandLoadTLSGDAddress(MBB, MBBI, NextMBBI);
+  case RISCV::PseudoVSETVLI:
+    return expandVSetVL(MBB, MBBI);
   }
 
   return false;
 }
 
 bool RISCVExpandPseudo::expandAuipcInstPair(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
     MachineBasicBlock::iterator &NextMBBI, unsigned FlagsHi,

 
 bool RISCVExpandPseudo::expandLoadTLSGDAddress(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
     MachineBasicBlock::iterator &NextMBBI) {
   return expandAuipcInstPair(MBB, MBBI, NextMBBI, RISCVII::MO_TLS_GD_HI,
                              RISCV::ADDI);
 }
 
+bool RISCVExpandPseudo::expandVSetVL(MachineBasicBlock &MBB,
+                                     MachineBasicBlock::iterator MBBI) {
+  assert(MBBI->getNumOperands() == 5 && "Unexpected instruction format");
+
+  DebugLoc DL = MBBI->getDebugLoc();
+
+  assert(MBBI->getOpcode() == RISCV::PseudoVSETVLI &&
+         "Unexpected pseudo instruction");
+  const MCInstrDesc &Desc = TII->get(RISCV::VSETVLI);
+  assert(Desc.getNumOperands() == 3 && "Unexpected instruction format");
+
+  Register DstReg = MBBI->getOperand(0).getReg();
+  bool DstIsDead = MBBI->getOperand(0).isDead();
+  BuildMI(MBB, MBBI, DL, Desc)
+      .addReg(DstReg, RegState::Define | getDeadRegState(DstIsDead))
+      .add(MBBI->getOperand(1))  // VL
+      .add(MBBI->getOperand(2)); // VType
+
+  MBBI->eraseFromParent(); // The pseudo instruction is gone now.
+  return true;
+}
+
 } // end of anonymous namespace
 
 INITIALIZE_PASS(RISCVExpandPseudo, "riscv-expand-pseudo",
                 RISCV_EXPAND_PSEUDO_NAME, false, false)
 namespace llvm {
 
 FunctionPass *createRISCVExpandPseudoPass() { return new RISCVExpandPseudo(); }
 
 } // end of namespace llvm

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
 
   if (Subtarget.hasStdExtZfh())
     addRegisterClass(MVT::f16, &RISCV::FPR16RegClass);
   if (Subtarget.hasStdExtF())
     addRegisterClass(MVT::f32, &RISCV::FPR32RegClass);
   if (Subtarget.hasStdExtD())
     addRegisterClass(MVT::f64, &RISCV::FPR64RegClass);
 
+  if (Subtarget.hasStdExtV()) {
+    addRegisterClass(RISCVVMVTs::vbool64_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vbool32_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vbool16_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vbool8_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vbool4_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vbool2_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vbool1_t, &RISCV::VRRegClass);
+
+    addRegisterClass(RISCVVMVTs::vint8mf8_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint8mf4_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint8mf2_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint8m1_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint8m2_t, &RISCV::VRM2RegClass);
+    addRegisterClass(RISCVVMVTs::vint8m4_t, &RISCV::VRM4RegClass);
+    addRegisterClass(RISCVVMVTs::vint8m8_t, &RISCV::VRM8RegClass);
+
+    addRegisterClass(RISCVVMVTs::vint16mf4_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint16mf2_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint16m1_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint16m2_t, &RISCV::VRM2RegClass);
+    addRegisterClass(RISCVVMVTs::vint16m4_t, &RISCV::VRM4RegClass);
+    addRegisterClass(RISCVVMVTs::vint16m8_t, &RISCV::VRM8RegClass);
+
+    addRegisterClass(RISCVVMVTs::vint32mf2_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint32m1_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint32m2_t, &RISCV::VRM2RegClass);
+    addRegisterClass(RISCVVMVTs::vint32m4_t, &RISCV::VRM4RegClass);
+    addRegisterClass(RISCVVMVTs::vint32m8_t, &RISCV::VRM8RegClass);
+
+    addRegisterClass(RISCVVMVTs::vint64m1_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vint64m2_t, &RISCV::VRM2RegClass);
+    addRegisterClass(RISCVVMVTs::vint64m4_t, &RISCV::VRM4RegClass);
+    addRegisterClass(RISCVVMVTs::vint64m8_t, &RISCV::VRM8RegClass);
+
+    addRegisterClass(RISCVVMVTs::vfloat32mf2_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vfloat32m1_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vfloat32m2_t, &RISCV::VRM2RegClass);
+    addRegisterClass(RISCVVMVTs::vfloat32m4_t, &RISCV::VRM4RegClass);
+    addRegisterClass(RISCVVMVTs::vfloat32m8_t, &RISCV::VRM8RegClass);
+
+    addRegisterClass(RISCVVMVTs::vfloat64m1_t, &RISCV::VRRegClass);
+    addRegisterClass(RISCVVMVTs::vfloat64m2_t, &RISCV::VRM2RegClass);
+    addRegisterClass(RISCVVMVTs::vfloat64m4_t, &RISCV::VRM4RegClass);
+    addRegisterClass(RISCVVMVTs::vfloat64m8_t, &RISCV::VRM8RegClass);
+  }
+
   // Compute derived properties from the register classes.
   computeRegisterProperties(STI.getRegisterInfo());
 
   setStackPointerRegisterToSaveRestore(RISCV::X2);
 
   for (auto N : {ISD::EXTLOAD, ISD::SEXTLOAD, ISD::ZEXTLOAD})
     setLoadExtAction(N, XLenVT, MVT::i1, Promote);
 
if (Subtarget.hasStdExtA()) {
     setMaxAtomicSizeInBitsSupported(Subtarget.getXLen());
     setMinCmpXchgSizeInBits(32);
   } else {
     setMaxAtomicSizeInBitsSupported(0);
   }
 
   setBooleanContents(ZeroOrOneBooleanContent);
 
+  if (Subtarget.hasStdExtV())
+    setBooleanVectorContents(ZeroOrOneBooleanContent);
+
   // Function alignments.
   const Align FunctionAlignment(Subtarget.hasStdExtC() ? 2 : 4);
   setMinFunctionAlignment(FunctionAlignment);
   setPrefFunctionAlignment(FunctionAlignment);
 
   // Effectively disable jump table generation.
   setMinimumJumpTableEntries(INT_MAX);
 
while (SelectMBBI != SelectEnd) {
     }
     SelectMBBI = Next;
   }
 
   F->getProperties().reset(MachineFunctionProperties::Property::NoPHIs);
   return TailMBB;
 }
 
+static MachineBasicBlock *addVSetVL(MachineInstr &MI, MachineBasicBlock *BB,
+                                    int VLIndex, unsigned SEWIndex,
+                                    unsigned VLMul) {
+  MachineFunction &MF = *BB->getParent();
+  DebugLoc DL = MI.getDebugLoc();
+  const TargetInstrInfo &TII = *MF.getSubtarget().getInstrInfo();
+
+  unsigned SEW = MI.getOperand(SEWIndex).getImm();
+  RISCVVLengthMultiplier::LengthMultiplier Multiplier;
+
+  switch (VLMul) {
+  default:
+    llvm_unreachable("Unexpected LMUL for instruction");
+  case 0:
+  case 1:
+  case 2:
+  case 3:
+  case 5:
+  case 6:
+  case 7:
+    Multiplier = static_cast<RISCVVLengthMultiplier::LengthMultiplier>(VLMul);
+    break;
+  }
+
+  RISCVVStandardElementWidth::StandardElementWidth ElementWidth;
+  switch (SEW) {
+  default:
+    llvm_unreachable("Unexpected SEW for instruction");
+  case 8:
+    ElementWidth = RISCVVStandardElementWidth::ElementWidth8;
+    break;
+  case 16:
+    ElementWidth = RISCVVStandardElementWidth::ElementWidth16;
+    break;
+  case 32:
+    ElementWidth = RISCVVStandardElementWidth::ElementWidth32;
+    break;
+  case 64:
+    ElementWidth = RISCVVStandardElementWidth::ElementWidth64;
+    break;
+  }
+
+  MachineRegisterInfo &MRI = MF.getRegInfo();
+
+  // VL and VTYPE are alive here.
+  MachineInstrBuilder MIB = BuildMI(*BB, MI, DL, TII.get(RISCV::PseudoVSETVLI));
+
+  if (VLIndex >= 0) {
+    // Set VL (rs1 != X0).
+    unsigned DestReg = MRI.createVirtualRegister(&RISCV::GPRRegClass);
+    MIB.addReg(DestReg, RegState::Define | RegState::Dead)
+       .addReg(MI.getOperand(VLIndex).getReg());
+  } else
+    // With no VL operator in the pseudo, do not modify VL (rd = X0, rs1 = X0).
+    MIB.addReg(RISCV::X0, RegState::Dead)
+       .addReg(RISCV::X0, RegState::Kill);
+
+  // For simplicity we reuse the vtype representation here.
+  // Bits | Name       | Description
+  // -----+------------+------------------------------------------------
+  // 5    | vlmul[2]   | Fractional lmul?
+  // 4:2  | vsew[2:0]  | Standard element width (SEW) setting
+  // 1:0  | vlmul[1:0] | Vector register group multiplier (LMUL) setting
+  MIB.addImm(((Multiplier & 0x4) << 3) | ((ElementWidth & 0x3) << 2) |
+             (Multiplier & 0x3));
+
+  // Remove (now) redundant operands from pseudo
+  MI.getOperand(SEWIndex).setImm(-1);
+  if (VLIndex >= 0) {
+    MI.getOperand(VLIndex).setReg(RISCV::NoRegister);
+    MI.getOperand(VLIndex).setIsKill(false);
+  }
+
+  return BB;
+}
+
 MachineBasicBlock *
 RISCVTargetLowering::EmitInstrWithCustomInserter(MachineInstr &MI,
                                                  MachineBasicBlock *BB) const {
+
+  if (const RISCVVPseudosTable::PseudoInfo *RVV =
+          RISCVVPseudosTable::getPseudoInfo(MI.getOpcode())) {
+    int VLIndex = RVV->getVLIndex();
+    int SEWIndex = RVV->getSEWIndex();
+
+    assert(SEWIndex >= 0 && "SEWIndex must be >= 0");
+    return addVSetVL(MI, BB, VLIndex, SEWIndex, RVV->VLMul);
+  }
+
   switch (MI.getOpcode()) {
   default:
     llvm_unreachable("Unexpected instr type to insert");
   case RISCV::ReadCycleWide:
     assert(!Subtarget.is64Bit() &&
            "ReadCycleWrite is only to be used on riscv32");
     return emitReadCycleWidePseudo(MI, BB);
   case RISCV::Select_GPR_Using_CC_GPR:

llvm/lib/Target/RISCV/RISCVInstrInfoV.td

let Predicates = [HasStdExtZvamo, HasStdExtA, IsRV64] in {
   defm VAMOXOREI64 : VAMO<AMOOPVamoXor, LSWidth64, "vamoxorei64.v">;
   defm VAMOANDEI64 : VAMO<AMOOPVamoAnd, LSWidth64, "vamoandei64.v">;
   defm VAMOOREI64 : VAMO<AMOOPVamoOr, LSWidth64, "vamoorei64.v">;
   defm VAMOMINEI64 : VAMO<AMOOPVamoMin, LSWidth64, "vamominei64.v">;
   defm VAMOMAXEI64 : VAMO<AMOOPVamoMax, LSWidth64, "vamomaxei64.v">;
   defm VAMOMINUEI64 : VAMO<AMOOPVamoMinu, LSWidth64, "vamominuei64.v">;
   defm VAMOMAXUEI64 : VAMO<AMOOPVamoMaxu, LSWidth64, "vamomaxuei64.v">;
 } // Predicates = [HasStdExtZvamo, HasStdExtA, IsRV64]
+
+include "RISCVInstrInfoVPseudos.td"

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

+//===-- RISCVInstrInfoVPseudos.td - RISC-V 'V' Pseudos -----*- tablegen -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// This file contains the required infrastructure to support code generation
+/// for the standard 'V' (Vector) extension, version 0.9.  This version is still
+/// experimental as the 'V' extension hasn't been ratified yet.
+///
+/// This file is included from RISCVInstrInfoV.td
+///
+//===----------------------------------------------------------------------===//
+
+//===----------------------------------------------------------------------===//
+// Utilities.
+//===----------------------------------------------------------------------===//
+
+// This class describes information associated to the LMUL.
+class LMULInfo<int lmul, VReg regclass, string mx> {
+  bits<3> value = lmul; // This is encoded as the vlmul field of vtype.
+  VReg vrclass = regclass;
+  string MX = mx;
+}
+
+// Associate LMUL with tablegen records of register classes.
+def V_M1  : LMULInfo<0b000,   VR,  "M1">;
+def V_M2  : LMULInfo<0b001, VRM2,  "M2">;
+def V_M4  : LMULInfo<0b010, VRM4,  "M4">;
+def V_M8  : LMULInfo<0b011, VRM8,  "M8">;
+
+def V_MF8 : LMULInfo<0b101,   VR, "MF8">;
+def V_MF4 : LMULInfo<0b110,   VR, "MF4">;
+def V_MF2 : LMULInfo<0b111,   VR, "MF2">;
+
+// Used to iterate over all possible LMULs.
+def MxList {
+  list<LMULInfo> m = [V_MF8, V_MF4, V_MF2, V_M1, V_M2, V_M4, V_M8];
+}
+
+class shift_amount<int num> {
+  int val = !if(!eq(num, 1), 0, !add(1, shift_amount<!srl(num, 1)>.val));
+}
+
+// Output pattern for X0 used to represent VLMAX in the pseudo instructions.
+def VLMax : OutPatFrag<(ops), (XLenVT X0)>;
+
+// List of EEW.
+defvar EEWList = [8, 16, 32, 64];
+
+//===----------------------------------------------------------------------===//
+// Vector register and vector group type information.
+//===----------------------------------------------------------------------===//
+
+class VectorTypeInfo<ValueType Vec, ValueType Mas, int Sew, VReg Reg, LMULInfo M>
+{
+  ValueType Vector = Vec;
+  ValueType Mask = Mas;
+  int SEW = Sew;
+  VReg RegClass = Reg;
+  LMULInfo LMul = M;
+}
+
+class GroupVectorTypeInfo<ValueType Vec, ValueType VecM1, ValueType Mas,
+                          int Sew, VReg Reg, LMULInfo M>
+    : VectorTypeInfo<Vec, Mas, Sew, Reg, M>
+{
+  ValueType VectorM1 = VecM1;
+}
+
+defset list<VectorTypeInfo> AllVectors = {
+  defset list<VectorTypeInfo> AllIntegerVectors = {
+    def VtypeInt8MF8  : VectorTypeInfo<vint8mf8_t,  vbool64_t,  8, VR, V_MF8>;
+    def VtypeInt8MF4  : VectorTypeInfo<vint8mf4_t,  vbool32_t,  8, VR, V_MF4>;
+    def VtypeInt8MF2  : VectorTypeInfo<vint8mf2_t,  vbool16_t,  8, VR, V_MF2>;
+    def VtypeInt8M1   : VectorTypeInfo<vint8m1_t,   vbool8_t,   8, VR, V_M1>;
+    def VtypeInt16MF4 : VectorTypeInfo<vint16mf4_t, vbool64_t, 16, VR, V_MF4>;
+    def VtypeInt16MF2 : VectorTypeInfo<vint16mf2_t, vbool32_t, 16, VR, V_MF2>;
+    def VtypeInt16M1  : VectorTypeInfo<vint16m1_t,  vbool16_t, 16, VR, V_M1>;
+    def VtypeInt32MF2 : VectorTypeInfo<vint32mf2_t, vbool64_t, 32, VR, V_MF2>;
+    def VtypeInt32M1  : VectorTypeInfo<vint32m1_t,  vbool32_t, 32, VR, V_M1>;
+    def VtypeInt64M1  : VectorTypeInfo<vint64m1_t,  vbool64_t, 64, VR, V_M1>;
+
+    def VtypeInt8M2  : GroupVectorTypeInfo<vint8m2_t,  vint8m1_t,  vbool4_t, 8,  VRM2, V_M2>;
+    def VtypeInt8M4  : GroupVectorTypeInfo<vint8m4_t,  vint8m1_t,  vbool2_t, 8,  VRM4, V_M4>;
+    def VtypeInt8M8  : GroupVectorTypeInfo<vint8m8_t,  vint8m1_t,  vbool1_t, 8,  VRM8, V_M8>;
+
+    def VtypeInt16M2  : GroupVectorTypeInfo<vint16m2_t, vint16m1_t, vbool8_t, 16, VRM2, V_M2>;
+    def VtypeInt16M4 : GroupVectorTypeInfo<vint16m4_t, vint16m1_t, vbool4_t, 16, VRM4, V_M4>;
+    def VtypeInt16M8 : GroupVectorTypeInfo<vint16m8_t, vint16m1_t, vbool2_t, 16, VRM8, V_M8>;
+
+    def VtypeInt32M2  : GroupVectorTypeInfo<vint32m2_t, vint32m1_t, vbool16_t, 32, VRM2, V_M2>;
+    def VtypeInt32M4  : GroupVectorTypeInfo<vint32m4_t, vint32m1_t, vbool8_t,  32, VRM4, V_M4>;
+    def VtypeInt32M8 : GroupVectorTypeInfo<vint32m8_t, vint32m1_t, vbool4_t,  32, VRM8, V_M8>;
+
+    def VtypeInt64M2  : GroupVectorTypeInfo<vint64m2_t, vint64m1_t, vbool32_t, 64, VRM2, V_M2>;
+    def VtypeInt64M4  : GroupVectorTypeInfo<vint64m4_t, vint64m1_t, vbool16_t, 64, VRM4, V_M4>;
+    def VtypeInt64M8  : GroupVectorTypeInfo<vint64m8_t, vint64m1_t, vbool8_t,  64, VRM8, V_M8>;
+  }
+}
+
+// This class holds the record of the RISCVVPseudoTable below.
+// This represents the information we need in codegen for each pseudo.
+class RISCVVPseudo {
+  Pseudo Pseudo = !cast<Pseudo>(NAME); // Used as a key.
+  Instruction BaseInstr;
+  bits<8> VLIndex;
+  bits<8> SEWIndex;
+  bits<8> MergeOpIndex;
+  bits<3> VLMul;
+}
+
+// The actual table.
+def RISCVVPseudosTable : GenericTable {
+  let FilterClass = "RISCVVPseudo";
+  let CppTypeName = "PseudoInfo";
+  let Fields = [ "Pseudo", "BaseInstr", "VLIndex", "SEWIndex", "MergeOpIndex",
+                 "VLMul" ];
+  let PrimaryKey = [ "Pseudo" ];
+  let PrimaryKeyName = "getPseudoInfo";
+}
+
+//===----------------------------------------------------------------------===//
+// Helpers to define the different pseudo instructions.
+//===----------------------------------------------------------------------===//
+
+multiclass pseudo_binary<VReg result_reg_class,
+                         VReg op1_reg_class,
+                         DAGOperand op2_kind,
+                         LMULInfo vlmul > {
+  let Constraints = "$rd = $merge",
+      Uses = [VL, VTYPE], VLIndex = 5, SEWIndex = 6, MergeOpIndex = 1,
+      BaseInstr = !cast<Instruction>(!subst("Pseudo", "", NAME)) in
+    def "_"# vlmul.MX : Pseudo<(outs result_reg_class:$rd),
+                            (ins result_reg_class:$merge,
+                                 op1_reg_class:$rs2, op2_kind:$rs1,
+                                 VMaskOp:$vm, GPR:$vl, ixlenimm:$sew),
+                            []>,
+                     RISCVVPseudo;
+}
+
+multiclass pseudo_binary_v_vv_vx_vi<Operand imm_type = simm5,
+                                    bit force_earlyclobber = 0> {
+  let mayLoad = 0, mayStore = 0, hasSideEffects = 0, usesCustomInserter = 1 in
+  foreach m = MxList.m in
+  {
+    let VLMul = m.value in
+    {
+      defvar evr = m.vrclass;
+      defm _VV : pseudo_binary<evr, evr, evr, m>;
+      defm _VX : pseudo_binary<evr, evr, GPR, m>;
+      defm _VI : pseudo_binary<evr, evr, imm_type, m>;
+    }
+  }
+}
+
+//===----------------------------------------------------------------------===//
+// Helpers to define the different patterns.
+//===----------------------------------------------------------------------===//
+
+multiclass pat_vop_binary<SDNode vop,
+                          string instruction_name,
+                          ValueType result_type,
+                          ValueType op_type,
+                          ValueType mask_type,
+                          int sew,
+                          LMULInfo vlmul,
+                          VReg result_reg_class,
+                          VReg op_reg_class,
+                          bit swap = 0>
+{
+  defvar instruction = !cast<Instruction>(instruction_name#"_VV_"# vlmul.MX);
+  def : Pat<(result_type (vop
+                          (op_type op_reg_class:$rs1),
+                          (op_type op_reg_class:$rs2))),
+            (instruction (result_type (IMPLICIT_DEF)),
+                         op_reg_class:$rs1,
+                         op_reg_class:$rs2,
+                         (mask_type zero_reg),
+                         VLMax, sew)>;
+}
+
+multiclass pat_vop_binary_common<SDNode vop,
+                                 string instruction_name,
+                                 list<VectorTypeInfo> vtilist>
+{
+  foreach vti = vtilist in
+  defm : pat_vop_binary<vop, instruction_name,
+                        vti.Vector, vti.Vector, vti.Mask, vti.SEW,
+                        vti.LMul, vti.RegClass, vti.RegClass>;
+}
+
+//===----------------------------------------------------------------------===//
+// Pseudo instructions and patterns.
+//===----------------------------------------------------------------------===//
+
+let Predicates = [HasStdExtV] in {
+
+//===----------------------------------------------------------------------===//
+// 6. Configuration-Setting Instructions
+//===----------------------------------------------------------------------===//
+
+// Pseudos.
+let hasSideEffects = 1, mayLoad = 0, mayStore = 0, Defs = [VL, VTYPE] in {
+def PseudoVSETVLI : Pseudo<(outs GPR:$rd), (ins GPR:$rs1, VTypeIOp:$vtypei), []>;
+
+}
+
+//===----------------------------------------------------------------------===//
+// 7. Vector Loads and Stores
+//===----------------------------------------------------------------------===//
+
+// Pseudos.
+foreach eew = EEWList in {
+  foreach lmul = MxList.m in {
+    defvar LInfo = lmul.MX;
+    defvar vreg = lmul.vrclass;
+    defvar vlmul = lmul.value;
+    defvar constraint = "$rd = $merge";
+
+    let mayLoad = 1, mayStore = 0, hasSideEffects = 0,
+        usesCustomInserter = 1,
+        VLMul = vlmul in
+    {
+      let Uses = [VL, VTYPE], VLIndex = 4, SEWIndex = 5, MergeOpIndex = 1,
+          Constraints = constraint,
+          BaseInstr = !cast<Instruction>("VLE" # eew # "_V") in
+          def "PseudoVLE" # eew # "_V_" # LInfo
+            : Pseudo<(outs vreg:$rd),
+                     (ins vreg:$merge, GPR:$rs1, VMaskOp:$mask, GPR:$vl,
+                      ixlenimm:$sew),
+                     []>,
+              RISCVVPseudo;
+    }
+
+    let mayLoad = 0, mayStore = 1, hasSideEffects = 0,
+        usesCustomInserter = 1,
+        VLMul = vlmul in
+    {
+      // Masked stores do not have a merge operand as merge is done in memory
+      let Uses = [VL, VTYPE],
+          VLIndex = 3, SEWIndex = 4, MergeOpIndex = -1,
+          BaseInstr = !cast<Instruction>("VSE" # eew # "_V") in
+        def "PseudoVSE" # eew # "_V_" # LInfo
+            : Pseudo<(outs),
+                     (ins vreg:$rd, GPR:$rs1, VMaskOp:$mask, GPR:$vl,
+                          ixlenimm:$sew),
+                     []>,
+              RISCVVPseudo;
+    }
+  }
+}
+
+// Patterns.
+multiclass pat_load_store<LLVMType type,
+                          LLVMType mask_type,
+                          int sew,
+                          LMULInfo vlmul,
+                          VReg reg_class>
+{
+  defvar load_instr = !cast<Instruction>("PseudoVLE" # sew # "_V_"# vlmul.MX);
+  defvar store_instr = !cast<Instruction>("PseudoVSE" # sew # "_V_"# vlmul.MX);
+  // Load
+  def : Pat<(type (load GPR:$rs1)),
+            (load_instr (type (IMPLICIT_DEF)),
+             GPR:$rs1,
+             (mask_type zero_reg),
+             VLMax, sew)>;
+  def : Pat<(type (load AddrFI:$rs1)),
+             (load_instr (type (IMPLICIT_DEF)),
+             AddrFI:$rs1,
+             (mask_type zero_reg),
+             VLMax, sew)>;
+
+  // Store
+  def : Pat<(store type:$rs2, GPR:$rs1),
+            (store_instr reg_class:$rs2, GPR:$rs1,
+             (mask_type zero_reg),
+              VLMax, sew)>;
+  def : Pat<(store type:$rs2, AddrFI:$rs1),
+            (store_instr reg_class:$rs2, AddrFI:$rs1,
+             (mask_type zero_reg),
+             VLMax, sew)>;
+}
+
+foreach vti = AllVectors in
+{
+  defm : pat_load_store<vti.Vector, vti.Mask,
+                        vti.SEW, vti.LMul, vti.RegClass>;
+}
+
+//===----------------------------------------------------------------------===//
+// 12. Vector Integer Arithmetic Instructions
+//===----------------------------------------------------------------------===//
+
+//===----------------------------------------------------------------------===//
+// 12.1. Vector Single-Width Integer Add and Subtract
+//===----------------------------------------------------------------------===//
+
+// Pseudo instructions.
+defm PseudoVADD        : pseudo_binary_v_vv_vx_vi;
+
+// Whole-register vector patterns.
+defm "" : pat_vop_binary_common<add, "PseudoVADD", AllIntegerVectors>;
+
+} // Predicates = [HasStdExtV]

llvm/lib/Target/RISCV/RISCVMCInstLower.cpp

 //===-- RISCVMCInstLower.cpp - Convert RISCV MachineInstr to an MCInst ------=//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file contains code to lower RISCV MachineInstrs to their corresponding
 // MCInst records.
 //
 //===----------------------------------------------------------------------===//
 
 #include "RISCV.h"
+#include "RISCVSubtarget.h"
 #include "MCTargetDesc/RISCVMCExpr.h"
 #include "llvm/CodeGen/AsmPrinter.h"
 #include "llvm/CodeGen/MachineBasicBlock.h"
 #include "llvm/CodeGen/MachineInstr.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
case MachineOperand::MO_ExternalSymbol:
     break;
   case MachineOperand::MO_ConstantPoolIndex:
     MCOp = lowerSymbolOperand(MO, AP.GetCPISymbol(MO.getIndex()), AP);
     break;
   }
   return true;
 }
 
+static bool lowerRISCVVMachineInstrToMCInst(const MachineInstr *MI,
+                                            MCInst &OutMI) {
+  const RISCVVPseudosTable::PseudoInfo *RVV =
+      RISCVVPseudosTable::getPseudoInfo(MI->getOpcode());
+  if (!RVV)
+    return false;
+
+  OutMI.setOpcode(RVV->BaseInstr);
+
+  const MachineBasicBlock *MBB = MI->getParent();
+  assert(MBB && "MI expected to be in a basic block");
+  const MachineFunction *MF = MBB->getParent();
+  assert(MF && "MBB expected to be in a machine function");
+
+  const TargetRegisterInfo *TRI =
+      MF->getSubtarget<RISCVSubtarget>().getRegisterInfo();
+  assert(TRI && "TargetRegisterInfo expected");
+
+  for (const MachineOperand &MO : MI->explicit_operands()) {
+    int OpNo = (int)MI->getOperandNo(&MO);
+    assert(OpNo >= 0 && "Operand number doesn't fit in an 'int' type");
+
+    // Skip VL, SEW and MergeOp operands
+    if (OpNo == RVV->getVLIndex() || OpNo == RVV->getSEWIndex() ||
+        OpNo == RVV->getMergeOpIndex())
+      continue;
+
+    MCOperand MCOp;
+    switch (MO.getType()) {
+    default:
+      llvm_unreachable("Unknown operand type");
+    case MachineOperand::MO_Register: {
+      unsigned Reg = MO.getReg();
+
+      // Nothing to do on NoRegister operands (used as vector mask operand on
+      // unmasked instructions)
+      if (Reg == RISCV::NoRegister) {
+        MCOp = MCOperand::createReg(Reg);
+        break;
+      }
+
+      const TargetRegisterClass *RC = TRI->getMinimalPhysRegClass(Reg);
+      if (RC->hasSuperClassEq(&RISCV::VRM2RegClass) ||
+          RC->hasSuperClassEq(&RISCV::VRM4RegClass) ||
+          RC->hasSuperClassEq(&RISCV::VRM8RegClass)) {
+        Reg = TRI->getSubReg(Reg, RISCV::sub_vrm2);
+        assert(Reg && "Subregister does not exist");
+      }
+
+      MCOp = MCOperand::createReg(Reg);
+      break;
+    }
+    case MachineOperand::MO_Immediate:
+      MCOp = MCOperand::createImm(MO.getImm());
+      break;
+    }
+    OutMI.addOperand(MCOp);
+  }
+  return true;
+}
+
 void llvm::LowerRISCVMachineInstrToMCInst(const MachineInstr *MI, MCInst &OutMI,
                                           const AsmPrinter &AP) {
+  if (lowerRISCVVMachineInstrToMCInst(MI, OutMI))
+    return;
+
   OutMI.setOpcode(MI->getOpcode());
 
   for (const MachineOperand &MO : MI->operands()) {
     MCOperand MCOp;
     if (LowerRISCVMachineOperandToMCOperand(MO, MCOp, AP))
       OutMI.addOperand(MCOp);
   }
 }

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

BitVector RISCVRegisterInfo::getReservedRegs(const MachineFunction &MF) const {
   markSuperRegs(Reserved, RISCV::X3); // gp
   markSuperRegs(Reserved, RISCV::X4); // tp
   if (TFI->hasFP(MF))
     markSuperRegs(Reserved, RISCV::X8); // fp
   // Reserve the base register if we need to realign the stack and allocate
   // variable-sized objects at runtime.
   if (TFI->hasBP(MF))
     markSuperRegs(Reserved, RISCVABI::getBPReg()); // bp
+
+  // V registers for code generation. We handle them manually.
+  markSuperRegs(Reserved, RISCV::VL);
+  markSuperRegs(Reserved, RISCV::VTYPE);
+
   assert(checkAllSuperRegsMarked(Reserved));
   return Reserved;
 }
 
 bool RISCVRegisterInfo::isAsmClobberable(const MachineFunction &MF,
                                          MCRegister PhysReg) const {
   return !MF.getSubtarget<RISCVSubtarget>().isRegisterReservedByUser(PhysReg);
 }

llvm/lib/Target/RISCV/RISCVRegisterInfo.td

def FPR64 : RegisterClass<"RISCV", [f64], 64, (add
     (sequence "F%u_D", 18, 27)
 )>;
 
 def FPR64C : RegisterClass<"RISCV", [f64], 64, (add
   (sequence "F%u_D", 10, 15),
   (sequence "F%u_D", 8, 9)
 )>;
 
+// Vector type mapping to LLVM types.
+//
+// Though the V extension allows that VLEN be as small as 8,
+// this approach assumes that VLEN>=64.
+// Additionally, the only supported ELEN values are 32 and 64,
+// thus `vscale` can be defined as VLEN/64,
+// allowing the same types with either ELEN value.
+//
+//         MF8    MF4     MF2     M1      M2      M4       M8
+// i64*    N/A    N/A     N/A     nxv1i64 nxv2i64 nxv4i64  nxv8i64
+// i32     N/A    N/A     nxv1i32 nxv2i32 nxv4i32 nxv8i32  nxv16i32
+// i16     N/A    nxv1i16 nxv2i16 nxv4i16 nxv8i16 nxv16i16 nxv32i16
+// i8      nxv1i8 nxv2i8  nxv4i8  nxv8i8  nxv16i8 nxv32i8  nxv64i8
+// double* N/A    N/A     N/A     nxv1f64 nxv2f64 nxv4f64  nxv8f64
+// float   N/A    N/A     nxv1f32 nxv2f32 nxv4f32 nxv8f32  nxv16f32
+// half    N/A    nxv1f16 nxv2f16 nxv4f16 nxv8f16 nxv16f16 nxv32f16
+// * ELEN=64
+
+defvar vint8mf8_t = nxv1i8;
+defvar vint8mf4_t = nxv2i8;
+defvar vint8mf2_t = nxv4i8;
+defvar vint8m1_t = nxv8i8;
+defvar vint8m2_t = nxv16i8;
+defvar vint8m4_t = nxv32i8;
+defvar vint8m8_t = nxv64i8;
+
+defvar vint16mf4_t = nxv1i16;
+defvar vint16mf2_t = nxv2i16;
+defvar vint16m1_t  = nxv4i16;
+defvar vint16m2_t  = nxv8i16;
+defvar vint16m4_t  = nxv16i16;
+defvar vint16m8_t  = nxv32i16;
+
+defvar vint32mf2_t = nxv1i32;
+defvar vint32m1_t  = nxv2i32;
+defvar vint32m2_t  = nxv4i32;
+defvar vint32m4_t  = nxv8i32;
+defvar vint32m8_t  = nxv16i32;
+
+defvar vint64m1_t = nxv1i64;
+defvar vint64m2_t = nxv2i64;
+defvar vint64m4_t = nxv4i64;
+defvar vint64m8_t = nxv8i64;
+
+defvar vfloat32mf2_t = nxv1f32;
+defvar vfloat32m1_t  = nxv2f32;
+defvar vfloat32m2_t  = nxv4f32;
+defvar vfloat32m4_t  = nxv8f32;
+defvar vfloat32m8_t  = nxv16f32;
+
+defvar vfloat64m1_t = nxv1f64;
+defvar vfloat64m2_t = nxv2f64;
+defvar vfloat64m4_t = nxv4f64;
+defvar vfloat64m8_t = nxv8f64;
+
+defvar vbool1_t  = nxv64i1;
+defvar vbool2_t  = nxv32i1;
+defvar vbool4_t  = nxv16i1;
+defvar vbool8_t  = nxv8i1;
+defvar vbool16_t = nxv4i1;
+defvar vbool32_t = nxv2i1;
+defvar vbool64_t = nxv1i1;
+
 // Vector registers
 let RegAltNameIndices = [ABIRegAltName] in {
   foreach Index = 0-31 in {
     def V#Index : RISCVReg<Index, "v"#Index, ["v"#Index]>, DwarfRegNum<[!add(Index, 64)]>;
   }
 
   foreach Index = [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22,
                    24, 26, 28, 30] in {
let RegAltNameIndices = [ABIRegAltName] in {
   def VTYPE  : RISCVReg<0, "vtype", ["vtype"]>;
   def VL     : RISCVReg<0, "vl", ["vl"]>;
 }
 
 class RegisterTypes<list<ValueType> reg_types> {
   list<ValueType> types = reg_types;
 }
 
-// The order of registers represents the preferred allocation sequence,
-// meaning caller-save regs are listed before callee-save.
-def VR : RegisterClass<"RISCV", [nxv8i8, nxv4i16, nxv2i32, nxv1i64],
-                       64, (add
-    (sequence "V%u", 25, 31),
+class VReg<list<ValueType> regTypes, dag regList, int Vlmul>
+  : RegisterClass<"RISCV",
+                  regTypes,
+                  // FIXME: Spill alignment set to 16 bytes.
+                  128,
+                  regList> {
+  int VLMul = Vlmul;
+  int Size = !mul(Vlmul, 64); // FIXME: assuming ELEN=64
+}
+
+def VR : VReg<[vint8mf2_t, vint8mf4_t, vint8mf8_t,
+               vint16mf2_t, vint16mf4_t, vint32mf2_t,
+               vint8m1_t, vint16m1_t, vint32m1_t, vint64m1_t,
+               vfloat32mf2_t, vfloat32m1_t, vfloat64m1_t,
+               vbool64_t, vbool32_t, vbool16_t, vbool8_t, vbool4_t,
+               vbool2_t, vbool1_t],
+           (add (sequence "V%u", 25, 31),
                 (sequence "V%u", 8, 24),
-    (sequence "V%u", 0, 7)
-  )> {
-  let Size = 64;
-}
+                (sequence "V%u", 0, 7)), 1>;
 
-def VRNoV0 : RegisterClass<"RISCV", [nxv8i8, nxv4i16, nxv2i32, nxv1i64],
-                           64, (add
-    (sequence "V%u", 25, 31),
+def VRNoV0 : VReg<[vint8mf2_t, vint8mf4_t, vint8mf8_t,
+                   vint16mf2_t, vint16mf4_t, vint32mf2_t,
+                   vint8m1_t, vint16m1_t, vint32m1_t, vint64m1_t,
+                   vfloat32mf2_t, vfloat32m1_t, vfloat64m1_t,
+                   vbool64_t, vbool32_t, vbool16_t, vbool8_t, vbool4_t,
+                   vbool2_t, vbool1_t],
+               (add (sequence "V%u", 25, 31),
                     (sequence "V%u", 8, 24),
-    (sequence "V%u", 1, 7)
-  )> {
-  let Size = 64;
-}
+                    (sequence "V%u", 1, 7)), 1>;
 
-def VRM2 : RegisterClass<"RISCV", [nxv16i8, nxv8i16, nxv4i32, nxv2i64], 64,
-                         (add V26M2, V28M2, V30M2, V8M2, V10M2, V12M2, V14M2, V16M2,
-                              V18M2, V20M2, V22M2, V24M2, V0M2, V2M2, V4M2, V6M2)> {
-  let Size = 128;
-}
+def VRM2 : VReg<[vint8m2_t, vint16m2_t, vint32m2_t, vint64m2_t,
+                 vfloat32m2_t, vfloat64m2_t],
+                  V18M2, V20M2, V22M2, V24M2, V0M2, V2M2, V4M2, V6M2), 2>;
 
-def VRM4 : RegisterClass<"RISCV", [nxv32i8, nxv16i16, nxv8i32, nxv4i64], 64,
-                         (add V28M4, V8M4, V12M4, V16M4, V20M4, V24M4, V0M4, V4M4)> {
-  let Size = 256;
-}
+def VRM4 : VReg<[vint8m4_t, vint16m4_t, vint32m4_t, vint64m4_t,
+                 vfloat32m4_t, vfloat64m4_t],
+             (add V28M4, V8M4, V12M4, V16M4, V20M4, V24M4, V0M4, V4M4), 4>;
 
-def VRM8 : RegisterClass<"RISCV", [nxv32i16, nxv16i32, nxv8i64], 64,
-                         (add V8M8, V16M8, V24M8, V0M8)> {
-  let Size = 512;
-}
+def VRM8 : VReg<[vint8m8_t, vint16m8_t, vint32m8_t, vint64m8_t,
+                 vfloat32m8_t, vfloat64m8_t],
+             (add V8M8, V16M8, V24M8, V0M8), 8>;
 
-def VMaskVT : RegisterTypes<[nxv1i1, nxv2i1, nxv4i1, nxv8i1, nxv16i1, nxv32i1]>;
+defvar VMaskVTs = [vbool64_t, vbool32_t, vbool16_t, vbool8_t,
+                   vbool4_t, vbool2_t, vbool1_t];
 
-def VMV0 : RegisterClass<"RISCV", VMaskVT.types, 64, (add V0)> {
+def VMV0 : RegisterClass<"RISCV", VMaskVTs, 64, (add V0)> {
   let Size = 64;
 }

llvm/lib/Target/RISCV/Utils/RISCVBaseInfo.h

 #ifndef LLVM_LIB_TARGET_RISCV_MCTARGETDESC_RISCVBASEINFO_H
 #define LLVM_LIB_TARGET_RISCV_MCTARGETDESC_RISCVBASEINFO_H
 
 #include "MCTargetDesc/RISCVMCTargetDesc.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/MC/MCInstrDesc.h"
 #include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Support/MachineValueType.h"
 
 namespace llvm {
 
 // RISCVII - This namespace holds all of the target specific flags that
 // instruction info tracks. All definitions must match RISCVInstrFormats.td.
 namespace RISCVII {
 enum {
   InstFormatPseudo = 0,

 namespace RISCVFeatures {
 
 // Validates if the given combination of features are valid for the target
 // triple. Exits with report_fatal_error if not.
 void validate(const Triple &TT, const FeatureBitset &FeatureBits);
 
 } // namespace RISCVFeatures
 
+namespace RISCVVMVTs {
+
+constexpr MVT vint8mf8_t = MVT::nxv1i8;
+constexpr MVT vint8mf4_t = MVT::nxv2i8;
+constexpr MVT vint8mf2_t = MVT::nxv4i8;
+constexpr MVT vint8m1_t = MVT::nxv8i8;
+constexpr MVT vint8m2_t = MVT::nxv16i8;
+constexpr MVT vint8m4_t = MVT::nxv32i8;
+constexpr MVT vint8m8_t = MVT::nxv64i8;
+
+constexpr MVT vint16mf4_t = MVT::nxv1i16;
+constexpr MVT vint16mf2_t = MVT::nxv2i16;
+constexpr MVT vint16m1_t = MVT::nxv4i16;
+constexpr MVT vint16m2_t = MVT::nxv8i16;
+constexpr MVT vint16m4_t = MVT::nxv16i16;
+constexpr MVT vint16m8_t = MVT::nxv32i16;
+
+constexpr MVT vint32mf2_t = MVT::nxv1i32;
+constexpr MVT vint32m1_t = MVT::nxv2i32;
+constexpr MVT vint32m2_t = MVT::nxv4i32;
+constexpr MVT vint32m4_t = MVT::nxv8i32;
+constexpr MVT vint32m8_t = MVT::nxv16i32;
+
+constexpr MVT vint64m1_t = MVT::nxv1i64;
+constexpr MVT vint64m2_t = MVT::nxv2i64;
+constexpr MVT vint64m4_t = MVT::nxv4i64;
+constexpr MVT vint64m8_t = MVT::nxv8i64;
+
+constexpr MVT vfloat16mf4_t = MVT::nxv1f16;
+constexpr MVT vfloat16mf2_t = MVT::nxv2f16;
+constexpr MVT vfloat16m1_t = MVT::nxv4f16;
+constexpr MVT vfloat16m2_t = MVT::nxv8f16;
+constexpr MVT vfloat16m4_t = MVT::nxv16f16;
+constexpr MVT vfloat16m8_t = MVT::nxv32f16;
+
+constexpr MVT vfloat32mf2_t = MVT::nxv1f32;
+constexpr MVT vfloat32m1_t = MVT::nxv2f32;
+constexpr MVT vfloat32m2_t = MVT::nxv4f32;
+constexpr MVT vfloat32m4_t = MVT::nxv8f32;
+constexpr MVT vfloat32m8_t = MVT::nxv16f32;
+
+constexpr MVT vfloat64m1_t = MVT::nxv1f64;
+constexpr MVT vfloat64m2_t = MVT::nxv2f64;
+constexpr MVT vfloat64m4_t = MVT::nxv4f64;
+constexpr MVT vfloat64m8_t = MVT::nxv8f64;
+
+constexpr MVT vbool1_t = MVT::nxv64i1;
+constexpr MVT vbool2_t = MVT::nxv32i1;
+constexpr MVT vbool4_t = MVT::nxv16i1;
+constexpr MVT vbool8_t = MVT::nxv8i1;
+constexpr MVT vbool16_t = MVT::nxv4i1;
+constexpr MVT vbool32_t = MVT::nxv2i1;
+constexpr MVT vbool64_t = MVT::nxv1i1;
+
+} // namespace RISCVVMVTs
+
+namespace RISCVVLengthMultiplier {
+
+enum LengthMultiplier {
+  LMul1 = 0,
+  LMul2 = 1,
+  LMul4 = 2,
+  LMul8 = 3,
+  LMulF8 = 5,
+  LMulF4 = 6,
+  LMulF2 = 7
+};
+
+}
+
+namespace RISCVVStandardElementWidth {
+
+enum StandardElementWidth {
+  ElementWidth8 = 0,
+  ElementWidth16 = 1,
+  ElementWidth32 = 2,
+  ElementWidth64 = 3
+};
+
+}
+
+namespace RISCVVPseudosTable {
+
+struct PseudoInfo {
+  unsigned int Pseudo;
+  unsigned int BaseInstr;
+  uint8_t VLIndex;
+  uint8_t SEWIndex;
+  uint8_t MergeOpIndex;
+  uint8_t VLMul;
+
+  int getVLIndex() const { return static_cast<int8_t>(VLIndex); }
+
+  int getSEWIndex() const { return static_cast<int8_t>(SEWIndex); }
+
+  int getMergeOpIndex() const { return static_cast<int8_t>(MergeOpIndex); }
+};
+
+using namespace RISCV;
+
+#define GET_RISCVVPseudosTable_DECL
+#include "RISCVGenSearchableTables.inc"
+
+} // end namespace RISCVVPseudosTable
+
 } // namespace llvm
 
 #endif

llvm/lib/Target/RISCV/Utils/RISCVBaseInfo.cpp

 //===-- RISCVBaseInfo.cpp - Top level definitions for RISCV MC ------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file contains small standalone enum definitions for the RISCV target
 // useful for the compiler back-end and the MC libraries.
 //
 //===----------------------------------------------------------------------===//
 
 #include "RISCVBaseInfo.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/Support/raw_ostream.h"
 
 namespace llvm {
 namespace RISCVSysReg {
 #define GET_SysRegsList_IMPL
 #include "RISCVGenSearchableTables.inc"
 } // namespace RISCVSysReg
 
 namespace RISCVABI {

 
 void validate(const Triple &TT, const FeatureBitset &FeatureBits) {
   if (TT.isArch64Bit() && FeatureBits[RISCV::FeatureRV32E])
     report_fatal_error("RV32E can't be enabled for an RV64 target");
 }
 
 } // namespace RISCVFeatures
 
+namespace RISCVVPseudosTable {
+
+#define GET_RISCVVPseudosTable_IMPL
+#include "RISCVGenSearchableTables.inc"
+
+} // namespace RISCVVPseudosTable
+
 } // namespace llvm

llvm/test/CodeGen/RISCV/rvv/add-vsetvli-gpr.mir

+# RUN: llc -mtriple riscv64 -mattr=+experimental-v %s  \
+# RUN:     -start-before=finalize-isel -stop-after=finalize-isel -o - \
+# RUN:     | FileCheck --check-prefix=POST-INSERTER %s
+
+# RUN: llc -mtriple riscv64 -mattr=+experimental-v %s  \
+# RUN:     -start-before=finalize-isel -o - \
+# RUN:     | FileCheck --check-prefix=CODEGEN %s
+
+--- |
+  define void @vadd_vint64m1(
+            <vscale x 1 x i64> *%pc,
+            <vscale x 1 x i64> *%pa,
+            <vscale x 1 x i64> *%pb,
+            i64 %vl)
+  {
+    ret void
+  }
+...
+---
+name: vadd_vint64m1
+tracksRegLiveness: true
+body:             |
+  bb.0 (%ir-block.0):
+    liveins: $x10, $x11, $x12, $x13
+
+    %3:gpr = COPY $x13
+    %2:gpr = COPY $x12
+    %1:gpr = COPY $x11
+    %0:gpr = COPY $x10
+    %5:vr = IMPLICIT_DEF
+    %4:vr = PseudoVLE64_V_M1 %5, %1, $noreg, %3, 64, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pa, align 8)
+    %7:vr = IMPLICIT_DEF
+    %6:vr = PseudoVLE64_V_M1 %7, %2, $noreg, %3, 64, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pb, align 8)
+    %9:vr = IMPLICIT_DEF
+    %8:vr = PseudoVADD_VV_M1 %9, killed %4, killed %6, $noreg, %3, 64, implicit $vl, implicit $vtype
+    PseudoVSE64_V_M1 killed %8, %0, $noreg, %3, 64, implicit $vl, implicit $vtype :: (store unknown-size into %ir.pc, align 8)
+    PseudoRET
+
+...
+
+# POST-INSERTER: %0:gpr = COPY $x13
+# POST-INSERTER: %4:vr = IMPLICIT_DEF
+# POST-INSERTER: dead %10:gpr = PseudoVSETVLI %0, 12, implicit-def $vl, implicit-def $vtype
+# POST-INSERTER: %5:vr = PseudoVLE64_V_M1 %4, %2, $noreg, $noreg, -1, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pa, align 8)
+# POST-INSERTER: %6:vr = IMPLICIT_DEF
+# POST-INSERTER: dead %11:gpr = PseudoVSETVLI %0, 12, implicit-def $vl, implicit-def $vtype
+# POST-INSERTER: %7:vr = PseudoVLE64_V_M1 %6, %1, $noreg, $noreg, -1, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pb, align 8)
+# POST-INSERTER: %8:vr = IMPLICIT_DEF
+# POST-INSERTER: dead %12:gpr = PseudoVSETVLI %0, 12, implicit-def $vl, implicit-def $vtype
+# POST-INSERTER: %9:vr = PseudoVADD_VV_M1 %8, killed %5, killed %7, $noreg, $noreg, -1, implicit $vl, implicit $vtype
+# POST-INSERTER: dead %13:gpr = PseudoVSETVLI %0, 12, implicit-def $vl, implicit-def $vtype
+# POST-INSERTER: PseudoVSE64_V_M1 killed %9, %3, $noreg, $noreg, -1, implicit $vl, implicit $vtype :: (store unknown-size into %ir.pc, align 8)
+
+# CODEGEN: vsetvli  a4, a3, e64,m1,tu,mu
+# CODEGEN-NEXT: vle64.v v25, (a1)
+# CODEGEN-NEXT: vsetvli a1, a3, e64,m1,tu,mu
+# CODEGEN-NEXT: vle64.v v26, (a2)
+# CODEGEN-NEXT: vsetvli a1, a3, e64,m1,tu,mu
+# CODEGEN-NEXT: vadd.vv v25, v25, v26
+# CODEGEN-NEXT: vsetvli a1, a3, e64,m1,tu,mu
+# CODEGEN-NEXT: vse64.v v25, (a0)
+# CODEGEN-NEXT: ret

llvm/test/CodeGen/RISCV/rvv/add-vsetvli-vlmax.ll

+; This test shows the evolution of RVV pseudo instructions within isel.
+
+; RUN: llc -mtriple riscv64 -mattr=+experimental-v %s -o %t.pre.mir \
+; RUN:     -stop-before=finalize-isel
+; RUN: cat %t.pre.mir | FileCheck --check-prefix=PRE-INSERTER %s
+
+; RUN: llc -mtriple riscv64 -mattr=+experimental-v %t.pre.mir -o %t.post.mir \
+; RUN:     -start-before=finalize-isel -stop-after=finalize-isel
+; RUN: cat %t.post.mir | FileCheck --check-prefix=POST-INSERTER %s
+
+define void @vadd_vint64m1(
+          <vscale x 1 x i64> *%pc,
+          <vscale x 1 x i64> *%pa,
+          <vscale x 1 x i64> *%pb)
+{
+  %va = load <vscale x 1 x i64>, <vscale x 1 x i64>* %pa
+  %vb = load <vscale x 1 x i64>, <vscale x 1 x i64>* %pb
+  %vc = add <vscale x 1 x i64> %va, %vb
+  store <vscale x 1 x i64> %vc, <vscale x 1 x i64> *%pc
+  ret void
+}
+
+; PRE-INSERTER: %4:vr = IMPLICIT_DEF
+; PRE-INSERTER: %3:vr = PseudoVLE64_V_M1 %4, %1, $noreg, $x0, 64, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pa, align 8)
+; PRE-INSERTER: %6:vr = IMPLICIT_DEF
+; PRE-INSERTER: %5:vr = PseudoVLE64_V_M1 %6, %2, $noreg, $x0, 64, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pb, align 8)
+; PRE-INSERTER: %8:vr = IMPLICIT_DEF
+; PRE-INSERTER: %7:vr = PseudoVADD_VV_M1 %8, killed %3, killed %5, $noreg, $x0, 64, implicit $vl, implicit $vtype
+; PRE-INSERTER: PseudoVSE64_V_M1 killed %7, %0, $noreg, $x0, 64, implicit $vl, implicit $vtype :: (store unknown-size into %ir.pc, align 8)
+
+; POST-INSERTER: %4:vr = IMPLICIT_DEF
+; POST-INSERTER: dead %9:gpr = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
+; POST-INSERTER: %3:vr = PseudoVLE64_V_M1 %4, %1, $noreg, $noreg, -1, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pa, align 8)
+; POST-INSERTER: %6:vr = IMPLICIT_DEF
+; POST-INSERTER: dead %10:gpr = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
+; POST-INSERTER: %5:vr = PseudoVLE64_V_M1 %6, %2, $noreg, $noreg, -1, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pb, align 8)
+; POST-INSERTER: %8:vr = IMPLICIT_DEF
+; POST-INSERTER: dead %11:gpr = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
+; POST-INSERTER: %7:vr = PseudoVADD_VV_M1 %8, killed %3, killed %5, $noreg, $noreg, -1, implicit $vl, implicit $vtype
+; POST-INSERTER: dead %12:gpr = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
+; POST-INSERTER: PseudoVSE64_V_M1 killed %7, %0, $noreg, $noreg, -1, implicit $vl, implicit $vtype :: (store unknown-size into %ir.pc, align 8)

llvm/test/CodeGen/RISCV/rvv/load-add-store-16.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple riscv32 -mattr=+experimental-v %s -o - \
+; RUN:     -verify-machineinstrs | FileCheck %s
+; RUN: llc -mtriple riscv64 -mattr=+experimental-v %s -o - \
+; RUN:     -verify-machineinstrs | FileCheck %s
+
+define void @vadd_vint16m1(<vscale x 4 x i16> *%pc, <vscale x 4 x i16> *%pa, <vscale x 4 x i16> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint16m1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e16,m1,tu,mu
+; CHECK-NEXT:    vle16.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e16,m1,tu,mu
+; CHECK-NEXT:    vle16.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e16,m1,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e16,m1,tu,mu
+; CHECK-NEXT:    vse16.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 4 x i16>, <vscale x 4 x i16>* %pa
+  %vb = load <vscale x 4 x i16>, <vscale x 4 x i16>* %pb
+  %vc = add <vscale x 4 x i16> %va, %vb
+  store <vscale x 4 x i16> %vc, <vscale x 4 x i16> *%pc
+  ret void
+}
+
+define void @vadd_vint16m2(<vscale x 8 x i16> *%pc, <vscale x 8 x i16> *%pa, <vscale x 8 x i16> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint16m2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e16,m2,tu,mu
+; CHECK-NEXT:    vle16.v v26, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e16,m2,tu,mu
+; CHECK-NEXT:    vle16.v v28, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e16,m2,tu,mu
+; CHECK-NEXT:    vadd.vv v26, v26, v28
+; CHECK-NEXT:    vsetvli a1, zero, e16,m2,tu,mu
+; CHECK-NEXT:    vse16.v v26, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 8 x i16>, <vscale x 8 x i16>* %pa
+  %vb = load <vscale x 8 x i16>, <vscale x 8 x i16>* %pb
+  %vc = add <vscale x 8 x i16> %va, %vb
+  store <vscale x 8 x i16> %vc, <vscale x 8 x i16> *%pc
+  ret void
+}
+
+define void @vadd_vint16m4(<vscale x 16 x i16> *%pc, <vscale x 16 x i16> *%pa, <vscale x 16 x i16> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint16m4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e16,m4,tu,mu
+; CHECK-NEXT:    vle16.v v28, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e16,m4,tu,mu
+; CHECK-NEXT:    vle16.v v8, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e16,m4,tu,mu
+; CHECK-NEXT:    vadd.vv v28, v28, v8
+; CHECK-NEXT:    vsetvli a1, zero, e16,m4,tu,mu
+; CHECK-NEXT:    vse16.v v28, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 16 x i16>, <vscale x 16 x i16>* %pa
+  %vb = load <vscale x 16 x i16>, <vscale x 16 x i16>* %pb
+  %vc = add <vscale x 16 x i16> %va, %vb
+  store <vscale x 16 x i16> %vc, <vscale x 16 x i16> *%pc
+  ret void
+}
+
+define void @vadd_vint16m8(<vscale x 32 x i16> *%pc, <vscale x 32 x i16> *%pa, <vscale x 32 x i16> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint16m8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e16,m8,tu,mu
+; CHECK-NEXT:    vle16.v v8, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e16,m8,tu,mu
+; CHECK-NEXT:    vle16.v v16, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e16,m8,tu,mu
+; CHECK-NEXT:    vadd.vv v8, v8, v16
+; CHECK-NEXT:    vsetvli a1, zero, e16,m8,tu,mu
+; CHECK-NEXT:    vse16.v v8, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 32 x i16>, <vscale x 32 x i16>* %pa
+  %vb = load <vscale x 32 x i16>, <vscale x 32 x i16>* %pb
+  %vc = add <vscale x 32 x i16> %va, %vb
+  store <vscale x 32 x i16> %vc, <vscale x 32 x i16> *%pc
+  ret void
+}
+
+define void @vadd_vint16mf2(<vscale x 2 x i16> *%pc, <vscale x 2 x i16> *%pa, <vscale x 2 x i16> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint16mf2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e16,mf2,tu,mu
+; CHECK-NEXT:    vle16.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf2,tu,mu
+; CHECK-NEXT:    vle16.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf2,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf2,tu,mu
+; CHECK-NEXT:    vse16.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 2 x i16>, <vscale x 2 x i16>* %pa
+  %vb = load <vscale x 2 x i16>, <vscale x 2 x i16>* %pb
+  %vc = add <vscale x 2 x i16> %va, %vb
+  store <vscale x 2 x i16> %vc, <vscale x 2 x i16> *%pc
+  ret void
+}
+
+define void @vadd_vint16mf4(<vscale x 1 x i16> *%pc, <vscale x 1 x i16> *%pa, <vscale x 1 x i16> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint16mf4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e16,mf4,tu,mu
+; CHECK-NEXT:    vle16.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf4,tu,mu
+; CHECK-NEXT:    vle16.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf4,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf4,tu,mu
+; CHECK-NEXT:    vse16.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 1 x i16>, <vscale x 1 x i16>* %pa
+  %vb = load <vscale x 1 x i16>, <vscale x 1 x i16>* %pb
+  %vc = add <vscale x 1 x i16> %va, %vb
+  store <vscale x 1 x i16> %vc, <vscale x 1 x i16> *%pc
+  ret void
+}

llvm/test/CodeGen/RISCV/rvv/load-add-store-32.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple riscv32 -mattr=+experimental-v %s -o - \
+; RUN:     -verify-machineinstrs | FileCheck %s
+; RUN: llc -mtriple riscv64 -mattr=+experimental-v %s -o - \
+; RUN:     -verify-machineinstrs | FileCheck %s
+
+define void @vadd_vint32m1(<vscale x 2 x i32> *%pc, <vscale x 2 x i32> *%pa, <vscale x 2 x i32> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint32m1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e32,m1,tu,mu
+; CHECK-NEXT:    vle32.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e32,m1,tu,mu
+; CHECK-NEXT:    vle32.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e32,m1,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e32,m1,tu,mu
+; CHECK-NEXT:    vse32.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 2 x i32>, <vscale x 2 x i32>* %pa
+  %vb = load <vscale x 2 x i32>, <vscale x 2 x i32>* %pb
+  %vc = add <vscale x 2 x i32> %va, %vb
+  store <vscale x 2 x i32> %vc, <vscale x 2 x i32> *%pc
+  ret void
+}
+
+define void @vadd_vint32m2(<vscale x 4 x i32> *%pc, <vscale x 4 x i32> *%pa, <vscale x 4 x i32> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint32m2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e32,m2,tu,mu
+; CHECK-NEXT:    vle32.v v26, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e32,m2,tu,mu
+; CHECK-NEXT:    vle32.v v28, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e32,m2,tu,mu
+; CHECK-NEXT:    vadd.vv v26, v26, v28
+; CHECK-NEXT:    vsetvli a1, zero, e32,m2,tu,mu
+; CHECK-NEXT:    vse32.v v26, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 4 x i32>, <vscale x 4 x i32>* %pa
+  %vb = load <vscale x 4 x i32>, <vscale x 4 x i32>* %pb
+  %vc = add <vscale x 4 x i32> %va, %vb
+  store <vscale x 4 x i32> %vc, <vscale x 4 x i32> *%pc
+  ret void
+}
+
+define void @vadd_vint32m4(<vscale x 8 x i32> *%pc, <vscale x 8 x i32> *%pa, <vscale x 8 x i32> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint32m4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e32,m4,tu,mu
+; CHECK-NEXT:    vle32.v v28, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e32,m4,tu,mu
+; CHECK-NEXT:    vle32.v v8, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e32,m4,tu,mu
+; CHECK-NEXT:    vadd.vv v28, v28, v8
+; CHECK-NEXT:    vsetvli a1, zero, e32,m4,tu,mu
+; CHECK-NEXT:    vse32.v v28, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 8 x i32>, <vscale x 8 x i32>* %pa
+  %vb = load <vscale x 8 x i32>, <vscale x 8 x i32>* %pb
+  %vc = add <vscale x 8 x i32> %va, %vb
+  store <vscale x 8 x i32> %vc, <vscale x 8 x i32> *%pc
+  ret void
+}
+
+define void @vadd_vint32m8(<vscale x 16 x i32> *%pc, <vscale x 16 x i32> *%pa, <vscale x 16 x i32> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint32m8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e32,m8,tu,mu
+; CHECK-NEXT:    vle32.v v8, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e32,m8,tu,mu
+; CHECK-NEXT:    vle32.v v16, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e32,m8,tu,mu
+; CHECK-NEXT:    vadd.vv v8, v8, v16
+; CHECK-NEXT:    vsetvli a1, zero, e32,m8,tu,mu
+; CHECK-NEXT:    vse32.v v8, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 16 x i32>, <vscale x 16 x i32>* %pa
+  %vb = load <vscale x 16 x i32>, <vscale x 16 x i32>* %pb
+  %vc = add <vscale x 16 x i32> %va, %vb
+  store <vscale x 16 x i32> %vc, <vscale x 16 x i32> *%pc
+  ret void
+}
+
+define void @vadd_vint32mf2(<vscale x 1 x i32> *%pc, <vscale x 1 x i32> *%pa, <vscale x 1 x i32> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint32mf2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e32,mf2,tu,mu
+; CHECK-NEXT:    vle32.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e32,mf2,tu,mu
+; CHECK-NEXT:    vle32.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e32,mf2,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e32,mf2,tu,mu
+; CHECK-NEXT:    vse32.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 1 x i32>, <vscale x 1 x i32>* %pa
+  %vb = load <vscale x 1 x i32>, <vscale x 1 x i32>* %pb
+  %vc = add <vscale x 1 x i32> %va, %vb
+  store <vscale x 1 x i32> %vc, <vscale x 1 x i32> *%pc
+  ret void
+}

llvm/test/CodeGen/RISCV/rvv/load-add-store-64.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple riscv32 -mattr=+experimental-v %s -o - \
+; RUN:     -verify-machineinstrs | FileCheck %s
+; RUN: llc -mtriple riscv64 -mattr=+experimental-v %s -o - \
+; RUN:     -verify-machineinstrs | FileCheck %s
+
+define void @vadd_vint64m1(<vscale x 1 x i64> *%pc, <vscale x 1 x i64> *%pa, <vscale x 1 x i64> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint64m1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e64,m1,tu,mu
+; CHECK-NEXT:    vle64.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e64,m1,tu,mu
+; CHECK-NEXT:    vle64.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e64,m1,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e64,m1,tu,mu
+; CHECK-NEXT:    vse64.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 1 x i64>, <vscale x 1 x i64>* %pa
+  %vb = load <vscale x 1 x i64>, <vscale x 1 x i64>* %pb
+  %vc = add <vscale x 1 x i64> %va, %vb
+  store <vscale x 1 x i64> %vc, <vscale x 1 x i64> *%pc
+  ret void
+}
+
+define void @vadd_vint64m2(<vscale x 2 x i64> *%pc, <vscale x 2 x i64> *%pa, <vscale x 2 x i64> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint64m2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e64,m2,tu,mu
+; CHECK-NEXT:    vle64.v v26, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e64,m2,tu,mu
+; CHECK-NEXT:    vle64.v v28, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e64,m2,tu,mu
+; CHECK-NEXT:    vadd.vv v26, v26, v28
+; CHECK-NEXT:    vsetvli a1, zero, e64,m2,tu,mu
+; CHECK-NEXT:    vse64.v v26, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 2 x i64>, <vscale x 2 x i64>* %pa
+  %vb = load <vscale x 2 x i64>, <vscale x 2 x i64>* %pb
+  %vc = add <vscale x 2 x i64> %va, %vb
+  store <vscale x 2 x i64> %vc, <vscale x 2 x i64> *%pc
+  ret void
+}
+
+define void @vadd_vint64m4(<vscale x 4 x i64> *%pc, <vscale x 4 x i64> *%pa, <vscale x 4 x i64> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint64m4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e64,m4,tu,mu
+; CHECK-NEXT:    vle64.v v28, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e64,m4,tu,mu
+; CHECK-NEXT:    vle64.v v8, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e64,m4,tu,mu
+; CHECK-NEXT:    vadd.vv v28, v28, v8
+; CHECK-NEXT:    vsetvli a1, zero, e64,m4,tu,mu
+; CHECK-NEXT:    vse64.v v28, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 4 x i64>, <vscale x 4 x i64>* %pa
+  %vb = load <vscale x 4 x i64>, <vscale x 4 x i64>* %pb
+  %vc = add <vscale x 4 x i64> %va, %vb
+  store <vscale x 4 x i64> %vc, <vscale x 4 x i64> *%pc
+  ret void
+}
+
+define void @vadd_vint64m8(<vscale x 8 x i64> *%pc, <vscale x 8 x i64> *%pa, <vscale x 8 x i64> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint64m8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e64,m8,tu,mu
+; CHECK-NEXT:    vle64.v v8, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e64,m8,tu,mu
+; CHECK-NEXT:    vle64.v v16, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e64,m8,tu,mu
+; CHECK-NEXT:    vadd.vv v8, v8, v16
+; CHECK-NEXT:    vsetvli a1, zero, e64,m8,tu,mu
+; CHECK-NEXT:    vse64.v v8, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 8 x i64>, <vscale x 8 x i64>* %pa
+  %vb = load <vscale x 8 x i64>, <vscale x 8 x i64>* %pb
+  %vc = add <vscale x 8 x i64> %va, %vb
+  store <vscale x 8 x i64> %vc, <vscale x 8 x i64> *%pc
+  ret void
+}

llvm/test/CodeGen/RISCV/rvv/load-add-store-8.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple riscv32 -mattr=+experimental-v %s -o - \
+; RUN:     -verify-machineinstrs | FileCheck %s
+; RUN: llc -mtriple riscv64 -mattr=+experimental-v %s -o - \
+; RUN:     -verify-machineinstrs | FileCheck %s
+
+define void @vadd_vint8m1(<vscale x 8 x i8> *%pc, <vscale x 8 x i8> *%pa, <vscale x 8 x i8> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint8m1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e8,m1,tu,mu
+; CHECK-NEXT:    vle8.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e8,m1,tu,mu
+; CHECK-NEXT:    vle8.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e8,m1,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e8,m1,tu,mu
+; CHECK-NEXT:    vse8.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 8 x i8>, <vscale x 8 x i8>* %pa
+  %vb = load <vscale x 8 x i8>, <vscale x 8 x i8>* %pb
+  %vc = add <vscale x 8 x i8> %va, %vb
+  store <vscale x 8 x i8> %vc, <vscale x 8 x i8> *%pc
+  ret void
+}
+
+define void @vadd_vint8m2(<vscale x 16 x i8> *%pc, <vscale x 16 x i8> *%pa, <vscale x 16 x i8> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint8m2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e8,m2,tu,mu
+; CHECK-NEXT:    vle8.v v26, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e8,m2,tu,mu
+; CHECK-NEXT:    vle8.v v28, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e8,m2,tu,mu
+; CHECK-NEXT:    vadd.vv v26, v26, v28
+; CHECK-NEXT:    vsetvli a1, zero, e8,m2,tu,mu
+; CHECK-NEXT:    vse8.v v26, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 16 x i8>, <vscale x 16 x i8>* %pa
+  %vb = load <vscale x 16 x i8>, <vscale x 16 x i8>* %pb
+  %vc = add <vscale x 16 x i8> %va, %vb
+  store <vscale x 16 x i8> %vc, <vscale x 16 x i8> *%pc
+  ret void
+}
+
+define void @vadd_vint8m4(<vscale x 32 x i8> *%pc, <vscale x 32 x i8> *%pa, <vscale x 32 x i8> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint8m4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e8,m4,tu,mu
+; CHECK-NEXT:    vle8.v v28, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e8,m4,tu,mu
+; CHECK-NEXT:    vle8.v v8, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e8,m4,tu,mu
+; CHECK-NEXT:    vadd.vv v28, v28, v8
+; CHECK-NEXT:    vsetvli a1, zero, e8,m4,tu,mu
+; CHECK-NEXT:    vse8.v v28, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 32 x i8>, <vscale x 32 x i8>* %pa
+  %vb = load <vscale x 32 x i8>, <vscale x 32 x i8>* %pb
+  %vc = add <vscale x 32 x i8> %va, %vb
+  store <vscale x 32 x i8> %vc, <vscale x 32 x i8> *%pc
+  ret void
+}
+
+define void @vadd_vint8m8(<vscale x 64 x i8> *%pc, <vscale x 64 x i8> *%pa, <vscale x 64 x i8> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint8m8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e8,m8,tu,mu
+; CHECK-NEXT:    vle8.v v8, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e8,m8,tu,mu
+; CHECK-NEXT:    vle8.v v16, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e8,m8,tu,mu
+; CHECK-NEXT:    vadd.vv v8, v8, v16
+; CHECK-NEXT:    vsetvli a1, zero, e8,m8,tu,mu
+; CHECK-NEXT:    vse8.v v8, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 64 x i8>, <vscale x 64 x i8>* %pa
+  %vb = load <vscale x 64 x i8>, <vscale x 64 x i8>* %pb
+  %vc = add <vscale x 64 x i8> %va, %vb
+  store <vscale x 64 x i8> %vc, <vscale x 64 x i8> *%pc
+  ret void
+}
+
+define void @vadd_vint8mf2(<vscale x 4 x i8> *%pc, <vscale x 4 x i8> *%pa, <vscale x 4 x i8> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint8mf2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e8,mf2,tu,mu
+; CHECK-NEXT:    vle8.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf2,tu,mu
+; CHECK-NEXT:    vle8.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf2,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf2,tu,mu
+; CHECK-NEXT:    vse8.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 4 x i8>, <vscale x 4 x i8>* %pa
+  %vb = load <vscale x 4 x i8>, <vscale x 4 x i8>* %pb
+  %vc = add <vscale x 4 x i8> %va, %vb
+  store <vscale x 4 x i8> %vc, <vscale x 4 x i8> *%pc
+  ret void
+}
+
+define void @vadd_vint8mf4(<vscale x 2 x i8> *%pc, <vscale x 2 x i8> *%pa, <vscale x 2 x i8> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint8mf4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e8,mf4,tu,mu
+; CHECK-NEXT:    vle8.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf4,tu,mu
+; CHECK-NEXT:    vle8.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf4,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf4,tu,mu
+; CHECK-NEXT:    vse8.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 2 x i8>, <vscale x 2 x i8>* %pa
+  %vb = load <vscale x 2 x i8>, <vscale x 2 x i8>* %pb
+  %vc = add <vscale x 2 x i8> %va, %vb
+  store <vscale x 2 x i8> %vc, <vscale x 2 x i8> *%pc
+  ret void
+}
+
+define void @vadd_vint8mf8(<vscale x 1 x i8> *%pc, <vscale x 1 x i8> *%pa, <vscale x 1 x i8> *%pb) nounwind {
+; CHECK-LABEL: vadd_vint8mf8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a3, zero, e8,mf8,tu,mu
+; CHECK-NEXT:    vle8.v v25, (a1)
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf8,tu,mu
+; CHECK-NEXT:    vle8.v v26, (a2)
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf8,tu,mu
+; CHECK-NEXT:    vadd.vv v25, v25, v26
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf8,tu,mu
+; CHECK-NEXT:    vse8.v v25, (a0)
+; CHECK-NEXT:    ret
+  %va = load <vscale x 1 x i8>, <vscale x 1 x i8>* %pa
+  %vb = load <vscale x 1 x i8>, <vscale x 1 x i8>* %pb
+  %vc = add <vscale x 1 x i8> %va, %vb
+  store <vscale x 1 x i8> %vc, <vscale x 1 x i8> *%pc
+  ret void
+}

llvm/utils/TableGen/GlobalISelEmitter.cpp

 std::set<LLTCodeGen> KnownTypes;
 
 class InstructionMatcher;
 /// Convert an MVT to an equivalent LLT if possible, or the invalid LLT() for
 /// MVTs that don't map cleanly to an LLT (e.g., iPTR, *any, ...).
 static Optional<LLTCodeGen> MVTToLLT(MVT::SimpleValueType SVT) {
   MVT VT(SVT);
 
-  if (VT.isVector() && VT.getVectorNumElements() != 1)
+  if (VT.isScalableVector())
+    return None;
+
+  if (VT.isFixedLengthVector() && VT.getVectorNumElements() != 1)
     return LLTCodeGen(
         LLT::vector(VT.getVectorNumElements(), VT.getScalarSizeInBits()));
 
   if (VT.isInteger() || VT.isFloatingPoint())
     return LLTCodeGen(LLT::scalar(VT.getSizeInBits()));
+
   return None;
 }
 
 static std::string explainPredicates(const TreePatternNode *N) {
   std::string Explanation = "";
   StringRef Separator = "";
   for (const TreePredicateCall &Call : N->getPredicateCalls()) {
     const TreePredicateFn &P = Call.Fn;