[RISCV] Frame handling for RISC-V V extension. (2nd. version)

[HsiangKai]

Summary

This patch proposes how to deal with RISC-V vector frame objects. The
layout of RISC-V vector frame will look like

|---------------------------------|
| scalar callee-saved registers   |
|---------------------------------|
| scalar local variables          |
|---------------------------------|
| scalar outgoing arguments       |
|---------------------------------|
| RVV local variables &&          |
| RVV outgoing arguments          |
|---------------------------------| <- end of frame (sp)

If there is realignment or variable length array in the stack, we will use
frame pointer to access fixed objects and stack pointer to access
non-fixed objects.

|---------------------------------| <- frame pointer (fp)
| scalar callee-saved registers   |
|---------------------------------|
| scalar local variables          |
|---------------------------------|
| ///// realignment /////         |
|---------------------------------|
| scalar outgoing arguments       |
|---------------------------------|
| RVV local variables &&          |
| RVV outgoing arguments          |
|---------------------------------| <- end of frame (sp)

If there are both realignment and variable length array in the stack, we
will use frame pointer to access fixed objects and base pointer to access
non-fixed objects.

|---------------------------------| <- frame pointer (fp)
| scalar callee-saved registers   |
|---------------------------------|
| scalar local variables          |
|---------------------------------|
| ///// realignment /////         |
|---------------------------------| <- base pointer (bp)
| RVV local variables &&          |
| RVV outgoing arguments          |
|---------------------------------|
| /////////////////////////////// |
| variable length array           |
| /////////////////////////////// |
|---------------------------------|
| scalar outgoing arguments       |
|---------------------------------| <- end of frame (sp)

In this version, we do not save the addresses of RVV objects in the
stack. We access them directly through the polynomial expression
(a x VLENB + b). We do not reserve frame pointer when there is any RVV
object in the stack. So, we also access the scalar frame objects through the
polynomial expression (a x VLENB + b) if the access across RVV stack
area.

[jrtc27]

llvm/include/llvm/CodeGen/MIRYamlMapping.h

350–351 ↗

(On Diff #315988)

The handling of RISCVVector is always the same as SVEVector when in generic code. Can we not merge the two as they're the same thing just for different architectures?

[rogfer01]

I prefer this approach. Computing the address is a bit more involved but doesn't need an extra memory access.

It looks to me less hardware machinery gets involved (in practice) to reach the stack slot.

[HsiangKai]

Address @jrtc27's comments.

[HsiangKai]

I also think one direct memory access is better than two memory accesses. I abandoned D93750 first. If there is anyone oppose it, I could reactivate it.

[craig.topper]

llvm/include/llvm/CodeGen/TargetFrameLowering.h

27 ↗

(On Diff #316011)

Is this just a formatting change?

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

162

Is Amount expected to be a multiple of 8 or does the division need to take the ceiling?

[craig.topper]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

167

Is Amount and NumOfVReg always positive? Should we check or assert that NumOfVReg fits in 32 bits before using isPowerOf2_32?

171

Drop this else, the if already returned.

180

Assert that NumOfVReg fits in 12 bits?

192

Is this if the same condition as the one at line 208? If so, how can this be true if we already signalled a fatal_error above?

[HsiangKai]

llvm/include/llvm/CodeGen/TargetFrameLowering.h

27 ↗

(On Diff #316011)

I will revert it.

[HsiangKai]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

162

Amount is expected to be a multiple of 8. If the object size is less than 8, it will be set to 8 at assignRVVStackObjectOffsets() in RISCVFrameLowering.cpp.

167

Amount and NumOfVReg are always positive. I will add assertions for the condition.

[HsiangKai]

• Address @craig.topper's comments.
• Update the test cases to use v8-v23 as arguments.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVFrameLowering.cpp

393

nit: It's a bit odd to have one == 0 comparison and one ! comparison in the same line. I'd favour consistency here.

623

else ...? I think we should probably assert here.

783

This is implicitly converting from int to unsigned. Is using int okay?

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

154

This parameter named Amount implies the function can be used generically, but the variables and assertions within specifically imply the amount is taken to be "the number of registers". If this function is truly generic I think things like NumOfVReg should be renamed, but if the function makes assumptions about what Amount is then that should be clarified.

Also, if the function is generic maybe it should live in RISCVInstrInfo

[HsiangKai]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

154

Amount could be the stack size occupied by vector objects or the offset of the vector objects. I think NumOfVReg is appropriate because the stack size or offset are equivalent to the number of vector times vector size(VLENB).

I will move this function to RISCVInstrInfo.

[HsiangKai]

Address @frasercrmck's comments.

[HsiangKai]

Update test cases for vector argument passing by reference.

[rogfer01]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

209

I'm not sure whether we won't get some cases where the frame index is fixed but the instruction is actually a RVV load/store.

Something from IR like below (assume the size of the array is correct for the runtime vscale value)

%array = alloca [64 x double]
%vptr = bitcast [64 x double]* %array to <vscale x 1 x double>*
%v = load <vscale x 1 x double>, <vscale x 1 x double>* %vptr

[HsiangKai]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

209

We may need to add an option to tell the compiler what vector length we are compiling for. Otherwise, how do we confirm the VLS length equals to RVV vector length? Another question is which LMUL do we support to do VLS load/store through RVV instructions, LMUL = 1 only?

I will leave a FIXME here.

[HsiangKai]

Address @rogfer01's comments.

[craig.topper]

Assuming the alloca is at least as large as the size of the load implied by vlmax, does anything break? Vscale vectorized code for code using a stack array could generate code like Roger's example. I think a runtime check would be emitted to make sure the array (really the bounds of the loop accessing the array) is at least as large vlmax.

[rogfer01]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

209

I'm not sure I'm reading the condition right:

if (!((RISCVVPseudosTable::getPseudoInfo(MI.getOpcode()) != nullptr) && MI.mayLoadOrStore()))

should be the same as

if (((RISCVVPseudosTable::getPseudoInfo(MI.getOpcode()) == nullptr) || !MI.mayLoadOrStore()))

or slightly simplified

if (!RISCVVPseudosTable::getPseudoInfo(MI.getOpcode()) || !MI.mayLoadOrStore()))

So my question is: what happens if !RISCVVPseudosTable::getPseudoInfo(MI.getOpcode()) is false (i.e. this is a RVV pseudo) and !MI.mayLoadOrStore() is false too (say it is RISCV::PseudoVLE64_V)?

I understand the LLVM IR snippet above can generate a case like this one: fixed offset but an RVV pseudo that does loads/store.

We can address this case in a later change. If we do, I suggest we add a report_fatal_error to clearly state this is not supported yet.

What do you think?

[jrtc27]

llvm/lib/Target/RISCV/RISCVFrameLowering.cpp

311

Currently it reads as an absolute statement regardless of Amount.

556

Caller can do MF.getFrameInfo().getCalleeSaveInfo() now, no need to pass it.

556

Eh, I guess sometimes it's already lying around, and there's a lot of redundancy in some of the arguments passed around elsewhere.

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

913

Does V require M? V without M would be a stupid combination, but if it's legal we shouldn't assume otherwise.

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

209

Your simplified form is much easier to parse; I'd prefer the patch use that instead.

221

This should never happen given we're in the else for if (!Offset.getScalable())

237

This step doesn't get a number/summary?

llvm/test/CodeGen/RISCV/rvv/allocate-lmul-2-4-8.ll

10

This should only be read once per function, especially since CSR accesses can be full pipeline flushes on some microarchitectures (though one would hope that something like vlenb wouldn't, being a constant CSR, in a high performance chip, but less code is still good). Though you'd still likely want it to be rematerialisable rather than spilled to the stack.

llvm/test/CodeGen/RISCV/rvv/localvar.ll

123

Ugh what ugly syntax they chose, should have gone with | or something (or at least put the whole thing in parentheses if they really want to use commas).

[HsiangKai]

llvm/lib/Target/RISCV/RISCVFrameLowering.cpp

556

I prefer to keep it as so in this patch.

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

237

I didn't understand this comment.

[craig.topper]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

237

There are comments like "1. Get vlenb..." and "2. Calculate address..." and "3. Replace address..." but this step didn't get a similar numbered comment.

[HsiangKai]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

237

Got it. I will refine it. Thanks.

[HsiangKai]

Address @craig.topper and @rogfer01's comments.

[rogfer01]

llvm/test/CodeGen/RISCV/rvv/access-fixed-objects-by-rvv.ll

5 ↗

(On Diff #319212)

I think you want nounwind here and in the function definition below.

[rogfer01]

llvm/test/CodeGen/RISCV/rvv/access-fixed-objects-by-rvv.ll

5 ↗

(On Diff #319212)

Oh nevermind, I was confused by the fact some of the other tests had nounwind.

[HsiangKai]

Fix failed cases.

[craig.topper]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

237

Should we add a kill flag here if FrameRegIsKill is true?

[HsiangKai]

Address @craig.topper's comments.

[craig.topper]

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

215

I think this also need getKillRegState(FrameRegIsKill)

237

Use getKillRegState(FrameRegIsKill)

[HsiangKai]

Ping. Is there anything I need to address or I missed?

[HsiangKai]

Rebase.

[rogfer01]

I'm happy with the patch as it stands now. At least it gives us a baseline to improve if that is needed.

Thanks @HsiangKai. LGTM.

[craig.topper]

LGTM

[SuH]

I have a question about access incoming args.

|---------------------------------|
| scalar callee-saved registers   |
|---------------------------------|
| scalar local variables          |
|---------------------------------|
| scalar outgoing arguments       |
|---------------------------------|
| RVV local variables &&          |
| RVV outgoing arguments          |
|---------------------------------| <- end of frame (sp)

In this frame layout design, if we have some args pass through stack(more than 8 scalar arguments or more than 32 vector registers passing from caller),
how to access incoming arguments frame objects across "RVV local variables" ?
(RVV local variables is currecnt frame and incoming arguments is previous frame.)

[rogfer01]

Hi @SuH,

I have a question about access incoming args.

We changed the RVV frame layout in D97111, does your question still hold under that new layout?

Kind regards,

[SuH]

Hi @rogfer01

We changed the RVV frame layout in D97111, does your question still hold under that new layout?

Yes, the problem still exist.

|---------------------------------|------
| scalar incoming arguments       |      |
|---------------------------------|      |  Previous frame
| RVV previous local variables && |      | <-----how to know this size or how to count count # of vectors?
| RVV incoming arguments          |------
|---------------------------------| - start of frame (fp)
| scalar callee-saved registers   |
|---------------------------------|
| scalar local variables          |
|---------------------------------|
| scalar outgoing arguments       |
|---------------------------------|
| RVV local variables &&          |
| RVV outgoing arguments          |
|---------------------------------| <- end of frame (sp)

My question is how to count the scalar incoming argument with this frame layout,
because scalar incoming arguments cross RVV previous local variables frame objects.
In the current frame(callee), we don't know the RVV previous local variables count.
So we can't get the real stack offset because callee not know the caller vector local variables size.

[rogfer01]

Hi @SuH,

My question is how to count the scalar incoming argument with this frame layout,
because scalar incoming arguments cross RVV previous local variables frame objects.
In the current frame(callee), we don't know the RVV previous local variables count.
So we can't get the real stack offset because callee not know the caller vector local variables size.

My understanding is that the FrameIndexes associated to the RVV incoming arguments will have positive offsets in the MachineFrameInfo before we start emitting the frame layout (local variables should have negative offsets). FrameIndexes for RVV objects should have a different StackID too.

That said, I might be mixing things here so if you have an example showing the issue, this would help a lot.

llvm/lib/Target/RISCV/RISCVFrameLowering.h

public:
   // Get the first stack adjustment amount for SplitSPAdjust.
   // Return 0 if we don't want to to split the SP adjustment in prologue and
   // epilogue.
   uint64_t getFirstSPAdjustAmount(const MachineFunction &MF) const;
 
   bool canUseAsPrologue(const MachineBasicBlock &MBB) const override;
   bool canUseAsEpilogue(const MachineBasicBlock &MBB) const override;
 
+  bool isSupportedStackID(TargetStackID::Value ID) const override;
+  TargetStackID::Value getStackIDForScalableVectors() const override;
+
 protected:
   const RISCVSubtarget &STI;
 
 private:
   void determineFrameLayout(MachineFunction &MF) const;
   void adjustReg(MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
                  const DebugLoc &DL, Register DestReg, Register SrcReg,
                  int64_t Val, MachineInstr::MIFlag Flag) const;
+  void adjustStackForRVV(MachineFunction &MF, MachineBasicBlock &MBB,
+                         MachineBasicBlock::iterator MBBI, const DebugLoc &DL,
+                         int64_t Amount) const;
+  int64_t assignRVVStackObjectOffsets(MachineFrameInfo &MFI) const;
 };
 }
 #endif

llvm/lib/Target/RISCV/RISCVFrameLowering.cpp

void RISCVFrameLowering::determineFrameLayout(MachineFunction &MF) const {
   MachineFrameInfo &MFI = MF.getFrameInfo();
 
   // Get the number of bytes to allocate from the FrameInfo.
   uint64_t FrameSize = MFI.getStackSize();
 
   // Get the alignment.
   Align StackAlign = getStackAlign();
 
-  // Set Max Call Frame Size
-  uint64_t MaxCallSize = alignTo(MFI.getMaxCallFrameSize(), StackAlign);
-  MFI.setMaxCallFrameSize(MaxCallSize);
-
   // Make sure the frame is aligned.
   FrameSize = alignTo(FrameSize, StackAlign);
 
   // Update frame info.
   MFI.setStackSize(FrameSize);
 }
 
 void RISCVFrameLowering::adjustReg(MachineBasicBlock &MBB,

 
 // Returns the register used to hold the frame pointer.
 static Register getFPReg(const RISCVSubtarget &STI) { return RISCV::X8; }
 
 // Returns the register used to hold the stack pointer.
 static Register getSPReg(const RISCVSubtarget &STI) { return RISCV::X2; }
 
 static SmallVector<CalleeSavedInfo, 8>
-getNonLibcallCSI(const std::vector<CalleeSavedInfo> &CSI) {
+getNonLibcallCSI(const MachineFunction &MF,
+                 const std::vector<CalleeSavedInfo> &CSI) {
+  const MachineFrameInfo &MFI = MF.getFrameInfo();
   SmallVector<CalleeSavedInfo, 8> NonLibcallCSI;
 
-  for (auto &CS : CSI)
-    if (CS.getFrameIdx() >= 0)
+  for (auto &CS : CSI) {
+    int FI = CS.getFrameIdx();
+    if (FI >= 0 && MFI.getStackID(FI) == TargetStackID::Default)
       NonLibcallCSI.push_back(CS);
+  }
 
   return NonLibcallCSI;
 }
 
+void RISCVFrameLowering::adjustStackForRVV(MachineFunction &MF,
+                                           MachineBasicBlock &MBB,
+                                           MachineBasicBlock::iterator MBBI,
+                                           const DebugLoc &DL,
+                                           int64_t Amount) const {
+  assert(Amount != 0 && "There is no need to adjust stack pointer for RVV.");
+
+  const RISCVInstrInfo *TII = STI.getInstrInfo();
+  Register SPReg = getSPReg(STI);
+  unsigned Opc = RISCV::ADD;
+  if (Amount < 0) {
+    Amount = -Amount;
+    Opc = RISCV::SUB;
+  }
+
+  // 1. Multiply the number of v-slots to the length of registers
+  Register FactorRegister = TII->getVLENFactoredAmount(MF, MBB, MBBI, Amount);
+  // 2. SP = SP - RVV stack size
+  BuildMI(MBB, MBBI, DL, TII->get(Opc), SPReg)
+      .addReg(SPReg)
+      .addReg(FactorRegister);
+}
+
 void RISCVFrameLowering::emitPrologue(MachineFunction &MF,
                                       MachineBasicBlock &MBB) const {
   MachineFrameInfo &MFI = MF.getFrameInfo();
   auto *RVFI = MF.getInfo<RISCVMachineFunctionInfo>();
   const RISCVRegisterInfo *RI = STI.getRegisterInfo();
   const RISCVInstrInfo *TII = STI.getInstrInfo();
   MachineBasicBlock::iterator MBBI = MBB.begin();
 
if (int LibCallRegs = getLibCallID(MF, MFI.getCalleeSavedInfo()) + 1) {
     unsigned LibCallFrameSize = alignTo((STI.getXLen() / 8) * LibCallRegs, 16);
     RVFI->setLibCallStackSize(LibCallFrameSize);
   }
 
   // FIXME (note copied from Lanai): This appears to be overallocating.  Needs
   // investigation. Get the number of bytes to allocate from the FrameInfo.
   uint64_t StackSize = MFI.getStackSize();
   uint64_t RealStackSize = StackSize + RVFI->getLibCallStackSize();
+  uint64_t RVVStackSize = RVFI->getRVVStackSize();
 
   // Early exit if there is no need to allocate on the stack
-  if (RealStackSize == 0 && !MFI.adjustsStack())
+  if (RealStackSize == 0 && !MFI.adjustsStack() && RVVStackSize == 0)
     return;
 
   // If the stack pointer has been marked as reserved, then produce an error if
   // the frame requires stack allocation
   if (STI.isRegisterReservedByUser(SPReg))
     MF.getFunction().getContext().diagnose(DiagnosticInfoUnsupported{
         MF.getFunction(), "Stack pointer required, but has been reserved."});
 
void RISCVFrameLowering::emitPrologue(MachineFunction &MF,
   const auto &CSI = MFI.getCalleeSavedInfo();
 
   // The frame pointer is callee-saved, and code has been generated for us to
   // save it to the stack. We need to skip over the storing of callee-saved
   // registers as the frame pointer must be modified after it has been saved
   // to the stack, not before.
   // FIXME: assumes exactly one instruction is used to save each callee-saved
   // register.
-  std::advance(MBBI, getNonLibcallCSI(CSI).size());
+  std::advance(MBBI, getNonLibcallCSI(MF, CSI).size());
 
   // Iterate over list of callee-saved registers and emit .cfi_offset
   // directives.
   for (const auto &Entry : CSI) {
     int FrameIdx = Entry.getFrameIdx();
     int64_t Offset;
     // Offsets for objects with fixed locations (IE: those saved by libcall) are
     // simply calculated from the frame index.
if (RI->needsStackRealignment(MF)) {
       if (hasBP(MF)) {
         // move BP, SP
         BuildMI(MBB, MBBI, DL, TII->get(RISCV::ADDI), BPReg)
             .addReg(SPReg)
             .addImm(0);
       }
     }
   }
+
+  if (RVVStackSize)
+    adjustStackForRVV(MF, MBB, MBBI, DL, -RVVStackSize);
 }
 
 void RISCVFrameLowering::emitEpilogue(MachineFunction &MF,
                                       MachineBasicBlock &MBB) const {
   const RISCVRegisterInfo *RI = STI.getRegisterInfo();
   MachineFrameInfo &MFI = MF.getFrameInfo();
   auto *RVFI = MF.getInfo<RISCVMachineFunctionInfo>();
   Register FPReg = getFPReg(STI);
if (!MBB.empty()) {
 
     // If callee-saved registers are saved via libcall, place stack adjustment
     // before this call.
     while (MBBI != MBB.begin() &&
            std::prev(MBBI)->getFlag(MachineInstr::FrameDestroy))
       --MBBI;
   }
 
-  const auto &CSI = getNonLibcallCSI(MFI.getCalleeSavedInfo());
+  const auto &CSI = getNonLibcallCSI(MF, MFI.getCalleeSavedInfo());
 
   // Skip to before the restores of callee-saved registers
   // FIXME: assumes exactly one instruction is used to restore each
   // callee-saved register.
   auto LastFrameDestroy = MBBI;
   if (!CSI.empty())
     LastFrameDestroy = std::prev(MBBI, CSI.size());
 
   uint64_t StackSize = MFI.getStackSize();
   uint64_t RealStackSize = StackSize + RVFI->getLibCallStackSize();
   uint64_t FPOffset = RealStackSize - RVFI->getVarArgsSaveSize();
+  uint64_t RVVStackSize = RVFI->getRVVStackSize();
 
   // Restore the stack pointer using the value of the frame pointer. Only
   // necessary if the stack pointer was modified, meaning the stack size is
   // unknown.
   if (RI->needsStackRealignment(MF) || MFI.hasVarSizedObjects()) {
     assert(hasFP(MF) && "frame pointer should not have been eliminated");
     adjustReg(MBB, LastFrameDestroy, DL, SPReg, FPReg, -FPOffset,
               MachineInstr::FrameDestroy);
+  } else {
+    if (RVVStackSize)
+      adjustStackForRVV(MF, MBB, LastFrameDestroy, DL, RVVStackSize);
   }
 
   uint64_t FirstSPAdjustAmount = getFirstSPAdjustAmount(MF);
   if (FirstSPAdjustAmount) {
     uint64_t SecondSPAdjustAmount = MFI.getStackSize() - FirstSPAdjustAmount;
     assert(SecondSPAdjustAmount > 0 &&
            "SecondSPAdjustAmount should be greater than zero");
 
RISCVFrameLowering::getFrameIndexReference(const MachineFunction &MF, int FI,
                                            Register &FrameReg) const {
   const MachineFrameInfo &MFI = MF.getFrameInfo();
   const TargetRegisterInfo *RI = MF.getSubtarget().getRegisterInfo();
   const auto *RVFI = MF.getInfo<RISCVMachineFunctionInfo>();
 
   // Callee-saved registers should be referenced relative to the stack
   // pointer (positive offset), otherwise use the frame pointer (negative
   // offset).
-  const auto &CSI = getNonLibcallCSI(MFI.getCalleeSavedInfo());
+  const auto &CSI = getNonLibcallCSI(MF, MFI.getCalleeSavedInfo());
   int MinCSFI = 0;
   int MaxCSFI = -1;
+  StackOffset Offset;
+  auto StackID = MFI.getStackID(FI);
 
-  int Offset = MFI.getObjectOffset(FI) - getOffsetOfLocalArea() +
-               MFI.getOffsetAdjustment();
+  assert((StackID == TargetStackID::Default ||
+          StackID == TargetStackID::ScalableVector) &&
+         "Unexpected stack ID for the frame object.");
+  if (StackID == TargetStackID::Default) {
+    Offset =
+        StackOffset::getFixed(MFI.getObjectOffset(FI) - getOffsetOfLocalArea() +
+                              MFI.getOffsetAdjustment());
+  } else if (StackID == TargetStackID::ScalableVector) {
+    Offset = StackOffset::getScalable(MFI.getObjectOffset(FI));
+  }
 
   uint64_t FirstSPAdjustAmount = getFirstSPAdjustAmount(MF);
 
   if (CSI.size()) {
     MinCSFI = CSI[0].getFrameIdx();
     MaxCSFI = CSI[CSI.size() - 1].getFrameIdx();
   }
 
   if (FI >= MinCSFI && FI <= MaxCSFI) {
     FrameReg = RISCV::X2;
 
     if (FirstSPAdjustAmount)
-      Offset += FirstSPAdjustAmount;
+      Offset += StackOffset::getFixed(FirstSPAdjustAmount);
     else
-      Offset += MFI.getStackSize();
+      Offset += StackOffset::getFixed(MFI.getStackSize());
   } else if (RI->needsStackRealignment(MF) && !MFI.isFixedObjectIndex(FI)) {
     // If the stack was realigned, the frame pointer is set in order to allow
     // SP to be restored, so we need another base register to record the stack
     // after realignment.
-    if (hasBP(MF))
+    if (hasBP(MF)) {
       FrameReg = RISCVABI::getBPReg();
-    else
+      // |--------------------------| -- <-- FP
+      // | callee-saved registers   | |
+      // |--------------------------| | MFI.getStackSize()
+      // | scalar local variables   | |
+      // |--------------------------| --
+      // | Realignment              | |
+      // |--------------------------| -- <-- BP
+      // | RVV objects              | | RVFI->getRVVStackSize()
+      // |--------------------------| --
+      // | VarSize objects          | |
+      // |--------------------------| -- <-- SP
+    } else {
       FrameReg = RISCV::X2;
-    Offset += MFI.getStackSize();
+      // When using SP to access frame objects, we need to add RVV stack size.
+      //
+      // |--------------------------| -- <-- FP
+      // | callee-saved registers   | |
+      // |--------------------------| | MFI.getStackSize()
+      // | scalar local variables   | |
+      // |--------------------------| --
+      // | Realignment              | |
+      // |--------------------------| --
+      // | RVV objects              | | RVFI->getRVVStackSize()
+      // |--------------------------| -- <-- SP
+      Offset += StackOffset::getScalable(RVFI->getRVVStackSize());
+    }
+    if (MFI.getStackID(FI) == TargetStackID::Default) {
+      Offset += StackOffset::getFixed(MFI.getStackSize());
       if (FI < 0)
-      Offset += RVFI->getLibCallStackSize();
+        Offset += StackOffset::getFixed(RVFI->getLibCallStackSize());
+    }
   } else {
     FrameReg = RI->getFrameRegister(MF);
     if (hasFP(MF)) {
-      Offset += RVFI->getVarArgsSaveSize();
+      Offset += StackOffset::getFixed(RVFI->getVarArgsSaveSize());
       if (FI >= 0)
-        Offset -= RVFI->getLibCallStackSize();
+        Offset -= StackOffset::getFixed(RVFI->getLibCallStackSize());
+      // When using FP to access scalable vector objects, we need to minus
+      // the frame size.
+      //
+      // |--------------------------| -- <-- FP
+      // | callee-saved registers   | |
+      // |--------------------------| | MFI.getStackSize()
+      // | scalar local variables   | |
+      // |--------------------------| -- (Offset of RVV objects is from here.)
+      // | RVV objects              |
+      // |--------------------------|
+      // | VarSize objects          |
+      // |--------------------------| <-- SP
+      if (MFI.getStackID(FI) == TargetStackID::ScalableVector)
+        Offset -= StackOffset::getFixed(MFI.getStackSize());
     } else {
-      Offset += MFI.getStackSize();
+      // When using SP to access frame objects, we need to add RVV stack size.
+      //
+      // |--------------------------| -- <-- FP
+      // | callee-saved registers   | |
+      // |--------------------------| | MFI.getStackSize()
+      // | scalar local variables   | |
+      // |--------------------------| --
+      // | RVV objects              | | RVFI->getRVVStackSize()
+      // |--------------------------| -- <-- SP
+      Offset += StackOffset::getScalable(RVFI->getRVVStackSize());
+      if (MFI.getStackID(FI) == TargetStackID::Default) {
+        Offset += StackOffset::getFixed(MFI.getStackSize());
         if (FI < 0)
-        Offset += RVFI->getLibCallStackSize();
+          Offset += StackOffset::getFixed(RVFI->getLibCallStackSize());
+      }
     }
   }
-  return StackOffset::getFixed(Offset);
+
+  return Offset;
 }
 
 void RISCVFrameLowering::determineCalleeSaves(MachineFunction &MF,
                                               BitVector &SavedRegs,
                                               RegScavenger *RS) const {
   TargetFrameLowering::determineCalleeSaves(MF, SavedRegs, RS);
   // Unconditionally spill RA and FP only if the function uses a frame
   // pointer.
if (MF.getSubtarget<RISCVSubtarget>().hasStdExtF()) {
         if (RISCV::FPR16RegClass.contains(Regs[i]) ||
             RISCV::FPR32RegClass.contains(Regs[i]) ||
             RISCV::FPR64RegClass.contains(Regs[i]))
           SavedRegs.set(Regs[i]);
     }
   }
 }
 
+int64_t
+RISCVFrameLowering::assignRVVStackObjectOffsets(MachineFrameInfo &MFI) const {
+  int64_t Offset = 0;
+  // Create a buffer of RVV objects to allocate.
+  SmallVector<int, 8> ObjectsToAllocate;
+  for (int I = 0, E = MFI.getObjectIndexEnd(); I != E; ++I) {
+    unsigned StackID = MFI.getStackID(I);
+    if (StackID != TargetStackID::ScalableVector)
+      continue;
+    if (MFI.isDeadObjectIndex(I))
+      continue;
+
+    ObjectsToAllocate.push_back(I);
+  }
+
+  // Allocate all RVV locals and spills
+  for (int FI : ObjectsToAllocate) {
+    // ObjectSize in bytes.
+    int64_t ObjectSize = MFI.getObjectSize(FI);
+    // If the data type is the fractional vector type, reserve one vector
+    // register for it.
+    if (ObjectSize < 8)
+      ObjectSize = 8;
+    // Currently, all scalable vector types are aligned to 8 bytes.
+    Offset = alignTo(Offset + ObjectSize, 8);
+    MFI.setObjectOffset(FI, -Offset);
+  }
+
+  return Offset;
+}
+
 void RISCVFrameLowering::processFunctionBeforeFrameFinalized(
     MachineFunction &MF, RegScavenger *RS) const {
   const TargetRegisterInfo *RegInfo = MF.getSubtarget().getRegisterInfo();
   MachineFrameInfo &MFI = MF.getFrameInfo();
   const TargetRegisterClass *RC = &RISCV::GPRRegClass;
   // estimateStackSize has been observed to under-estimate the final stack
   // size, so give ourselves wiggle-room by checking for stack size
   // representable an 11-bit signed field rather than 12-bits.
   // FIXME: It may be possible to craft a function with a small stack that
   // still needs an emergency spill slot for branch relaxation. This case
   // would currently be missed.
   if (!isInt<11>(MFI.estimateStackSize(MF))) {
     int RegScavFI = MFI.CreateStackObject(RegInfo->getSpillSize(*RC),
                                           RegInfo->getSpillAlign(*RC), false);
     RS->addScavengingFrameIndex(RegScavFI);
   }
+
+  auto *RVFI = MF.getInfo<RISCVMachineFunctionInfo>();
+  int64_t RVVStackSize = assignRVVStackObjectOffsets(MFI);
+  RVFI->setRVVStackSize(RVVStackSize);
 }
 
 // Not preserve stack space within prologue for outgoing variables when the
 // function contains variable size objects and let eliminateCallFramePseudoInstr
 // preserve stack space for it.
 bool RISCVFrameLowering::hasReservedCallFrame(const MachineFunction &MF) const {
   return !MF.getFrameInfo().hasVarSizedObjects();
 }
BuildMI(MBB, MI, DL, TII.get(RISCV::PseudoCALLReg), RISCV::X5)
         .setMIFlag(MachineInstr::FrameSetup);
 
     // Add registers spilled in libcall as liveins.
     for (auto &CS : CSI)
       MBB.addLiveIn(CS.getReg());
   }
 
   // Manually spill values not spilled by libcall.
-  const auto &NonLibcallCSI = getNonLibcallCSI(CSI);
+  const auto &NonLibcallCSI = getNonLibcallCSI(*MF, CSI);
   for (auto &CS : NonLibcallCSI) {
     // Insert the spill to the stack frame.
     Register Reg = CS.getReg();
     const TargetRegisterClass *RC = TRI->getMinimalPhysRegClass(Reg);
     TII.storeRegToStackSlot(MBB, MI, Reg, true, CS.getFrameIdx(), RC, TRI);
   }
 
   return true;
 }
 
 bool RISCVFrameLowering::restoreCalleeSavedRegisters(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MI,
     MutableArrayRef<CalleeSavedInfo> CSI, const TargetRegisterInfo *TRI) const {
   if (CSI.empty())
     return true;
 
   MachineFunction *MF = MBB.getParent();
   const TargetInstrInfo &TII = *MF->getSubtarget().getInstrInfo();
   DebugLoc DL;
   if (MI != MBB.end() && !MI->isDebugInstr())
     DL = MI->getDebugLoc();
 
   // Manually restore values not restored by libcall. Insert in reverse order.
   // loadRegFromStackSlot can insert multiple instructions.
-  const auto &NonLibcallCSI = getNonLibcallCSI(CSI);
+  const auto &NonLibcallCSI = getNonLibcallCSI(*MF, CSI);
   for (auto &CS : reverse(NonLibcallCSI)) {
     Register Reg = CS.getReg();
     const TargetRegisterClass *RC = TRI->getMinimalPhysRegClass(Reg);
     TII.loadRegFromStackSlot(MBB, MI, Reg, CS.getFrameIdx(), RC, TRI);
     assert(MI != MBB.begin() && "loadRegFromStackSlot didn't insert any code!");
   }
 
   const char *RestoreLibCall = getRestoreLibCallName(*MF, CSI);
bool RISCVFrameLowering::canUseAsEpilogue(const MachineBasicBlock &MBB) const {
   // restore will be eliminated regardless.
   if (!SuccMBB)
     return true;
 
   // The successor can only contain a return, since we would effectively be
   // replacing the successor with our own tail return at the end of our block.
   return SuccMBB->isReturnBlock() && SuccMBB->size() == 1;
 }
+
+bool RISCVFrameLowering::isSupportedStackID(TargetStackID::Value ID) const {
+  switch (ID) {
+  case TargetStackID::Default:
+  case TargetStackID::ScalableVector:
+    return true;
+  case TargetStackID::NoAlloc:
+  case TargetStackID::SGPRSpill:
+    return false;
+  }
+  llvm_unreachable("Invalid TargetStackID::Value");
+}
+
+TargetStackID::Value RISCVFrameLowering::getStackIDForScalableVectors() const {
+  return TargetStackID::ScalableVector;
+}

llvm/lib/Target/RISCV/RISCVInstrInfo.h

public:
   buildOutlinedFrame(MachineBasicBlock &MBB, MachineFunction &MF,
                      const outliner::OutlinedFunction &OF) const override;
 
   // Insert a call to an outlined function into a given basic block.
   virtual MachineBasicBlock::iterator
   insertOutlinedCall(Module &M, MachineBasicBlock &MBB,
                      MachineBasicBlock::iterator &It, MachineFunction &MF,
                      const outliner::Candidate &C) const override;
+
+  Register getVLENFactoredAmount(MachineFunction &MF, MachineBasicBlock &MBB,
+                                 MachineBasicBlock::iterator II,
+                                 int64_t Amount) const;
+
 protected:
   const RISCVSubtarget &STI;
 };
 
 } // end namespace llvm
 #endif

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

MachineBasicBlock::iterator RISCVInstrInfo::insertOutlinedCall(
 
   // Add in a call instruction to the outlined function at the given location.
   It = MBB.insert(It,
                   BuildMI(MF, DebugLoc(), get(RISCV::PseudoCALLReg), RISCV::X5)
                       .addGlobalAddress(M.getNamedValue(MF.getName()), 0,
                                         RISCVII::MO_CALL));
   return It;
 }
+
+Register RISCVInstrInfo::getVLENFactoredAmount(MachineFunction &MF,
+                                               MachineBasicBlock &MBB,
+                                               MachineBasicBlock::iterator II,
+                                               int64_t Amount) const {
+  assert(Amount > 0 && "There is no need to get VLEN scaled value.");
+  assert(Amount % 8 == 0 &&
+         "Reserve the stack by the multiple of one vector size.");
+
+  MachineRegisterInfo &MRI = MF.getRegInfo();
+  const RISCVInstrInfo *TII = MF.getSubtarget<RISCVSubtarget>().getInstrInfo();
+  DebugLoc DL = II->getDebugLoc();
+  int64_t NumOfVReg = Amount / 8;
+
+  Register SizeOfVector = MRI.createVirtualRegister(&RISCV::GPRRegClass);
+  BuildMI(MBB, II, DL, TII->get(RISCV::PseudoReadVLENB), SizeOfVector);
+  Register FactorRegister = MRI.createVirtualRegister(&RISCV::GPRRegClass);
+  assert(isInt<12>(NumOfVReg) &&
+         "Expect the number of vector registers within 12-bits.");
+  if (isPowerOf2_32(NumOfVReg)) {
+    uint32_t ShiftAmount = Log2_32(NumOfVReg);
+    if (ShiftAmount == 0)
+      return SizeOfVector;
+    BuildMI(MBB, II, DL, TII->get(RISCV::SLLI), FactorRegister)
+        .addReg(SizeOfVector)
+        .addImm(ShiftAmount);
+  } else {
+    Register VN = MRI.createVirtualRegister(&RISCV::GPRRegClass);
+    BuildMI(MBB, II, DL, TII->get(RISCV::ADDI), VN)
+        .addReg(RISCV::X0)
+        .addImm(NumOfVReg);
+    BuildMI(MBB, II, DL, TII->get(RISCV::MUL), FactorRegister)
+        .addReg(SizeOfVector)
+        .addReg(VN, RegState::Kill);
+  }
+
+  return FactorRegister;
+}

llvm/lib/Target/RISCV/RISCVMachineFunctionInfo.h

private:
   int VarArgsFrameIndex = 0;
   /// Size of the save area used for varargs
   int VarArgsSaveSize = 0;
   /// FrameIndex used for transferring values between 64-bit FPRs and a pair
   /// of 32-bit GPRs via the stack.
   int MoveF64FrameIndex = -1;
   /// Size of any opaque stack adjustment due to save/restore libcalls.
   unsigned LibCallStackSize = 0;
+  /// Size of RVV stack.
+  uint64_t RVVStackSize = 0;
 
 public:
   RISCVMachineFunctionInfo(const MachineFunction &MF) {}
 
   int getVarArgsFrameIndex() const { return VarArgsFrameIndex; }
   void setVarArgsFrameIndex(int Index) { VarArgsFrameIndex = Index; }
 
   unsigned getVarArgsSaveSize() const { return VarArgsSaveSize; }
public:
   void setLibCallStackSize(unsigned Size) { LibCallStackSize = Size; }
 
   bool useSaveRestoreLibCalls(const MachineFunction &MF) const {
     // We cannot use fixed locations for the callee saved spill slots if the
     // function uses a varargs save area.
     return MF.getSubtarget<RISCVSubtarget>().enableSaveRestore() &&
            VarArgsSaveSize == 0 && !MF.getFrameInfo().hasTailCall();
   }
+
+  uint64_t getRVVStackSize() const { return RVVStackSize; }
+  void setRVVStackSize(uint64_t Size) { RVVStackSize = Size; }
 };
 
 } // end namespace llvm
 
 #endif // LLVM_LIB_TARGET_RISCV_RISCVMACHINEFUNCTIONINFO_H

llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp

bool RISCVRegisterInfo::hasReservedSpillSlot(const MachineFunction &MF,
   auto FII = FixedCSRFIMap.find(Reg);
   if (FII == FixedCSRFIMap.end())
     return false;
 
   FrameIdx = FII->second;
   return true;
 }
 
 void RISCVRegisterInfo::eliminateFrameIndex(MachineBasicBlock::iterator II,
                                             int SPAdj, unsigned FIOperandNum,
                                             RegScavenger *RS) const {
   assert(SPAdj == 0 && "Unexpected non-zero SPAdj value");
 
   MachineInstr &MI = *II;
   MachineFunction &MF = *MI.getParent()->getParent();
+  const MachineFrameInfo &MFI = MF.getFrameInfo();
   MachineRegisterInfo &MRI = MF.getRegInfo();
   const RISCVInstrInfo *TII = MF.getSubtarget<RISCVSubtarget>().getInstrInfo();
   DebugLoc DL = MI.getDebugLoc();
 
   int FrameIndex = MI.getOperand(FIOperandNum).getIndex();
   Register FrameReg;
-  int Offset = getFrameLowering(MF)
-                   ->getFrameIndexReference(MF, FrameIndex, FrameReg)
-                   .getFixed() +
-               MI.getOperand(FIOperandNum + 1).getImm();
+  StackOffset Offset =
+      getFrameLowering(MF)->getFrameIndexReference(MF, FrameIndex, FrameReg);
+  if (MFI.getStackID(FrameIndex) == TargetStackID::Default)
+    Offset += StackOffset::getFixed(MI.getOperand(FIOperandNum + 1).getImm());
+
+  // FIXME: The frame index is fixed but the instruction is actually a
+  // RVV load/store.
+  // Something from IR like below:
+  //
+  // %array = alloca [64 x double]
+  // %vptr = bitcast [64 x double]* %array to <vscale x 1 x double>*
+  // %v = load <vscale x 1 x double>, <vscale x 1 x double>* %vptr
+  assert(((RISCVVPseudosTable::getPseudoInfo(MI.getOpcode()) == nullptr) ||
+          (MFI.getStackID(FrameIndex) != TargetStackID::Default)) &&
+         "No support for VLS frame objects read/written by RVV load/store.");
 
-  if (!isInt<32>(Offset)) {
+  if (!isInt<32>(Offset.getFixed())) {
     report_fatal_error(
         "Frame offsets outside of the signed 32-bit range not supported");
   }
 
   MachineBasicBlock &MBB = *MI.getParent();
   bool FrameRegIsKill = false;
 
-  if (!isInt<12>(Offset)) {
-    assert(isInt<32>(Offset) && "Int32 expected");
+  if (!isInt<12>(Offset.getFixed())) {
     // The offset won't fit in an immediate, so use a scratch register instead
     // Modify Offset and FrameReg appropriately
     Register ScratchReg = MRI.createVirtualRegister(&RISCV::GPRRegClass);
-    TII->movImm(MBB, II, DL, ScratchReg, Offset);
+    TII->movImm(MBB, II, DL, ScratchReg, Offset.getFixed());
     BuildMI(MBB, II, DL, TII->get(RISCV::ADD), ScratchReg)
         .addReg(FrameReg)
         .addReg(ScratchReg, RegState::Kill);
-    Offset = 0;
+    Offset = StackOffset::get(0, Offset.getScalable());
     FrameReg = ScratchReg;
     FrameRegIsKill = true;
   }
 
+  if (!Offset.getScalable()) {
+    // Offset = (fixed offset, 0)
     MI.getOperand(FIOperandNum)
         .ChangeToRegister(FrameReg, false, false, FrameRegIsKill);
-  MI.getOperand(FIOperandNum + 1).ChangeToImmediate(Offset);
+    if (!((RISCVVPseudosTable::getPseudoInfo(MI.getOpcode()) != nullptr) &&
+          MI.mayLoadOrStore()))
+      MI.getOperand(FIOperandNum + 1).ChangeToImmediate(Offset.getFixed());
+  } else {
+    // Offset = (fixed offset, scalable offset)
+    unsigned Opc = RISCV::ADD;
+    int64_t ScalableValue = Offset.getScalable();
+    if (ScalableValue < 0) {
+      ScalableValue = -ScalableValue;
+      Opc = RISCV::SUB;
+    }
+
+    if (ScalableValue == 0) {
+      MI.getOperand(FIOperandNum)
+          .ChangeToRegister(FrameReg, false, false, true);
+      return;
+    }
+
+    // 1. Get vlenb && multiply vlen with number of vector register.
+    Register FactorRegister =
+        TII->getVLENFactoredAmount(MF, MBB, II, ScalableValue);
+
+    // 2. Calculate address: FrameReg + result of multiply
+    Register VL = MRI.createVirtualRegister(&RISCV::GPRRegClass);
+    BuildMI(MBB, II, DL, TII->get(Opc), VL)
+        .addReg(FrameReg)
+        .addReg(FactorRegister);
+
+    // Scalable load/store has no immediate field.
+    if (MFI.getStackID(FrameIndex) == TargetStackID::ScalableVector &&
+        Offset.getFixed()) {
+      BuildMI(MBB, II, DL, TII->get(RISCV::ADDI), VL)
+          .addReg(VL)
+          .addImm(Offset.getFixed());
+    }
+
+    // 3. Replace address register with calculated address register
+    MI.getOperand(FIOperandNum).ChangeToRegister(VL, false, false, true);
+    if (MFI.getStackID(FrameIndex) == TargetStackID::Default)
+      MI.getOperand(FIOperandNum + 1).ChangeToImmediate(Offset.getFixed());
+  }
 }
 
 Register RISCVRegisterInfo::getFrameRegister(const MachineFunction &MF) const {
   const TargetFrameLowering *TFI = getFrameLowering(MF);
   return TFI->hasFP(MF) ? RISCV::X8 : RISCV::X2;
 }
 
 const uint32_t *

llvm/test/CodeGen/RISCV/rvv/allocate-lmul-2-4-8.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-v -verify-machineinstrs < %s \
+; RUN:    | FileCheck %s
+
+define void @lmul1() nounwind {
+; CHECK-LABEL: lmul1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    ret
+  %v = alloca <vscale x 1 x i64>
+  ret void
+}
+
+define void @lmul2() nounwind {
+; CHECK-LABEL: lmul2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 1
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    ret
+  %v = alloca <vscale x 2 x i64>
+  ret void
+}
+
+define void @lmul4() nounwind {
+; CHECK-LABEL: lmul4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -32
+; CHECK-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 16(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 32
+; CHECK-NEXT:    andi sp, sp, -32
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 2
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -32
+; CHECK-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 32
+; CHECK-NEXT:    ret
+  %v = alloca <vscale x 4 x i64>
+  ret void
+}
+
+define void @lmul8() nounwind {
+; CHECK-LABEL: lmul8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -64
+; CHECK-NEXT:    sd ra, 56(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 48(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 64
+; CHECK-NEXT:    andi sp, sp, -64
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 3
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -64
+; CHECK-NEXT:    ld s0, 48(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 56(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    ret
+  %v = alloca <vscale x 8 x i64>
+  ret void
+}
+
+define void @lmul1_and_2() nounwind {
+; CHECK-LABEL: lmul1_and_2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 3
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 3
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 1 x i64>
+  %v2 = alloca <vscale x 2 x i64>
+  ret void
+}
+
+define void @lmul2_and_4() nounwind {
+; CHECK-LABEL: lmul2_and_4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -32
+; CHECK-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 16(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 32
+; CHECK-NEXT:    andi sp, sp, -32
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 6
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -32
+; CHECK-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 32
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 2 x i64>
+  %v2 = alloca <vscale x 4 x i64>
+  ret void
+}
+
+define void @lmul1_and_4() nounwind {
+; CHECK-LABEL: lmul1_and_4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -32
+; CHECK-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 16(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 32
+; CHECK-NEXT:    andi sp, sp, -32
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 5
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -32
+; CHECK-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 32
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 1 x i64>
+  %v2 = alloca <vscale x 4 x i64>
+  ret void
+}
+
+define void @lmul2_and_1() nounwind {
+; CHECK-LABEL: lmul2_and_1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 3
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 3
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 2 x i64>
+  %v2 = alloca <vscale x 1 x i64>
+  ret void
+}
+
+define void @lmul4_and_1() nounwind {
+; CHECK-LABEL: lmul4_and_1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -32
+; CHECK-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 16(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 32
+; CHECK-NEXT:    andi sp, sp, -32
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 5
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -32
+; CHECK-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 32
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 4 x i64>
+  %v2 = alloca <vscale x 1 x i64>
+  ret void
+}
+
+define void @lmul4_and_2() nounwind {
+; CHECK-LABEL: lmul4_and_2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -32
+; CHECK-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 16(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 32
+; CHECK-NEXT:    andi sp, sp, -32
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 6
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -32
+; CHECK-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 32
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 4 x i64>
+  %v2 = alloca <vscale x 2 x i64>
+  ret void
+}
+
+define void @lmul4_and_2_x2_0() nounwind {
+; CHECK-LABEL: lmul4_and_2_x2_0:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -32
+; CHECK-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 16(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 32
+; CHECK-NEXT:    andi sp, sp, -32
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 12
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -32
+; CHECK-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 32
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 4 x i64>
+  %v2 = alloca <vscale x 2 x i64>
+  %v3 = alloca <vscale x 4 x i64>
+  %v4 = alloca <vscale x 2 x i64>
+  ret void
+}
+
+define void @lmul4_and_2_x2_1() nounwind {
+; CHECK-LABEL: lmul4_and_2_x2_1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -32
+; CHECK-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 16(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 32
+; CHECK-NEXT:    andi sp, sp, -32
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 12
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -32
+; CHECK-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 32
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 4 x i64>
+  %v3 = alloca <vscale x 4 x i64>
+  %v2 = alloca <vscale x 2 x i64>
+  %v4 = alloca <vscale x 2 x i64>
+  ret void
+}
+
+
+define void @gpr_and_lmul1_and_2() nounwind {
+; CHECK-LABEL: gpr_and_lmul1_and_2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -16
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 3
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi a0, zero, 3
+; CHECK-NEXT:    csrr a1, vlenb
+; CHECK-NEXT:    addi a2, zero, 3
+; CHECK-NEXT:    mul a1, a1, a2
+; CHECK-NEXT:    add a1, sp, a1
+; CHECK-NEXT:    sd a0, 8(a1)
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 3
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    addi sp, sp, 16
+; CHECK-NEXT:    ret
+  %x1 = alloca i64
+  %v1 = alloca <vscale x 1 x i64>
+  %v2 = alloca <vscale x 2 x i64>
+  store volatile i64 3, i64* %x1
+  ret void
+}
+
+define void @gpr_and_lmul1_and_4() nounwind {
+; CHECK-LABEL: gpr_and_lmul1_and_4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -32
+; CHECK-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 16(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 32
+; CHECK-NEXT:    andi sp, sp, -32
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 5
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi a0, zero, 3
+; CHECK-NEXT:    csrr a1, vlenb
+; CHECK-NEXT:    addi a2, zero, 5
+; CHECK-NEXT:    mul a1, a1, a2
+; CHECK-NEXT:    add a1, sp, a1
+; CHECK-NEXT:    sd a0, 8(a1)
+; CHECK-NEXT:    addi sp, s0, -32
+; CHECK-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 32
+; CHECK-NEXT:    ret
+  %x1 = alloca i64
+  %v1 = alloca <vscale x 1 x i64>
+  %v2 = alloca <vscale x 4 x i64>
+  store volatile i64 3, i64* %x1
+  ret void
+}
+
+define void @lmul_1_2_4_8() nounwind {
+; CHECK-LABEL: lmul_1_2_4_8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -64
+; CHECK-NEXT:    sd ra, 56(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 48(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 64
+; CHECK-NEXT:    andi sp, sp, -64
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 15
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -64
+; CHECK-NEXT:    ld s0, 48(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 56(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 1 x i64>
+  %v2 = alloca <vscale x 2 x i64>
+  %v4 = alloca <vscale x 4 x i64>
+  %v8 = alloca <vscale x 8 x i64>
+  ret void
+}
+
+define void @lmul_1_2_4_8_x2_0() nounwind {
+; CHECK-LABEL: lmul_1_2_4_8_x2_0:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -64
+; CHECK-NEXT:    sd ra, 56(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 48(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 64
+; CHECK-NEXT:    andi sp, sp, -64
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 30
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -64
+; CHECK-NEXT:    ld s0, 48(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 56(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 1 x i64>
+  %v2 = alloca <vscale x 1 x i64>
+  %v3 = alloca <vscale x 2 x i64>
+  %v4 = alloca <vscale x 2 x i64>
+  %v5 = alloca <vscale x 4 x i64>
+  %v6 = alloca <vscale x 4 x i64>
+  %v7 = alloca <vscale x 8 x i64>
+  %v8 = alloca <vscale x 8 x i64>
+  ret void
+}
+
+define void @lmul_1_2_4_8_x2_1() nounwind {
+; CHECK-LABEL: lmul_1_2_4_8_x2_1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    addi sp, sp, -64
+; CHECK-NEXT:    sd ra, 56(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    sd s0, 48(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi s0, sp, 64
+; CHECK-NEXT:    andi sp, sp, -64
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a1, zero, 30
+; CHECK-NEXT:    mul a0, a0, a1
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    addi sp, s0, -64
+; CHECK-NEXT:    ld s0, 48(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    ld ra, 56(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    ret
+  %v8 = alloca <vscale x 8 x i64>
+  %v7 = alloca <vscale x 8 x i64>
+  %v6 = alloca <vscale x 4 x i64>
+  %v5 = alloca <vscale x 4 x i64>
+  %v4 = alloca <vscale x 2 x i64>
+  %v3 = alloca <vscale x 2 x i64>
+  %v2 = alloca <vscale x 1 x i64>
+  %v1 = alloca <vscale x 1 x i64>
+  ret void
+}
+
+define void @masks() nounwind {
+; CHECK-LABEL: masks:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 2
+; CHECK-NEXT:    sub sp, sp, a0
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 2
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    ret
+  %v1 = alloca <vscale x 1 x i1>
+  %v2 = alloca <vscale x 2 x i1>
+  %v4 = alloca <vscale x 4 x i1>
+  %v8 = alloca <vscale x 8 x i1>
+  ret void
+}

llvm/test/CodeGen/RISCV/rvv/localvar.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-v < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IV
+
+define void @local_var_mf8() {
+; RV64IV-LABEL: local_var_mf8:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    .cfi_def_cfa_offset 0
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 1
+; RV64IV-NEXT:    sub sp, sp, a0
+; RV64IV-NEXT:    vsetvli a0, zero, e8,mf8,ta,mu
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    vle8.v v25, (a0)
+; RV64IV-NEXT:    vle8.v v25, (sp)
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 1
+; RV64IV-NEXT:    add sp, sp, a0
+; RV64IV-NEXT:    ret
+  %local0 = alloca <vscale x 1 x i8>
+  %local1 = alloca <vscale x 1 x i8>
+  load volatile <vscale x 1 x i8>, <vscale x 1 x i8>* %local0
+  load volatile <vscale x 1 x i8>, <vscale x 1 x i8>* %local1
+  ret void
+}
+
+define void @local_var_m1() {
+; RV64IV-LABEL: local_var_m1:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    .cfi_def_cfa_offset 0
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 1
+; RV64IV-NEXT:    sub sp, sp, a0
+; RV64IV-NEXT:    vsetvli a0, zero, e8,m1,ta,mu
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    vle8.v v25, (a0)
+; RV64IV-NEXT:    vle8.v v25, (sp)
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 1
+; RV64IV-NEXT:    add sp, sp, a0
+; RV64IV-NEXT:    ret
+  %local0 = alloca <vscale x 8 x i8>
+  %local1 = alloca <vscale x 8 x i8>
+  load volatile <vscale x 8 x i8>, <vscale x 8 x i8>* %local0
+  load volatile <vscale x 8 x i8>, <vscale x 8 x i8>* %local1
+  ret void
+}
+
+define void @local_var_m2() {
+; RV64IV-LABEL: local_var_m2:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    .cfi_def_cfa_offset 0
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 2
+; RV64IV-NEXT:    sub sp, sp, a0
+; RV64IV-NEXT:    vsetvli a0, zero, e8,m2,ta,mu
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 1
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    vle8.v v26, (a0)
+; RV64IV-NEXT:    vle8.v v26, (sp)
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 2
+; RV64IV-NEXT:    add sp, sp, a0
+; RV64IV-NEXT:    ret
+  %local0 = alloca <vscale x 16 x i8>
+  %local1 = alloca <vscale x 16 x i8>
+  load volatile <vscale x 16 x i8>, <vscale x 16 x i8>* %local0
+  load volatile <vscale x 16 x i8>, <vscale x 16 x i8>* %local1
+  ret void
+}
+
+define void @local_var_m4() {
+; RV64IV-LABEL: local_var_m4:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    addi sp, sp, -32
+; RV64IV-NEXT:    .cfi_def_cfa_offset 32
+; RV64IV-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    sd s0, 16(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    .cfi_offset ra, -8
+; RV64IV-NEXT:    .cfi_offset s0, -16
+; RV64IV-NEXT:    addi s0, sp, 32
+; RV64IV-NEXT:    .cfi_def_cfa s0, 0
+; RV64IV-NEXT:    andi sp, sp, -32
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 3
+; RV64IV-NEXT:    sub sp, sp, a0
+; RV64IV-NEXT:    vsetvli a0, zero, e8,m4,ta,mu
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 2
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    vle8.v v28, (a0)
+; RV64IV-NEXT:    vle8.v v28, (sp)
+; RV64IV-NEXT:    addi sp, s0, -32
+; RV64IV-NEXT:    ld s0, 16(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    addi sp, sp, 32
+; RV64IV-NEXT:    ret
+  %local0 = alloca <vscale x 32 x i8>
+  %local1 = alloca <vscale x 32 x i8>
+  load volatile <vscale x 32 x i8>, <vscale x 32 x i8>* %local0
+  load volatile <vscale x 32 x i8>, <vscale x 32 x i8>* %local1
+  ret void
+}
+
+define void @local_var_m8() {
+; RV64IV-LABEL: local_var_m8:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    addi sp, sp, -64
+; RV64IV-NEXT:    .cfi_def_cfa_offset 64
+; RV64IV-NEXT:    sd ra, 56(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    sd s0, 48(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    .cfi_offset ra, -8
+; RV64IV-NEXT:    .cfi_offset s0, -16
+; RV64IV-NEXT:    addi s0, sp, 64
+; RV64IV-NEXT:    .cfi_def_cfa s0, 0
+; RV64IV-NEXT:    andi sp, sp, -64
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 4
+; RV64IV-NEXT:    sub sp, sp, a0
+; RV64IV-NEXT:    vsetvli a0, zero, e8,m8,ta,mu
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 3
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    vle8.v v8, (a0)
+; RV64IV-NEXT:    vle8.v v8, (sp)
+; RV64IV-NEXT:    addi sp, s0, -64
+; RV64IV-NEXT:    ld s0, 48(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    ld ra, 56(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    addi sp, sp, 64
+; RV64IV-NEXT:    ret
+  %local0 = alloca <vscale x 64 x i8>
+  %local1 = alloca <vscale x 64 x i8>
+  load volatile <vscale x 64 x i8>, <vscale x 64 x i8>* %local0
+  load volatile <vscale x 64 x i8>, <vscale x 64 x i8>* %local1
+  ret void
+}
+
+define void @local_var_m2_mix_local_scalar() {
+; RV64IV-LABEL: local_var_m2_mix_local_scalar:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    addi sp, sp, -16
+; RV64IV-NEXT:    .cfi_def_cfa_offset 16
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 2
+; RV64IV-NEXT:    sub sp, sp, a0
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 2
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    lw a0, 12(a0)
+; RV64IV-NEXT:    vsetvli a0, zero, e8,m2,ta,mu
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 1
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    vle8.v v26, (a0)
+; RV64IV-NEXT:    vle8.v v26, (sp)
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 2
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    lw a0, 8(a0)
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 2
+; RV64IV-NEXT:    add sp, sp, a0
+; RV64IV-NEXT:    addi sp, sp, 16
+; RV64IV-NEXT:    ret
+  %local_scalar0 = alloca i32
+  %local0 = alloca <vscale x 16 x i8>
+  %local1 = alloca <vscale x 16 x i8>
+  %local_scalar1 = alloca i32
+  load volatile i32, i32* %local_scalar0
+  load volatile <vscale x 16 x i8>, <vscale x 16 x i8>* %local0
+  load volatile <vscale x 16 x i8>, <vscale x 16 x i8>* %local1
+  load volatile i32, i32* %local_scalar1
+  ret void
+}
+
+define void @local_var_m2_with_varsize_object(i64 %n) {
+; RV64IV-LABEL: local_var_m2_with_varsize_object:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    addi sp, sp, -16
+; RV64IV-NEXT:    .cfi_def_cfa_offset 16
+; RV64IV-NEXT:    sd ra, 8(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    sd s0, 0(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    .cfi_offset ra, -8
+; RV64IV-NEXT:    .cfi_offset s0, -16
+; RV64IV-NEXT:    addi s0, sp, 16
+; RV64IV-NEXT:    .cfi_def_cfa s0, 0
+; RV64IV-NEXT:    csrr a1, vlenb
+; RV64IV-NEXT:    slli a1, a1, 2
+; RV64IV-NEXT:    sub sp, sp, a1
+; RV64IV-NEXT:    addi a0, a0, 15
+; RV64IV-NEXT:    andi a0, a0, -16
+; RV64IV-NEXT:    sub a0, sp, a0
+; RV64IV-NEXT:    mv sp, a0
+; RV64IV-NEXT:    csrr a1, vlenb
+; RV64IV-NEXT:    slli a1, a1, 1
+; RV64IV-NEXT:    sub a1, s0, a1
+; RV64IV-NEXT:    addi a1, a1, -16
+; RV64IV-NEXT:    mv a1, a1
+; RV64IV-NEXT:    call notdead@plt
+; RV64IV-NEXT:    vsetvli a0, zero, e8,m2,ta,mu
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 1
+; RV64IV-NEXT:    sub a0, s0, a0
+; RV64IV-NEXT:    addi a0, a0, -16
+; RV64IV-NEXT:    vle8.v v26, (a0)
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 2
+; RV64IV-NEXT:    sub a0, s0, a0
+; RV64IV-NEXT:    addi a0, a0, -16
+; RV64IV-NEXT:    vle8.v v26, (a0)
+; RV64IV-NEXT:    addi sp, s0, -16
+; RV64IV-NEXT:    ld s0, 0(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    ld ra, 8(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    addi sp, sp, 16
+; RV64IV-NEXT:    ret
+  %1 = alloca i8, i64 %n
+  %2 = alloca <vscale x 16 x i8>
+  %3 = alloca <vscale x 16 x i8>
+  call void @notdead(i8* %1, <vscale x 16 x i8>* %2)
+  load volatile <vscale x 16 x i8>, <vscale x 16 x i8>* %2
+  load volatile <vscale x 16 x i8>, <vscale x 16 x i8>* %3
+  ret void
+}
+
+define void @local_var_m2_with_bp(i64 %n) {
+; RV64IV-LABEL: local_var_m2_with_bp:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    addi sp, sp, -256
+; RV64IV-NEXT:    .cfi_def_cfa_offset 256
+; RV64IV-NEXT:    sd ra, 248(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    sd s0, 240(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    sd s1, 232(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    .cfi_offset ra, -8
+; RV64IV-NEXT:    .cfi_offset s0, -16
+; RV64IV-NEXT:    .cfi_offset s1, -24
+; RV64IV-NEXT:    addi s0, sp, 256
+; RV64IV-NEXT:    .cfi_def_cfa s0, 0
+; RV64IV-NEXT:    andi sp, sp, -128
+; RV64IV-NEXT:    mv s1, sp
+; RV64IV-NEXT:    csrr a1, vlenb
+; RV64IV-NEXT:    slli a1, a1, 2
+; RV64IV-NEXT:    sub sp, sp, a1
+; RV64IV-NEXT:    addi a0, a0, 15
+; RV64IV-NEXT:    andi a0, a0, -16
+; RV64IV-NEXT:    sub a0, sp, a0
+; RV64IV-NEXT:    mv sp, a0
+; RV64IV-NEXT:    addi a1, s1, 128
+; RV64IV-NEXT:    csrr a2, vlenb
+; RV64IV-NEXT:    slli a2, a2, 1
+; RV64IV-NEXT:    sub a2, s1, a2
+; RV64IV-NEXT:    mv a2, a2
+; RV64IV-NEXT:    call notdead2@plt
+; RV64IV-NEXT:    lw a0, 124(s1)
+; RV64IV-NEXT:    vsetvli a0, zero, e8,m2,ta,mu
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 1
+; RV64IV-NEXT:    sub a0, s1, a0
+; RV64IV-NEXT:    vle8.v v26, (a0)
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 2
+; RV64IV-NEXT:    sub a0, s1, a0
+; RV64IV-NEXT:    vle8.v v26, (a0)
+; RV64IV-NEXT:    lw a0, 120(s1)
+; RV64IV-NEXT:    addi sp, s0, -256
+; RV64IV-NEXT:    ld s1, 232(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    ld s0, 240(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    ld ra, 248(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    addi sp, sp, 256
+; RV64IV-NEXT:    ret
+  %1 = alloca i8, i64 %n
+  %2 = alloca i32, align 128
+  %local_scalar0 = alloca i32
+  %local0 = alloca <vscale x 16 x i8>
+  %local1 = alloca <vscale x 16 x i8>
+  %local_scalar1 = alloca i32
+  call void @notdead2(i8* %1, i32* %2, <vscale x 16 x i8>* %local0)
+  load volatile i32, i32* %local_scalar0
+  load volatile <vscale x 16 x i8>, <vscale x 16 x i8>* %local0
+  load volatile <vscale x 16 x i8>, <vscale x 16 x i8>* %local1
+  load volatile i32, i32* %local_scalar1
+  ret void
+}
+
+declare void @notdead(i8*, <vscale x 16 x i8>*)
+declare void @notdead2(i8*, i32*, <vscale x 16 x i8>*)

llvm/test/CodeGen/RISCV/rvv/memory-args.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-v -O2 < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IV
+
+declare <vscale x 64 x i8> @llvm.riscv.vmacc.nxv64i8.nxv64i8(
+  <vscale x 64 x i8>,
+  <vscale x 64 x i8>,
+  <vscale x 64 x i8>,
+  i64);
+
+define <vscale x 64 x i8> @callee(<vscale x 64 x i8> %arg0, <vscale x 64 x i8> %arg1, <vscale x 64 x i8> %arg2) {
+; RV64IV-LABEL: callee:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    vsetvli a1, zero, e8,m8,ta,mu
+; RV64IV-NEXT:    vle8.v v24, (a0)
+; RV64IV-NEXT:    addi a0, zero, 1024
+; RV64IV-NEXT:    vsetvli a0, a0, e8,m8,tu,mu
+; RV64IV-NEXT:    vmacc.vv v8, v16, v24
+; RV64IV-NEXT:    ret
+  %ret = call <vscale x 64 x i8> @llvm.riscv.vmacc.nxv64i8.nxv64i8(
+                                  <vscale x 64 x i8> %arg0,
+                                  <vscale x 64 x i8> %arg1,
+                                  <vscale x 64 x i8> %arg2, i64 1024)
+  ret <vscale x 64 x i8> %ret
+}
+
+define <vscale x 64 x i8> @caller() {
+; RV64IV-LABEL: caller:
+; RV64IV:       # %bb.0:
+; RV64IV-NEXT:    addi sp, sp, -64
+; RV64IV-NEXT:    .cfi_def_cfa_offset 64
+; RV64IV-NEXT:    sd ra, 56(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    sd s0, 48(sp) # 8-byte Folded Spill
+; RV64IV-NEXT:    .cfi_offset ra, -8
+; RV64IV-NEXT:    .cfi_offset s0, -16
+; RV64IV-NEXT:    addi s0, sp, 64
+; RV64IV-NEXT:    .cfi_def_cfa s0, 0
+; RV64IV-NEXT:    andi sp, sp, -64
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 5
+; RV64IV-NEXT:    sub sp, sp, a0
+; RV64IV-NEXT:    vsetvli a0, zero, e8,m8,ta,mu
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    addi a1, zero, 24
+; RV64IV-NEXT:    mul a0, a0, a1
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    vle8.v v8, (a0)
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 4
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    vle8.v v16, (a0)
+; RV64IV-NEXT:    csrr a0, vlenb
+; RV64IV-NEXT:    slli a0, a0, 3
+; RV64IV-NEXT:    add a0, sp, a0
+; RV64IV-NEXT:    vle8.v v24, (a0)
+; RV64IV-NEXT:    mv a0, sp
+; RV64IV-NEXT:    vse8.v v24, (sp)
+; RV64IV-NEXT:    call callee@plt
+; RV64IV-NEXT:    addi sp, s0, -64
+; RV64IV-NEXT:    ld s0, 48(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    ld ra, 56(sp) # 8-byte Folded Reload
+; RV64IV-NEXT:    addi sp, sp, 64
+; RV64IV-NEXT:    ret
+  %local0 = alloca <vscale x 64 x i8>
+  %local1 = alloca <vscale x 64 x i8>
+  %local2 = alloca <vscale x 64 x i8>
+  %arg0 = load volatile <vscale x 64 x i8>, <vscale x 64 x i8>* %local0
+  %arg1 = load volatile <vscale x 64 x i8>, <vscale x 64 x i8>* %local1
+  %arg2 = load volatile <vscale x 64 x i8>, <vscale x 64 x i8>* %local2
+  %ret = call <vscale x 64 x i8> @callee(<vscale x 64 x i8> %arg0,
+                                         <vscale x 64 x i8> %arg1,
+                                         <vscale x 64 x i8> %arg2)
+  ret <vscale x 64 x i8> %ret
+}