[RISCV][LegalizeTypes][TargetLowering] Add a target hook to limit alignment of scalable vectors in SelectionDAG::getReducedAlign.

[craig.topper]

Summary

getReducedAlign is used by LegalizeTypes to when creating stack
temporaries for some vector operations that can't be easily split
into vectors with fewer elements. For example, insert and extract
elements/subvectors.

Scalable vector stack temporaries do not update the maximum stack
alignment in MachineFunctionInfo. If a scalable vector temporary is
created with alignment larger than the maximum stack alignment and
larger than the 16 byte guaranteed for RISC-V, we will fail to
reserve the frame pointer prior to register allocation. But prologue
epilogue emitter will try to use it to align the stack for the vectors.
Since it wasn't reserved for register allocation, it might be used
for something else before the epilogue.

I've also limited some stack temporaries created by argument lowering
and have added a check to ensure the FP was reserved for regalloc. I'll
probably split that into separate patch, but wanted everything together
for initial discussion.

Fixes PR57939.

[craig.topper]

Upload the complete patch

[craig.topper]

clang-format

[reames]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

2304

Looking at this code structure, it really seems like what you're doing here is indirectly implementing a lower alignment requirement.

Digging through the DataLayout code, I'm struggling to tell where our alignment for scalable vectors is actually coming from. It doesn't appear to be the default data layout string from RISCVTargetMachine, and if we hit the default, we'd seem to be getting values less than 16. How'd we get an perf alignment of greater than 16 to start with? Is this maybe a high LMUL case hitting the fallthrough logic in DataLayout::getAlignment? That's the only thing I can think of.

If so, I think this is fixing the issue in the wrong place. I think we need to decide on abi and perf alignments for scalable vector types, and then ensure that DataLayout returns those values. If I'm reading the code correctly, we probably end up with sane values for LMUL=1 and LMUL=2 types, but get bogus results for LMUL>=4.

Only once we've decided on what the preferred alignment should be does it make sense to add a cap here. And even then, it can only cap preferred alignment, not abi alignment.

llvm/lib/Target/RISCV/RISCVFrameLowering.cpp

518

Good idea!

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

11734

Your change here looks correct to me, and could be separated into it's own patch. I would recognize the code a bit. The way I would explain what this is doing is as follows:

This code is creating a stack slot whose address will be passed to the callee. The alignment of the stack slot must satisfy the abi alignment for type. The psABI document appears to be silent on the required abi alignment for a vector type. Given hardware requirements, I believe having the abi alignment follow the alignment of the element type is reasonable. The largest such abi alignment would be 8.

We would prefer to use a larger alignment if profitable - i.e. the perf alignment - but can not use an alignment larger than the stack alignment without additional stack alignment (which is both unimplemented, and likely unprofitable.) Thus we chose some value greater than 8 and less than equal to the stack alignment.

Does the above match your understanding of what's going on here?

[frasercrmck]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

2304

The alignment for scalable vectors comes from the known minimum size. The DataLayout's default settings for vectors mean that this just returns the type size for the alignment. So any vector (fixed or scalable) with a minimum size of > 16 means the stack realignment triggers.

I had a patch which was intended to make it easier for us to specify vector alignments in the DataLayout without listing all possible min sizes we support (which is quite a few): D112531.

As for applying that to RISC-V, I was never sure about using an RVV-enabled DataLayout in non-RVV contexts (as vectors would be scalarized, and we don't want to unnecessarily introduce non-XLEN alignments to vectors. I couldn't see an example of targets which change DataLayouts based on feature flags as RVV is. Maybe if our alignments were always XLEN it would be okay, but legally in RVV we could have a 1-byte alignment for i8 vectors, 2-byte for i16 vectors, etc.

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

2304

Looking at this code structure, it really seems like what you're doing here is indirectly implementing a lower alignment requirement.

Digging through the DataLayout code, I'm struggling to tell where our alignment for scalable vectors is actually coming from. It doesn't appear to be the default data layout string from RISCVTargetMachine, and if we hit the default, we'd seem to be getting values less than 16. How'd we get an perf alignment of greater than 16 to start with? Is this maybe a high LMUL case hitting the fallthrough logic in DataLayout::getAlignment? That's the only thing I can think of.

If so, I think this is fixing the issue in the wrong place. I think we need to decide on abi and perf alignments for scalable vector types, and then ensure that DataLayout returns those values. If I'm reading the code correctly, we probably end up with sane values for LMUL=1 and LMUL=2 types, but get bogus results for LMUL>=4.

I think it is LMUL >= 4 cases that are failing because getKnownMinSize is larger than 16.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

11734

Your change here looks correct to me, and could be separated into it's own patch. I would recognize the code a bit. The way I would explain what this is doing is as follows:

This code is creating a stack slot whose address will be passed to the callee. The alignment of the stack slot must satisfy the abi alignment for type. The psABI document appears to be silent on the required abi alignment for a vector type. Given hardware requirements, I believe having the abi alignment follow the alignment of the element type is reasonable. The largest such abi alignment would be 8.

We would prefer to use a larger alignment if profitable - i.e. the perf alignment - but can not use an alignment larger than the stack alignment without additional stack alignment (which is both unimplemented, and likely unprofitable.) Thus we chose some value greater than 8 and less than equal to the stack alignment.

Does the above match your understanding of what's going on here?

llvm/include/llvm/CodeGen/TargetLowering.h

 //===- llvm/CodeGen/TargetLowering.h - Target Lowering Info -----*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
public:
   }
 
   /// Returns the type to be used for the EVL/AVL operand of VP nodes:
   /// ISD::VP_ADD, ISD::VP_SUB, etc. It must be a legal scalar integer type,
   /// and must be at least as large as i32. The EVL is implicitly zero-extended
   /// to any larger type.
   virtual MVT getVPExplicitVectorLengthTy() const { return MVT::i32; }
 
+  /// Returns a limit to apply to scalable vector temporaries created during
+  /// type legalization and lowering.
+  virtual MaybeAlign getMaxScalableVectorAlign() const { return None; }
+
   /// This callback is used to inspect load/store instructions and add
   /// target-specific MachineMemOperand flags to them.  The default
   /// implementation does nothing.
   virtual MachineMemOperand::Flags getTargetMMOFlags(const Instruction &I) const {
     return MachineMemOperand::MONone;
   }
 
   MachineMemOperand::Flags getLoadMemOperandFlags(const LoadInst &LI,

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

 //===- SelectionDAG.cpp - Implement the SelectionDAG data structures ------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
return getStore(Tmp1.getValue(1), dl, Tmp1, Node->getOperand(1),
                   MachinePointerInfo(VD));
 }
 
 Align SelectionDAG::getReducedAlign(EVT VT, bool UseABI) {
   const DataLayout &DL = getDataLayout();
   Type *Ty = VT.getTypeForEVT(*getContext());
   Align RedAlign = UseABI ? DL.getABITypeAlign(Ty) : DL.getPrefTypeAlign(Ty);
 
+  // Limit scalable vector alignment if requested by the target.
+  if (VT.isScalableVector())
+    if (MaybeAlign ScalableAlign = TLI->getMaxScalableVectorAlign())
+      RedAlign = std::min(RedAlign, *ScalableAlign);
+
   if (TLI->isTypeLegal(VT) || !VT.isVector())
     return RedAlign;
 
   const TargetFrameLowering *TFI = MF->getSubtarget().getFrameLowering();
   const Align StackAlign = TFI->getStackAlign();
 
   // See if we can choose a smaller ABI alignment in cases where it's an
   // illegal vector type that will get broken down.

llvm/lib/Target/RISCV/RISCVFrameLowering.cpp

 //===-- RISCVFrameLowering.cpp - RISCV Frame Information ------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
BuildMI(MBB, MBBI, DL, TII->get(TargetOpcode::CFI_INSTRUCTION))
         .setMIFlag(MachineInstr::FrameSetup);
   }
 
   // Generate new FP.
   if (hasFP(MF)) {
     if (STI.isRegisterReservedByUser(FPReg))
       MF.getFunction().getContext().diagnose(DiagnosticInfoUnsupported{
           MF.getFunction(), "Frame pointer required, but has been reserved."});
+    // The frame pointer does need to be reserved from register allocation.
+    assert(MF.getRegInfo().isReserved(FPReg) && "FP not reserved");
 
     adjustReg(MBB, MBBI, DL, FPReg, SPReg,
               RealStackSize - RVFI->getVarArgsSaveSize(),
               MachineInstr::FrameSetup);
 
     // Emit ".cfi_def_cfa $fp, RVFI->getVarArgsSaveSize()"
     unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::cfiDefCfa(
         nullptr, RI->getDwarfRegNum(FPReg, true), RVFI->getVarArgsSaveSize()));

llvm/lib/Target/RISCV/RISCVISelLowering.h

 //===-- RISCVISelLowering.h - RISCV DAG Lowering Interface ------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
private:
   void validateCCReservedRegs(
       const SmallVectorImpl<std::pair<llvm::Register, llvm::SDValue>> &Regs,
       MachineFunction &MF) const;
 
   bool useRVVForFixedLengthVectorVT(MVT VT) const;
 
   MVT getVPExplicitVectorLengthTy() const override;
 
+  MaybeAlign getMaxScalableVectorAlign() const override;
+
   /// RVV code generation for fixed length vectors does not lower all
   /// BUILD_VECTORs. This makes BUILD_VECTOR legalisation a source of stores to
   /// merge. However, merging them creates a BUILD_VECTOR that is just as
   /// illegal as the original, thus leading to an infinite legalisation loop.
   /// NOTE: Once BUILD_VECTOR can be custom lowered for all legal vector types,
   /// this override can be removed.
   bool mergeStoresAfterLegalization(EVT VT) const override;
 

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

 //===-- RISCVISelLowering.cpp - RISCV DAG Lowering Implementation  --------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
if (Subtarget.hasVInstructions() &&
     return EVT::getVectorVT(Context, MVT::i1, VT.getVectorElementCount());
   return VT.changeVectorElementTypeToInteger();
 }
 
 MVT RISCVTargetLowering::getVPExplicitVectorLengthTy() const {
   return Subtarget.getXLenVT();
 }
 
+MaybeAlign RISCVTargetLowering::getMaxScalableVectorAlign() const {
+  return Align(8);
+}
+
 bool RISCVTargetLowering::getTgtMemIntrinsic(IntrinsicInfo &Info,
                                              const CallInst &I,
                                              MachineFunction &MF,
                                              unsigned Intrinsic) const {
   auto &DL = I.getModule()->getDataLayout();
   switch (Intrinsic) {
   default:
     return false;
for (unsigned i = 0, j = 0, e = ArgLocs.size(); i != e; ++i) {
 
     // IsF64OnRV32DSoftABI && VA.isMemLoc() is handled below in the same way
     // as any other MemLoc.
 
     // Promote the value if needed.
     // For now, only handle fully promoted and indirect arguments.
     if (VA.getLocInfo() == CCValAssign::Indirect) {
       // Store the argument in a stack slot and pass its address.
       Align StackAlign =
           std::max(getPrefTypeAlign(Outs[i].ArgVT, DAG),
                    getPrefTypeAlign(ArgValue.getValueType(), DAG));
       TypeSize StoredSize = ArgValue.getValueType().getStoreSize();
+      // Limit alignment of scalable vector objects.
+      if (StoredSize.isScalable())
+        StackAlign = std::min(StackAlign, Align(8));
+
       // If the original argument was split (e.g. i128), we need
       // to store the required parts of it here (and pass just one address).
       // Vectors may be partly split to registers and partly to the stack, in
       // which case the base address is partly offset and subsequent stores are
       // relative to that.
       unsigned ArgIndex = Outs[i].OrigArgIndex;
       unsigned ArgPartOffset = Outs[i].PartOffset;
       assert(VA.getValVT().isVector() || ArgPartOffset == 0);
       // Calculate the total size to store. We don't have access to what we're
       // actually storing other than performing the loop and collecting the
       // info.
       SmallVector<std::pair<SDValue, SDValue>> Parts;
       while (i + 1 != e && Outs[i + 1].OrigArgIndex == ArgIndex) {
         SDValue PartValue = OutVals[i + 1];
         unsigned PartOffset = Outs[i + 1].PartOffset - ArgPartOffset;
         SDValue Offset = DAG.getIntPtrConstant(PartOffset, DL);
         EVT PartVT = PartValue.getValueType();
         if (PartVT.isScalableVector())
           Offset = DAG.getNode(ISD::VSCALE, DL, XLenVT, Offset);
         StoredSize += PartVT.getStoreSize();
-        StackAlign = std::max(StackAlign, getPrefTypeAlign(PartVT, DAG));
+        Align PartAlign = getPrefTypeAlign(PartVT, DAG);
+        // Limit alignment of scalable vector objects.
+        if (PartVT.isScalableVector())
+          PartAlign = std::min(PartAlign, Align(8));
+        StackAlign = std::max(StackAlign, PartAlign);
         Parts.push_back(std::make_pair(PartValue, Offset));
         ++i;
       }
       SDValue SpillSlot = DAG.CreateStackTemporary(StoredSize, StackAlign);
       int FI = cast<FrameIndexSDNode>(SpillSlot)->getIndex();
       MemOpChains.push_back(
           DAG.getStore(Chain, DL, ArgValue, SpillSlot,
                        MachinePointerInfo::getFixedStack(MF, FI)));

llvm/test/CodeGen/RISCV/rvv/calling-conv-fastcc.ll

 }
 
 declare <vscale x 32 x i32> @ext2(<vscale x 32 x i32>, <vscale x 32 x i32>, i32, i32)
 declare <vscale x 32 x i32> @ext3(<vscale x 32 x i32>, <vscale x 32 x i32>, <vscale x 32 x i32>, i32, i32)
 
 define fastcc <vscale x 32 x i32> @ret_nxv32i32_call_nxv32i32_nxv32i32_i32(<vscale x 32 x i32> %x, <vscale x 32 x i32> %y, i32 %w) {
 ; RV32-LABEL: ret_nxv32i32_call_nxv32i32_nxv32i32_i32:
 ; RV32:       # %bb.0:
-; RV32-NEXT:    addi sp, sp, -144
-; RV32-NEXT:    .cfi_def_cfa_offset 144
-; RV32-NEXT:    sw ra, 140(sp) # 4-byte Folded Spill
+; RV32-NEXT:    addi sp, sp, -32
+; RV32-NEXT:    .cfi_def_cfa_offset 32
+; RV32-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32-NEXT:    .cfi_offset ra, -4
-; RV32-NEXT:    addi s0, sp, 144
-; RV32-NEXT:    .cfi_def_cfa s0, 0
 ; RV32-NEXT:    csrr a1, vlenb
 ; RV32-NEXT:    slli a1, a1, 4
 ; RV32-NEXT:    sub sp, sp, a1
-; RV32-NEXT:    andi sp, sp, -128
 ; RV32-NEXT:    csrr a1, vlenb
 ; RV32-NEXT:    slli a1, a1, 3
 ; RV32-NEXT:    add a3, a0, a1
 ; RV32-NEXT:    vl8re32.v v24, (a3)
 ; RV32-NEXT:    vl8re32.v v0, (a0)
-; RV32-NEXT:    addi a0, sp, 128
+; RV32-NEXT:    addi a0, sp, 16
 ; RV32-NEXT:    vs8r.v v8, (a0)
 ; RV32-NEXT:    add a1, a0, a1
-; RV32-NEXT:    addi a0, sp, 128
+; RV32-NEXT:    addi a0, sp, 16
 ; RV32-NEXT:    li a3, 2
 ; RV32-NEXT:    vs8r.v v16, (a1)
 ; RV32-NEXT:    vmv8r.v v8, v0
 ; RV32-NEXT:    vmv8r.v v16, v24
 ; RV32-NEXT:    call ext2@plt
-; RV32-NEXT:    addi sp, s0, -144
-; RV32-NEXT:    lw ra, 140(sp) # 4-byte Folded Reload
-; RV32-NEXT:    addi sp, sp, 144
+; RV32-NEXT:    csrr a0, vlenb
+; RV32-NEXT:    slli a0, a0, 4
+; RV32-NEXT:    add sp, sp, a0
+; RV32-NEXT:    lw ra, 28(sp) # 4-byte Folded Reload
+; RV32-NEXT:    addi sp, sp, 32
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: ret_nxv32i32_call_nxv32i32_nxv32i32_i32:
 ; RV64:       # %bb.0:
-; RV64-NEXT:    addi sp, sp, -144
-; RV64-NEXT:    .cfi_def_cfa_offset 144
-; RV64-NEXT:    sd ra, 136(sp) # 8-byte Folded Spill
+; RV64-NEXT:    addi sp, sp, -32
+; RV64-NEXT:    .cfi_def_cfa_offset 32
+; RV64-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
 ; RV64-NEXT:    .cfi_offset ra, -8
-; RV64-NEXT:    addi s0, sp, 144
-; RV64-NEXT:    .cfi_def_cfa s0, 0
 ; RV64-NEXT:    csrr a1, vlenb
 ; RV64-NEXT:    slli a1, a1, 4
 ; RV64-NEXT:    sub sp, sp, a1
-; RV64-NEXT:    andi sp, sp, -128
 ; RV64-NEXT:    csrr a1, vlenb
 ; RV64-NEXT:    slli a1, a1, 3
 ; RV64-NEXT:    add a3, a0, a1
 ; RV64-NEXT:    vl8re32.v v24, (a3)
 ; RV64-NEXT:    vl8re32.v v0, (a0)
-; RV64-NEXT:    addi a0, sp, 128
+; RV64-NEXT:    addi a0, sp, 16
 ; RV64-NEXT:    vs8r.v v8, (a0)
 ; RV64-NEXT:    add a1, a0, a1
-; RV64-NEXT:    addi a0, sp, 128
+; RV64-NEXT:    addi a0, sp, 16
 ; RV64-NEXT:    li a3, 2
 ; RV64-NEXT:    vs8r.v v16, (a1)
 ; RV64-NEXT:    vmv8r.v v8, v0
 ; RV64-NEXT:    vmv8r.v v16, v24
 ; RV64-NEXT:    call ext2@plt
-; RV64-NEXT:    addi sp, s0, -144
-; RV64-NEXT:    ld ra, 136(sp) # 8-byte Folded Reload
-; RV64-NEXT:    addi sp, sp, 144
+; RV64-NEXT:    csrr a0, vlenb
+; RV64-NEXT:    slli a0, a0, 4
+; RV64-NEXT:    add sp, sp, a0
+; RV64-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; RV64-NEXT:    addi sp, sp, 32
 ; RV64-NEXT:    ret
   %t = call fastcc <vscale x 32 x i32> @ext2(<vscale x 32 x i32> %y, <vscale x 32 x i32> %x, i32 %w, i32 2)
   ret <vscale x 32 x i32> %t
 }
 
 define fastcc <vscale x 32 x i32> @ret_nxv32i32_call_nxv32i32_nxv32i32_nxv32i32_i32(<vscale x 32 x i32> %x, <vscale x 32 x i32> %y, <vscale x 32 x i32> %z, i32 %w) {
 ; RV32-LABEL: ret_nxv32i32_call_nxv32i32_nxv32i32_nxv32i32_i32:
 ; RV32:       # %bb.0:
-; RV32-NEXT:    addi sp, sp, -144
-; RV32-NEXT:    .cfi_def_cfa_offset 144
-; RV32-NEXT:    sw ra, 140(sp) # 4-byte Folded Spill
+; RV32-NEXT:    addi sp, sp, -32
+; RV32-NEXT:    .cfi_def_cfa_offset 32
+; RV32-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32-NEXT:    .cfi_offset ra, -4
-; RV32-NEXT:    addi s0, sp, 144
-; RV32-NEXT:    .cfi_def_cfa s0, 0
 ; RV32-NEXT:    csrr a1, vlenb
 ; RV32-NEXT:    li a3, 48
 ; RV32-NEXT:    mul a1, a1, a3
 ; RV32-NEXT:    sub sp, sp, a1
-; RV32-NEXT:    andi sp, sp, -128
 ; RV32-NEXT:    csrr a1, vlenb
 ; RV32-NEXT:    slli a1, a1, 3
 ; RV32-NEXT:    add a3, a2, a1
 ; RV32-NEXT:    vl8re32.v v24, (a3)
 ; RV32-NEXT:    csrr a3, vlenb
 ; RV32-NEXT:    slli a3, a3, 3
 ; RV32-NEXT:    add a3, sp, a3
-; RV32-NEXT:    addi a3, a3, 128
+; RV32-NEXT:    addi a3, a3, 16
 ; RV32-NEXT:    vs8r.v v24, (a3) # Unknown-size Folded Spill
 ; RV32-NEXT:    add a3, a0, a1
 ; RV32-NEXT:    vl8re32.v v24, (a3)
-; RV32-NEXT:    addi a3, sp, 128
+; RV32-NEXT:    addi a3, sp, 16
 ; RV32-NEXT:    vs8r.v v24, (a3) # Unknown-size Folded Spill
 ; RV32-NEXT:    vl8re32.v v0, (a2)
 ; RV32-NEXT:    vl8re32.v v24, (a0)
 ; RV32-NEXT:    csrr a0, vlenb
 ; RV32-NEXT:    slli a0, a0, 4
 ; RV32-NEXT:    add a0, sp, a0
-; RV32-NEXT:    addi a0, a0, 128
+; RV32-NEXT:    addi a0, a0, 16
 ; RV32-NEXT:    vs8r.v v8, (a0)
 ; RV32-NEXT:    csrr a2, vlenb
 ; RV32-NEXT:    slli a2, a2, 5
 ; RV32-NEXT:    add a2, sp, a2
-; RV32-NEXT:    addi a2, a2, 128
+; RV32-NEXT:    addi a2, a2, 16
 ; RV32-NEXT:    vs8r.v v24, (a2)
 ; RV32-NEXT:    add a0, a0, a1
 ; RV32-NEXT:    vs8r.v v16, (a0)
 ; RV32-NEXT:    add a1, a2, a1
 ; RV32-NEXT:    csrr a0, vlenb
 ; RV32-NEXT:    slli a0, a0, 5
 ; RV32-NEXT:    add a0, sp, a0
-; RV32-NEXT:    addi a0, a0, 128
+; RV32-NEXT:    addi a0, a0, 16
 ; RV32-NEXT:    csrr a2, vlenb
 ; RV32-NEXT:    slli a2, a2, 4
 ; RV32-NEXT:    add a2, sp, a2
-; RV32-NEXT:    addi a2, a2, 128
+; RV32-NEXT:    addi a2, a2, 16
 ; RV32-NEXT:    li a5, 42
-; RV32-NEXT:    addi a3, sp, 128
+; RV32-NEXT:    addi a3, sp, 16
 ; RV32-NEXT:    vl8re8.v v8, (a3) # Unknown-size Folded Reload
 ; RV32-NEXT:    vs8r.v v8, (a1)
 ; RV32-NEXT:    vmv8r.v v8, v0
 ; RV32-NEXT:    csrr a1, vlenb
 ; RV32-NEXT:    slli a1, a1, 3
 ; RV32-NEXT:    add a1, sp, a1
-; RV32-NEXT:    addi a1, a1, 128
+; RV32-NEXT:    addi a1, a1, 16
 ; RV32-NEXT:    vl8re8.v v16, (a1) # Unknown-size Folded Reload
 ; RV32-NEXT:    call ext3@plt
-; RV32-NEXT:    addi sp, s0, -144
-; RV32-NEXT:    lw ra, 140(sp) # 4-byte Folded Reload
-; RV32-NEXT:    addi sp, sp, 144
+; RV32-NEXT:    csrr a0, vlenb
+; RV32-NEXT:    li a1, 48
+; RV32-NEXT:    mul a0, a0, a1
+; RV32-NEXT:    add sp, sp, a0
+; RV32-NEXT:    lw ra, 28(sp) # 4-byte Folded Reload
+; RV32-NEXT:    addi sp, sp, 32
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: ret_nxv32i32_call_nxv32i32_nxv32i32_nxv32i32_i32:
 ; RV64:       # %bb.0:
-; RV64-NEXT:    addi sp, sp, -144
-; RV64-NEXT:    .cfi_def_cfa_offset 144
-; RV64-NEXT:    sd ra, 136(sp) # 8-byte Folded Spill
+; RV64-NEXT:    addi sp, sp, -48
+; RV64-NEXT:    .cfi_def_cfa_offset 48
+; RV64-NEXT:    sd ra, 40(sp) # 8-byte Folded Spill
 ; RV64-NEXT:    .cfi_offset ra, -8
-; RV64-NEXT:    addi s0, sp, 144
-; RV64-NEXT:    .cfi_def_cfa s0, 0
 ; RV64-NEXT:    csrr a1, vlenb
 ; RV64-NEXT:    li a3, 48
 ; RV64-NEXT:    mul a1, a1, a3
 ; RV64-NEXT:    sub sp, sp, a1
-; RV64-NEXT:    andi sp, sp, -128
 ; RV64-NEXT:    csrr a1, vlenb
 ; RV64-NEXT:    slli a1, a1, 3
 ; RV64-NEXT:    add a3, a2, a1
 ; RV64-NEXT:    vl8re32.v v24, (a3)
 ; RV64-NEXT:    csrr a3, vlenb
 ; RV64-NEXT:    slli a3, a3, 3
 ; RV64-NEXT:    add a3, sp, a3
-; RV64-NEXT:    addi a3, a3, 128
+; RV64-NEXT:    addi a3, a3, 32
 ; RV64-NEXT:    vs8r.v v24, (a3) # Unknown-size Folded Spill
 ; RV64-NEXT:    add a3, a0, a1
 ; RV64-NEXT:    vl8re32.v v24, (a3)
-; RV64-NEXT:    addi a3, sp, 128
+; RV64-NEXT:    addi a3, sp, 32
 ; RV64-NEXT:    vs8r.v v24, (a3) # Unknown-size Folded Spill
 ; RV64-NEXT:    vl8re32.v v0, (a2)
 ; RV64-NEXT:    vl8re32.v v24, (a0)
 ; RV64-NEXT:    csrr a0, vlenb
 ; RV64-NEXT:    slli a0, a0, 4
 ; RV64-NEXT:    add a0, sp, a0
-; RV64-NEXT:    addi a0, a0, 128
+; RV64-NEXT:    addi a0, a0, 32
 ; RV64-NEXT:    vs8r.v v8, (a0)
 ; RV64-NEXT:    csrr a2, vlenb
 ; RV64-NEXT:    slli a2, a2, 5
 ; RV64-NEXT:    add a2, sp, a2
-; RV64-NEXT:    addi a2, a2, 128
+; RV64-NEXT:    addi a2, a2, 32
 ; RV64-NEXT:    vs8r.v v24, (a2)
 ; RV64-NEXT:    add a0, a0, a1
 ; RV64-NEXT:    vs8r.v v16, (a0)
 ; RV64-NEXT:    add a1, a2, a1
 ; RV64-NEXT:    csrr a0, vlenb
 ; RV64-NEXT:    slli a0, a0, 5
 ; RV64-NEXT:    add a0, sp, a0
-; RV64-NEXT:    addi a0, a0, 128
+; RV64-NEXT:    addi a0, a0, 32
 ; RV64-NEXT:    csrr a2, vlenb
 ; RV64-NEXT:    slli a2, a2, 4
 ; RV64-NEXT:    add a2, sp, a2
-; RV64-NEXT:    addi a2, a2, 128
+; RV64-NEXT:    addi a2, a2, 32
 ; RV64-NEXT:    li a5, 42
-; RV64-NEXT:    addi a3, sp, 128
+; RV64-NEXT:    addi a3, sp, 32
 ; RV64-NEXT:    vl8re8.v v8, (a3) # Unknown-size Folded Reload
 ; RV64-NEXT:    vs8r.v v8, (a1)
 ; RV64-NEXT:    vmv8r.v v8, v0
 ; RV64-NEXT:    csrr a1, vlenb
 ; RV64-NEXT:    slli a1, a1, 3
 ; RV64-NEXT:    add a1, sp, a1
-; RV64-NEXT:    addi a1, a1, 128
+; RV64-NEXT:    addi a1, a1, 32
 ; RV64-NEXT:    vl8re8.v v16, (a1) # Unknown-size Folded Reload
 ; RV64-NEXT:    call ext3@plt
-; RV64-NEXT:    addi sp, s0, -144
-; RV64-NEXT:    ld ra, 136(sp) # 8-byte Folded Reload
-; RV64-NEXT:    addi sp, sp, 144
+; RV64-NEXT:    csrr a0, vlenb
+; RV64-NEXT:    li a1, 48
+; RV64-NEXT:    mul a0, a0, a1
+; RV64-NEXT:    add sp, sp, a0
+; RV64-NEXT:    ld ra, 40(sp) # 8-byte Folded Reload
+; RV64-NEXT:    addi sp, sp, 48
 ; RV64-NEXT:    ret
   %t = call fastcc <vscale x 32 x i32> @ext3(<vscale x 32 x i32> %z, <vscale x 32 x i32> %y, <vscale x 32 x i32> %x, i32 %w, i32 42)
   ret <vscale x 32 x i32> %t
 }
 
 ; A test case where the normal calling convention would pass directly via the
 ; stack, but with fastcc can pass indirectly with the extra GPR registers
 ; allowed.
; CHECK-NEXT:    ret
   %s = add <vscale x 32 x i32> %x, %z
   ret <vscale x 32 x i32> %s
 }
 
 ; Calling the function above. Ensure we pass the arguments correctly.
 define fastcc <vscale x 32 x i32> @pass_vector_arg_indirect_stack(<vscale x 32 x i32> %x, <vscale x 32 x i32> %y, <vscale x 32 x i32> %z) {
 ; RV32-LABEL: pass_vector_arg_indirect_stack:
 ; RV32:       # %bb.0:
-; RV32-NEXT:    addi sp, sp, -144
-; RV32-NEXT:    .cfi_def_cfa_offset 144
-; RV32-NEXT:    sw ra, 140(sp) # 4-byte Folded Spill
+; RV32-NEXT:    addi sp, sp, -32
+; RV32-NEXT:    .cfi_def_cfa_offset 32
+; RV32-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32-NEXT:    .cfi_offset ra, -4
-; RV32-NEXT:    addi s0, sp, 144
-; RV32-NEXT:    .cfi_def_cfa s0, 0
 ; RV32-NEXT:    csrr a0, vlenb
 ; RV32-NEXT:    slli a0, a0, 5
 ; RV32-NEXT:    sub sp, sp, a0
-; RV32-NEXT:    andi sp, sp, -128
 ; RV32-NEXT:    csrr a0, vlenb
 ; RV32-NEXT:    slli a0, a0, 3
 ; RV32-NEXT:    vsetvli a1, zero, e32, m8, ta, ma
 ; RV32-NEXT:    vmv.v.i v8, 0
-; RV32-NEXT:    addi a1, sp, 128
+; RV32-NEXT:    addi a1, sp, 16
 ; RV32-NEXT:    vs8r.v v8, (a1)
 ; RV32-NEXT:    csrr a2, vlenb
 ; RV32-NEXT:    slli a2, a2, 4
 ; RV32-NEXT:    add a2, sp, a2
-; RV32-NEXT:    addi a2, a2, 128
+; RV32-NEXT:    addi a2, a2, 16
 ; RV32-NEXT:    vs8r.v v8, (a2)
 ; RV32-NEXT:    add a1, a1, a0
 ; RV32-NEXT:    vs8r.v v8, (a1)
 ; RV32-NEXT:    add a0, a2, a0
 ; RV32-NEXT:    li a1, 1
 ; RV32-NEXT:    li a2, 2
 ; RV32-NEXT:    li a3, 3
 ; RV32-NEXT:    li a4, 4
 ; RV32-NEXT:    li a5, 5
 ; RV32-NEXT:    li a6, 6
 ; RV32-NEXT:    li a7, 7
 ; RV32-NEXT:    csrr t2, vlenb
 ; RV32-NEXT:    slli t2, t2, 4
 ; RV32-NEXT:    add t2, sp, t2
-; RV32-NEXT:    addi t2, t2, 128
-; RV32-NEXT:    addi t4, sp, 128
+; RV32-NEXT:    addi t2, t2, 16
+; RV32-NEXT:    addi t4, sp, 16
 ; RV32-NEXT:    li t6, 8
 ; RV32-NEXT:    vs8r.v v8, (a0)
 ; RV32-NEXT:    li a0, 0
 ; RV32-NEXT:    vmv.v.i v16, 0
 ; RV32-NEXT:    call vector_arg_indirect_stack@plt
-; RV32-NEXT:    addi sp, s0, -144
-; RV32-NEXT:    lw ra, 140(sp) # 4-byte Folded Reload
-; RV32-NEXT:    addi sp, sp, 144
+; RV32-NEXT:    csrr a0, vlenb
+; RV32-NEXT:    slli a0, a0, 5
+; RV32-NEXT:    add sp, sp, a0
+; RV32-NEXT:    lw ra, 28(sp) # 4-byte Folded Reload
+; RV32-NEXT:    addi sp, sp, 32
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: pass_vector_arg_indirect_stack:
 ; RV64:       # %bb.0:
-; RV64-NEXT:    addi sp, sp, -144
-; RV64-NEXT:    .cfi_def_cfa_offset 144
-; RV64-NEXT:    sd ra, 136(sp) # 8-byte Folded Spill
+; RV64-NEXT:    addi sp, sp, -32
+; RV64-NEXT:    .cfi_def_cfa_offset 32
+; RV64-NEXT:    sd ra, 24(sp) # 8-byte Folded Spill
 ; RV64-NEXT:    .cfi_offset ra, -8
-; RV64-NEXT:    addi s0, sp, 144
-; RV64-NEXT:    .cfi_def_cfa s0, 0
 ; RV64-NEXT:    csrr a0, vlenb
 ; RV64-NEXT:    slli a0, a0, 5
 ; RV64-NEXT:    sub sp, sp, a0
-; RV64-NEXT:    andi sp, sp, -128
 ; RV64-NEXT:    csrr a0, vlenb
 ; RV64-NEXT:    slli a0, a0, 3
 ; RV64-NEXT:    vsetvli a1, zero, e32, m8, ta, ma
 ; RV64-NEXT:    vmv.v.i v8, 0
-; RV64-NEXT:    addi a1, sp, 128
+; RV64-NEXT:    addi a1, sp, 16
 ; RV64-NEXT:    vs8r.v v8, (a1)
 ; RV64-NEXT:    csrr a2, vlenb
 ; RV64-NEXT:    slli a2, a2, 4
 ; RV64-NEXT:    add a2, sp, a2
-; RV64-NEXT:    addi a2, a2, 128
+; RV64-NEXT:    addi a2, a2, 16
 ; RV64-NEXT:    vs8r.v v8, (a2)
 ; RV64-NEXT:    add a1, a1, a0
 ; RV64-NEXT:    vs8r.v v8, (a1)
 ; RV64-NEXT:    add a0, a2, a0
 ; RV64-NEXT:    li a1, 1
 ; RV64-NEXT:    li a2, 2
 ; RV64-NEXT:    li a3, 3
 ; RV64-NEXT:    li a4, 4
 ; RV64-NEXT:    li a5, 5
 ; RV64-NEXT:    li a6, 6
 ; RV64-NEXT:    li a7, 7
 ; RV64-NEXT:    csrr t2, vlenb
 ; RV64-NEXT:    slli t2, t2, 4
 ; RV64-NEXT:    add t2, sp, t2
-; RV64-NEXT:    addi t2, t2, 128
-; RV64-NEXT:    addi t4, sp, 128
+; RV64-NEXT:    addi t2, t2, 16
+; RV64-NEXT:    addi t4, sp, 16
 ; RV64-NEXT:    li t6, 8
 ; RV64-NEXT:    vs8r.v v8, (a0)
 ; RV64-NEXT:    li a0, 0
 ; RV64-NEXT:    vmv.v.i v16, 0
 ; RV64-NEXT:    call vector_arg_indirect_stack@plt
-; RV64-NEXT:    addi sp, s0, -144
-; RV64-NEXT:    ld ra, 136(sp) # 8-byte Folded Reload
-; RV64-NEXT:    addi sp, sp, 144
+; RV64-NEXT:    csrr a0, vlenb
+; RV64-NEXT:    slli a0, a0, 5
+; RV64-NEXT:    add sp, sp, a0
+; RV64-NEXT:    ld ra, 24(sp) # 8-byte Folded Reload
+; RV64-NEXT:    addi sp, sp, 32
 ; RV64-NEXT:    ret
   %s = call fastcc <vscale x 32 x i32> @vector_arg_indirect_stack(i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7, <vscale x 32 x i32> zeroinitializer, <vscale x 32 x i32> zeroinitializer, <vscale x 32 x i32> zeroinitializer, i32 8)
   ret <vscale x 32 x i32> %s
 }

llvm/test/CodeGen/RISCV/rvv/calling-conv.ll

; CHECK-NEXT:    ret
   %a = add <vscale x 32 x i32> %x, %y
   ret <vscale x 32 x i32> %a
 }
 
 ; Call the function above. Check that we set the arguments correctly.
 define <vscale x 32 x i32> @caller_scalable_vector_split_indirect(<vscale x 32 x i32> %x) {
 ; RV32-LABEL: caller_scalable_vector_split_indirect:
 ; RV32:       # %bb.0:
-; RV32-NEXT:    addi sp, sp, -144
-; RV32-NEXT:    .cfi_def_cfa_offset 144
-; RV32-NEXT:    sw ra, 140(sp) # 4-byte Folded Spill
+; RV32-NEXT:    addi sp, sp, -48
+; RV32-NEXT:    .cfi_def_cfa_offset 48
+; RV32-NEXT:    sw ra, 44(sp) # 4-byte Folded Spill
 ; RV32-NEXT:    .cfi_offset ra, -4
-; RV32-NEXT:    addi s0, sp, 144
-; RV32-NEXT:    .cfi_def_cfa s0, 0
 ; RV32-NEXT:    csrr a0, vlenb
 ; RV32-NEXT:    slli a0, a0, 4
 ; RV32-NEXT:    sub sp, sp, a0
-; RV32-NEXT:    andi sp, sp, -128
-; RV32-NEXT:    addi a0, sp, 128
+; RV32-NEXT:    addi a0, sp, 32
 ; RV32-NEXT:    vs8r.v v8, (a0)
 ; RV32-NEXT:    csrr a1, vlenb
 ; RV32-NEXT:    slli a1, a1, 3
 ; RV32-NEXT:    add a0, a0, a1
 ; RV32-NEXT:    vs8r.v v16, (a0)
 ; RV32-NEXT:    vsetvli a0, zero, e32, m8, ta, ma
 ; RV32-NEXT:    vmv.v.i v8, 0
-; RV32-NEXT:    addi a0, sp, 128
+; RV32-NEXT:    addi a0, sp, 32
 ; RV32-NEXT:    vmv.v.i v16, 0
 ; RV32-NEXT:    call callee_scalable_vector_split_indirect@plt
-; RV32-NEXT:    addi sp, s0, -144
-; RV32-NEXT:    lw ra, 140(sp) # 4-byte Folded Reload
-; RV32-NEXT:    addi sp, sp, 144
+; RV32-NEXT:    csrr a0, vlenb
+; RV32-NEXT:    slli a0, a0, 4
+; RV32-NEXT:    add sp, sp, a0
+; RV32-NEXT:    lw ra, 44(sp) # 4-byte Folded Reload
+; RV32-NEXT:    addi sp, sp, 48
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: caller_scalable_vector_split_indirect:
 ; RV64:       # %bb.0:
-; RV64-NEXT:    addi sp, sp, -144
-; RV64-NEXT:    .cfi_def_cfa_offset 144
-; RV64-NEXT:    sd ra, 136(sp) # 8-byte Folded Spill
+; RV64-NEXT:    addi sp, sp, -48
+; RV64-NEXT:    .cfi_def_cfa_offset 48
+; RV64-NEXT:    sd ra, 40(sp) # 8-byte Folded Spill
 ; RV64-NEXT:    .cfi_offset ra, -8
-; RV64-NEXT:    addi s0, sp, 144
-; RV64-NEXT:    .cfi_def_cfa s0, 0
 ; RV64-NEXT:    csrr a0, vlenb
 ; RV64-NEXT:    slli a0, a0, 4
 ; RV64-NEXT:    sub sp, sp, a0
-; RV64-NEXT:    andi sp, sp, -128
-; RV64-NEXT:    addi a0, sp, 128
+; RV64-NEXT:    addi a0, sp, 32
 ; RV64-NEXT:    vs8r.v v8, (a0)
 ; RV64-NEXT:    csrr a1, vlenb
 ; RV64-NEXT:    slli a1, a1, 3
 ; RV64-NEXT:    add a0, a0, a1
 ; RV64-NEXT:    vs8r.v v16, (a0)
 ; RV64-NEXT:    vsetvli a0, zero, e32, m8, ta, ma
 ; RV64-NEXT:    vmv.v.i v8, 0
-; RV64-NEXT:    addi a0, sp, 128
+; RV64-NEXT:    addi a0, sp, 32
 ; RV64-NEXT:    vmv.v.i v16, 0
 ; RV64-NEXT:    call callee_scalable_vector_split_indirect@plt
-; RV64-NEXT:    addi sp, s0, -144
-; RV64-NEXT:    ld ra, 136(sp) # 8-byte Folded Reload
-; RV64-NEXT:    addi sp, sp, 144
+; RV64-NEXT:    csrr a0, vlenb
+; RV64-NEXT:    slli a0, a0, 4
+; RV64-NEXT:    add sp, sp, a0
+; RV64-NEXT:    ld ra, 40(sp) # 8-byte Folded Reload
+; RV64-NEXT:    addi sp, sp, 48
 ; RV64-NEXT:    ret
   %c = alloca i64
   %a = call <vscale x 32 x i32> @callee_scalable_vector_split_indirect(<vscale x 32 x i32> zeroinitializer, <vscale x 32 x i32> %x)
   ret <vscale x 32 x i32> %a
 }

llvm/test/CodeGen/RISCV/rvv/extractelt-fp.ll

 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 16 x double> %v, i32 0
   ret double %r
 }
 
 define double @extractelt_nxv16f64_neg1(<vscale x 16 x double> %v) {
 ; CHECK-LABEL: extractelt_nxv16f64_neg1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    addi sp, sp, -64
-; CHECK-NEXT:    .cfi_def_cfa_offset 64
-; CHECK-NEXT:    addi s0, sp, 64
-; CHECK-NEXT:    .cfi_def_cfa s0, 0
+; CHECK-NEXT:    addi sp, sp, -16
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    csrr a0, vlenb
 ; CHECK-NEXT:    slli a0, a0, 4
 ; CHECK-NEXT:    sub sp, sp, a0
-; CHECK-NEXT:    andi sp, sp, -64
-; CHECK-NEXT:    addi a0, sp, 64
+; CHECK-NEXT:    addi a0, sp, 16
 ; CHECK-NEXT:    vs8r.v v8, (a0)
 ; CHECK-NEXT:    csrr a1, vlenb
 ; CHECK-NEXT:    slli a2, a1, 3
 ; CHECK-NEXT:    add a2, a0, a2
 ; CHECK-NEXT:    vs8r.v v16, (a2)
 ; CHECK-NEXT:    slli a1, a1, 4
 ; CHECK-NEXT:    add a0, a1, a0
 ; CHECK-NEXT:    fld fa0, -8(a0)
-; CHECK-NEXT:    addi sp, s0, -64
-; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 4
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    addi sp, sp, 16
 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 16 x double> %v, i32 -1
   ret double %r
 }
 
 define double @extractelt_nxv16f64_imm(<vscale x 16 x double> %v) {
 ; CHECK-LABEL: extractelt_nxv16f64_imm:
 ; CHECK:       # %bb.0:

 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    csrr a1, vlenb
 ; CHECK-NEXT:    slli a2, a1, 1
 ; CHECK-NEXT:    addi a2, a2, -1
 ; CHECK-NEXT:    bltu a0, a2, .LBB54_2
 ; CHECK-NEXT:  # %bb.1:
 ; CHECK-NEXT:    mv a0, a2
 ; CHECK-NEXT:  .LBB54_2:
-; CHECK-NEXT:    addi sp, sp, -64
-; CHECK-NEXT:    .cfi_def_cfa_offset 64
-; CHECK-NEXT:    addi s0, sp, 64
-; CHECK-NEXT:    .cfi_def_cfa s0, 0
+; CHECK-NEXT:    addi sp, sp, -16
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    csrr a2, vlenb
 ; CHECK-NEXT:    slli a2, a2, 4
 ; CHECK-NEXT:    sub sp, sp, a2
-; CHECK-NEXT:    andi sp, sp, -64
 ; CHECK-NEXT:    slli a0, a0, 3
-; CHECK-NEXT:    addi a2, sp, 64
+; CHECK-NEXT:    addi a2, sp, 16
 ; CHECK-NEXT:    add a0, a2, a0
 ; CHECK-NEXT:    vs8r.v v8, (a2)
 ; CHECK-NEXT:    slli a1, a1, 3
 ; CHECK-NEXT:    add a1, a2, a1
 ; CHECK-NEXT:    vs8r.v v16, (a1)
 ; CHECK-NEXT:    fld fa0, 0(a0)
-; CHECK-NEXT:    addi sp, s0, -64
-; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 4
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    addi sp, sp, 16
 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 16 x double> %v, i32 %idx
   ret double %r
 }

llvm/test/CodeGen/RISCV/rvv/extractelt-i1.ll

 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    csrr a2, vlenb
 ; CHECK-NEXT:    slli a3, a2, 4
 ; CHECK-NEXT:    addi a3, a3, -1
 ; CHECK-NEXT:    bltu a1, a3, .LBB7_2
 ; CHECK-NEXT:  # %bb.1:
 ; CHECK-NEXT:    mv a1, a3
 ; CHECK-NEXT:  .LBB7_2:
-; CHECK-NEXT:    addi sp, sp, -64
-; CHECK-NEXT:    addi s0, sp, 64
+; CHECK-NEXT:    addi sp, sp, -16
 ; CHECK-NEXT:    csrr a3, vlenb
 ; CHECK-NEXT:    slli a3, a3, 4
 ; CHECK-NEXT:    sub sp, sp, a3
-; CHECK-NEXT:    andi sp, sp, -64
-; CHECK-NEXT:    addi a3, sp, 64
+; CHECK-NEXT:    addi a3, sp, 16
 ; CHECK-NEXT:    slli a2, a2, 3
 ; CHECK-NEXT:    add a4, a0, a2
 ; CHECK-NEXT:    vl8r.v v16, (a4)
 ; CHECK-NEXT:    vl8r.v v24, (a0)
 ; CHECK-NEXT:    add a0, a3, a1
 ; CHECK-NEXT:    vsetvli a1, zero, e8, m8, ta, ma
 ; CHECK-NEXT:    vmseq.vi v8, v16, 0
 ; CHECK-NEXT:    vmseq.vi v0, v24, 0
 ; CHECK-NEXT:    vmv.v.i v16, 0
 ; CHECK-NEXT:    vmerge.vim v24, v16, 1, v0
 ; CHECK-NEXT:    vs8r.v v24, (a3)
 ; CHECK-NEXT:    add a1, a3, a2
 ; CHECK-NEXT:    vmv1r.v v0, v8
 ; CHECK-NEXT:    vmerge.vim v8, v16, 1, v0
 ; CHECK-NEXT:    vs8r.v v8, (a1)
 ; CHECK-NEXT:    lb a0, 0(a0)
-; CHECK-NEXT:    addi sp, s0, -64
-; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    csrr a1, vlenb
+; CHECK-NEXT:    slli a1, a1, 4
+; CHECK-NEXT:    add sp, sp, a1
+; CHECK-NEXT:    addi sp, sp, 16
 ; CHECK-NEXT:    ret
   %a = load <vscale x 128 x i8>, <vscale x 128 x i8>* %x
   %b = icmp eq <vscale x 128 x i8> %a, zeroinitializer
   %c = extractelement <vscale x 128 x i1> %b, i64 %idx
   ret i1 %c
 }

llvm/test/CodeGen/RISCV/rvv/extractelt-int-rv32.ll

 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 32 x i32> %v, i32 0
   ret i32 %r
 }
 
 define i32 @extractelt_nxv32i32_neg1(<vscale x 32 x i32> %v) {
 ; CHECK-LABEL: extractelt_nxv32i32_neg1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    addi sp, sp, -64
-; CHECK-NEXT:    .cfi_def_cfa_offset 64
-; CHECK-NEXT:    addi s0, sp, 64
-; CHECK-NEXT:    .cfi_def_cfa s0, 0
+; CHECK-NEXT:    addi sp, sp, -16
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    csrr a0, vlenb
 ; CHECK-NEXT:    slli a0, a0, 4
 ; CHECK-NEXT:    sub sp, sp, a0
-; CHECK-NEXT:    andi sp, sp, -64
-; CHECK-NEXT:    addi a0, sp, 64
+; CHECK-NEXT:    addi a0, sp, 16
 ; CHECK-NEXT:    vs8r.v v8, (a0)
 ; CHECK-NEXT:    csrr a1, vlenb
 ; CHECK-NEXT:    slli a2, a1, 3
 ; CHECK-NEXT:    add a2, a0, a2
 ; CHECK-NEXT:    vs8r.v v16, (a2)
 ; CHECK-NEXT:    slli a1, a1, 4
 ; CHECK-NEXT:    add a0, a1, a0
 ; CHECK-NEXT:    lw a0, -4(a0)
-; CHECK-NEXT:    addi sp, s0, -64
-; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    csrr a1, vlenb
+; CHECK-NEXT:    slli a1, a1, 4
+; CHECK-NEXT:    add sp, sp, a1
+; CHECK-NEXT:    addi sp, sp, 16
 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 32 x i32> %v, i32 -1
   ret i32 %r
 }
 
 define i32 @extractelt_nxv32i32_imm(<vscale x 32 x i32> %v) {
 ; CHECK-LABEL: extractelt_nxv32i32_imm:
 ; CHECK:       # %bb.0:

 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    csrr a1, vlenb
 ; CHECK-NEXT:    slli a2, a1, 2
 ; CHECK-NEXT:    addi a2, a2, -1
 ; CHECK-NEXT:    bltu a0, a2, .LBB74_2
 ; CHECK-NEXT:  # %bb.1:
 ; CHECK-NEXT:    mv a0, a2
 ; CHECK-NEXT:  .LBB74_2:
-; CHECK-NEXT:    addi sp, sp, -64
-; CHECK-NEXT:    .cfi_def_cfa_offset 64
-; CHECK-NEXT:    addi s0, sp, 64
-; CHECK-NEXT:    .cfi_def_cfa s0, 0
+; CHECK-NEXT:    addi sp, sp, -16
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    csrr a2, vlenb
 ; CHECK-NEXT:    slli a2, a2, 4
 ; CHECK-NEXT:    sub sp, sp, a2
-; CHECK-NEXT:    andi sp, sp, -64
 ; CHECK-NEXT:    slli a0, a0, 2
-; CHECK-NEXT:    addi a2, sp, 64
+; CHECK-NEXT:    addi a2, sp, 16
 ; CHECK-NEXT:    add a0, a2, a0
 ; CHECK-NEXT:    vs8r.v v8, (a2)
 ; CHECK-NEXT:    slli a1, a1, 3
 ; CHECK-NEXT:    add a1, a2, a1
 ; CHECK-NEXT:    vs8r.v v16, (a1)
 ; CHECK-NEXT:    lw a0, 0(a0)
-; CHECK-NEXT:    addi sp, s0, -64
-; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    csrr a1, vlenb
+; CHECK-NEXT:    slli a1, a1, 4
+; CHECK-NEXT:    add sp, sp, a1
+; CHECK-NEXT:    addi sp, sp, 16
 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 32 x i32> %v, i32 %idx
   ret i32 %r
 }
 
 define i64 @extractelt_nxv16i64_0(<vscale x 16 x i64> %v) {
 ; CHECK-LABEL: extractelt_nxv16i64_0:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 1, e32, m8, ta, ma
 ; CHECK-NEXT:    vslidedown.vi v16, v8, 1
 ; CHECK-NEXT:    vmv.x.s a1, v16
 ; CHECK-NEXT:    vmv.x.s a0, v8
 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 16 x i64> %v, i32 0
   ret i64 %r
 }

llvm/test/CodeGen/RISCV/rvv/extractelt-int-rv64.ll

 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 16 x i64> %v, i32 0
   ret i64 %r
 }
 
 define i64 @extractelt_nxv16i64_neg1(<vscale x 16 x i64> %v) {
 ; CHECK-LABEL: extractelt_nxv16i64_neg1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    addi sp, sp, -64
-; CHECK-NEXT:    .cfi_def_cfa_offset 64
-; CHECK-NEXT:    addi s0, sp, 64
-; CHECK-NEXT:    .cfi_def_cfa s0, 0
+; CHECK-NEXT:    addi sp, sp, -16
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    csrr a0, vlenb
 ; CHECK-NEXT:    slli a0, a0, 4
 ; CHECK-NEXT:    sub sp, sp, a0
-; CHECK-NEXT:    andi sp, sp, -64
-; CHECK-NEXT:    addi a0, sp, 64
+; CHECK-NEXT:    addi a0, sp, 16
 ; CHECK-NEXT:    vs8r.v v8, (a0)
 ; CHECK-NEXT:    csrr a1, vlenb
 ; CHECK-NEXT:    slli a2, a1, 3
 ; CHECK-NEXT:    add a2, a0, a2
 ; CHECK-NEXT:    vs8r.v v16, (a2)
 ; CHECK-NEXT:    slli a1, a1, 4
 ; CHECK-NEXT:    add a0, a1, a0
 ; CHECK-NEXT:    ld a0, -8(a0)
-; CHECK-NEXT:    addi sp, s0, -64
-; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    csrr a1, vlenb
+; CHECK-NEXT:    slli a1, a1, 4
+; CHECK-NEXT:    add sp, sp, a1
+; CHECK-NEXT:    addi sp, sp, 16
 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 16 x i64> %v, i32 -1
   ret i64 %r
 }
 
 define i64 @extractelt_nxv16i64_imm(<vscale x 16 x i64> %v) {
 ; CHECK-LABEL: extractelt_nxv16i64_imm:
 ; CHECK:       # %bb.0:

 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    csrr a1, vlenb
 ; CHECK-NEXT:    slli a2, a1, 1
 ; CHECK-NEXT:    addi a2, a2, -1
 ; CHECK-NEXT:    bltu a0, a2, .LBB74_2
 ; CHECK-NEXT:  # %bb.1:
 ; CHECK-NEXT:    mv a0, a2
 ; CHECK-NEXT:  .LBB74_2:
-; CHECK-NEXT:    addi sp, sp, -64
-; CHECK-NEXT:    .cfi_def_cfa_offset 64
-; CHECK-NEXT:    addi s0, sp, 64
-; CHECK-NEXT:    .cfi_def_cfa s0, 0
+; CHECK-NEXT:    addi sp, sp, -16
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    csrr a2, vlenb
 ; CHECK-NEXT:    slli a2, a2, 4
 ; CHECK-NEXT:    sub sp, sp, a2
-; CHECK-NEXT:    andi sp, sp, -64
 ; CHECK-NEXT:    slli a0, a0, 3
-; CHECK-NEXT:    addi a2, sp, 64
+; CHECK-NEXT:    addi a2, sp, 16
 ; CHECK-NEXT:    add a0, a2, a0
 ; CHECK-NEXT:    vs8r.v v8, (a2)
 ; CHECK-NEXT:    slli a1, a1, 3
 ; CHECK-NEXT:    add a1, a2, a1
 ; CHECK-NEXT:    vs8r.v v16, (a1)
 ; CHECK-NEXT:    ld a0, 0(a0)
-; CHECK-NEXT:    addi sp, s0, -64
-; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    csrr a1, vlenb
+; CHECK-NEXT:    slli a1, a1, 4
+; CHECK-NEXT:    add sp, sp, a1
+; CHECK-NEXT:    addi sp, sp, 16
 ; CHECK-NEXT:    ret
   %r = extractelement <vscale x 16 x i64> %v, i32 %idx
   ret i64 %r
 }

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-insert-subvector.ll

; CHECK-NEXT:    ret
   ret void
 }
 
 ; Check we don't mistakenly optimize this: we don't know whether this is
 ; inserted into the low or high split vector.
 define void @insert_v2i64_nxv16i64_hi(<2 x i64>* %psv, <vscale x 16 x i64>* %out) {
 ; CHECK-LABEL: insert_v2i64_nxv16i64_hi:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    addi sp, sp, -64
-; CHECK-NEXT:    .cfi_def_cfa_offset 64
-; CHECK-NEXT:    addi s0, sp, 64
-; CHECK-NEXT:    .cfi_def_cfa s0, 0
+; CHECK-NEXT:    addi sp, sp, -16
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    csrr a2, vlenb
 ; CHECK-NEXT:    slli a2, a2, 4
 ; CHECK-NEXT:    sub sp, sp, a2
-; CHECK-NEXT:    andi sp, sp, -64
 ; CHECK-NEXT:    vsetivli zero, 2, e64, m1, ta, ma
 ; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    addi a0, sp, 128
+; CHECK-NEXT:    addi a0, sp, 80
 ; CHECK-NEXT:    vse64.v v8, (a0)
 ; CHECK-NEXT:    csrr a0, vlenb
 ; CHECK-NEXT:    slli a0, a0, 3
-; CHECK-NEXT:    addi a2, sp, 64
+; CHECK-NEXT:    addi a2, sp, 16
 ; CHECK-NEXT:    add a3, a2, a0
 ; CHECK-NEXT:    vl8re64.v v8, (a3)
 ; CHECK-NEXT:    vl8re64.v v16, (a2)
 ; CHECK-NEXT:    add a0, a1, a0
 ; CHECK-NEXT:    vs8r.v v8, (a0)
 ; CHECK-NEXT:    vs8r.v v16, (a1)
-; CHECK-NEXT:    addi sp, s0, -64
-; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 4
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    addi sp, sp, 16
 ; CHECK-NEXT:    ret
   %sv = load <2 x i64>, <2 x i64>* %psv
   %v = call <vscale x 16 x i64> @llvm.vector.insert.v2i64.nxv16i64(<vscale x 16 x i64> undef, <2 x i64> %sv, i64 8)
   store <vscale x 16 x i64> %v, <vscale x 16 x i64>* %out
   ret void
 }
 
 declare <8 x i1> @llvm.vector.insert.v4i1.v8i1(<8 x i1>, <4 x i1>, i64)

llvm/test/CodeGen/RISCV/rvv/named-vector-shuffle-reverse.ll

 ; CHECK-NEXT:    ret
   %res = call <vscale x 6 x i64> @llvm.experimental.vector.reverse.nxv6i64(<vscale x 6 x i64> %a)
   ret <vscale x 6 x i64> %res
 }
 
 define <vscale x 12 x i64> @reverse_nxv12i64(<vscale x 12 x i64> %a) {
 ; CHECK-LABEL: reverse_nxv12i64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    addi sp, sp, -64
-; CHECK-NEXT:    .cfi_def_cfa_offset 64
-; CHECK-NEXT:    addi s0, sp, 64
-; CHECK-NEXT:    .cfi_def_cfa s0, 0
+; CHECK-NEXT:    addi sp, sp, -16
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    csrr a0, vlenb
 ; CHECK-NEXT:    slli a0, a0, 4
 ; CHECK-NEXT:    sub sp, sp, a0
-; CHECK-NEXT:    andi sp, sp, -64
 ; CHECK-NEXT:    csrr a0, vlenb
 ; CHECK-NEXT:    addi a1, a0, -1
 ; CHECK-NEXT:    vsetvli a2, zero, e64, m8, ta, ma
 ; CHECK-NEXT:    vid.v v24
 ; CHECK-NEXT:    vrsub.vx v24, v24, a1
 ; CHECK-NEXT:    vrgather.vv v0, v16, v24
 ; CHECK-NEXT:    vmv4r.v v16, v4
 ; CHECK-NEXT:    vrgather.vv v0, v8, v24
 ; CHECK-NEXT:    vmv4r.v v20, v0
 ; CHECK-NEXT:    slli a0, a0, 3
-; CHECK-NEXT:    addi a1, sp, 64
+; CHECK-NEXT:    addi a1, sp, 16
 ; CHECK-NEXT:    add a0, a1, a0
 ; CHECK-NEXT:    vs4r.v v4, (a0)
 ; CHECK-NEXT:    vs8r.v v16, (a1)
 ; CHECK-NEXT:    vl8re64.v v16, (a0)
 ; CHECK-NEXT:    vl8re64.v v8, (a1)
-; CHECK-NEXT:    addi sp, s0, -64
-; CHECK-NEXT:    addi sp, sp, 64
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 4
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    addi sp, sp, 16
 ; CHECK-NEXT:    ret
   %res = call <vscale x 12 x i64> @llvm.experimental.vector.reverse.nxv12i64(<vscale x 12 x i64> %a)
   ret <vscale x 12 x i64> %res
 }
 
 declare <vscale x 2 x i1> @llvm.experimental.vector.reverse.nxv2i1(<vscale x 2 x i1>)
 declare <vscale x 4 x i1> @llvm.experimental.vector.reverse.nxv4i1(<vscale x 4 x i1>)
 declare <vscale x 8 x i1> @llvm.experimental.vector.reverse.nxv8i1(<vscale x 8 x i1>)

llvm/test/CodeGen/RISCV/rvv/rvv-args-by-mem.ll

; CHECK-NEXT:    ret
   %s1 = add <vscale x 16 x i32> %x, %z
   %s = add <vscale x 16 x i32> %s0, %s1
   ret <vscale x 16 x i32> %s
 }
 
 define <vscale x 16 x i32> @foo(i32 %0, i32 %1, i32 %2, i32 %3, i32 %4, i32 %5, i32 %6, i32 %7, <vscale x 16 x i32> %x) {
 ; CHECK-LABEL: foo:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    addi sp, sp, -80
-; CHECK-NEXT:    .cfi_def_cfa_offset 80
-; CHECK-NEXT:    sd ra, 72(sp) # 8-byte Folded Spill
+; CHECK-NEXT:    addi sp, sp, -48
+; CHECK-NEXT:    .cfi_def_cfa_offset 48
+; CHECK-NEXT:    sd ra, 40(sp) # 8-byte Folded Spill
 ; CHECK-NEXT:    .cfi_offset ra, -8
-; CHECK-NEXT:    addi s0, sp, 80
-; CHECK-NEXT:    .cfi_def_cfa s0, 0
 ; CHECK-NEXT:    csrr t0, vlenb
 ; CHECK-NEXT:    slli t0, t0, 4
 ; CHECK-NEXT:    sub sp, sp, t0
-; CHECK-NEXT:    andi sp, sp, -64
-; CHECK-NEXT:    mv s1, sp
-; CHECK-NEXT:    addi t0, s1, 64
+; CHECK-NEXT:    addi t0, sp, 32
 ; CHECK-NEXT:    vs8r.v v8, (t0)
 ; CHECK-NEXT:    csrr t1, vlenb
 ; CHECK-NEXT:    slli t1, t1, 3
-; CHECK-NEXT:    add t1, s1, t1
-; CHECK-NEXT:    addi t1, t1, 64
+; CHECK-NEXT:    add t1, sp, t1
+; CHECK-NEXT:    addi t1, t1, 32
 ; CHECK-NEXT:    vs8r.v v8, (t1)
 ; CHECK-NEXT:    sd t0, 8(sp)
 ; CHECK-NEXT:    sd t1, 0(sp)
 ; CHECK-NEXT:    vmv8r.v v16, v8
 ; CHECK-NEXT:    call bar@plt
-; CHECK-NEXT:    addi sp, s0, -80
-; CHECK-NEXT:    ld ra, 72(sp) # 8-byte Folded Reload
-; CHECK-NEXT:    addi sp, sp, 80
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 4
+; CHECK-NEXT:    add sp, sp, a0
+; CHECK-NEXT:    ld ra, 40(sp) # 8-byte Folded Reload
+; CHECK-NEXT:    addi sp, sp, 48
 ; CHECK-NEXT:    ret
   %ret = call <vscale x 16 x i32> @bar(i32 %0, i32 %1, i32 %2, i32 %3, i32 %4, i32 %5, i32 %6, i32 %7, <vscale x 16 x i32> %x, <vscale x 16 x i32> %x, <vscale x 16 x i32> %x, <vscale x 16 x i32> %x)
   ret <vscale x 16 x i32> %ret
 }