[RISCV] Initial version of a demand based vsetvli insertion pass.

[craig.topper]

Summary

This patch adds a new pass to conservativley insert vsetvli's.
This replaces the current approach of aggressively adding them
and then trying to remove unneeded.

I've removed the custom inserter and implicit uses of vl/vtype
from the pseudo instructions. The implicit uses will be added
when the vsetvli instruction is inserted.

I've placed this pass after the SSA optimizations but while we
are still in SSE form. I'm considering pushing it later to after
the machine scheduler, but that affects more tests and requires
live intervals to be handled. Moving it after the scheduler may
allow us to add heuristics to the scheduler to group same vtype
and vsetvli operations together.

Other goals for this are to enable more intelligent selection of
vtype for mask instructions like vmand/vmor/vmxor, etc which
don't use SEW. More intelligent vtype selection for vmv.x.s which
doesn't use LMUL. If we are able to see the previous vtypes we can
avoid inserting a vsetvli for these instructions in some cases. This
is hard to do in the current design since we can't track the users
of a vsetvli when we are trying to see if it is unneeded.

[HsiangKai]

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

151

Should we check the range of immediate? vsetivli only has 5-bits immediate field.

203

Is 'the' redundant in the comment?

[craig.topper]

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

151

I can assert it. It should have been checked by SelectionDAG when the instruction was created.

203

Yes. Thanks

[HsiangKai]

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

151

Got it. That's fine without the assertion.

[arcbbb]

LGTM.

I think https://reviews.llvm.org/D100932 is still useful here.
How do you think if I move it into RISCVInsertVSETVLI.cpp instead?

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

134

is it enough to say this instruction is not reading VL?
If yes, we don't have to add VL implicit use if HasVLOpMask is false.

[HsiangKai]

llvm/test/CodeGen/RISCV/rvv/addi-scalable-offset.mir

2

This test case needs to add -mattr=+experimental-v in the command line. Otherwise, the PseudoVSETVLI will disappear.

[craig.topper]

Don't add VL implicit use if VL isn't used.
Add VL implicit use to X0, X0 vsetvli.
Add experimental-v flag to addi-scalable-offset.mir

[craig.topper]

Remove extra word from comment

[craig.topper]

In D101938#2743763, @arcbbb wrote:

LGTM.

I think https://reviews.llvm.org/D100932 is still useful here.
How do you think if I move it into RISCVInsertVSETVLI.cpp instead?

I agree it's useful, but we should probably keep it in RISCVCleanupVSETVLI.cpp so we can keep the skipFunction() check in it's runOnMachineFunction and the getOptLevel() check in RISCVTargetMachine.

[craig.topper]

Fix up the mask-reg-alloc.mir test to not have noreg for AVL.

[craig.topper]

Rebase on recent test additions.

[craig.topper]

Remove implicit VL/VTYPE from another test.

[craig.topper]

Add a new MIR test to get more direct coverage and show the manipulation of the implicit operands.

[craig.topper]

Rebase

[craig.topper]

Don't add VSETVLI before vmv.x.s if the SEW matches. The other fields of VTYPE don't matter.

[craig.topper]

Rebase after removal of RISCVII:VSEW

[rogfer01]

Thanks @craig.topper. This is a great cleanup compared to our existing approach (which was actually very conservative).

Some higher level questions:

• I understand we still want a global cleanup after this pass, don't we?
• Would be this pass a good moment to try to optimise those cases where the AVL and SEW/LMUL are not changed and the instruction has explicit EEW. (Maybe we need to add some extra TSFlag for those)
  • For instance if we are under SEW=64 LMUL=2 and AVL=%avl a PseudoVLE32 that switches to SEW=32, LMUL=½, AVL=%avl I understand it can skip the vsetvli as it will behave as expected.

[frasercrmck]

LGTM, good stuff. One minor nit and a question.

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

236

Not for this patch but should we make sure calls are marked as clobbering VL/VTYPE? Or is there a reason to keep them separate?

252

ST has already been defined above

[craig.topper]

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

236

I think any call with a regmask operand probably does appear to clobber VL since the regmask marks non-clobbered registers and not include VL. But I think I wasn't sure all calls would end up with a regmask argument.

[craig.topper]

Address some review comments.
Clear dead flags for VL and VTYPE defs on existing vsetvli instructions in case
we make them live.

[craig.topper]

Abandoned in favor of D102737

llvm/lib/Target/RISCV/CMakeLists.txt

 tablegen(LLVM RISCVGenSearchableTables.inc -gen-searchable-tables)
 tablegen(LLVM RISCVGenSubtargetInfo.inc -gen-subtarget)
 
 add_public_tablegen_target(RISCVCommonTableGen)
 
 add_llvm_target(RISCVCodeGen
   RISCVAsmPrinter.cpp
   RISCVCallLowering.cpp
-  RISCVCleanupVSETVLI.cpp
   RISCVExpandAtomicPseudoInsts.cpp
   RISCVExpandPseudoInsts.cpp
   RISCVFrameLowering.cpp
+  RISCVInsertVSETVLI.cpp
   RISCVInstrInfo.cpp
   RISCVInstructionSelector.cpp
   RISCVISelDAGToDAG.cpp
   RISCVISelLowering.cpp
   RISCVLegalizerInfo.cpp
   RISCVMCInstLower.cpp
   RISCVMergeBaseOffset.cpp
   RISCVRegisterBankInfo.cpp

llvm/lib/Target/RISCV/RISCV.h

 void initializeRISCVMergeBaseOffsetOptPass(PassRegistry &);
 
 FunctionPass *createRISCVExpandPseudoPass();
 void initializeRISCVExpandPseudoPass(PassRegistry &);
 
 FunctionPass *createRISCVExpandAtomicPseudoPass();
 void initializeRISCVExpandAtomicPseudoPass(PassRegistry &);
 
-FunctionPass *createRISCVCleanupVSETVLIPass();
-void initializeRISCVCleanupVSETVLIPass(PassRegistry &);
+FunctionPass *createRISCVInsertVSETVLIPass();
+void initializeRISCVInsertVSETVLIPass(PassRegistry &);
 
 InstructionSelector *createRISCVInstructionSelector(const RISCVTargetMachine &,
                                                     RISCVSubtarget &,
                                                     RISCVRegisterBankInfo &);
 }
 
 #endif

llvm/lib/Target/RISCV/RISCVCleanupVSETVLI.cpp

-//===- RISCVCleanupVSETVLI.cpp - Cleanup unneeded VSETVLI instructions ----===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-//
-// This file implements a function pass that removes duplicate vsetvli
-// instructions within a basic block.
-//
-//===----------------------------------------------------------------------===//
-
-#include "RISCV.h"
-#include "RISCVSubtarget.h"
-#include "llvm/CodeGen/MachineFunctionPass.h"
-using namespace llvm;
-
-#define DEBUG_TYPE "riscv-cleanup-vsetvli"
-#define RISCV_CLEANUP_VSETVLI_NAME "RISCV Cleanup VSETVLI pass"
-
-namespace {
-
-class RISCVCleanupVSETVLI : public MachineFunctionPass {
-public:
-  static char ID;
-
-  RISCVCleanupVSETVLI() : MachineFunctionPass(ID) {
-    initializeRISCVCleanupVSETVLIPass(*PassRegistry::getPassRegistry());
-  }
-  bool runOnMachineFunction(MachineFunction &MF) override;
-  bool runOnMachineBasicBlock(MachineBasicBlock &MBB);
-
-  MachineFunctionProperties getRequiredProperties() const override {
-    return MachineFunctionProperties().set(
-        MachineFunctionProperties::Property::IsSSA);
-  }
-
-  // This pass modifies the program, but does not modify the CFG
-  void getAnalysisUsage(AnalysisUsage &AU) const override {
-    AU.setPreservesCFG();
-    MachineFunctionPass::getAnalysisUsage(AU);
-  }
-
-  StringRef getPassName() const override { return RISCV_CLEANUP_VSETVLI_NAME; }
-};
-
-} // end anonymous namespace
-
-char RISCVCleanupVSETVLI::ID = 0;
-
-INITIALIZE_PASS(RISCVCleanupVSETVLI, DEBUG_TYPE,
-                RISCV_CLEANUP_VSETVLI_NAME, false, false)
-
-static bool isRedundantVSETVLI(MachineInstr &MI, MachineInstr *PrevVSETVLI) {
-  // If we don't have a previous VSET{I}VLI or the VL output isn't dead, we
-  // can't remove this VSETVLI.
-  if (!PrevVSETVLI || !MI.getOperand(0).isDead())
-    return false;
-
-  // Does this VSET{I}VLI use the same VTYPE immediate.
-  int64_t PrevVTYPEImm = PrevVSETVLI->getOperand(2).getImm();
-  int64_t VTYPEImm = MI.getOperand(2).getImm();
-  if (PrevVTYPEImm != VTYPEImm)
-    return false;
-
-  if (MI.getOpcode() == RISCV::PseudoVSETIVLI) {
-    // If the previous opcode wasn't vsetivli we can't compare them.
-    if (PrevVSETVLI->getOpcode() != RISCV::PseudoVSETIVLI)
-      return false;
-
-    // For VSETIVLI, we can just compare the immediates.
-    return PrevVSETVLI->getOperand(1).getImm() == MI.getOperand(1).getImm();
-  }
-
-  assert(MI.getOpcode() == RISCV::PseudoVSETVLI);
-  Register AVLReg = MI.getOperand(1).getReg();
-  Register PrevOutVL = PrevVSETVLI->getOperand(0).getReg();
-
-  // If this VSETVLI isn't changing VL, it is redundant.
-  if (AVLReg == RISCV::X0 && MI.getOperand(0).getReg() == RISCV::X0)
-    return true;
-
-  // If the previous VSET{I}VLI's output (which isn't X0) is fed into this
-  // VSETVLI, this one isn't changing VL so is redundant.
-  // Only perform this on virtual registers to avoid the complexity of having
-  // to work out if the physical register was clobbered somewhere in between.
-  if (AVLReg.isVirtual() && AVLReg == PrevOutVL)
-    return true;
-
-  // If the previous opcode isn't vsetvli we can't do any more comparison.
-  if (PrevVSETVLI->getOpcode() != RISCV::PseudoVSETVLI)
-    return false;
-
-  // Does this VSETVLI use the same AVL register?
-  if (AVLReg != PrevVSETVLI->getOperand(1).getReg())
-    return false;
-
-  // If the AVLReg is X0 we must be setting VL to VLMAX. Keeping VL unchanged
-  // was handled above.
-  if (AVLReg == RISCV::X0) {
-    // This instruction is setting VL to VLMAX, this is redundant if the
-    // previous VSETVLI was also setting VL to VLMAX. But it is not redundant
-    // if they were setting it to any other value or leaving VL unchanged.
-    return PrevOutVL != RISCV::X0;
-  }
-
-  // This vsetvli is redundant.
-  return true;
-}
-
-bool RISCVCleanupVSETVLI::runOnMachineBasicBlock(MachineBasicBlock &MBB) {
-  bool Changed = false;
-  MachineInstr *PrevVSETVLI = nullptr;
-
-  for (auto MII = MBB.begin(), MIE = MBB.end(); MII != MIE;) {
-    MachineInstr &MI = *MII++;
-
-    if (MI.getOpcode() != RISCV::PseudoVSETVLI &&
-        MI.getOpcode() != RISCV::PseudoVSETIVLI) {
-      if (PrevVSETVLI &&
-          (MI.isCall() || MI.modifiesRegister(RISCV::VL) ||
-           MI.modifiesRegister(RISCV::VTYPE))) {
-        // Old VL/VTYPE is overwritten.
-        PrevVSETVLI = nullptr;
-      }
-      continue;
-    }
-
-    if (isRedundantVSETVLI(MI, PrevVSETVLI)) {
-      // This VSETVLI is redundant, remove it.
-      MI.eraseFromParent();
-      Changed = true;
-    } else {
-      // Otherwise update VSET{I}VLI for the next iteration.
-      PrevVSETVLI = &MI;
-    }
-  }
-
-  return Changed;
-}
-
-bool RISCVCleanupVSETVLI::runOnMachineFunction(MachineFunction &MF) {
-  if (skipFunction(MF.getFunction()))
-    return false;
-
-  // Skip if the vector extension is not enabled.
-  const RISCVSubtarget &ST = MF.getSubtarget<RISCVSubtarget>();
-  if (!ST.hasStdExtV())
-    return false;
-
-  bool Changed = false;
-
-  for (MachineBasicBlock &MBB : MF)
-    Changed |= runOnMachineBasicBlock(MBB);
-
-  return Changed;
-}
-
-/// Returns an instance of the Cleanup VSETVLI pass.
-FunctionPass *llvm::createRISCVCleanupVSETVLIPass() {
-  return new RISCVCleanupVSETVLI();
-}

llvm/lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

bool RISCVExpandPseudo::expandLoadTLSGDAddress(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
     MachineBasicBlock::iterator &NextMBBI) {
   return expandAuipcInstPair(MBB, MBBI, NextMBBI, RISCVII::MO_TLS_GD_HI,
                              RISCV::ADDI);
 }
 
 bool RISCVExpandPseudo::expandVSetVL(MachineBasicBlock &MBB,
                                      MachineBasicBlock::iterator MBBI) {
-  assert(MBBI->getNumOperands() == 5 && "Unexpected instruction format");
+  assert(MBBI->getNumExplicitOperands() == 3 && MBBI->getNumOperands() >= 5 &&
+         "Unexpected instruction format");
 
   DebugLoc DL = MBBI->getDebugLoc();
 
   assert((MBBI->getOpcode() == RISCV::PseudoVSETVLI ||
           MBBI->getOpcode() == RISCV::PseudoVSETIVLI) &&
          "Unexpected pseudo instruction");
   unsigned Opcode;
   if (MBBI->getOpcode() == RISCV::PseudoVSETVLI)

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

while (SelectMBBI != SelectEnd) {
     }
     SelectMBBI = Next;
   }
 
   F->getProperties().reset(MachineFunctionProperties::Property::NoPHIs);
   return TailMBB;
 }
 
-static MachineInstr *elideCopies(MachineInstr *MI,
-                                 const MachineRegisterInfo &MRI) {
-  while (true) {
-    if (!MI->isFullCopy())
-      return MI;
-    if (!Register::isVirtualRegister(MI->getOperand(1).getReg()))
-      return nullptr;
-    MI = MRI.getVRegDef(MI->getOperand(1).getReg());
-    if (!MI)
-      return nullptr;
-  }
-}
-
-static MachineBasicBlock *addVSetVL(MachineInstr &MI, MachineBasicBlock *BB,
-                                    int VLIndex, unsigned SEWIndex,
-                                    RISCVVLMUL VLMul, bool ForceTailAgnostic) {
-  MachineFunction &MF = *BB->getParent();
-  DebugLoc DL = MI.getDebugLoc();
-  const TargetInstrInfo &TII = *MF.getSubtarget().getInstrInfo();
-
-  unsigned Log2SEW = MI.getOperand(SEWIndex).getImm();
-  assert(RISCVVType::isValidSEW(1 << Log2SEW) && "Unexpected SEW");
-  RISCVVSEW ElementWidth = static_cast<RISCVVSEW>(Log2SEW - 3);
-
-  MachineRegisterInfo &MRI = MF.getRegInfo();
-
-  auto BuildVSETVLI = [&]() {
-    if (VLIndex >= 0) {
-      Register DestReg = MRI.createVirtualRegister(&RISCV::GPRRegClass);
-      const MachineOperand &VLOp = MI.getOperand(VLIndex);
-
-      // VL can be a register or an immediate.
-      if (VLOp.isImm())
-        return BuildMI(*BB, MI, DL, TII.get(RISCV::PseudoVSETIVLI))
-            .addReg(DestReg, RegState::Define | RegState::Dead)
-            .addImm(VLOp.getImm());
-
-      Register VLReg = MI.getOperand(VLIndex).getReg();
-      return BuildMI(*BB, MI, DL, TII.get(RISCV::PseudoVSETVLI))
-          .addReg(DestReg, RegState::Define | RegState::Dead)
-          .addReg(VLReg);
-    }
-
-    // With no VL operator in the pseudo, do not modify VL (rd = X0, rs1 = X0).
-    return BuildMI(*BB, MI, DL, TII.get(RISCV::PseudoVSETVLI))
-        .addReg(RISCV::X0, RegState::Define | RegState::Dead)
-        .addReg(RISCV::X0, RegState::Kill);
-  };
-
-  MachineInstrBuilder MIB = BuildVSETVLI();
-
-  // Default to tail agnostic unless the destination is tied to a source. In
-  // that case the user would have some control over the tail values. The tail
-  // policy is also ignored on instructions that only update element 0 like
-  // vmv.s.x or reductions so use agnostic there to match the common case.
-  // FIXME: This is conservatively correct, but we might want to detect that
-  // the input is undefined.
-  bool TailAgnostic = true;
-  unsigned UseOpIdx;
-  if (!ForceTailAgnostic && MI.isRegTiedToUseOperand(0, &UseOpIdx)) {
-    TailAgnostic = false;
-    // If the tied operand is an IMPLICIT_DEF we can keep TailAgnostic.
-    const MachineOperand &UseMO = MI.getOperand(UseOpIdx);
-    MachineInstr *UseMI = MRI.getVRegDef(UseMO.getReg());
-    if (UseMI) {
-      UseMI = elideCopies(UseMI, MRI);
-      if (UseMI && UseMI->isImplicitDef())
-        TailAgnostic = true;
-    }
-  }
-
-  // For simplicity we reuse the vtype representation here.
-  MIB.addImm(RISCVVType::encodeVTYPE(VLMul, ElementWidth,
-                                     /*TailAgnostic*/ TailAgnostic,
-                                     /*MaskAgnostic*/ false));
-
-  // Remove (now) redundant operands from pseudo
-  if (VLIndex >= 0 && MI.getOperand(VLIndex).isReg()) {
-    MI.getOperand(VLIndex).setReg(RISCV::NoRegister);
-    MI.getOperand(VLIndex).setIsKill(false);
-  }
-
-  return BB;
-}
-
 MachineBasicBlock *
 RISCVTargetLowering::EmitInstrWithCustomInserter(MachineInstr &MI,
                                                  MachineBasicBlock *BB) const {
-  uint64_t TSFlags = MI.getDesc().TSFlags;
-
-  if (TSFlags & RISCVII::HasSEWOpMask) {
-    unsigned NumOperands = MI.getNumExplicitOperands();
-    int VLIndex = (TSFlags & RISCVII::HasVLOpMask) ? NumOperands - 2 : -1;
-    unsigned SEWIndex = NumOperands - 1;
-    bool ForceTailAgnostic = TSFlags & RISCVII::ForceTailAgnosticMask;
-
-    RISCVVLMUL VLMul = static_cast<RISCVVLMUL>((TSFlags & RISCVII::VLMulMask) >>
-                                               RISCVII::VLMulShift);
-    return addVSetVL(MI, BB, VLIndex, SEWIndex, VLMul, ForceTailAgnostic);
-  }
-
   switch (MI.getOpcode()) {
   default:
     llvm_unreachable("Unexpected instr type to insert");
   case RISCV::ReadCycleWide:
     assert(!Subtarget.is64Bit() &&
            "ReadCycleWrite is only to be used on riscv32");
     return emitReadCycleWidePseudo(MI, BB);
   case RISCV::Select_GPR_Using_CC_GPR:

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

+//===- RISCVInsertVSETVLI.cpp - Insert VSETVLI instructions ---------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements a function pass that inserts VSETVLI instructions where
+// needed.
+//
+//===----------------------------------------------------------------------===//
+
+#include "RISCV.h"
+#include "RISCVSubtarget.h"
+#include "llvm/CodeGen/LiveIntervals.h"
+#include "llvm/CodeGen/MachineFunctionPass.h"
+using namespace llvm;
+
+#define DEBUG_TYPE "riscv-insert-vsetvli"
+#define RISCV_INSERT_VSETVLI_NAME "RISCV Insert VSETVLI pass"
+
+namespace {
+
+class RISCVInsertVSETVLI : public MachineFunctionPass {
+  const TargetInstrInfo *TII;
+  MachineRegisterInfo *MRI;
+
+public:
+  static char ID;
+
+  RISCVInsertVSETVLI() : MachineFunctionPass(ID) {
+    initializeRISCVInsertVSETVLIPass(*PassRegistry::getPassRegistry());
+  }
+  bool runOnMachineFunction(MachineFunction &MF) override;
+  bool runOnMachineBasicBlock(MachineBasicBlock &MBB);
+
+  void getAnalysisUsage(AnalysisUsage &AU) const override {
+    AU.setPreservesCFG();
+    MachineFunctionPass::getAnalysisUsage(AU);
+  }
+
+  StringRef getPassName() const override { return RISCV_INSERT_VSETVLI_NAME; }
+
+private:
+  bool maybeInsertVSETVLI(MachineBasicBlock &MBB,
+                          MachineBasicBlock::iterator MII,
+                          const MachineInstr *&PrevVSETVLI);
+};
+
+} // end anonymous namespace
+
+char RISCVInsertVSETVLI::ID = 0;
+
+INITIALIZE_PASS(RISCVInsertVSETVLI, DEBUG_TYPE, RISCV_INSERT_VSETVLI_NAME,
+                false, false)
+
+static MachineInstr *elideCopies(MachineInstr *MI,
+                                 const MachineRegisterInfo *MRI) {
+  while (true) {
+    if (!MI->isFullCopy())
+      return MI;
+    if (!Register::isVirtualRegister(MI->getOperand(1).getReg()))
+      return nullptr;
+    MI = MRI->getVRegDef(MI->getOperand(1).getReg());
+    if (!MI)
+      return nullptr;
+  }
+}
+
+static unsigned computeVType(const MachineInstr &MI, uint64_t TSFlags,
+                             const MachineRegisterInfo *MRI) {
+  unsigned NumOperands = MI.getNumExplicitOperands();
+
+  RISCVVLMUL VLMul = static_cast<RISCVVLMUL>((TSFlags & RISCVII::VLMulMask) >>
+                                             RISCVII::VLMulShift);
+
+  unsigned Log2SEW = MI.getOperand(NumOperands - 1).getImm();
+  assert(RISCVVType::isValidSEW(1 << Log2SEW) && "Unexpected SEW");
+  RISCVVSEW ElementWidth = static_cast<RISCVVSEW>(Log2SEW - 3);
+
+  // Default to tail agnostic unless the destination is tied to a source.
+  // Unless the source is undef. In that case the user would have some control
+  // over the tail values. The tail policy is also ignored on instructions
+  // that only update element 0 like vmv.s.x or reductions so use agnostic
+  // there to match the common case.
+  // FIXME: This is conservatively correct, but we might want to detect that
+  // the input is undefined.
+  bool ForceTailAgnostic = TSFlags & RISCVII::ForceTailAgnosticMask;
+  bool TailAgnostic = true;
+  unsigned UseOpIdx;
+  if (!ForceTailAgnostic && MI.isRegTiedToUseOperand(0, &UseOpIdx)) {
+    TailAgnostic = false;
+    // If the tied operand is an IMPLICIT_DEF we can keep TailAgnostic.
+    const MachineOperand &UseMO = MI.getOperand(UseOpIdx);
+    MachineInstr *UseMI = MRI->getVRegDef(UseMO.getReg());
+    if (UseMI) {
+      UseMI = elideCopies(UseMI, MRI);
+      if (UseMI && UseMI->isImplicitDef())
+        TailAgnostic = true;
+    }
+  }
+
+  return RISCVVType::encodeVTYPE(VLMul, ElementWidth,
+                                 /*TailAgnostic*/ TailAgnostic,
+                                 /*MaskAgnostic*/ false);
+}
+
+bool RISCVInsertVSETVLI::maybeInsertVSETVLI(MachineBasicBlock &MBB,
+                                            MachineBasicBlock::iterator MII,
+                                            const MachineInstr *&PrevVSETVLI) {
+  MachineInstr &MI = *MII;
+
+  uint64_t TSFlags = MI.getDesc().TSFlags;
+  if (!(TSFlags & RISCVII::HasSEWOpMask))
+    return false;
+
+  // Add VL/VTYPE implicit uses.
+  if (TSFlags & RISCVII::HasVLOpMask) {
+    MI.addOperand(
+        MachineOperand::CreateReg(RISCV::VL, /*isDef*/ false, /*isImp*/ true));
+  }
+  MI.addOperand(
+      MachineOperand::CreateReg(RISCV::VTYPE, /*isDef*/ false, /*isImp*/ true));
+
+  unsigned VTypeImm = computeVType(MI, TSFlags, MRI);
+
+  // Invalidate PrevVSETVLI if the vtype doesn't match.
+  if (PrevVSETVLI && PrevVSETVLI->getOperand(2).getImm() != VTypeImm)
+    PrevVSETVLI = nullptr;
+
+  DebugLoc DL = MI.getDebugLoc();
+
+  if (!(TSFlags & RISCVII::HasVLOpMask)) {
+    // This instruction isn't changing VL, if it has the same VTYPE as the last
+    // VSETVLI we're done.
+    if (PrevVSETVLI)
+      return false;
+
+    auto MIB = BuildMI(MBB, MII, DL, TII->get(RISCV::PseudoVSETVLI))
+                   .addReg(RISCV::X0, RegState::Define | RegState::Dead)
+                   .addReg(RISCV::X0, RegState::Kill)
+                   .addImm(VTypeImm)
+                   .addReg(RISCV::VL, RegState::Implicit);
+    PrevVSETVLI = &*MIB;
+    return true;
+  }
+
+  MachineOperand &VLOp = MI.getOperand(MI.getNumExplicitOperands() - 2);
+
+  // First handle the VSETIVLI case.
+  if (VLOp.isImm()) {
+    int64_t Imm = VLOp.getImm();
+
+    // If we have a previous VSETIVLI, we don't need to insert one. The VTYPE
+    // was checked earlier, so just compare the immediate.
+    if (PrevVSETVLI && PrevVSETVLI->getOpcode() == RISCV::PseudoVSETIVLI &&
+        PrevVSETVLI->getOperand(1).getImm() == Imm)
+      return false;
+
+    // TODO: Use X0 as the destination.
+    Register DestReg = MRI->createVirtualRegister(&RISCV::GPRRegClass);
+    auto MIB = BuildMI(MBB, MII, DL, TII->get(RISCV::PseudoVSETIVLI))
+                   .addReg(DestReg, RegState::Define | RegState::Dead)
+                   .addImm(Imm)
+                   .addImm(VTypeImm);
+    PrevVSETVLI = &*MIB;
+
+    return true;
+  }
+
+  // Otherwise we expect VL to be a GPR.
+  Register AVLReg = VLOp.getReg();
+
+  auto needVSETVLI = [](const MachineInstr *PrevVSETVLI, Register AVLReg) {
+    if (!PrevVSETVLI)
+      return true;
+
+    Register PrevOutVL = PrevVSETVLI->getOperand(0).getReg();
+    // If the previous VSET{I}VLI's output (which isn't X0) is fed into this
+    // VSETVLI, this one isn't changing VL so is redundant.
+    // Only perform this on virtual registers to avoid the complexity of having
+    // to work out if the physical register was clobbered somewhere in between.
+    if (AVLReg.isVirtual() && AVLReg == PrevOutVL)
+      return false;
+
+    if (PrevVSETVLI->getOpcode() == RISCV::PseudoVSETVLI &&
+        AVLReg == PrevVSETVLI->getOperand(1).getReg() &&
+        (AVLReg != RISCV::X0 || PrevOutVL != RISCV::X0))
+      return false;
+
+    return true;
+  };
+
+  if (needVSETVLI(PrevVSETVLI, AVLReg)) {
+    // TODO: Use X0 as the destination if AVLReg is not X0.
+    Register DestReg = MRI->createVirtualRegister(&RISCV::GPRRegClass);
+    auto MIB = BuildMI(MBB, MII, DL, TII->get(RISCV::PseudoVSETVLI))
+                   .addReg(DestReg, RegState::Define | RegState::Dead)
+                   .addReg(AVLReg)
+                   .addImm(VTypeImm);
+    PrevVSETVLI = &*MIB;
+  }
+
+  // Clear the GPR, we don't need it anymore
+  VLOp.setReg(RISCV::NoRegister);
+  VLOp.setIsKill(false);
+
+  return true;
+}
+
+bool RISCVInsertVSETVLI::runOnMachineBasicBlock(MachineBasicBlock &MBB) {
+  bool Changed = false;
+  const MachineInstr *PrevVSETVLI = nullptr;
+
+  for (auto MII = MBB.begin(), MIE = MBB.end(); MII != MIE; ++MII) {
+    MachineInstr &MI = *MII;
+
+    // If this is an explicit VSETVLI/VSETIVILI instruction, cache it as our
+    // previous VL/VTYPE update.
+    if (MI.getOpcode() == RISCV::PseudoVSETVLI ||
+        MI.getOpcode() == RISCV::PseudoVSETIVLI) {
+      PrevVSETVLI = &MI;
+      continue;
+    }
+
+    Changed |= maybeInsertVSETVLI(MBB, MII, PrevVSETVLI);
+
+    if (PrevVSETVLI && (MI.isCall() || MI.modifiesRegister(RISCV::VL) ||
+                        MI.modifiesRegister(RISCV::VTYPE))) {
+      // Old VL/VTYPE is overwritten.
+      PrevVSETVLI = nullptr;
+    }
+  }
+
+  return Changed;
+}
+
+bool RISCVInsertVSETVLI::runOnMachineFunction(MachineFunction &MF) {
+  // Skip if the vector extension is not enabled.
+  const RISCVSubtarget &ST = MF.getSubtarget<RISCVSubtarget>();
+  if (!ST.hasStdExtV())
+    return false;
+
+  const RISCVSubtarget &Subtarget = MF.getSubtarget<RISCVSubtarget>();
+  TII = Subtarget.getInstrInfo();
+  MRI = &MF.getRegInfo();
+
+  bool Changed = false;
+  for (MachineBasicBlock &MBB : MF)
+    Changed |= runOnMachineBasicBlock(MBB);
+
+  return Changed;
+}
+
+/// Returns an instance of the Insert VSETVLI pass.
+FunctionPass *llvm::createRISCVInsertVSETVLIPass() {
+  return new RISCVInsertVSETVLI();
+}

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

 class VPseudoUSLoadNoMask<VReg RetClass, bits<7> EEW, bit isFF> :
       Pseudo<(outs RetClass:$rd),
              (ins GPR:$rs1, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLE</*Masked*/0, /*Strided*/0, /*FF*/isFF, EEW, VLMul> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoUSLoadMask<VReg RetClass, bits<7> EEW, bit isFF> :
       Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
               (ins GetVRegNoV0<RetClass>.R:$merge,
                    GPR:$rs1,
                    VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLE</*Masked*/1, /*Strided*/0, /*FF*/isFF, EEW, VLMul> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = "$rd = $merge";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoSLoadNoMask<VReg RetClass, bits<7> EEW>:
       Pseudo<(outs RetClass:$rd),
              (ins GPR:$rs1, GPR:$rs2, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLE</*Masked*/0, /*Strided*/1, /*FF*/0, EEW, VLMul> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoSLoadMask<VReg RetClass, bits<7> EEW>:
       Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
               (ins GetVRegNoV0<RetClass>.R:$merge,
                    GPR:$rs1, GPR:$rs2,
                    VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLE</*Masked*/1, /*Strided*/1, /*FF*/0, EEW, VLMul> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = "$rd = $merge";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoILoadNoMask<VReg RetClass, VReg IdxClass, bits<7> EEW, bits<3> LMUL,
                          bit Ordered, bit EarlyClobber>:
       Pseudo<(outs RetClass:$rd),
              (ins GPR:$rs1, IdxClass:$rs2, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLX</*Masked*/0, Ordered, EEW, VLMul, LMUL> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let Constraints = !if(!eq(EarlyClobber, 1), "@earlyclobber $rd", "");
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoILoadMask<VReg RetClass, VReg IdxClass, bits<7> EEW, bits<3> LMUL,
                        bit Ordered, bit EarlyClobber>:
       Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
               (ins GetVRegNoV0<RetClass>.R:$merge,
                    GPR:$rs1, IdxClass:$rs2,
                    VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLX</*Masked*/1, Ordered, EEW, VLMul, LMUL> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = !if(!eq(EarlyClobber, 1), "@earlyclobber $rd, $rd = $merge", "$rd = $merge");
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoUSStoreNoMask<VReg StClass, bits<7> EEW>:
       Pseudo<(outs),
               (ins StClass:$rd, GPR:$rs1, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSE</*Masked*/0, /*Strided*/0, EEW, VLMul> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoUSStoreMask<VReg StClass, bits<7> EEW>:
       Pseudo<(outs),
               (ins StClass:$rd, GPR:$rs1, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSE</*Masked*/1, /*Strided*/0, EEW, VLMul> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoSStoreNoMask<VReg StClass, bits<7> EEW>:
       Pseudo<(outs),
               (ins StClass:$rd, GPR:$rs1, GPR:$rs2, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSE</*Masked*/0, /*Strided*/1, EEW, VLMul> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoSStoreMask<VReg StClass, bits<7> EEW>:
       Pseudo<(outs),
               (ins StClass:$rd, GPR:$rs1, GPR:$rs2, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSE</*Masked*/1, /*Strided*/1, EEW, VLMul> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 // Unary instruction that is never masked so HasDummyMask=0.
 class VPseudoUnaryNoDummyMask<VReg RetClass,
                               DAGOperand Op2Class> :
         Pseudo<(outs RetClass:$rd),
                (ins Op2Class:$rs1, AVL:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoNullaryNoMask<VReg RegClass>:
       Pseudo<(outs RegClass:$rd),
              (ins AVL:$vl, ixlenimm:$sew),
              []>, RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoNullaryMask<VReg RegClass>:
       Pseudo<(outs GetVRegNoV0<RegClass>.R:$rd),
              (ins GetVRegNoV0<RegClass>.R:$merge, VMaskOp:$vm, AVL:$vl,
               ixlenimm:$sew), []>, RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints ="$rd = $merge";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 // Nullary for pseudo instructions. They are expanded in
 // RISCVExpandPseudoInsts pass.
 class VPseudoNullaryPseudoM<string BaseInst>
        : Pseudo<(outs VR:$rd), (ins AVL:$vl, ixlenimm:$sew), []>,
        RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   // BaseInstr is not used in RISCVExpandPseudoInsts pass.
   // Just fill a corresponding real v-inst to pass tablegen check.
   let BaseInstr = !cast<Instruction>(BaseInst);
 }
 
 // RetClass could be GPR or VReg.
 class VPseudoUnaryNoMask<DAGOperand RetClass, VReg OpClass, string Constraint = ""> :
         Pseudo<(outs RetClass:$rd),
                (ins OpClass:$rs2, AVL:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = Constraint;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoUnaryMask<VReg RetClass, VReg OpClass, string Constraint = ""> :
         Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
                (ins GetVRegNoV0<RetClass>.R:$merge, OpClass:$rs2,
                     VMaskOp:$vm, AVL:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 // mask unary operation without maskedoff
 class VPseudoMaskUnarySOutMask:
         Pseudo<(outs GPR:$rd),
                (ins VR:$rs1, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 // Mask can be V0~V31
 class VPseudoUnaryAnyMask<VReg RetClass,
                           VReg Op1Class> :
       Pseudo<(outs RetClass:$rd),
              (ins RetClass:$merge,
                   Op1Class:$rs2,
                   VR:$vm, AVL:$vl, ixlenimm:$sew),
              []>,
       RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = "@earlyclobber $rd, $rd = $merge";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoBinaryNoMask<VReg RetClass,
                           VReg Op1Class,
                           DAGOperand Op2Class,
                           string Constraint> :
         Pseudo<(outs RetClass:$rd),
                (ins Op1Class:$rs2, Op2Class:$rs1, AVL:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = Constraint;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoIStoreNoMask<VReg StClass, VReg IdxClass, bits<7> EEW, bits<3> LMUL,
                           bit Ordered>:
       Pseudo<(outs),
               (ins StClass:$rd, GPR:$rs1, IdxClass:$rs2, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSX</*Masked*/0, Ordered, EEW, VLMul, LMUL> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoIStoreMask<VReg StClass, VReg IdxClass, bits<7> EEW, bits<3> LMUL,
                         bit Ordered>:
       Pseudo<(outs),
               (ins StClass:$rd, GPR:$rs1, IdxClass:$rs2, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSX</*Masked*/1, Ordered, EEW, VLMul, LMUL> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoBinaryMask<VReg RetClass,
                         RegisterClass Op1Class,
                         DAGOperand Op2Class,
                         string Constraint> :
         Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
                 (ins GetVRegNoV0<RetClass>.R:$merge,
                      Op1Class:$rs2, Op2Class:$rs1,
                      VMaskOp:$vm, AVL:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 // Like VPseudoBinaryMask, but output can be V0.
 class VPseudoBinaryMOutMask<VReg RetClass,
                             RegisterClass Op1Class,
                             DAGOperand Op2Class,
                             string Constraint> :
         Pseudo<(outs RetClass:$rd),
                 (ins RetClass:$merge,
                      Op1Class:$rs2, Op2Class:$rs1,
                      VMaskOp:$vm, AVL:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = Join<[Constraint, "$rd = $merge"], ",">.ret;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoBinaryCarryIn<VReg RetClass,
                            VReg Op1Class,
                            DAGOperand Op2Class,
                            LMULInfo MInfo,
                            bit CarryIn,
                            string Constraint> :
         Pseudo<(outs RetClass:$rd),
                !if(CarryIn,
                   (ins Op1Class:$rs2, Op2Class:$rs1, VMV0:$carry, AVL:$vl,
                        ixlenimm:$sew),
                   (ins Op1Class:$rs2, Op2Class:$rs1, AVL:$vl, ixlenimm:$sew)), []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = Constraint;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 0;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
   let VLMul = MInfo.value;
 }
 
 class VPseudoTernaryNoMask<VReg RetClass,
                            RegisterClass Op1Class,
                            DAGOperand Op2Class,
                            string Constraint> :
         Pseudo<(outs RetClass:$rd),
                (ins RetClass:$rs3, Op1Class:$rs1, Op2Class:$rs2,
                     AVL:$vl, ixlenimm:$sew),
                []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = Join<[Constraint, "$rd = $rs3"], ",">.ret;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoAMOWDNoMask<VReg RetClass,
                          VReg Op1Class> :
         Pseudo<(outs GetVRegNoV0<RetClass>.R:$vd_wd),
                (ins GPR:$rs1,
                     Op1Class:$vs2,
                     GetVRegNoV0<RetClass>.R:$vd,
                     AVL:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 1;
   let mayStore = 1;
   let hasSideEffects = 1;
-  let usesCustomInserter = 1;
   let Constraints = "$vd_wd = $vd";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoAMOWDMask<VReg RetClass,
                        VReg Op1Class> :
         Pseudo<(outs GetVRegNoV0<RetClass>.R:$vd_wd),
                (ins GPR:$rs1,
                     Op1Class:$vs2,
                     GetVRegNoV0<RetClass>.R:$vd,
                     VMaskOp:$vm, AVL:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 1;
   let mayStore = 1;
   let hasSideEffects = 1;
-  let usesCustomInserter = 1;
   let Constraints = "$vd_wd = $vd";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 multiclass VPseudoAMOEI<int eew> {
   // Standard scalar AMO supports 32, 64, and 128 Mem data bits,
   // and in the base vector "V" extension, only SEW up to ELEN = max(XLEN, FLEN)

 class VPseudoUSSegLoadNoMask<VReg RetClass, bits<7> EEW, bits<4> NF, bit isFF>:
       Pseudo<(outs RetClass:$rd),
              (ins GPR:$rs1, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLSEG<NF, /*Masked*/0, /*Strided*/0, /*FF*/isFF, EEW, VLMul> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoUSSegLoadMask<VReg RetClass, bits<7> EEW, bits<4> NF, bit isFF>:
       Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
              (ins GetVRegNoV0<RetClass>.R:$merge, GPR:$rs1,
                   VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLSEG<NF, /*Masked*/1, /*Strided*/0, /*FF*/isFF, EEW, VLMul> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = "$rd = $merge";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoSSegLoadNoMask<VReg RetClass, bits<7> EEW, bits<4> NF>:
       Pseudo<(outs RetClass:$rd),
              (ins GPR:$rs1, GPR:$offset, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLSEG<NF, /*Masked*/0, /*Strided*/1, /*FF*/0, EEW, VLMul> {
   let mayLoad = 1;
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoSSegLoadMask<VReg RetClass, bits<7> EEW, bits<4> NF>:
       Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
              (ins GetVRegNoV0<RetClass>.R:$merge, GPR:$rs1,
                   GPR:$offset, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLSEG<NF, /*Masked*/1, /*Strided*/1, /*FF*/0, EEW, VLMul> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   let Constraints = "$rd = $merge";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoISegLoadNoMask<VReg RetClass, VReg IdxClass, bits<7> EEW, bits<3> LMUL,
                             bits<4> NF, bit Ordered>:
       Pseudo<(outs RetClass:$rd),
              (ins GPR:$rs1, IdxClass:$offset, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLXSEG<NF, /*Masked*/0, Ordered, EEW, VLMul, LMUL> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   // For vector indexed segment loads, the destination vector register groups
   // cannot overlap the source vector register group
   let Constraints = "@earlyclobber $rd";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoISegLoadMask<VReg RetClass, VReg IdxClass, bits<7> EEW, bits<3> LMUL,
                           bits<4> NF, bit Ordered>:
       Pseudo<(outs GetVRegNoV0<RetClass>.R:$rd),
              (ins GetVRegNoV0<RetClass>.R:$merge, GPR:$rs1,
                   IdxClass:$offset, VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVLXSEG<NF, /*Masked*/1, Ordered, EEW, VLMul, LMUL> {
   let mayLoad = 1;
   let mayStore = 0;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
   // For vector indexed segment loads, the destination vector register groups
   // cannot overlap the source vector register group
   let Constraints = "@earlyclobber $rd, $rd = $merge";
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasMergeOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoUSSegStoreNoMask<VReg ValClass, bits<7> EEW, bits<4> NF>:
       Pseudo<(outs),
              (ins ValClass:$rd, GPR:$rs1, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSSEG<NF, /*Masked*/0, /*Strided*/0, EEW, VLMul> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoUSSegStoreMask<VReg ValClass, bits<7> EEW, bits<4> NF>:
       Pseudo<(outs),
              (ins ValClass:$rd, GPR:$rs1,
                   VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSSEG<NF, /*Masked*/1, /*Strided*/0, EEW, VLMul> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoSSegStoreNoMask<VReg ValClass, bits<7> EEW, bits<4> NF>:
       Pseudo<(outs),
              (ins ValClass:$rd, GPR:$rs1, GPR: $offset, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSSEG<NF, /*Masked*/0, /*Strided*/1, EEW, VLMul> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoSSegStoreMask<VReg ValClass, bits<7> EEW, bits<4> NF>:
       Pseudo<(outs),
              (ins ValClass:$rd, GPR:$rs1, GPR: $offset,
                   VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSSEG<NF, /*Masked*/1, /*Strided*/1, EEW, VLMul> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoISegStoreNoMask<VReg ValClass, VReg IdxClass, bits<7> EEW, bits<3> LMUL,
                              bits<4> NF, bit Ordered>:
       Pseudo<(outs),
              (ins ValClass:$rd, GPR:$rs1, IdxClass: $index,
                   AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSXSEG<NF, /*Masked*/0, Ordered, EEW, VLMul, LMUL> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let HasDummyMask = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 class VPseudoISegStoreMask<VReg ValClass, VReg IdxClass, bits<7> EEW, bits<3> LMUL,
                            bits<4> NF, bit Ordered>:
       Pseudo<(outs),
              (ins ValClass:$rd, GPR:$rs1, IdxClass: $index,
                   VMaskOp:$vm, AVL:$vl, ixlenimm:$sew),[]>,
       RISCVVPseudo,
       RISCVVSXSEG<NF, /*Masked*/1, Ordered, EEW, VLMul, LMUL> {
   let mayLoad = 0;
   let mayStore = 1;
   let hasSideEffects = 0;
-  let usesCustomInserter = 1;
-  let Uses = [VL, VTYPE];
   let HasVLOp = 1;
   let HasSEWOp = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
 multiclass VPseudoUSLoad<bit isFF> {
   foreach eew = EEWList in {
     foreach lmul = MxSet<eew>.m in {
let Defs = [VXSAT], hasSideEffects = 1 in {
   defm PseudoVSADD       : VPseudoBinaryV_VV_VX_VI;
   defm PseudoVSSUBU      : VPseudoBinaryV_VV_VX;
   defm PseudoVSSUB       : VPseudoBinaryV_VV_VX;
 }
 
 //===----------------------------------------------------------------------===//
 // 13.2. Vector Single-Width Averaging Add and Subtract
 //===----------------------------------------------------------------------===//
-let Uses = [VL, VTYPE, VXRM], hasSideEffects = 1 in {
+let Uses = [VXRM], hasSideEffects = 1 in {
   defm PseudoVAADDU      : VPseudoBinaryV_VV_VX;
   defm PseudoVAADD       : VPseudoBinaryV_VV_VX;
   defm PseudoVASUBU      : VPseudoBinaryV_VV_VX;
   defm PseudoVASUB       : VPseudoBinaryV_VV_VX;
 }
 
 //===----------------------------------------------------------------------===//
 // 13.3. Vector Single-Width Fractional Multiply with Rounding and Saturation
 //===----------------------------------------------------------------------===//
-let Uses = [VL, VTYPE, VXRM], Defs = [VXSAT], hasSideEffects = 1 in {
+let Uses = [VXRM], Defs = [VXSAT], hasSideEffects = 1 in {
   defm PseudoVSMUL      : VPseudoBinaryV_VV_VX;
 }
 
 //===----------------------------------------------------------------------===//
 // 13.4. Vector Single-Width Scaling Shift Instructions
 //===----------------------------------------------------------------------===//
-let Uses = [VL, VTYPE, VXRM], hasSideEffects = 1 in {
+let Uses = [VXRM], hasSideEffects = 1 in {
   defm PseudoVSSRL        : VPseudoBinaryV_VV_VX_VI<uimm5>;
   defm PseudoVSSRA        : VPseudoBinaryV_VV_VX_VI<uimm5>;
 }
 
 //===----------------------------------------------------------------------===//
 // 13.5. Vector Narrowing Fixed-Point Clip Instructions
 //===----------------------------------------------------------------------===//
-let Uses = [VL, VTYPE, VXRM], Defs = [VXSAT], hasSideEffects = 1 in {
+let Uses = [VXRM], Defs = [VXSAT], hasSideEffects = 1 in {
   defm PseudoVNCLIP     : VPseudoBinaryV_WV_WX_WI;
   defm PseudoVNCLIPU    : VPseudoBinaryV_WV_WX_WI;
 }
 
 } // Predicates = [HasStdExtV]
 
 let Predicates = [HasStdExtV, HasStdExtF] in {
 //===----------------------------------------------------------------------===//

 // 17. Vector Permutation Instructions
 //===----------------------------------------------------------------------===//
 
 //===----------------------------------------------------------------------===//
 // 17.1. Integer Scalar Move Instructions
 //===----------------------------------------------------------------------===//
 
 let Predicates = [HasStdExtV] in {
-let mayLoad = 0, mayStore = 0, hasSideEffects = 0, usesCustomInserter = 1,
-    Uses = [VL, VTYPE] in {
+let mayLoad = 0, mayStore = 0, hasSideEffects = 0 in {
   foreach m = MxList.m in {
     let VLMul = m.value in {
       let HasSEWOp = 1, BaseInstr = VMV_X_S in
       def PseudoVMV_X_S # "_" # m.MX: Pseudo<(outs GPR:$rd),
                                              (ins m.vrclass:$rs2, ixlenimm:$sew),
                                              []>, RISCVVPseudo;
       let HasVLOp = 1, HasSEWOp = 1, BaseInstr = VMV_S_X,
           ForceTailAgnostic = true, Constraints = "$rd = $rs1" in
       def PseudoVMV_S_X # "_" # m.MX: Pseudo<(outs m.vrclass:$rd),
                                              (ins m.vrclass:$rs1, GPR:$rs2,
                                                   AVL:$vl, ixlenimm:$sew),
                                              []>, RISCVVPseudo;
     }
   }
 }
 } // Predicates = [HasStdExtV]
 
 //===----------------------------------------------------------------------===//
 // 17.2. Floating-Point Scalar Move Instructions
 //===----------------------------------------------------------------------===//
 
 let Predicates = [HasStdExtV, HasStdExtF] in {
-let mayLoad = 0, mayStore = 0, hasSideEffects = 0, usesCustomInserter = 1,
-    Uses = [VL, VTYPE] in {
+let mayLoad = 0, mayStore = 0, hasSideEffects = 0 in {
   foreach m = MxList.m in {
     foreach f = FPList.fpinfo in {
       let VLMul = m.value in {
         let HasSEWOp = 1, BaseInstr = VFMV_F_S in
         def "PseudoVFMV_" # f.FX # "_S_" # m.MX :
                                           Pseudo<(outs f.fprclass:$rd),
                                                  (ins m.vrclass:$rs2,
                                                       ixlenimm:$sew),

llvm/lib/Target/RISCV/RISCVTargetMachine.cpp

 
 extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeRISCVTarget() {
   RegisterTargetMachine<RISCVTargetMachine> X(getTheRISCV32Target());
   RegisterTargetMachine<RISCVTargetMachine> Y(getTheRISCV64Target());
   auto *PR = PassRegistry::getPassRegistry();
   initializeGlobalISel(*PR);
   initializeRISCVMergeBaseOffsetOptPass(*PR);
   initializeRISCVExpandPseudoPass(*PR);
-  initializeRISCVCleanupVSETVLIPass(*PR);
+  initializeRISCVInsertVSETVLIPass(*PR);
 }
 
 static StringRef computeDataLayout(const Triple &TT) {
   if (TT.isArch64Bit())
     return "e-m:e-p:64:64-i64:64-i128:128-n64-S128";
   assert(TT.isArch32Bit() && "only RV32 and RV64 are currently supported");
   return "e-m:e-p:32:32-i64:64-n32-S128";
 }
void RISCVPassConfig::addPreEmitPass2() {
   addPass(createRISCVExpandPseudoPass());
   // Schedule the expansion of AMOs at the last possible moment, avoiding the
   // possibility for other passes to break the requirements for forward
   // progress in the LR/SC block.
   addPass(createRISCVExpandAtomicPseudoPass());
 }
 
 void RISCVPassConfig::addPreRegAlloc() {
-  if (TM->getOptLevel() != CodeGenOpt::None) {
+  if (TM->getOptLevel() != CodeGenOpt::None)
     addPass(createRISCVMergeBaseOffsetOptPass());
-    addPass(createRISCVCleanupVSETVLIPass());
-  }
+  addPass(createRISCVInsertVSETVLIPass());
 }

llvm/test/CodeGen/RISCV/rvv/add-vsetvli-gpr.mir

-# RUN: llc -mtriple riscv64 -mattr=+experimental-v %s  \
-# RUN:     -start-before=finalize-isel -stop-after=finalize-isel -o - \
-# RUN:     | FileCheck --check-prefix=POST-INSERTER %s
-
-# RUN: llc -mtriple riscv64 -mattr=+experimental-v %s  \
-# RUN:     -start-before=finalize-isel -o - \
-# RUN:     | FileCheck --check-prefix=CODEGEN %s
-
---- |
-  define void @vadd_vint64m1(
-            <vscale x 1 x i64> *%pc,
-            <vscale x 1 x i64> *%pa,
-            <vscale x 1 x i64> *%pb,
-            i64 %vl)
-  {
-    ret void
-  }
-...
----
-name: vadd_vint64m1
-tracksRegLiveness: true
-body:             |
-  bb.0 (%ir-block.0):
-    liveins: $x10, $x11, $x12, $x13
-
-    %3:gpr = COPY $x13
-    %2:gpr = COPY $x12
-    %1:gpr = COPY $x11
-    %0:gpr = COPY $x10
-    %4:vr = PseudoVLE64_V_M1 %1, %3, 6, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pa, align 8)
-    %5:vr = PseudoVLE64_V_M1 %2, %3, 6, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pb, align 8)
-    %6:vr = PseudoVADD_VV_M1 killed %4, killed %5, %3, 6, implicit $vl, implicit $vtype
-    PseudoVSE64_V_M1 killed %6, %0, %3, 6, implicit $vl, implicit $vtype :: (store unknown-size into %ir.pc, align 8)
-    PseudoRET
-
-...
-
-# POST-INSERTER: %0:gpr = COPY $x13
-# POST-INSERTER: %1:gpr = COPY $x12
-# POST-INSERTER: %2:gpr = COPY $x11
-# POST-INSERTER: %3:gpr = COPY $x10
-# POST-INSERTER: dead %7:gpr = PseudoVSETVLI %0, 88, implicit-def $vl, implicit-def $vtype
-# POST-INSERTER: %4:vr = PseudoVLE64_V_M1 %2, $noreg, 6, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pa, align 8)
-# POST-INSERTER: dead %8:gpr = PseudoVSETVLI %0, 88, implicit-def $vl, implicit-def $vtype
-# POST-INSERTER: %5:vr = PseudoVLE64_V_M1 %1, $noreg, 6, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pb, align 8)
-# POST-INSERTER: dead %9:gpr = PseudoVSETVLI %0, 88, implicit-def $vl, implicit-def $vtype
-# POST-INSERTER: %6:vr = PseudoVADD_VV_M1 killed %4, killed %5, $noreg, 6, implicit $vl, implicit $vtype
-# POST-INSERTER: dead %10:gpr = PseudoVSETVLI %0, 88, implicit-def $vl, implicit-def $vtype
-# POST-INSERTER: PseudoVSE64_V_M1 killed %6, %3, $noreg, 6, implicit $vl, implicit $vtype :: (store unknown-size into %ir.pc, align 8)
-
-# CODEGEN: vsetvli  a3, a3, e64,m1,ta,mu
-# CODEGEN-NEXT: vle64.v v25, (a1)
-# CODEGEN-NEXT: vle64.v v26, (a2)
-# CODEGEN-NEXT: vadd.vv v25, v25, v26
-# CODEGEN-NEXT: vse64.v v25, (a0)
-# CODEGEN-NEXT: ret

llvm/test/CodeGen/RISCV/rvv/add-vsetvli-vlmax.ll

-; This test shows the evolution of RVV pseudo instructions within isel.
-
-; RUN: llc -mtriple riscv64 -mattr=+experimental-v %s -o %t.pre.mir \
-; RUN:     -stop-before=finalize-isel
-; RUN: cat %t.pre.mir | FileCheck --check-prefix=PRE-INSERTER %s
-
-; RUN: llc -mtriple riscv64 -mattr=+experimental-v %t.pre.mir -o %t.post.mir \
-; RUN:     -start-before=finalize-isel -stop-after=finalize-isel
-; RUN: cat %t.post.mir | FileCheck --check-prefix=POST-INSERTER %s
-
-define void @vadd_vint64m1(
-          <vscale x 1 x i64> *%pc,
-          <vscale x 1 x i64> *%pa,
-          <vscale x 1 x i64> *%pb)
-{
-  %va = load <vscale x 1 x i64>, <vscale x 1 x i64>* %pa
-  %vb = load <vscale x 1 x i64>, <vscale x 1 x i64>* %pb
-  %vc = add <vscale x 1 x i64> %va, %vb
-  store <vscale x 1 x i64> %vc, <vscale x 1 x i64> *%pc
-  ret void
-}
-
-; PRE-INSERTER: %3:vr = VL1RE64_V %1 :: (load unknown-size from %ir.pa, align 8)
-; PRE-INSERTER: %4:vr = VL1RE64_V %2 :: (load unknown-size from %ir.pb, align 8)
-; PRE-INSERTER: %5:vr = PseudoVADD_VV_M1 killed %3, killed %4, $x0, 6, implicit $vl, implicit $vtype
-; PRE-INSERTER:  VS1R_V killed %5, %0 :: (store unknown-size into %ir.pc, align 8)
-
-; POST-INSERTER: %3:vr = VL1RE64_V %1 :: (load unknown-size from %ir.pa, align 8)
-; POST-INSERTER: %4:vr = VL1RE64_V %2 :: (load unknown-size from %ir.pb, align 8)
-; POST-INSERTER: dead %6:gpr = PseudoVSETVLI $x0, 88, implicit-def $vl, implicit-def $vtype
-; POST-INSERTER: %5:vr = PseudoVADD_VV_M1 killed %3, killed %4, $noreg, 6, implicit $vl, implicit $vtype
-; POST-INSERTER: VS1R_V killed %5, %0 :: (store unknown-size into %ir.pc, align 8)

llvm/test/CodeGen/RISCV/rvv/addi-scalable-offset.mir

 # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-# RUN: llc -march=riscv64 -stop-after=prologepilog %s -o - 2>&1 | FileCheck %s
+# RUN: llc -march=riscv64 -mattr=+experimental-v -stop-after=prologepilog %s -o - 2>&1 | FileCheck %s
 
 --- |
   define void @add_scalable_offset(
             <vscale x 1 x i64> *%pa,
             i64 %vl)
   {
     ret void
   }
bb.0:
     ; CHECK: $x2 = ADD $x2, killed $x10
     ; CHECK: $x2 = frame-destroy ADDI $x2, 240
     ; CHECK: $x8 = LD $x2, 2016 :: (load 8 from %stack.4)
     ; CHECK: $x1 = LD $x2, 2024 :: (load 8 from %stack.3)
     ; CHECK: $x2 = frame-destroy ADDI $x2, 2032
     ; CHECK: PseudoRET
     %1:gpr = COPY $x11
     %0:gpr = COPY $x10
-    %2:vr = PseudoVLE64_V_M1 %0, %1, 6, implicit $vl, implicit $vtype :: (load unknown-size from %ir.pa, align 8)
+    %2:vr = PseudoVLE64_V_M1 %0, %1, 6 :: (load unknown-size from %ir.pa, align 8)
     %3:gpr = ADDI %stack.2, 0
     VS1R_V killed %2:vr, %3:gpr
     PseudoRET
 
 ...

llvm/test/CodeGen/RISCV/rvv/cleanup-vsetivli.mir

-# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-# RUN: llc %s -mtriple=riscv64 -run-pass=riscv-cleanup-vsetvli -o - | FileCheck %s
-
-# Make sure we don't combine these VSET{I}VLIs in the cleanup pass. We could not
-# differentiate AVL values if the opcode of the previous one is different from
-# current one.
-
---- |
-  ; ModuleID = '../llvm/test/CodeGen/RISCV/rvv/add-vsetivli.ll'
-  source_filename = "../llvm/test/CodeGen/RISCV/rvv/add-vsetivli.ll"
-  target datalayout = "e-m:e-p:64:64-i64:64-i128:128-n64-S128"
-  target triple = "riscv64"
-
-  define void @cleanup_vsetivli() #0 {
-    ret void
-  }
-
-  attributes #0 = { "target-features"="+experimental-v" }
-
-...
----
-name:            cleanup_vsetivli
-alignment:       4
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr }
-frameInfo:
-  maxAlignment:    1
-machineFunctionInfo: {}
-body:             |
-  bb.0 (%ir-block.0):
-    ; CHECK-LABEL: name: cleanup_vsetivli
-    ; CHECK: dead %0:gpr = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: dead %1:gpr = PseudoVSETIVLI 5, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: dead %3:gpr = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: dead %5:gpr = PseudoVSETIVLI 5, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: PseudoRET
-    dead %0:gpr  = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    dead %1:gpr  = PseudoVSETIVLI 5, 12, implicit-def $vl, implicit-def $vtype
-    dead %2:gpr  = PseudoVSETIVLI 5, 12, implicit-def $vl, implicit-def $vtype
-    dead %3:gpr  = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    dead %4:gpr  = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    dead %5:gpr  = PseudoVSETIVLI 5, 12, implicit-def $vl, implicit-def $vtype
-    PseudoRET
-
-...

llvm/test/CodeGen/RISCV/rvv/cleanup-vsetvli.mir

-# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-# RUN: llc %s -mtriple=riscv64 -run-pass=riscv-cleanup-vsetvli -o - | FileCheck %s
-
---- |
-  ; ModuleID = '../llvm/test/CodeGen/RISCV/rvv/add-vsetvli-vlmax.ll'
-  source_filename = "../llvm/test/CodeGen/RISCV/rvv/add-vsetvli-vlmax.ll"
-  target datalayout = "e-m:e-p:64:64-i64:64-i128:128-n64-S128"
-  target triple = "riscv64"
-
-  define void @cleanup_vsetvli0() #0 {
-    ret void
-  }
-
-  define void @cleanup_vsetvli1() #0 {
-    ret void
-  }
-
-  attributes #0 = { "target-features"="+experimental-v" }
-
-...
----
-# Make sure we don't combine these two VSETVLIs in the cleanup pass. The first
-# keeps the previous value of VL, the second sets it to VLMAX. We can't remove
-# the first since we can't tell if this is a change of VL.
-name:            cleanup_vsetvli0
-alignment:       4
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr }
-frameInfo:
-  maxAlignment:    1
-machineFunctionInfo: {}
-body:             |
-  bb.0 (%ir-block.0):
-    ; CHECK-LABEL: name: cleanup_vsetvli0
-    ; CHECK: dead $x0 = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: dead %0:gpr = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: PseudoRET
-    dead $x0  = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    dead %0:gpr  = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    PseudoRET
-
-...
----
-# 1. Ensure we can remove the second VSETVLI which takes its AVL from the first VSETVLI.
-# 2. Ensure we can remove the fourth VSETVLI which takes its AVL from the VSETIVLI.
-# 3. Make sure we don't combine the latter two VSETVLIs; the first outputs to a
-# physical register which is clobbered by a later instruction.
-name:            cleanup_vsetvli1
-alignment:       4
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr }
-frameInfo:
-  maxAlignment:    1
-machineFunctionInfo: {}
-body:             |
-  bb.0 (%ir-block.0):
-    liveins: $x3
-    ; CHECK-LABEL: name: cleanup_vsetvli1
-    ; CHECK: liveins: $x3
-    ; CHECK: [[PseudoVSETVLI:%[0-9]+]]:gpr = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: [[PseudoVSETIVLI:%[0-9]+]]:gpr = PseudoVSETIVLI 4, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: $x1 = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: $x1 = COPY $x3
-    ; CHECK: dead %4:gpr = PseudoVSETVLI $x1, 12, implicit-def $vl, implicit-def $vtype
-    ; CHECK: PseudoRET
-    %0:gpr  = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    dead %1:gpr  = PseudoVSETVLI %0, 12, implicit-def $vl, implicit-def $vtype
-
-    %2:gpr  = PseudoVSETIVLI 4, 12, implicit-def $vl, implicit-def $vtype
-    dead %3:gpr  = PseudoVSETVLI %2, 12, implicit-def $vl, implicit-def $vtype
-
-    $x1  = PseudoVSETVLI $x0, 12, implicit-def $vl, implicit-def $vtype
-    $x1 = COPY $x3
-    dead %4:gpr  = PseudoVSETVLI $x1, 12, implicit-def $vl, implicit-def $vtype
-    PseudoRET
-
-...

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-ctlz.ll

 ; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v25, a6
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v26
 ; LMULMAX2-RV32-NEXT:    lui a1, 349525
 ; LMULMAX2-RV32-NEXT:    addi a4, a1, 1365
 ; LMULMAX2-RV32-NEXT:    lui a1, 209715
 ; LMULMAX2-RV32-NEXT:    addi a3, a1, 819
 ; LMULMAX2-RV32-NEXT:    lui a1, 61681
 ; LMULMAX2-RV32-NEXT:    addi a7, a1, -241
-; LMULMAX2-RV32-NEXT:    lui a1, 4112
-; LMULMAX2-RV32-NEXT:    addi a2, a1, 257
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
+; LMULMAX2-RV32-NEXT:    lui a2, 4112
+; LMULMAX2-RV32-NEXT:    addi a2, a2, 257
 ; LMULMAX2-RV32-NEXT:    bnez a5, .LBB3_2
 ; LMULMAX2-RV32-NEXT:  # %bb.1:
+; LMULMAX2-RV32-NEXT:    vsetvli zero, zero, e64,m1,ta,mu
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5

 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB3_3:
 ; LMULMAX2-RV32-NEXT:    sw a5, 0(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v25, v25, 1
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v25, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v25
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB3_5
+; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v26
+; LMULMAX2-RV32-NEXT:    bnez a5, .LBB3_5
 ; LMULMAX2-RV32-NEXT:  # %bb.4:
-; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
-; LMULMAX2-RV32-NEXT:    or a1, a5, a1
+; LMULMAX2-RV32-NEXT:    vsetvli zero, zero, e64,m1,ta,mu
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
+; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    not a1, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a4, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a4
 ; LMULMAX2-RV32-NEXT:    and a4, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    add a1, a4, a1
 ; LMULMAX2-RV32-NEXT:    srli a3, a1, 4
 ; LMULMAX2-RV32-NEXT:    add a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB3_6
 ; LMULMAX2-RV32-NEXT:  .LBB3_5:
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
+; LMULMAX2-RV32-NEXT:    or a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5

 ; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a6
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a5, v26
 ; LMULMAX1-RV32-NEXT:    lui a1, 349525
 ; LMULMAX1-RV32-NEXT:    addi a4, a1, 1365
 ; LMULMAX1-RV32-NEXT:    lui a1, 209715
 ; LMULMAX1-RV32-NEXT:    addi a3, a1, 819
 ; LMULMAX1-RV32-NEXT:    lui a1, 61681
 ; LMULMAX1-RV32-NEXT:    addi a7, a1, -241
-; LMULMAX1-RV32-NEXT:    lui a1, 4112
-; LMULMAX1-RV32-NEXT:    addi a2, a1, 257
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
+; LMULMAX1-RV32-NEXT:    lui a2, 4112
+; LMULMAX1-RV32-NEXT:    addi a2, a2, 257
 ; LMULMAX1-RV32-NEXT:    bnez a5, .LBB3_2
 ; LMULMAX1-RV32-NEXT:  # %bb.1:
+; LMULMAX1-RV32-NEXT:    vsetvli zero, zero, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5

 ; LMULMAX1-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB3_3:
 ; LMULMAX1-RV32-NEXT:    sw a5, 0(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v25, v25, 1
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a6
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV32-NEXT:    vmv.x.s a5, v25
-; LMULMAX1-RV32-NEXT:    bnez a1, .LBB3_5
+; LMULMAX1-RV32-NEXT:    vmv.x.s a5, v26
+; LMULMAX1-RV32-NEXT:    bnez a5, .LBB3_5
 ; LMULMAX1-RV32-NEXT:  # %bb.4:
-; LMULMAX1-RV32-NEXT:    srli a1, a5, 1
-; LMULMAX1-RV32-NEXT:    or a1, a5, a1
+; LMULMAX1-RV32-NEXT:    vsetvli zero, zero, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
+; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
+; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    not a1, a1
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a4, a5, a4
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    and a4, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    add a1, a4, a1
 ; LMULMAX1-RV32-NEXT:    srli a3, a1, 4
 ; LMULMAX1-RV32-NEXT:    add a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX1-RV32-NEXT:    j .LBB3_6
 ; LMULMAX1-RV32-NEXT:  .LBB3_5:
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
+; LMULMAX1-RV32-NEXT:    srli a1, a5, 1
+; LMULMAX1-RV32-NEXT:    or a1, a5, a1
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a5

 ; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v26, a6
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v28
 ; LMULMAX2-RV32-NEXT:    lui a1, 349525
 ; LMULMAX2-RV32-NEXT:    addi a4, a1, 1365
 ; LMULMAX2-RV32-NEXT:    lui a1, 209715
 ; LMULMAX2-RV32-NEXT:    addi a3, a1, 819
 ; LMULMAX2-RV32-NEXT:    lui a1, 61681
 ; LMULMAX2-RV32-NEXT:    addi a7, a1, -241
-; LMULMAX2-RV32-NEXT:    lui a1, 4112
-; LMULMAX2-RV32-NEXT:    addi a2, a1, 257
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX2-RV32-NEXT:    lui a2, 4112
+; LMULMAX2-RV32-NEXT:    addi a2, a2, 257
 ; LMULMAX2-RV32-NEXT:    bnez a5, .LBB7_2
 ; LMULMAX2-RV32-NEXT:  # %bb.1:
+; LMULMAX2-RV32-NEXT:    vsetvli zero, zero, e64,m2,ta,mu
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5

 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_3:
 ; LMULMAX2-RV32-NEXT:    sw a5, 0(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v28, v26, 3
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v30, v28, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v30
-; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v28
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB7_5
+; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v30
+; LMULMAX2-RV32-NEXT:    bnez a5, .LBB7_5
 ; LMULMAX2-RV32-NEXT:  # %bb.4:
-; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
-; LMULMAX2-RV32-NEXT:    or a1, a5, a1
+; LMULMAX2-RV32-NEXT:    vsetvli zero, zero, e64,m2,ta,mu
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
+; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    not a1, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    add a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    add a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    addi a5, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB7_6
 ; LMULMAX2-RV32-NEXT:  .LBB7_5:
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
+; LMULMAX2-RV32-NEXT:    or a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5

 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_6:
 ; LMULMAX2-RV32-NEXT:    sw a5, 24(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v28, v26, 2
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v30, v28, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v30
-; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v28
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB7_8
+; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v30
+; LMULMAX2-RV32-NEXT:    bnez a5, .LBB7_8
 ; LMULMAX2-RV32-NEXT:  # %bb.7:
-; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
-; LMULMAX2-RV32-NEXT:    or a1, a5, a1
+; LMULMAX2-RV32-NEXT:    vsetvli zero, zero, e64,m2,ta,mu
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
+; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    not a1, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    add a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    add a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    addi a5, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB7_9
 ; LMULMAX2-RV32-NEXT:  .LBB7_8:
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
+; LMULMAX2-RV32-NEXT:    or a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5

 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_9:
 ; LMULMAX2-RV32-NEXT:    sw a5, 16(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v26, v26, 1
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v26, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v26
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB7_11
+; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v28
+; LMULMAX2-RV32-NEXT:    bnez a5, .LBB7_11
 ; LMULMAX2-RV32-NEXT:  # %bb.10:
-; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
-; LMULMAX2-RV32-NEXT:    or a1, a5, a1
+; LMULMAX2-RV32-NEXT:    vsetvli zero, zero, e64,m2,ta,mu
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    not a1, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a4, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a4
 ; LMULMAX2-RV32-NEXT:    and a4, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    add a1, a4, a1
 ; LMULMAX2-RV32-NEXT:    srli a3, a1, 4
 ; LMULMAX2-RV32-NEXT:    add a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB7_12
 ; LMULMAX2-RV32-NEXT:  .LBB7_11:
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
+; LMULMAX2-RV32-NEXT:    or a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a5

 ; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a7
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v27
 ; LMULMAX1-RV32-NEXT:    lui a2, 349525
 ; LMULMAX1-RV32-NEXT:    addi a5, a2, 1365
 ; LMULMAX1-RV32-NEXT:    lui a2, 209715
 ; LMULMAX1-RV32-NEXT:    addi a4, a2, 819
 ; LMULMAX1-RV32-NEXT:    lui a2, 61681
 ; LMULMAX1-RV32-NEXT:    addi t0, a2, -241
-; LMULMAX1-RV32-NEXT:    lui a2, 4112
-; LMULMAX1-RV32-NEXT:    addi a3, a2, 257
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV32-NEXT:    lui a3, 4112
+; LMULMAX1-RV32-NEXT:    addi a3, a3, 257
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_2
 ; LMULMAX1-RV32-NEXT:  # %bb.1:
-; LMULMAX1-RV32-NEXT:    srli a1, a2, 1
-; LMULMAX1-RV32-NEXT:    or a1, a2, a1
+; LMULMAX1-RV32-NEXT:    vsetvli zero, zero, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2

 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_3:
 ; LMULMAX1-RV32-NEXT:    sw a1, 16(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v26, v26, 1
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a7
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v27
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v26
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_5
 ; LMULMAX1-RV32-NEXT:  # %bb.4:
-; LMULMAX1-RV32-NEXT:    srli a1, a2, 1
-; LMULMAX1-RV32-NEXT:    or a1, a2, a1
+; LMULMAX1-RV32-NEXT:    vsetvli zero, zero, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2

 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_6:
 ; LMULMAX1-RV32-NEXT:    sw a1, 24(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 12(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 4(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a7
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v25
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_8
 ; LMULMAX1-RV32-NEXT:  # %bb.7:
-; LMULMAX1-RV32-NEXT:    srli a1, a2, 1
-; LMULMAX1-RV32-NEXT:    or a1, a2, a1
+; LMULMAX1-RV32-NEXT:    vsetvli zero, zero, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2

 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_9:
 ; LMULMAX1-RV32-NEXT:    sw a1, 0(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v25, v25, 1
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a7
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v25
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_11
 ; LMULMAX1-RV32-NEXT:  # %bb.10:
-; LMULMAX1-RV32-NEXT:    srli a1, a2, 1
-; LMULMAX1-RV32-NEXT:    or a1, a2, a1
+; LMULMAX1-RV32-NEXT:    vsetvli zero, zero, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-cttz.ll

 ; LMULMAX2-RV32:       # %bb.0:
 ; LMULMAX2-RV32-NEXT:    addi sp, sp, -16
 ; LMULMAX2-RV32-NEXT:    .cfi_def_cfa_offset 16
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 2, e64,m1,ta,mu
 ; LMULMAX2-RV32-NEXT:    vle64.v v25, (a0)
 ; LMULMAX2-RV32-NEXT:    sw zero, 12(sp)
 ; LMULMAX2-RV32-NEXT:    sw zero, 4(sp)
 ; LMULMAX2-RV32-NEXT:    addi a6, zero, 32
-; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
-; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v25, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v26
 ; LMULMAX2-RV32-NEXT:    lui a1, 349525
 ; LMULMAX2-RV32-NEXT:    addi a4, a1, 1365
 ; LMULMAX2-RV32-NEXT:    lui a1, 209715
 ; LMULMAX2-RV32-NEXT:    addi a3, a1, 819
 ; LMULMAX2-RV32-NEXT:    lui a1, 61681
 ; LMULMAX2-RV32-NEXT:    addi a7, a1, -241
 ; LMULMAX2-RV32-NEXT:    lui a2, 4112
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
+; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v25
 ; LMULMAX2-RV32-NEXT:    addi a2, a2, 257
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB3_2
+; LMULMAX2-RV32-NEXT:    bnez a5, .LBB3_2
 ; LMULMAX2-RV32-NEXT:  # %bb.1:
-; LMULMAX2-RV32-NEXT:    addi a1, a5, -1
-; LMULMAX2-RV32-NEXT:    not a5, a5
-; LMULMAX2-RV32-NEXT:    and a1, a5, a1
+; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
+; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v25, a6
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX2-RV32-NEXT:    addi a5, a1, -1
+; LMULMAX2-RV32-NEXT:    not a1, a1
+; LMULMAX2-RV32-NEXT:    and a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    add a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    add a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    addi a5, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB3_3
 ; LMULMAX2-RV32-NEXT:  .LBB3_2:
-; LMULMAX2-RV32-NEXT:    addi a5, a1, -1
-; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    and a1, a1, a5
+; LMULMAX2-RV32-NEXT:    addi a1, a5, -1
+; LMULMAX2-RV32-NEXT:    not a5, a5
+; LMULMAX2-RV32-NEXT:    and a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    add a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    add a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB3_3:
 ; LMULMAX2-RV32-NEXT:    sw a5, 0(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v25, v25, 1
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v25
-; LMULMAX2-RV32-NEXT:    vsrl.vx v25, v25, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX2-RV32-NEXT:    bnez a5, .LBB3_5
 ; LMULMAX2-RV32-NEXT:  # %bb.4:
+; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
+; LMULMAX2-RV32-NEXT:    vsrl.vx v25, v25, a6
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX2-RV32-NEXT:    addi a5, a1, -1
 ; LMULMAX2-RV32-NEXT:    not a1, a1
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a4, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a4
 ; LMULMAX2-RV32-NEXT:    and a4, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2

 ; LMULMAX1-RV32:       # %bb.0:
 ; LMULMAX1-RV32-NEXT:    addi sp, sp, -16
 ; LMULMAX1-RV32-NEXT:    .cfi_def_cfa_offset 16
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 2, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vle64.v v25, (a0)
 ; LMULMAX1-RV32-NEXT:    sw zero, 12(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 4(sp)
 ; LMULMAX1-RV32-NEXT:    addi a6, zero, 32
-; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
-; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a6
-; LMULMAX1-RV32-NEXT:    vmv.x.s a5, v26
 ; LMULMAX1-RV32-NEXT:    lui a1, 349525
 ; LMULMAX1-RV32-NEXT:    addi a4, a1, 1365
 ; LMULMAX1-RV32-NEXT:    lui a1, 209715
 ; LMULMAX1-RV32-NEXT:    addi a3, a1, 819
 ; LMULMAX1-RV32-NEXT:    lui a1, 61681
 ; LMULMAX1-RV32-NEXT:    addi a7, a1, -241
 ; LMULMAX1-RV32-NEXT:    lui a2, 4112
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
+; LMULMAX1-RV32-NEXT:    vmv.x.s a5, v25
 ; LMULMAX1-RV32-NEXT:    addi a2, a2, 257
-; LMULMAX1-RV32-NEXT:    bnez a1, .LBB3_2
+; LMULMAX1-RV32-NEXT:    bnez a5, .LBB3_2
 ; LMULMAX1-RV32-NEXT:  # %bb.1:
-; LMULMAX1-RV32-NEXT:    addi a1, a5, -1
-; LMULMAX1-RV32-NEXT:    not a5, a5
-; LMULMAX1-RV32-NEXT:    and a1, a5, a1
+; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a6
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX1-RV32-NEXT:    addi a5, a1, -1
+; LMULMAX1-RV32-NEXT:    not a1, a1
+; LMULMAX1-RV32-NEXT:    and a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    add a1, a5, a1
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX1-RV32-NEXT:    add a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:    addi a5, a1, 32
 ; LMULMAX1-RV32-NEXT:    j .LBB3_3
 ; LMULMAX1-RV32-NEXT:  .LBB3_2:
-; LMULMAX1-RV32-NEXT:    addi a5, a1, -1
-; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    and a1, a1, a5
+; LMULMAX1-RV32-NEXT:    addi a1, a5, -1
+; LMULMAX1-RV32-NEXT:    not a5, a5
+; LMULMAX1-RV32-NEXT:    and a1, a5, a1
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    add a1, a5, a1
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX1-RV32-NEXT:    add a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB3_3:
 ; LMULMAX1-RV32-NEXT:    sw a5, 0(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v25, v25, 1
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a5, v25
-; LMULMAX1-RV32-NEXT:    vsrl.vx v25, v25, a6
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX1-RV32-NEXT:    bnez a5, .LBB3_5
 ; LMULMAX1-RV32-NEXT:  # %bb.4:
+; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vsrl.vx v25, v25, a6
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX1-RV32-NEXT:    addi a5, a1, -1
 ; LMULMAX1-RV32-NEXT:    not a1, a1
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a5
 ; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a4, a5, a4
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    and a4, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2

 ; LMULMAX2-RV32-NEXT:    andi sp, sp, -32
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 4, e64,m2,ta,mu
 ; LMULMAX2-RV32-NEXT:    vle64.v v26, (a0)
 ; LMULMAX2-RV32-NEXT:    sw zero, 28(sp)
 ; LMULMAX2-RV32-NEXT:    sw zero, 20(sp)
 ; LMULMAX2-RV32-NEXT:    sw zero, 12(sp)
 ; LMULMAX2-RV32-NEXT:    sw zero, 4(sp)
 ; LMULMAX2-RV32-NEXT:    addi a6, zero, 32
-; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
-; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v26, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v28
 ; LMULMAX2-RV32-NEXT:    lui a1, 349525
 ; LMULMAX2-RV32-NEXT:    addi a4, a1, 1365
 ; LMULMAX2-RV32-NEXT:    lui a1, 209715
 ; LMULMAX2-RV32-NEXT:    addi a3, a1, 819
 ; LMULMAX2-RV32-NEXT:    lui a1, 61681
 ; LMULMAX2-RV32-NEXT:    addi a7, a1, -241
 ; LMULMAX2-RV32-NEXT:    lui a2, 4112
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v26
 ; LMULMAX2-RV32-NEXT:    addi a2, a2, 257
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB7_2
+; LMULMAX2-RV32-NEXT:    bnez a5, .LBB7_2
 ; LMULMAX2-RV32-NEXT:  # %bb.1:
-; LMULMAX2-RV32-NEXT:    addi a1, a5, -1
-; LMULMAX2-RV32-NEXT:    not a5, a5
-; LMULMAX2-RV32-NEXT:    and a1, a5, a1
+; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
+; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v26, a6
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
+; LMULMAX2-RV32-NEXT:    addi a5, a1, -1
+; LMULMAX2-RV32-NEXT:    not a1, a1
+; LMULMAX2-RV32-NEXT:    and a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    add a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    add a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    addi a5, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB7_3
 ; LMULMAX2-RV32-NEXT:  .LBB7_2:
-; LMULMAX2-RV32-NEXT:    addi a5, a1, -1
-; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    and a1, a1, a5
+; LMULMAX2-RV32-NEXT:    addi a1, a5, -1
+; LMULMAX2-RV32-NEXT:    not a5, a5
+; LMULMAX2-RV32-NEXT:    and a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    add a1, a5, a1
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
 ; LMULMAX2-RV32-NEXT:    add a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_3:
 ; LMULMAX2-RV32-NEXT:    sw a5, 0(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v28, v26, 3
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v28
-; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v28, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
 ; LMULMAX2-RV32-NEXT:    bnez a5, .LBB7_5
 ; LMULMAX2-RV32-NEXT:  # %bb.4:
+; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
+; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v28, a6
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
 ; LMULMAX2-RV32-NEXT:    addi a5, a1, -1
 ; LMULMAX2-RV32-NEXT:    not a1, a1
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2

 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_6:
 ; LMULMAX2-RV32-NEXT:    sw a5, 24(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v28, v26, 2
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v28
-; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v28, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
 ; LMULMAX2-RV32-NEXT:    bnez a5, .LBB7_8
 ; LMULMAX2-RV32-NEXT:  # %bb.7:
+; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
+; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v28, a6
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
 ; LMULMAX2-RV32-NEXT:    addi a5, a1, -1
 ; LMULMAX2-RV32-NEXT:    not a1, a1
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a5, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    and a5, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2

 ; LMULMAX2-RV32-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_9:
 ; LMULMAX2-RV32-NEXT:    sw a5, 16(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v26, v26, 1
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v26
-; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v26, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX2-RV32-NEXT:    bnez a5, .LBB7_11
 ; LMULMAX2-RV32-NEXT:  # %bb.10:
+; LMULMAX2-RV32-NEXT:    vsetivli a1, 1, e64,m2,ta,mu
+; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v26, a6
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX2-RV32-NEXT:    addi a5, a1, -1
 ; LMULMAX2-RV32-NEXT:    not a1, a1
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a5
 ; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
 ; LMULMAX2-RV32-NEXT:    and a4, a5, a4
 ; LMULMAX2-RV32-NEXT:    sub a1, a1, a4
 ; LMULMAX2-RV32-NEXT:    and a4, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2

 ; LMULMAX2-RV64-NEXT:    ret
 ;
 ; LMULMAX1-RV32-LABEL: cttz_v4i64:
 ; LMULMAX1-RV32:       # %bb.0:
 ; LMULMAX1-RV32-NEXT:    addi sp, sp, -32
 ; LMULMAX1-RV32-NEXT:    .cfi_def_cfa_offset 32
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 2, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vle64.v v25, (a0)
-; LMULMAX1-RV32-NEXT:    addi a6, a0, 16
-; LMULMAX1-RV32-NEXT:    vle64.v v26, (a6)
+; LMULMAX1-RV32-NEXT:    addi a7, a0, 16
+; LMULMAX1-RV32-NEXT:    vle64.v v26, (a7)
 ; LMULMAX1-RV32-NEXT:    sw zero, 28(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 20(sp)
-; LMULMAX1-RV32-NEXT:    addi a7, zero, 32
-; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
-; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a7
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v27
-; LMULMAX1-RV32-NEXT:    lui a2, 349525
-; LMULMAX1-RV32-NEXT:    addi a5, a2, 1365
-; LMULMAX1-RV32-NEXT:    lui a2, 209715
-; LMULMAX1-RV32-NEXT:    addi a4, a2, 819
-; LMULMAX1-RV32-NEXT:    lui a2, 61681
-; LMULMAX1-RV32-NEXT:    addi t0, a2, -241
+; LMULMAX1-RV32-NEXT:    addi a6, zero, 32
+; LMULMAX1-RV32-NEXT:    lui a1, 349525
+; LMULMAX1-RV32-NEXT:    addi a5, a1, 1365
+; LMULMAX1-RV32-NEXT:    lui a1, 209715
+; LMULMAX1-RV32-NEXT:    addi a4, a1, 819
+; LMULMAX1-RV32-NEXT:    lui a1, 61681
+; LMULMAX1-RV32-NEXT:    addi t0, a1, -241
 ; LMULMAX1-RV32-NEXT:    lui a3, 4112
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX1-RV32-NEXT:    addi a3, a3, 257
-; LMULMAX1-RV32-NEXT:    bnez a2, .LBB7_2
+; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_2
 ; LMULMAX1-RV32-NEXT:  # %bb.1:
+; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a6
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v27
 ; LMULMAX1-RV32-NEXT:    addi a2, a1, -1
 ; LMULMAX1-RV32-NEXT:    not a1, a1
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    add a1, a2, a1
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    add a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, t0
 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX1-RV32-NEXT:    j .LBB7_3
 ; LMULMAX1-RV32-NEXT:  .LBB7_2:
-; LMULMAX1-RV32-NEXT:    addi a1, a2, -1
-; LMULMAX1-RV32-NEXT:    not a2, a2
-; LMULMAX1-RV32-NEXT:    and a1, a2, a1
+; LMULMAX1-RV32-NEXT:    addi a2, a1, -1
+; LMULMAX1-RV32-NEXT:    not a1, a1
+; LMULMAX1-RV32-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    add a1, a2, a1
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    add a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, t0
 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_3:
 ; LMULMAX1-RV32-NEXT:    sw a1, 16(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v26, v26, 1
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v26, a7
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v26
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_5
 ; LMULMAX1-RV32-NEXT:  # %bb.4:
-; LMULMAX1-RV32-NEXT:    addi a1, a2, -1
-; LMULMAX1-RV32-NEXT:    not a2, a2
-; LMULMAX1-RV32-NEXT:    and a1, a2, a1
+; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v26, a6
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX1-RV32-NEXT:    addi a2, a1, -1
+; LMULMAX1-RV32-NEXT:    not a1, a1
+; LMULMAX1-RV32-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    add a1, a2, a1
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4

 ; LMULMAX1-RV32-NEXT:    add a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, t0
 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_6:
 ; LMULMAX1-RV32-NEXT:    sw a1, 24(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 12(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 4(sp)
-; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
-; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a7
+; LMULMAX1-RV32-NEXT:    vsetvli zero, zero, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v26
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_8
 ; LMULMAX1-RV32-NEXT:  # %bb.7:
-; LMULMAX1-RV32-NEXT:    addi a1, a2, -1
-; LMULMAX1-RV32-NEXT:    not a2, a2
-; LMULMAX1-RV32-NEXT:    and a1, a2, a1
+; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a6
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX1-RV32-NEXT:    addi a2, a1, -1
+; LMULMAX1-RV32-NEXT:    not a1, a1
+; LMULMAX1-RV32-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    add a1, a2, a1
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4

 ; LMULMAX1-RV32-NEXT:    and a1, a1, t0
 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_9:
 ; LMULMAX1-RV32-NEXT:    sw a1, 0(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v25, v25, 1
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
-; LMULMAX1-RV32-NEXT:    vsrl.vx v25, v25, a7
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v25
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_11
 ; LMULMAX1-RV32-NEXT:  # %bb.10:
-; LMULMAX1-RV32-NEXT:    addi a1, a2, -1
-; LMULMAX1-RV32-NEXT:    not a2, a2
-; LMULMAX1-RV32-NEXT:    and a1, a2, a1
+; LMULMAX1-RV32-NEXT:    vsetivli a1, 1, e64,m1,ta,mu
+; LMULMAX1-RV32-NEXT:    vsrl.vx v25, v25, a6
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
+; LMULMAX1-RV32-NEXT:    addi a2, a1, -1
+; LMULMAX1-RV32-NEXT:    not a1, a1
+; LMULMAX1-RV32-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    add a1, a2, a1
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4

 ; LMULMAX1-RV32-NEXT:  .LBB7_12:
 ; LMULMAX1-RV32-NEXT:    sw a1, 8(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 4, e32,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vle32.v v25, (sp)
 ; LMULMAX1-RV32-NEXT:    addi a1, sp, 16
 ; LMULMAX1-RV32-NEXT:    vle32.v v26, (a1)
 ; LMULMAX1-RV32-NEXT:    vsetivli a1, 2, e64,m1,ta,mu
 ; LMULMAX1-RV32-NEXT:    vse64.v v25, (a0)
-; LMULMAX1-RV32-NEXT:    vse64.v v26, (a6)
+; LMULMAX1-RV32-NEXT:    vse64.v v26, (a7)
 ; LMULMAX1-RV32-NEXT:    addi sp, sp, 32
 ; LMULMAX1-RV32-NEXT:    ret
 ;
 ; LMULMAX1-RV64-LABEL: cttz_v4i64:
 ; LMULMAX1-RV64:       # %bb.0:
 ; LMULMAX1-RV64-NEXT:    vsetivli a1, 2, e64,m1,ta,mu
 ; LMULMAX1-RV64-NEXT:    addi a6, a0, 16
 ; LMULMAX1-RV64-NEXT:    vle64.v v26, (a6)

llvm/test/CodeGen/RISCV/rvv/frameindex-addr.ll

declare void @llvm.riscv.vse.nxv1i64(
   <vscale x 1 x i64>*,
   i64);
 
 define i64 @test(<vscale x 1 x i64> %0) nounwind {
   ; CHECK-LABEL: name: test
   ; CHECK: bb.0.entry:
   ; CHECK:   liveins: $v8
   ; CHECK:   [[COPY:%[0-9]+]]:vr = COPY $v8
-  ; CHECK:   dead %2:gpr = PseudoVSETIVLI 1, 88, implicit-def $vl, implicit-def $vtype
-  ; CHECK:   PseudoVSE64_V_M1 [[COPY]], %stack.0.a, 1, 6, implicit $vl, implicit $vtype
+  ; CHECK:   PseudoVSE64_V_M1 [[COPY]], %stack.0.a, 1, 6
   ; CHECK:   [[LD:%[0-9]+]]:gpr = LD %stack.0.a, 0 :: (dereferenceable load 8 from %ir.a)
   ; CHECK:   $x10 = COPY [[LD]]
   ; CHECK:   PseudoRET implicit $x10
 entry:
   %a = alloca i64
   %b = bitcast i64* %a to <vscale x 1 x i64>*
   call void @llvm.riscv.vse.nxv1i64(
     <vscale x 1 x i64> %0,
     <vscale x 1 x i64>* %b,
     i64 1)
   %c = load i64, i64* %a
   ret i64 %c
 }

llvm/test/CodeGen/RISCV/rvv/mask-reg-alloc.mir

 ---
 name: mask_reg_alloc
 tracksRegLiveness: true
 body:             |
   bb.0 (%ir-block.0):
     liveins: $v0, $v1, $v2, $v3
     ; CHECK-LABEL: name: mask_reg_alloc
     ; CHECK: liveins: $v0, $v1, $v2, $v3
-    ; CHECK: renamable $v25 = PseudoVMERGE_VIM_M1 killed renamable $v2, 1, killed renamable $v0, $noreg, -1, implicit $vl, implicit $vtype
+    ; CHECK: dead renamable $x10 = PseudoVSETVLI $noreg, 64, implicit-def $vl, implicit-def $vtype
+    ; CHECK: renamable $v25 = PseudoVMERGE_VIM_M1 killed renamable $v2, 1, killed renamable $v0, $noreg, 3, implicit $vl, implicit $vtype
     ; CHECK: renamable $v0 = COPY killed renamable $v1
-    ; CHECK: renamable $v26 = PseudoVMERGE_VIM_M1 killed renamable $v3, 1, killed renamable $v0, $noreg, -1, implicit $vl, implicit $vtype
-    ; CHECK: renamable $v0 = PseudoVADD_VV_M1 killed renamable $v25, killed renamable $v26, $noreg, -1, implicit $vl, implicit $vtype
+    ; CHECK: renamable $v26 = PseudoVMERGE_VIM_M1 killed renamable $v3, 1, killed renamable $v0, $noreg, 3, implicit $vl, implicit $vtype
+    ; CHECK: renamable $v0 = PseudoVADD_VV_M1 killed renamable $v25, killed renamable $v26, $noreg, 3, implicit $vl, implicit $vtype
     ; CHECK: PseudoRET implicit $v0
     %0:vr = COPY $v0
     %1:vr = COPY $v1
     %2:vr = COPY $v2
     %3:vr = COPY $v3
     %4:vmv0 = COPY %0
-    %5:vrnov0 = PseudoVMERGE_VIM_M1 killed %2, 1, %4, $noreg, -1, implicit $vl, implicit $vtype
+    %5:vrnov0 = PseudoVMERGE_VIM_M1 killed %2, 1, %4, $noreg, 3, implicit $vl, implicit $vtype
     %6:vmv0 = COPY %1
-    %7:vrnov0 = PseudoVMERGE_VIM_M1 killed %3, 1, %6, $noreg, -1, implicit $vl, implicit $vtype
-    %8:vr = PseudoVADD_VV_M1 killed %5, killed %7, $noreg, -1, implicit $vl, implicit $vtype
+    %7:vrnov0 = PseudoVMERGE_VIM_M1 killed %3, 1, %6, $noreg, 3, implicit $vl, implicit $vtype
+    %8:vr = PseudoVADD_VV_M1 killed %5, killed %7, $noreg, 3, implicit $vl, implicit $vtype
     $v0 = COPY %8
     PseudoRET implicit $v0
 ...

llvm/test/CodeGen/RISCV/rvv/tail-agnostic-impdef-copy.mir

bb.0 (%ir-block.0):
 
     ; CHECK-LABEL: name: masked_load_nxv8i64
     ; CHECK: liveins: $x10, $v0
     ; CHECK: [[COPY:%[0-9]+]]:vr = COPY $v0
     ; CHECK: [[COPY1:%[0-9]+]]:gpr = COPY $x10
     ; CHECK: $v0 = COPY [[COPY]]
     ; CHECK: [[DEF:%[0-9]+]]:vrm8 = IMPLICIT_DEF
     ; CHECK: [[COPY2:%[0-9]+]]:vrm8nov0 = COPY [[DEF]]
-    ; CHECK: dead %5:gpr = PseudoVSETVLI $x0, 91, implicit-def $vl, implicit-def $vtype
-    ; CHECK: [[PseudoVLE64_V_M8_MASK:%[0-9]+]]:vrm8nov0 = PseudoVLE64_V_M8_MASK [[COPY2]], [[COPY1]], $v0, $noreg, 6, implicit $vl, implicit $vtype :: (load 64 from %ir.a, align 8)
+    ; CHECK: [[PseudoVLE64_V_M8_MASK:%[0-9]+]]:vrm8nov0 = PseudoVLE64_V_M8_MASK [[COPY2]], [[COPY1]], $v0, $x0, 6, implicit $vl, implicit $vtype :: (load 64 from %ir.a, align 8)
     ; CHECK: $v8m8 = COPY [[PseudoVLE64_V_M8_MASK]]
     ; CHECK: PseudoRET implicit $v8m8
     %1:vr = COPY $v0
     %0:gpr = COPY $x10
     $v0 = COPY %1
     %3:vrm8 = IMPLICIT_DEF
     %4:vrm8nov0 = COPY %3
     %2:vrm8nov0 = PseudoVLE64_V_M8_MASK %4, %0, $v0, $x0, 6, implicit $vl, implicit $vtype :: (load 64 from %ir.a, align 8)
     $v8m8 = COPY %2
     PseudoRET implicit $v8m8
 
 ...