[RISCV] Fix VSETVLI insertion by syncing phases 2 and 3

[frasercrmck]

Summary

This patch attempts to make the VSETVLI insertion pass more stable. A
lot of bugs are ultimately introduced due to phases being out of sync.

This is most notable when we skip VSETVLI insertion in phase 3 by re-using a
previous config, but still rely on phase 2 information which was
calculated without full knowledge of what we're going to skip.

The idea used in this patch is something @rogfer01 suggested in D124089.
By ensuring that VSETVLI "skips" are calculated in one re-usable place
(a new overload of needVSETVLI) we can use it during phases 1 and 2 to
produce more accurate information. In addition, during phase 2, we now
re-compute block-local VTYPE changes once predecessors have been
computed.

[frasercrmck]

adjust comments

[reames]

Have you tried adding asserts to the end of emitVSETVLIs to ensure the final block state is as expected? I strongly suspect this change is insufficient to fix all the cases.

I've been working on the same problem, but with a different approach. I've been trying to separate the PHI case and the mutation case into a separate phase 4 which works over the fully materialized form. I've got all the transforms (messily) implemented, but even with those out of line, I'm still getting failures of the expected block end state assert. I don't yet know why.

One really nasty case I came across - that I don't think your change handles - is that mutation of the prior vsetvli can change whether the current state is compatible with a following instruction. The compatibility logic includes a bit which looks back at the defining vsetvli for a vreg operand, and since we mutated it, the result can change between phase 1 and phase 3. At the moment, I'm thinking we can't have both that compatibility rule and the mutation in use in the same pass.

Now despite all that, I think your changes here really help in terms of code structure.

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

1070

This bit on it's own is super important. It's slower, but it avoids the whole need for the merge and Change state entirely.

I think this is a good idea, and probably stands on it's own.

[reames]

Here (https://reviews.llvm.org/D125035) is the assertion I mentioned above as a flag we can use for testing.

[frasercrmck]

In D125021#3494767, @reames wrote:

Have you tried adding asserts to the end of emitVSETVLIs to ensure the final block state is as expected? I strongly suspect this change is insufficient to fix all the cases.

It's certainly not enough to fix all cases, as I'm seeing new regressions when testing this internally. Looks like we might be missing coverage on loops. I'll get to work on reducing those failures and committing them. Your asserts might help with that.

I've been working on the same problem, but with a different approach. I've been trying to separate the PHI case and the mutation case into a separate phase 4 which works over the fully materialized form. I've got all the transforms (messily) implemented, but even with those out of line, I'm still getting failures of the expected block end state assert. I don't yet know why.

One really nasty case I came across - that I don't think your change handles - is that mutation of the prior vsetvli can change whether the current state is compatible with a following instruction. The compatibility logic includes a bit which looks back at the defining vsetvli for a vreg operand, and since we mutated it, the result can change between phase 1 and phase 3. At the moment, I'm thinking we can't have both that compatibility rule and the mutation in use in the same pass.

Yeah I only really clocked this mutation thing yesterday. I thought that by including the "mutation" in needVSETVLI (now in phases 1 and 2) we might be able to get away with it being synced up?

Now despite all that, I think your changes here really help in terms of code structure.

Thanks. I do think having the one method to decide the compatibility and using it consistently across phases is logical and less error-prone. Unfortunately it seems we're not able to make what should be an NFC change without breaking things.

[frasercrmck]

rebase on new test which now fails

[frasercrmck]

llvm/test/CodeGen/RISCV/rvv/vsetvli-insert-crossbb.mir

856

This is a bug, but I'm not yet sure if it's dormant and exposed by this patch or actually caused by it.

[frasercrmck]

remove need for TmpStatus (and BBInfo.Change, really)

[frasercrmck]

pre-emptively include D125133

[frasercrmck]

This is all largely irrelevant now. I'll open a new revision for the API refactor.

llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp

public:
   }
 
   // Determine whether the vector instructions requirements represented by
   // InstrInfo are compatible with the previous vsetvli instruction represented
   // by this.
   bool isCompatible(const VSETVLIInfo &InstrInfo, bool Strict) const {
     assert(isValid() && InstrInfo.isValid() &&
            "Can't compare invalid VSETVLIInfos");
-    assert(!InstrInfo.SEWLMULRatioOnly &&
-           "Expected a valid VTYPE for instruction!");
     // Nothing is compatible with Unknown.
     if (isUnknown() || InstrInfo.isUnknown())
       return false;
 
     // If only our VLMAX ratio is valid, then this isn't compatible.
-    if (SEWLMULRatioOnly)
+    if (SEWLMULRatioOnly || InstrInfo.SEWLMULRatioOnly)
       return false;
 
     // If the instruction doesn't need an AVLReg and the SEW matches, consider
     // it compatible.
     if (!Strict && InstrInfo.hasAVLReg() &&
         InstrInfo.AVLReg == RISCV::NoRegister) {
       if (SEW == InstrInfo.SEW)
         return true;
struct BlockData {
   VSETVLIInfo Pred;
 
   // Keeps track of whether the block is already in the queue.
   bool InQueue = false;
 
   BlockData() = default;
 };
 
+// The different kinds of VSETVLI insertion we support.
+enum VSETVLIInsertionKind {
+  // No VSETVLI required (e.g., compatible with previous)
+  None,
+  // A new VSETVLI is required
+  Required,
+  // A previous VSETVLI may safely be mutated.
+  MutatePrevious,
+};
+
 class RISCVInsertVSETVLI : public MachineFunctionPass {
   const TargetInstrInfo *TII;
   MachineRegisterInfo *MRI;
 
   std::vector<BlockData> BlockInfo;
   std::queue<const MachineBasicBlock *> WorkList;
 
 public:
   static char ID;
 
   RISCVInsertVSETVLI() : MachineFunctionPass(ID) {
     initializeRISCVInsertVSETVLIPass(*PassRegistry::getPassRegistry());
   }
   bool runOnMachineFunction(MachineFunction &MF) override;
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.setPreservesCFG();
     MachineFunctionPass::getAnalysisUsage(AU);
   }
 
   StringRef getPassName() const override { return RISCV_INSERT_VSETVLI_NAME; }
 
 private:
   bool needVSETVLI(const VSETVLIInfo &Require, const VSETVLIInfo &CurInfo);
+  VSETVLIInsertionKind needVSETVLI(const MachineInstr &MI,
+                                   const VSETVLIInfo &Require,
+                                   const VSETVLIInfo &CurInfo,
+                                   const MachineInstr *PrevVSETVLIMI = nullptr);
   bool needVSETVLIPHI(const VSETVLIInfo &Require, const MachineBasicBlock &MBB);
   void insertVSETVLI(MachineBasicBlock &MBB, MachineInstr &MI,
                      const VSETVLIInfo &Info, const VSETVLIInfo &PrevInfo);
   void insertVSETVLI(MachineBasicBlock &MBB,
                      MachineBasicBlock::iterator InsertPt, DebugLoc DL,
                      const VSETVLIInfo &Info, const VSETVLIInfo &PrevInfo);
 
   bool computeVLVTYPEChanges(const MachineBasicBlock &MBB);
bool canSkipVSETVLIForLoadStore(const MachineInstr &MI,
   case RISCV::PseudoVSSE64_V_M8_MASK:
     EEW = 64;
     break;
   }
 
   return CurInfo.isCompatibleWithLoadStoreEEW(EEW, Require);
 }
 
+VSETVLIInsertionKind RISCVInsertVSETVLI::needVSETVLI(
+    const MachineInstr &MI, const VSETVLIInfo &Require,
+    const VSETVLIInfo &CurInfo, const MachineInstr *PrevVSETVLIMI) {
+  if (!needVSETVLI(Require, CurInfo))
+    return VSETVLIInsertionKind::None;
+
+  // If this is a unit-stride or strided load/store, we may be able to use
+  // the EMUL=(EEW/SEW)*LMUL relationship to avoid changing vtype.
+  if (canSkipVSETVLIForLoadStore(MI, Require, CurInfo))
+    return VSETVLIInsertionKind::None;
+
+  // If the previous VL/VTYPE is set by VSETVLI and do not use, Merge it
+  // with current VL/VTYPE.
+  if (PrevVSETVLIMI) {
+    bool HasSameAVL =
+        CurInfo.hasSameAVL(Require) ||
+        (Require.hasAVLReg() && Require.getAVLReg().isVirtual() &&
+         Require.getAVLReg() == PrevVSETVLIMI->getOperand(0).getReg());
+    // If these two VSETVLI have the same AVL and the same VLMAX,
+    // we could merge these two VSETVLI.
+    if (HasSameAVL && CurInfo.getSEWLMULRatio() == Require.getSEWLMULRatio())
+      return VSETVLIInsertionKind::MutatePrevious;
+
+    if (isScalarMoveInstr(MI) &&
+        ((CurInfo.hasNonZeroAVL() && Require.hasNonZeroAVL()) ||
+         (CurInfo.hasZeroAVL() && Require.hasZeroAVL())) &&
+        Require.hasSameVLMAX(CurInfo))
+      return VSETVLIInsertionKind::MutatePrevious;
+  }
+
+  return VSETVLIInsertionKind::Required;
+}
+
 bool RISCVInsertVSETVLI::computeVLVTYPEChanges(const MachineBasicBlock &MBB) {
   bool HadVectorOp = false;
 
+  // Only set if current VSETVLIInfo is from an explicit VSET(I)VLI.
+  const MachineInstr *PrevVSETVLIMI = nullptr;
+
   BlockData &BBInfo = BlockInfo[MBB.getNumber()];
   for (const MachineInstr &MI : MBB) {
     // If this is an explicit VSETVLI or VSETIVLI, update our state.
     if (isVectorConfigInstr(MI)) {
       HadVectorOp = true;
+      PrevVSETVLIMI = &MI;
       BBInfo.Change = getInfoForVSETVLI(MI);
       continue;
     }
 
     uint64_t TSFlags = MI.getDesc().TSFlags;
     if (RISCVII::hasSEWOp(TSFlags)) {
       HadVectorOp = true;
 
       VSETVLIInfo NewInfo = computeInfoForInstr(MI, TSFlags, MRI);
 
       if (!BBInfo.Change.isValid()) {
         BBInfo.Change = NewInfo;
       } else {
         // If this instruction isn't compatible with the previous VL/VTYPE
         // we need to insert a VSETVLI.
-        // If this is a unit-stride or strided load/store, we may be able to use
-        // the EMUL=(EEW/SEW)*LMUL relationship to avoid changing vtype.
         // NOTE: We only do this if the vtype we're comparing against was
         // created in this block. We need the first and third phase to treat
         // the store the same way.
-        if (!canSkipVSETVLIForLoadStore(MI, NewInfo, BBInfo.Change) &&
-            needVSETVLI(NewInfo, BBInfo.Change))
+        if (needVSETVLI(MI, NewInfo, BBInfo.Change, PrevVSETVLIMI) !=
+            VSETVLIInsertionKind::None) {
           BBInfo.Change = NewInfo;
         }
       }
+      PrevVSETVLIMI = nullptr;
+    }
 
     // If this is something that updates VL/VTYPE that we don't know about, set
     // the state to unknown.
     if (MI.isCall() || MI.isInlineAsm() || MI.modifiesRegister(RISCV::VL) ||
         MI.modifiesRegister(RISCV::VTYPE)) {
       BBInfo.Change = VSETVLIInfo::getUnknown();
+      PrevVSETVLIMI = nullptr;
     }
   }
 
   // Initial exit state is whatever change we found in the block.
   BBInfo.Exit = BBInfo.Change;
 
   return HadVectorOp;
 }
void RISCVInsertVSETVLI::computeIncomingVLVTYPE(const MachineBasicBlock &MBB) {
   // If no change, no need to rerun block
   if (InInfo == BBInfo.Pred)
     return;
 
   BBInfo.Pred = InInfo;
   LLVM_DEBUG(dbgs() << "Entry state of " << printMBBReference(MBB)
                     << " changed to " << BBInfo.Pred << "\n");
 
-  VSETVLIInfo TmpStatus = BBInfo.Pred.merge(BBInfo.Change);
+  // Cache the exit state before recomputation.
+  // FIXME: We can drop 'Change' and just use 'Exit'.
+  // FIXME: We shouldn't have to set 'Change' as an input
+  auto CachedExit = BBInfo.Exit;
+
+  // Now that we've computed the info for predecessors, recompute the VTYPE
+  // changes on this block. The predecessors may have changed the incoming
+  // vtype and we must be in sync with phase 3.
+  BBInfo.Change = InInfo;
+  computeVLVTYPEChanges(MBB);
 
   // If the new exit value matches the old exit value, we don't need to revisit
   // any blocks.
-  if (BBInfo.Exit == TmpStatus)
+  if (CachedExit == BBInfo.Exit)
     return;
 
-  BBInfo.Exit = TmpStatus;
   LLVM_DEBUG(dbgs() << "Exit state of " << printMBBReference(MBB)
                     << " changed to " << BBInfo.Exit << "\n");
 
   // Add the successors to the work list so we can propagate the changed exit
   // status.
   for (MachineBasicBlock *S : MBB.successors())
     if (!BlockInfo[S->getNumber()].InQueue)
       WorkList.push(S);
bool RISCVInsertVSETVLI::needVSETVLIPHI(const VSETVLIInfo &Require,
   }
 
   // If all the incoming values to the PHI checked out, we don't need
   // to insert a VSETVLI.
   return false;
 }
 
 void RISCVInsertVSETVLI::emitVSETVLIs(MachineBasicBlock &MBB) {
-  VSETVLIInfo CurInfo;
-  // Only be set if current VSETVLIInfo is from an explicit VSET(I)VLI.
+  BlockData &BBInfo = BlockInfo[MBB.getNumber()];
+  VSETVLIInfo CurInfo = BBInfo.Pred;
+
+  // Only set if current VSETVLIInfo is from an explicit VSET(I)VLI.
   MachineInstr *PrevVSETVLIMI = nullptr;
 
   for (MachineInstr &MI : MBB) {
     // If this is an explicit VSETVLI or VSETIVLI, update our state.
     if (isVectorConfigInstr(MI)) {
       // Conservatively, mark the VL and VTYPE as live.
       assert(MI.getOperand(3).getReg() == RISCV::VL &&
              MI.getOperand(4).getReg() == RISCV::VTYPE &&
if (RISCVII::hasSEWOp(TSFlags)) {
       MI.addOperand(MachineOperand::CreateReg(RISCV::VTYPE, /*isDef*/ false,
                                               /*isImp*/ true));
 
       if (!CurInfo.isValid()) {
         // We haven't found any vector instructions or VL/VTYPE changes yet,
         // use the predecessor information.
         assert(BlockInfo[MBB.getNumber()].Pred.isValid() &&
                "Expected a valid predecessor state.");
-        if (needVSETVLI(NewInfo, BlockInfo[MBB.getNumber()].Pred) &&
-            needVSETVLIPHI(NewInfo, MBB)) {
+        if (needVSETVLI(NewInfo, BlockInfo[MBB.getNumber()].Pred)) {
+          if (needVSETVLIPHI(NewInfo, MBB)) {
             insertVSETVLI(MBB, MI, NewInfo, BlockInfo[MBB.getNumber()].Pred);
+          } else {
+            // IF this is the first implicit state change, and the state change
+            // requested can be proven to produce the same register contents, we
+            // can skip emitting the actual state change and continue as if we
+            // had since we know the GPR result of the implicit state change
+            // wouldn't be used and VL/VTYPE registers are correct.  Note that
+            // we *do* need to model the state as if it changed as while the
+            // register contents are unchanged, the abstract model can change.
+          }
+          PrevVSETVLIMI = nullptr;
           CurInfo = NewInfo;
         }
       } else {
         // If this instruction isn't compatible with the previous VL/VTYPE
         // we need to insert a VSETVLI.
-        // If this is a unit-stride or strided load/store, we may be able to use
-        // the EMUL=(EEW/SEW)*LMUL relationship to avoid changing vtype.
         // NOTE: We can't use predecessor information for the store. We must
         // treat it the same as the first phase so that we produce the correct
-        // vl/vtype for succesor blocks.
-        if (!canSkipVSETVLIForLoadStore(MI, NewInfo, CurInfo) &&
-            needVSETVLI(NewInfo, CurInfo)) {
-          // If the previous VL/VTYPE is set by VSETVLI and do not use, Merge it
-          // with current VL/VTYPE.
-          bool NeedInsertVSETVLI = true;
-          if (PrevVSETVLIMI) {
-            bool HasSameAVL =
-                CurInfo.hasSameAVL(NewInfo) ||
-                (NewInfo.hasAVLReg() && NewInfo.getAVLReg().isVirtual() &&
-                 NewInfo.getAVLReg() == PrevVSETVLIMI->getOperand(0).getReg());
-            // If these two VSETVLI have the same AVL and the same VLMAX,
-            // we could merge these two VSETVLI.
-            if (HasSameAVL &&
-                CurInfo.getSEWLMULRatio() == NewInfo.getSEWLMULRatio()) {
-              PrevVSETVLIMI->getOperand(2).setImm(NewInfo.encodeVTYPE());
-              NeedInsertVSETVLI = false;
-            }
-            if (isScalarMoveInstr(MI) &&
-                ((CurInfo.hasNonZeroAVL() && NewInfo.hasNonZeroAVL()) ||
-                 (CurInfo.hasZeroAVL() && NewInfo.hasZeroAVL())) &&
-                NewInfo.hasSameVLMAX(CurInfo)) {
-              PrevVSETVLIMI->getOperand(2).setImm(NewInfo.encodeVTYPE());
-              NeedInsertVSETVLI = false;
-            }
-          }
-          if (NeedInsertVSETVLI)
+        // vl/vtype for successor blocks.
+        auto Kind = needVSETVLI(MI, NewInfo, CurInfo, PrevVSETVLIMI);
+        if (Kind != VSETVLIInsertionKind::None) {
+          if (Kind == VSETVLIInsertionKind::Required)
             insertVSETVLI(MBB, MI, NewInfo, CurInfo);
+          else if (Kind == VSETVLIInsertionKind::MutatePrevious)
+            PrevVSETVLIMI->getOperand(2).setImm(NewInfo.encodeVTYPE());
           CurInfo = NewInfo;
         }
       }
       PrevVSETVLIMI = nullptr;
     }
 
     // If this is something updates VL/VTYPE that we don't know about, set
     // the state to unknown.

llvm/test/CodeGen/RISCV/rvv/vsetvli-insert-crossbb.ll

 
 define <vscale x 1 x double> @test3(i64 %avl, i8 zeroext %cond, <vscale x 1 x double> %a, <vscale x 1 x double> %b) nounwind {
 ; CHECK-LABEL: test3:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    beqz a1, .LBB2_2
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ; CHECK-NEXT:    vsetvli a0, a0, e64, m1, ta, mu
 ; CHECK-NEXT:    vfadd.vv v9, v8, v9
-; CHECK-NEXT:    vfmul.vv v8, v9, v8
-; CHECK-NEXT:    ret
+; CHECK-NEXT:    j .LBB2_3
 ; CHECK-NEXT:  .LBB2_2: # %if.else
 ; CHECK-NEXT:    vsetvli a0, a0, e64, m1, ta, mu
 ; CHECK-NEXT:    vfsub.vv v9, v8, v9
+; CHECK-NEXT:  .LBB2_3: # %if.end
+; CHECK-NEXT:    vsetvli zero, a0, e64, m1, ta, mu
 ; CHECK-NEXT:    vfmul.vv v8, v9, v8
 ; CHECK-NEXT:    ret
 entry:
   %tobool = icmp eq i8 %cond, 0
   br i1 %tobool, label %if.else, label %if.then
 
 if.then:                                          ; preds = %entry
   %0 = tail call i64 @llvm.riscv.vsetvli(i64 %avl, i64 3, i64 0)

 
 define void @saxpy_vec(i64 %n, float %a, float* nocapture readonly %x, float* nocapture %y) {
 ; CHECK-LABEL: saxpy_vec:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    vsetvli a3, a0, e32, m8, ta, mu
 ; CHECK-NEXT:    beqz a3, .LBB8_2
 ; CHECK-NEXT:  .LBB8_1: # %for.body
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
+; CHECK-NEXT:    vsetvli zero, a3, e32, m8, ta, mu
 ; CHECK-NEXT:    vle32.v v8, (a1)
 ; CHECK-NEXT:    vle32.v v16, (a2)
 ; CHECK-NEXT:    slli a4, a3, 2
 ; CHECK-NEXT:    add a1, a1, a4
-; CHECK-NEXT:    vsetvli zero, a3, e32, m8, tu, mu
+; CHECK-NEXT:    vsetvli zero, zero, e32, m8, tu, mu
 ; CHECK-NEXT:    vfmacc.vf v16, fa0, v8
 ; CHECK-NEXT:    vse32.v v16, (a2)
 ; CHECK-NEXT:    sub a0, a0, a3
 ; CHECK-NEXT:    vsetvli a3, a0, e32, m8, ta, mu
 ; CHECK-NEXT:    add a2, a2, a4
 ; CHECK-NEXT:    bnez a3, .LBB8_1
 ; CHECK-NEXT:  .LBB8_2: # %for.end
 ; CHECK-NEXT:    ret

 define <vscale x 2 x i32> @test_vsetvli_x0_x0(<vscale x 2 x i32>* %x, <vscale x 2 x i16>* %y, <vscale x 2 x i32> %z, i64 %vl, i1 %cond) nounwind {
 ; CHECK-LABEL: test_vsetvli_x0_x0:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    vsetvli zero, a2, e32, m1, ta, mu
 ; CHECK-NEXT:    vle32.v v9, (a0)
 ; CHECK-NEXT:    andi a0, a3, 1
 ; CHECK-NEXT:    beqz a0, .LBB9_2
 ; CHECK-NEXT:  # %bb.1: # %if
-; CHECK-NEXT:    vsetvli zero, zero, e16, mf2, ta, mu
 ; CHECK-NEXT:    vle16.v v10, (a1)
+; CHECK-NEXT:    vsetvli zero, zero, e16, mf2, ta, mu
 ; CHECK-NEXT:    vwcvt.x.x.v v8, v10
 ; CHECK-NEXT:  .LBB9_2: # %if.end
 ; CHECK-NEXT:    vsetvli zero, zero, e32, m1, ta, mu
 ; CHECK-NEXT:    vadd.vv v8, v9, v8
 ; CHECK-NEXT:    ret
 entry:
   %a = call <vscale x 2 x i32> @llvm.riscv.vle.nxv2i32(<vscale x 2 x i32> undef, <vscale x 2 x i32>* %x, i64 %vl)
   br i1 %cond, label %if, label %if.end

 define <vscale x 2 x i32> @test_vsetvli_x0_x0_2(<vscale x 2 x i32>* %x, <vscale x 2 x i16>* %y, <vscale x 2 x i16>* %z, i64 %vl, i1 %cond, i1 %cond2, <vscale x 2 x i32> %w) nounwind {
 ; CHECK-LABEL: test_vsetvli_x0_x0_2:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    vsetvli zero, a3, e32, m1, ta, mu
 ; CHECK-NEXT:    vle32.v v9, (a0)
 ; CHECK-NEXT:    andi a0, a4, 1
 ; CHECK-NEXT:    beqz a0, .LBB10_2
 ; CHECK-NEXT:  # %bb.1: # %if
-; CHECK-NEXT:    vsetvli zero, zero, e16, mf2, ta, mu
 ; CHECK-NEXT:    vle16.v v10, (a1)
+; CHECK-NEXT:    vsetvli zero, zero, e16, mf2, ta, mu
 ; CHECK-NEXT:    vwadd.wv v9, v9, v10
 ; CHECK-NEXT:  .LBB10_2: # %if.end
 ; CHECK-NEXT:    andi a0, a5, 1
 ; CHECK-NEXT:    beqz a0, .LBB10_4
 ; CHECK-NEXT:  # %bb.3: # %if2
 ; CHECK-NEXT:    vsetvli zero, zero, e16, mf2, ta, mu
 ; CHECK-NEXT:    vle16.v v10, (a2)
 ; CHECK-NEXT:    vwadd.wv v9, v9, v10

llvm/test/CodeGen/RISCV/rvv/vsetvli-insert-crossbb.mir

body:             |
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1:
   ; CHECK-NEXT:   successors: %bb.1(0x40000000), %bb.2(0x40000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[PHI:%[0-9]+]]:gpr = PHI [[COPY2]], %bb.0, %10, %bb.1
   ; CHECK-NEXT:   [[PseudoVADD_VX_M1_:%[0-9]+]]:vr = PseudoVADD_VX_M1 [[PseudoVID_V_M1_]], [[PHI]], -1, 6 /* e64 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[MUL:%[0-9]+]]:gpr = MUL [[PHI]], [[SRLI]]
   ; CHECK-NEXT:   [[ADD:%[0-9]+]]:gpr = ADD [[COPY]], [[MUL]]
-  ; CHECK-NEXT:   dead $x0 = PseudoVSETVLIX0 killed $x0, 87 /* e32, mf2, ta, mu */, implicit-def $vl, implicit-def $vtype, implicit $vl
   ; CHECK-NEXT:   PseudoVSE32_V_MF2 killed [[PseudoVADD_VX_M1_]], killed [[ADD]], -1, 5 /* e32 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[ADDI:%[0-9]+]]:gpr = ADDI [[PHI]], 1
-  ; CHECK-NEXT:   dead $x0 = PseudoVSETVLIX0 killed $x0, 88 /* e64, m1, ta, mu */, implicit-def $vl, implicit-def $vtype, implicit $vl
   ; CHECK-NEXT:   BLTU [[ADDI]], [[COPY1]], %bb.1
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2:
   ; CHECK-NEXT:   PseudoRET
   bb.0:
     liveins: $x10, $x11
     %0:gpr = COPY $x10
bb.2.middle.block:
     %20:vr = PseudoVMV_S_X_M1 %21, %19, 1, 5
     %24:vr = IMPLICIT_DEF
     %23:vr = PseudoVREDSUM_VS_M1 %24, %16, killed %20, 4, 5
     PseudoVSE32_V_M1 killed %23, %8, 1, 5 :: (store (s32) into %ir.res)
     PseudoRET
 
 ...
 ---
-# FIXME: This test shows incorrect VSETVLI insertion. The VLUXEI64 needs
-# configuration for SEW=8 but it instead inherits a SEW=64 from the entry
-# block.
 name:            vsetvli_vluxei64_regression
 tracksRegLiveness: true
 body:             |
   ; CHECK-LABEL: name: vsetvli_vluxei64_regression
   ; CHECK: bb.0:
   ; CHECK-NEXT:   successors: %bb.1(0x80000000)
   ; CHECK-NEXT:   liveins: $x10, $x11, $x12, $v0, $v1, $v2, $v3
   ; CHECK-NEXT: {{  $}}
body:             |
   ; CHECK-NEXT:   %t2:gpr = COPY $x0
   ; CHECK-NEXT:   BEQ %a, %t2, %bb.3
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2:
   ; CHECK-NEXT:   successors: %bb.3(0x80000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   $v0 = COPY %mask
+  ; CHECK-NEXT:   dead $x0 = PseudoVSETVLIX0 killed $x0, 69 /* e8, mf8, ta, mu */, implicit-def $vl, implicit-def $vtype, implicit $vl
   ; CHECK-NEXT:   early-clobber %t0:vrnov0 = PseudoVLUXEI64_V_M1_MF8_MASK %t5, killed %inaddr, %idxs, $v0, -1, 3 /* e8 */, 1, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   %ldval:vr = COPY %t0
   ; CHECK-NEXT:   PseudoBR %bb.3
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3:
   ; CHECK-NEXT:   %stval:vr = PHI %t4, %bb.1, %ldval, %bb.2
   ; CHECK-NEXT:   $v0 = COPY %mask
+  ; CHECK-NEXT:   dead $x0 = PseudoVSETVLIX0 killed $x0, 69 /* e8, mf8, ta, mu */, implicit-def $vl, implicit-def $vtype, implicit $vl
   ; CHECK-NEXT:   PseudoVSOXEI64_V_M1_MF8_MASK killed %stval, killed %b, %idxs, $v0, -1, 3 /* e8 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   PseudoRET
   bb.0:
     successors: %bb.1
     liveins: $x10, $x11, $x12, $v0, $v1, $v2, $v3
 
     %a:gpr = COPY $x10
     %b:gpr = COPY $x11
body:             |
   ; CHECK-NEXT:   [[COPY3:%[0-9]+]]:gpr = COPY $x0
   ; CHECK-NEXT:   PseudoBR %bb.1
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1:
   ; CHECK-NEXT:   successors: %bb.2(0x40000000), %bb.3(0x40000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[PHI:%[0-9]+]]:gpr = PHI [[COPY3]], %bb.0, %11, %bb.3
   ; CHECK-NEXT:   [[ADD:%[0-9]+]]:gpr = ADD [[COPY2]], [[PHI]]
   ; CHECK-NEXT:   dead $x0 = PseudoVSETVLIX0 killed $x0, 88 /* e64, m1, ta, mu */, implicit-def $vl, implicit-def $vtype, implicit $vl
   ; CHECK-NEXT:   [[PseudoVADD_VX_M1_:%[0-9]+]]:vr = PseudoVADD_VX_M1 [[PseudoVID_V_M1_]], killed [[ADD]], -1, 6 /* e64 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[PseudoVMSLTU_VX_M1_:%[0-9]+]]:vr = PseudoVMSLTU_VX_M1 [[PseudoVADD_VX_M1_]], [[COPY1]], -1, 6 /* e64 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[PseudoVCPOP_M_B1_:%[0-9]+]]:gpr = PseudoVCPOP_M_B1 [[PseudoVMSLTU_VX_M1_]], -1, 0 /* e8 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[COPY4:%[0-9]+]]:gpr = COPY $x0
   ; CHECK-NEXT:   BEQ killed [[PseudoVCPOP_M_B1_]], [[COPY4]], %bb.3
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2:
   ; CHECK-NEXT:   successors: %bb.3(0x80000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[ADD1:%[0-9]+]]:gpr = ADD %src, [[PHI]]
-  ; CHECK-NEXT:   dead $x0 = PseudoVSETVLIX0 killed $x0, 69 /* e8, mf8, ta, mu */, implicit-def $vl, implicit-def $vtype, implicit $vl
   ; CHECK-NEXT:   [[PseudoVLE8_V_MF8_:%[0-9]+]]:vrnov0 = PseudoVLE8_V_MF8 killed [[ADD1]], -1, 3 /* e8 */, implicit $vl, implicit $vtype
+  ; CHECK-NEXT:   dead $x0 = PseudoVSETVLIX0 killed $x0, 69 /* e8, mf8, ta, mu */, implicit-def $vl, implicit-def $vtype, implicit $vl
   ; CHECK-NEXT:   [[PseudoVADD_VI_MF8_:%[0-9]+]]:vrnov0 = PseudoVADD_VI_MF8 [[PseudoVLE8_V_MF8_]], 4, -1, 3 /* e8 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[ADD2:%[0-9]+]]:gpr = ADD %dst, [[PHI]]
   ; CHECK-NEXT:   PseudoVSE8_V_MF8 killed [[PseudoVADD_VI_MF8_]], killed [[ADD2]], -1, 3 /* e8 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3:
   ; CHECK-NEXT:   successors: %bb.1(0x7c000000), %bb.4(0x04000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[ADD3:%[0-9]+]]:gpr = ADD [[PHI]], %inc

llvm/test/CodeGen/RISCV/rvv/vsetvli-insert.ll

 }
 declare <vscale x 1 x i1> @llvm.riscv.vmseq.nxv1i64.i64(<vscale x 1 x i64>, <vscale x 1 x i64>, i64)
 declare <vscale x 1 x i1> @llvm.riscv.vmand.nxv1i1.i64(<vscale x 1 x i1>, <vscale x 1 x i1>, i64)
 
 ; Make sure we don't insert a vsetvli for the vmor instruction.
 define void @test6(i32* nocapture readonly %A, i32* nocapture %B, i64 %n) {
 ; CHECK-LABEL: test6:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    vsetvli a6, a2, e32, m1, ta, mu
-; CHECK-NEXT:    beqz a6, .LBB5_3
+; CHECK-NEXT:    vsetvli a3, a2, e32, m1, ta, mu
+; CHECK-NEXT:    beqz a3, .LBB5_3
 ; CHECK-NEXT:  # %bb.1: # %for.body.preheader
 ; CHECK-NEXT:    li a4, 0
 ; CHECK-NEXT:  .LBB5_2: # %for.body
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    slli a5, a4, 2
-; CHECK-NEXT:    add a3, a0, a5
-; CHECK-NEXT:    vle32.v v8, (a3)
+; CHECK-NEXT:    add a6, a0, a5
+; CHECK-NEXT:    vsetvli zero, a3, e32, m1, ta, mu
+; CHECK-NEXT:    vle32.v v8, (a6)
 ; CHECK-NEXT:    vmsle.vi v9, v8, -3
 ; CHECK-NEXT:    vmsgt.vi v10, v8, 2
 ; CHECK-NEXT:    vmor.mm v0, v9, v10
-; CHECK-NEXT:    add a3, a1, a5
-; CHECK-NEXT:    vse32.v v8, (a3), v0.t
-; CHECK-NEXT:    add a4, a4, a6
-; CHECK-NEXT:    vsetvli a6, a2, e32, m1, ta, mu
-; CHECK-NEXT:    bnez a6, .LBB5_2
+; CHECK-NEXT:    add a5, a5, a1
+; CHECK-NEXT:    vse32.v v8, (a5), v0.t
+; CHECK-NEXT:    add a4, a4, a3
+; CHECK-NEXT:    vsetvli a3, a2, e32, m1, ta, mu
+; CHECK-NEXT:    bnez a3, .LBB5_2
 ; CHECK-NEXT:  .LBB5_3: # %for.cond.cleanup
 ; CHECK-NEXT:    ret
 entry:
   %0 = tail call i64 @llvm.riscv.vsetvli.i64(i64 %n, i64 2, i64 0)
   %cmp.not11 = icmp eq i64 %0, 0
   br i1 %cmp.not11, label %for.cond.cleanup, label %for.body
 
 for.cond.cleanup:                                 ; preds = %for.body, %entry