[RISCV][NFC] Replace the pseudos for instructions that depend on lmul with variants that encode the SEW into the name

[nitinjohnraj]

Summary

• vcompress
• vrgather.vv/vrgatherei16.vv (vrgather.vi/vx are fine)
• vdiv/vdivu/vfdiv
• vsqr

These instructions depend on vl, so we will create pseudos for these instructions that depend on SEW, since VLUpperBound = LMUL * VLEN / SEW. This will, in the future, help us reason about them more finely in the scheduler model.

These instructions also depend on vl, and will be dealt with in the next revision:

• strided and indexed load/stores.
• segment load/stores
• v(w)(f)red*

[michaelmaitland]

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

1919

If we're going to use suffix, should we use it in WriteVCompressV_MX_E and ReadVCompressV_MX_E too?

2041–2061

WriteVGatherV_MX_E and ReadVGatherV_MX_E dont depend on sew from EEWList. Can we pull out the loop over sews from SchedSEWSet<mx>.val to simplify?

2337

Use suffix in WriteVFSqrtV_MX and ReadVFSqrtV_MX. Also need to add _E to WriteVFSqrtV_MX and ReadVFSqrtV_MX

2350

Where did you land on making this SEW aware?

2468

nit: define sews closer to where you use it?

2472

Why did we change ReadVGatherV to ReadVGatherI?

[michaelmaitland]

llvm/lib/Target/RISCV/RISCVScheduleV.td

41

Why don't we make this case SEW aware?

79

Why don't we make this case SEW aware?

114

Why don't we make this case SEW aware?

157

Why don't we make this case SEW aware?

[frasercrmck]

Just a glancing review but, to me, "expand pseudos" has a more immediate connotation of the many pseudo expansion passes (e.g., RISCVExpandPseudoInsts) than "expanding the set of pseudos" as you seem to intend it.

[nitinjohnraj]

In D144925#4157772, @frasercrmck wrote:

Just a glancing review but, to me, "expand pseudos" has a more immediate connotation of the many pseudo expansion passes (e.g., RISCVExpandPseudoInsts) than "expanding the set of pseudos" as you seem to intend it.

Thanks for the comment, that makes sense. :)

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

1919

Good point, yeah.

2041–2061

Thanks for the review! We can certainly do that, but does it really simplify the code? It's a nested loop in either case, so I'm not clear which version performs better/looks simpler.

2337

Thanks, will do!

2468

Appreciated, I agree with that nit, seems better for clarity.

2472

After discussion with @craig.topper, we did this because otherwise we'd need to maintain both SEW-aware and SEW-unaware versions of ReadVGatherV:

1. The SEW-aware version on line 2481 for the _VV binary, which is SEW-aware itself
2. The SEW-unaware version for the _VI binary, which is not SEW aware

This change helps us generate only the SEW-aware ReadVGatherV SchedReads. I'd be glad to talk about this more if you think we should approach this differently though.

2476

I feel particularly uneasy about this change. I could leave it as is, or I could create ReadVGatherX_MX with no SEW in RISCVScheduleV.td.

llvm/lib/Target/RISCV/RISCVScheduleV.td

41

If I remember correctly, that was causing some other files to complain and I put this in as

[craig.topper]

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

2472

We should move this part to a separate patch. vrgather.vi, vrgather.vx, vrgather.vv are all sort of different.

Let's consider a uarch that splits a LMUL>1 into multiple subvectors acrosss multiple uops:
vrgather.vi knows at decode time which subvector of the full source is needed for the destination subvectors
vrgather.vx doesn't know until the scalar instruction executes which subvector of the full source is needed. But only one subvector will be needed.
vrgather.vv every subvector of the destination can be dependent on multiple subvectors of the source.

[michaelmaitland]

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

2041–2061

You are right. Good as you have it :)

llvm/lib/Target/RISCV/RISCVScheduleV.td

41

I think this should be sew aware here and below and we need to fix the other files that start complaining.

[nitinjohnraj]

Made the SchedReads and SchedWrites sew-aware

[craig.topper]

A lot of the content from your original patch is no longer showing in this review. For example, SchedSEWSet is no longer created in this diff.

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

2350

I think the answer is no.

[nitinjohnraj]

Created a diff from a commit range this time

[michaelmaitland]

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

2337

defvar WriteVFSqrtV_MX_E and defvar WriteVFSqrtV_MX_E

2350

If this does not need to be SEW aware, should we remove the TODO?

[nitinjohnraj]

Diff updated to add another commit that was missed last time (Sorry, never used commit ranges with arcanist before.)

[nitinjohnraj]

Refactoring and removing comments

[nitinjohnraj]

llvm/lib/Target/RISCV/RISCVScheduleV.td

114

Did we want to make UpperBound SEW-aware, or handle it in a different way? IIRC we discussed using it as a defined variable?

167

Same as line 62

176

Same as line 62

186

Same as line 62

[michaelmaitland]

llvm/lib/Target/RISCV/RISCVScheduleV.td

114

I think the best solution for now is to use the largest LMUL and the largest SEW for that given LMUL as the UpperBound SchedWrite. This assumes that the largest LMUL and SEW has the largest latency and uses the most resource cycles. This assumption may not always true, but I think is a good starting point.

[craig.topper]

llvm/lib/Target/RISCV/RISCVScheduleV.td

114

If the worst case is the number of elements rather than the size of elements you'd want to use smallest sew.

[nitinjohnraj]

Attempting to fix the diff

[michaelmaitland]

LGTM. As we discussed offline, today UpperBound refers to the largest LMUL supported. This should be changed in a future patch. I propose that it should be removed from MxList and its meaning should be tied the SchedWrite with the worst case behaviour. Now that we are supporting SchedWrite by LMUL + SEW or just LMUL depending on the instruction, the specific scheduler model can decide which LMUL or LMUL + SEW corresponds to the worst case behaviour. This means no assumption is made on whether the Largest LMUL or smallest SEW behaves the worst.

[michaelmaitland]

Is this still an open issue? I think you may have committed all the parts.

[michaelmaitland]

These changes have been committed.

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

 defvar TA_MA = 3;
 
 //===----------------------------------------------------------------------===//
 // Utilities.
 //===----------------------------------------------------------------------===//
 
 class PseudoToVInst<string PseudoInst> {
   defvar AffixSubsts = [["Pseudo", ""],
+                        ["_E64", ""],
+                        ["_E32", ""],
+                        ["_E16", ""],
+                        ["_E8", ""],
                         ["_F64", "_F"],
                         ["_F32", "_F"],
                         ["_F16", "_F"],
                         ["_VF64", "_VF"],
                         ["_VF32", "_VF"],
                         ["_VF16", "_VF"],
                         ["_WF64", "_WF"],
                         ["_WF32", "_WF"],
let VLMul = m.value in {
                                  Sched<[WriteVMIotV_MX, ReadVMIotV_MX, ReadVMask]>;
     }
   }
 }
 
 multiclass VPseudoVCPR_V {
   foreach m = MxList in {
     defvar mx = m.MX;
-    defvar WriteVCompressV_MX = !cast<SchedWrite>("WriteVCompressV_" # mx);
-    defvar ReadVCompressV_MX = !cast<SchedRead>("ReadVCompressV_" # mx);
-
+    defvar sews = SchedSEWSet<mx>.val;
     let VLMul = m.value in
-      def _VM # "_" # m.MX : VPseudoUnaryAnyMask<m.vrclass, m.vrclass>,
-                             Sched<[WriteVCompressV_MX, ReadVCompressV_MX, ReadVCompressV_MX]>;
+      foreach e = sews in {
+        defvar WriteVCompressV_MX_E = !cast<SchedWrite>("WriteVCompressV_" # mx # "_E" # e);
+        defvar ReadVCompressV_MX_E = !cast<SchedRead>("ReadVCompressV_" # mx # "_E" # e);
+
+        defvar suffix = "_" # m.MX # "_E" # e;
+        def _VM # suffix : VPseudoUnaryAnyMask<m.vrclass, m.vrclass>,
+                           Sched<[WriteVCompressV_MX_E, ReadVCompressV_MX_E, ReadVCompressV_MX_E]>;
+      }
   }
 }
 
 multiclass VPseudoBinary<VReg RetClass,
                          VReg Op1Class,
                          DAGOperand Op2Class,
                          LMULInfo MInfo,
                          string Constraint = ""> {
   let VLMul = MInfo.value in {
     def "_" # MInfo.MX : VPseudoBinaryNoMask<RetClass, Op1Class, Op2Class,
                                              Constraint>;
     def "_" # MInfo.MX # "_TU" : VPseudoBinaryNoMaskTU<RetClass, Op1Class, Op2Class,
                                                        Constraint>;
     def "_" # MInfo.MX # "_MASK" : VPseudoBinaryMaskPolicy<RetClass, Op1Class, Op2Class,
                                                            Constraint>,
                                    RISCVMaskedPseudo</*MaskOpIdx*/ 3>;
   }
 }
 
+multiclass VPseudoBinary_E<VReg RetClass,
+                           VReg Op1Class,
+                           DAGOperand Op2Class,
+                           LMULInfo MInfo,
+                           int sew,
+                           string Constraint = ""> {
+  let VLMul = MInfo.value in {
+    defvar suffix = "_" # MInfo.MX # "_E" # sew;
+    def suffix : VPseudoBinaryNoMask<RetClass, Op1Class, Op2Class,
+                                     Constraint>;
+    def suffix # "_TU" : VPseudoBinaryNoMaskTU<RetClass, Op1Class, Op2Class,
+                                               Constraint>;
+    def suffix # "_MASK" : VPseudoBinaryMaskPolicy<RetClass, Op1Class, Op2Class,
+                                                   Constraint>,
+                           RISCVMaskedPseudo</*MaskOpIdx*/ 3>;
+  }
+}
+
 multiclass VPseudoBinaryM<VReg RetClass,
                           VReg Op1Class,
                           DAGOperand Op2Class,
                           LMULInfo MInfo,
                           string Constraint = ""> {
   let VLMul = MInfo.value in {
     def "_" # MInfo.MX : VPseudoBinaryNoMask<RetClass, Op1Class, Op2Class,
                                              Constraint>;
let VLMul = lmul.value in {
     def "_" # lmul.MX # "_" # emul.MX # "_TU": VPseudoBinaryNoMaskTU<RetClass, Op1Class, Op2Class,
                                                                      Constraint>;
     def "_" # lmul.MX # "_" # emul.MX # "_MASK" : VPseudoBinaryMaskPolicy<RetClass, Op1Class, Op2Class,
                                                                           Constraint>,
                                                   RISCVMaskedPseudo</*MaskOpIdx*/ 3>;
   }
 }
 
+multiclass VPseudoBinaryEmul_E<VReg RetClass,
+                               VReg Op1Class,
+                               DAGOperand Op2Class,
+                               LMULInfo lmul,
+                               int sew,
+                               LMULInfo emul,
+                               string Constraint = ""> {
+  let VLMul = lmul.value in {
+    defvar suffix = "_" # lmul.MX # "_E" # sew # "_" # emul.MX;
+    def suffix : VPseudoBinaryNoMask<RetClass, Op1Class, Op2Class,
+                                     Constraint>;
+    def suffix # "_TU" : VPseudoBinaryNoMaskTU<RetClass, Op1Class, Op2Class,
+                                               Constraint>;
+    def suffix # "_MASK" : VPseudoBinaryMaskPolicy<RetClass, Op1Class, Op2Class,
+                                                   Constraint>,
+                           RISCVMaskedPseudo</*MaskOpIdx*/ 3>;
+  }
+}
+
 multiclass VPseudoTiedBinary<VReg RetClass,
                              DAGOperand Op2Class,
                              LMULInfo MInfo,
                              string Constraint = ""> {
   let VLMul = MInfo.value in {
     def "_" # MInfo.MX # "_TIED": VPseudoTiedBinaryNoMask<RetClass, Op2Class,
                                                           Constraint>;
     def "_" # MInfo.MX # "_MASK_TIED" : VPseudoTiedBinaryMask<RetClass, Op2Class,
                                                          Constraint>;
   }
 }
 
 multiclass VPseudoBinaryV_VV<LMULInfo m, string Constraint = ""> {
   defm _VV : VPseudoBinary<m.vrclass, m.vrclass, m.vrclass, m, Constraint>;
 }
 
+multiclass VPseudoBinaryV_VV_E<LMULInfo m, int sew, string Constraint = ""> {
+  defm _VV : VPseudoBinary_E<m.vrclass, m.vrclass, m.vrclass, m, sew, Constraint>;
+}
+
 // Similar to VPseudoBinaryV_VV, but uses MxListF.
 multiclass VPseudoBinaryFV_VV<LMULInfo m, string Constraint = ""> {
   defm _VV : VPseudoBinary<m.vrclass, m.vrclass, m.vrclass, m, Constraint>;
 }
 
+multiclass VPseudoBinaryFV_VV_E<LMULInfo m, int sew, string Constraint = ""> {
+  defm _VV : VPseudoBinary_E<m.vrclass, m.vrclass, m.vrclass, m, sew, Constraint>;
+}
+
 multiclass VPseudoVGTR_VV_EEW<int eew, string Constraint = ""> {
   foreach m = MxList in {
     defvar mx = m.MX;
-    defvar WriteVGatherV_MX = !cast<SchedWrite>("WriteVGatherV_" # mx);
-    defvar ReadVGatherV_MX = !cast<SchedRead>("ReadVGatherV_" # mx);
 
     foreach sew = EEWList in {
       defvar octuple_lmul = m.octuple;
       // emul = lmul * eew / sew
       defvar octuple_emul = !srl(!mul(octuple_lmul, eew), log2<sew>.val);
       if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then {
         defvar emulMX = octuple_to_str<octuple_emul>.ret;
         defvar emul = !cast<LMULInfo>("V_" # emulMX);
-        defm _VV : VPseudoBinaryEmul<m.vrclass, m.vrclass, emul.vrclass, m, emul, Constraint>,
-                   Sched<[WriteVGatherV_MX, ReadVGatherV_MX, ReadVGatherV_MX]>;
+        defvar sews = SchedSEWSet<mx>.val;
+        foreach e = sews in {
+          defvar WriteVGatherV_MX_E = !cast<SchedWrite>("WriteVGatherV_" # mx # "_E" # e);
+          defvar ReadVGatherV_MX_E = !cast<SchedRead>("ReadVGatherV_" # mx # "_E" # e);
+          defm _VV : VPseudoBinaryEmul_E<m.vrclass, m.vrclass, emul.vrclass, m, e, emul, Constraint>,
+                     Sched<[WriteVGatherV_MX_E, ReadVGatherV_MX_E, ReadVGatherV_MX_E]>;
+        }
       }
     }
   }
 }
 
 multiclass VPseudoBinaryV_VX<LMULInfo m, string Constraint = ""> {
   defm "_VX" : VPseudoBinary<m.vrclass, m.vrclass, GPR, m, Constraint>;
 }
 
+multiclass VPseudoBinaryV_VX_E<LMULInfo m, int sew, string Constraint = ""> {
+  defm "_VX" : VPseudoBinary_E<m.vrclass, m.vrclass, GPR, m, sew, Constraint>;
+}
+
 multiclass VPseudoVSLD1_VX<string Constraint = ""> {
   foreach m = MxList in {
     defvar mx = m.MX;
     defvar WriteVISlide1X_MX = !cast<SchedWrite>("WriteVISlide1X_" # mx);
     defvar ReadVISlideV_MX = !cast<SchedRead>("ReadVISlideV_" # mx);
     defvar ReadVISlideX_MX = !cast<SchedRead>("ReadVISlideX_" # mx);
 
     defm "_VX" : VPseudoBinary<m.vrclass, m.vrclass, GPR, m, Constraint>,
                  Sched<[WriteVISlide1X_MX, ReadVISlideV_MX, ReadVISlideX_MX, ReadVMask]>;
   }
 }
 
 multiclass VPseudoBinaryV_VF<LMULInfo m, FPR_Info f, string Constraint = ""> {
   defm "_V" # f.FX : VPseudoBinary<m.vrclass, m.vrclass,
                                    f.fprclass, m, Constraint>;
 }
 
+multiclass VPseudoBinaryV_VF_E<LMULInfo m, int sew, FPR_Info f,
+                               string Constraint = ""> {
+  defm "_V" # f.FX : VPseudoBinary_E<m.vrclass, m.vrclass,
+                                     f.fprclass, m, sew, Constraint>;
+}
+
 multiclass VPseudoVSLD1_VF<string Constraint = ""> {
   foreach f = FPList in {
     foreach m = f.MxList in {
       defvar mx = m.MX;
       defvar WriteVFSlide1F_MX = !cast<SchedWrite>("WriteVFSlide1F_" # mx);
       defvar ReadVFSlideV_MX = !cast<SchedRead>("ReadVFSlideV_" # mx);
       defvar ReadVFSlideF_MX = !cast<SchedRead>("ReadVFSlideF_" # mx);
 
let VLMul = m.value in {
                                  Sched<[WriteVFClassV_MX, ReadVFClassV_MX, ReadVMask]>;
     }
   }
 }
 
 multiclass VPseudoVSQR_V {
   foreach m = MxListF in {
     defvar mx = m.MX;
-    defvar WriteVFSqrtV_MX = !cast<SchedWrite>("WriteVFSqrtV_" # mx);
-    defvar ReadVFSqrtV_MX = !cast<SchedRead>("ReadVFSqrtV_" # mx);
+    defvar sews = SchedSEWSet<m.MX>.val;
 
-    let VLMul = m.value in {
-      def "_V_" # mx : VPseudoUnaryNoMask<m.vrclass, m.vrclass>,
+    let VLMul = m.value in
+      foreach e = sews in {
+        defvar WriteVFSqrtV_MX = !cast<SchedWrite>("WriteVFSqrtV_" # mx # "_E" # e);
+        defvar ReadVFSqrtV_MX = !cast<SchedRead>("ReadVFSqrtV_" # mx # "_E" # e);
+        defvar suffix = mx # "_E" # e;
+
+        def "_V_" # suffix : VPseudoUnaryNoMask<m.vrclass, m.vrclass>,
                               Sched<[WriteVFSqrtV_MX, ReadVFSqrtV_MX, ReadVMask]>;
-      def "_V_" # mx # "_TU": VPseudoUnaryNoMaskTU<m.vrclass, m.vrclass>,
+        def "_V_" # suffix # "_TU": VPseudoUnaryNoMaskTU<m.vrclass, m.vrclass>,
                                     Sched<[WriteVFSqrtV_MX, ReadVFSqrtV_MX, ReadVMask]>;
-      def "_V_" # mx # "_MASK" : VPseudoUnaryMaskTA<m.vrclass, m.vrclass>,
+        def "_V_" # suffix # "_MASK" : VPseudoUnaryMaskTA<m.vrclass, m.vrclass>,
                                        RISCVMaskedPseudo</*MaskOpIdx*/ 2>,
                                        Sched<[WriteVFSqrtV_MX, ReadVFSqrtV_MX, ReadVMask]>;
       }
   }
 }
 
+// TODO: Should I make this SEW aware?
 multiclass VPseudoVRCP_V {
   foreach m = MxListF in {
     defvar mx = m.MX;
     defvar WriteVFRecpV_MX = !cast<SchedWrite>("WriteVFRecpV_" # mx);
     defvar ReadVFRecpV_MX = !cast<SchedRead>("ReadVFRecpV_" # mx);
 
     let VLMul = m.value in {
       def "_V_" # mx : VPseudoUnaryNoMask<m.vrclass, m.vrclass>,

 multiclass VPseudoBinaryM_VI<LMULInfo m> {
   defm _VI : VPseudoBinaryM<VR, m.vrclass, simm5, m,
                             !if(!ge(m.octuple, 16), "@earlyclobber $rd", "")>;
 }
 
 multiclass VPseudoVGTR_VV_VX_VI<Operand ImmType = simm5, string Constraint = ""> {
   foreach m = MxList in {
     defvar mx = m.MX;
-    defvar WriteVGatherV_MX = !cast<SchedWrite>("WriteVGatherV_" # mx);
+    defvar sews = SchedSEWSet<mx>.val;
     defvar WriteVGatherX_MX = !cast<SchedWrite>("WriteVGatherX_" # mx);
     defvar WriteVGatherI_MX = !cast<SchedWrite>("WriteVGatherI_" # mx);
-    defvar ReadVGatherV_MX = !cast<SchedRead>("ReadVGatherV_" # mx);
     defvar ReadVGatherX_MX = !cast<SchedRead>("ReadVGatherX_" # mx);
+    defvar ReadVGatherI_MX = !cast<SchedRead>("ReadVGatherI_" # mx);
 
-    defm "" : VPseudoBinaryV_VV<m, Constraint>,
-              Sched<[WriteVGatherV_MX, ReadVGatherV_MX, ReadVGatherV_MX, ReadVMask]>;
+    // TODO: Is this change okay? V->X
     defm "" : VPseudoBinaryV_VX<m, Constraint>,
-              Sched<[WriteVGatherX_MX, ReadVGatherV_MX, ReadVGatherX_MX, ReadVMask]>;
+              Sched<[WriteVGatherX_MX, ReadVGatherX_MX, ReadVGatherX_MX, ReadVMask]>;
     defm "" : VPseudoBinaryV_VI<ImmType, m, Constraint>,
-              Sched<[WriteVGatherI_MX, ReadVGatherV_MX, ReadVMask]>;
+              Sched<[WriteVGatherI_MX, ReadVGatherI_MX, ReadVMask]>;
+
+    foreach e = sews in {
+      defvar WriteVGatherV_MX_E = !cast<SchedWrite>("WriteVGatherV_" # mx # "_E" # e);
+      defvar ReadVGatherV_MX_E = !cast<SchedRead>("ReadVGatherV_" # mx # "_E" # e);
+      defm "" : VPseudoBinaryV_VV_E<m, e, Constraint>,
+                Sched<[WriteVGatherV_MX_E, ReadVGatherV_MX_E, ReadVGatherV_MX_E, ReadVMask]>;
+    }
   }
 }
 
 multiclass VPseudoVSALU_VV_VX_VI<Operand ImmType = simm5, string Constraint = ""> {
   foreach m = MxList in {
     defvar mx = m.MX;
     defvar WriteVSALUV_MX = !cast<SchedWrite>("WriteVSALUV_" # mx);
     defvar WriteVSALUX_MX = !cast<SchedWrite>("WriteVSALUX_" # mx);
foreach m = MxList in {
     defm "" : VPseudoBinaryV_VX<m>,
               Sched<[WriteVIMulX_MX, ReadVIMulV_MX, ReadVIMulX_MX, ReadVMask]>;
   }
 }
 
 multiclass VPseudoVDIV_VV_VX {
   foreach m = MxList in {
     defvar mx = m.MX;
-    defvar WriteVIDivV_MX = !cast<SchedWrite>("WriteVIDivV_" # mx);
-    defvar WriteVIDivX_MX = !cast<SchedWrite>("WriteVIDivX_" # mx);
-    defvar ReadVIDivV_MX = !cast<SchedRead>("ReadVIDivV_" # mx);
-    defvar ReadVIDivX_MX = !cast<SchedRead>("ReadVIDivX_" # mx);
-
-    defm "" : VPseudoBinaryV_VV<m>,
-              Sched<[WriteVIDivV_MX, ReadVIDivV_MX, ReadVIDivV_MX, ReadVMask]>;
-    defm "" : VPseudoBinaryV_VX<m>,
-              Sched<[WriteVIDivX_MX, ReadVIDivV_MX, ReadVIDivX_MX, ReadVMask]>;
+    defvar sews = SchedSEWSet<mx>.val;
+    foreach e = sews in {
+      defvar WriteVIDivV_MX_E = !cast<SchedWrite>("WriteVIDivV_" # mx # "_E" # e);
+      defvar WriteVIDivX_MX_E = !cast<SchedWrite>("WriteVIDivX_" # mx # "_E" # e);
+      defvar ReadVIDivV_MX_E = !cast<SchedRead>("ReadVIDivV_" # mx # "_E" # e);
+      defvar ReadVIDivX_MX_E = !cast<SchedRead>("ReadVIDivX_" # mx # "_E" # e);
+
+      defm "" : VPseudoBinaryV_VV_E<m, e>,
+                Sched<[WriteVIDivV_MX_E, ReadVIDivV_MX_E, ReadVIDivV_MX_E, ReadVMask]>;
+      defm "" : VPseudoBinaryV_VX_E<m, e>,
+                Sched<[WriteVIDivX_MX_E, ReadVIDivV_MX_E, ReadVIDivX_MX_E, ReadVMask]>;
+    }
   }
 }
 
 multiclass VPseudoVFMUL_VV_VF {
   foreach m = MxListF in {
     defvar mx = m.MX;
     defvar WriteVFMulV_MX = !cast<SchedWrite>("WriteVFMulV_" # mx);
     defvar ReadVFMulV_MX = !cast<SchedRead>("ReadVFMulV_" # mx);
foreach m = f.MxList in {
                 Sched<[WriteVFMulF_MX, ReadVFMulV_MX, ReadVFMulF_MX, ReadVMask]>;
     }
   }
 }
 
 multiclass VPseudoVFDIV_VV_VF {
   foreach m = MxListF in {
     defvar mx = m.MX;
-    defvar WriteVFDivV_MX = !cast<SchedWrite>("WriteVFDivV_" # mx);
-    defvar ReadVFDivV_MX = !cast<SchedRead>("ReadVFDivV_" # mx);
+    defvar sews = SchedSEWSet<mx>.val;
+    foreach e = sews in {
+      defvar WriteVFDivV_MX_E = !cast<SchedWrite>("WriteVFDivV_" # mx # "_E" # e);
+      defvar ReadVFDivV_MX_E = !cast<SchedRead>("ReadVFDivV_" # mx # "_E" # e);
 
-    defm "" : VPseudoBinaryFV_VV<m>,
-              Sched<[WriteVFDivV_MX, ReadVFDivV_MX, ReadVFDivV_MX, ReadVMask]>;
+      defm "" : VPseudoBinaryFV_VV_E<m, e>,
+                Sched<[WriteVFDivV_MX_E, ReadVFDivV_MX_E, ReadVFDivV_MX_E, ReadVMask]>;
+    }
   }
 
   foreach f = FPList in {
     foreach m = f.MxList in {
       defvar mx = m.MX;
-      defvar WriteVFDivF_MX = !cast<SchedWrite>("WriteVFDivF_" # mx);
-      defvar ReadVFDivV_MX = !cast<SchedRead>("ReadVFDivV_" # mx);
-      defvar ReadVFDivF_MX = !cast<SchedRead>("ReadVFDivF_" # mx);
+      defvar sews = SchedSEWSet<mx>.val;
+      foreach e = sews in {
+        defvar WriteVFDivF_MX_E = !cast<SchedWrite>("WriteVFDivF_" # mx # "_E" # e);
+        defvar ReadVFDivV_MX_E = !cast<SchedRead>("ReadVFDivV_" # mx # "_E" # e);
+        defvar ReadVFDivF_MX_E = !cast<SchedRead>("ReadVFDivF_" # mx # "_E" # e);
 
-      defm "" : VPseudoBinaryV_VF<m, f>,
-                Sched<[WriteVFDivF_MX, ReadVFDivV_MX, ReadVFDivF_MX, ReadVMask]>;
+        defm "" : VPseudoBinaryV_VF_E<m, e, f>,
+                  Sched<[WriteVFDivF_MX_E, ReadVFDivV_MX_E, ReadVFDivF_MX_E, ReadVMask]>;
+      }
     }
   }
 }
 
 multiclass VPseudoVFRDIV_VF {
   foreach f = FPList in {
     foreach m = f.MxList in {
       defvar mx = m.MX;
-      defvar WriteVFDivF_MX = !cast<SchedWrite>("WriteVFDivF_" # mx);
-      defvar ReadVFDivV_MX = !cast<SchedRead>("ReadVFDivV_" # mx);
-      defvar ReadVFDivF_MX = !cast<SchedRead>("ReadVFDivF_" # mx);
-        defm "" : VPseudoBinaryV_VF<m, f>,
-                  Sched<[WriteVFDivF_MX, ReadVFDivV_MX, ReadVFDivF_MX, ReadVMask]>;
+      defvar sews = SchedSEWSet<mx>.val;
+      foreach e = sews in {
+        defvar WriteVFDivF_MX_E = !cast<SchedWrite>("WriteVFDivF_" # mx # "_E" # e);
+        defvar ReadVFDivV_MX_E = !cast<SchedRead>("ReadVFDivV_" # mx # "_E" # e);
+        defvar ReadVFDivF_MX_E = !cast<SchedRead>("ReadVFDivF_" # mx # "_E" # e);
+
+        defm "" : VPseudoBinaryV_VF_E<m, e, f>,
+                  Sched<[WriteVFDivF_MX_E, ReadVFDivV_MX_E, ReadVFDivF_MX_E, ReadVMask]>;
+      }
     }
   }
 }
 
 multiclass VPseudoVALU_VV_VX {
  foreach m = MxList in {
     defvar mx = m.MX;
     defvar WriteVIALUV_MX = !cast<SchedWrite>("WriteVIALUV_" # mx);
class VPatUnaryNoMask<string intrinsic_name,
   Pat<(result_type (!cast<Intrinsic>(intrinsic_name)
                    (result_type undef),
                    (op2_type op2_reg_class:$rs2),
                    VLOpFrag)),
                    (!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX)
                    (op2_type op2_reg_class:$rs2),
                    GPR:$vl, sew)>;
 
+class VPatUnaryNoMask_E<string intrinsic_name,
+                        string inst,
+                        string kind,
+                        ValueType result_type,
+                        ValueType op2_type,
+                        int log2sew,
+                        LMULInfo vlmul,
+                        int sew,
+                        VReg op2_reg_class> :
+  Pat<(result_type (!cast<Intrinsic>(intrinsic_name)
+                   (result_type undef),
+                   (op2_type op2_reg_class:$rs2),
+                   VLOpFrag)),
+                   (!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX#"_E"#sew)
+                   (op2_type op2_reg_class:$rs2),
+                   GPR:$vl, log2sew)>;
+
 class VPatUnaryNoMaskTU<string intrinsic_name,
                         string inst,
                         string kind,
                         ValueType result_type,
                         ValueType op2_type,
                         int sew,
                         LMULInfo vlmul,
                         VReg result_reg_class,
                         VReg op2_reg_class> :
   Pat<(result_type (!cast<Intrinsic>(intrinsic_name)
                    (result_type result_reg_class:$merge),
                    (op2_type op2_reg_class:$rs2),
                    VLOpFrag)),
                    (!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX#"_TU")
                    (result_type result_reg_class:$merge),
                    (op2_type op2_reg_class:$rs2),
                    GPR:$vl, sew)>;
 
+class VPatUnaryNoMaskTU_E<string intrinsic_name,
+                          string inst,
+                          string kind,
+                          ValueType result_type,
+                          ValueType op2_type,
+                          int log2sew,
+                          LMULInfo vlmul,
+                          int sew,
+                          VReg result_reg_class,
+                          VReg op2_reg_class> :
+  Pat<(result_type (!cast<Intrinsic>(intrinsic_name)
+                   (result_type result_reg_class:$merge),
+                   (op2_type op2_reg_class:$rs2),
+                   VLOpFrag)),
+                   (!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX#"_E"#sew#"_TU")
+                   (result_type result_reg_class:$merge),
+                   (op2_type op2_reg_class:$rs2),
+                   GPR:$vl, log2sew)>;
+
 class VPatUnaryMask<string intrinsic_name,
                     string inst,
                     string kind,
                     ValueType result_type,
                     ValueType op2_type,
                     ValueType mask_type,
                     int sew,
                     LMULInfo vlmul,
Pat<(result_type (!cast<Intrinsic>(intrinsic_name#"_mask")
                    (op2_type op2_reg_class:$rs2),
                    (mask_type V0),
                    VLOpFrag, (XLenVT timm:$policy))),
                    (!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX#"_MASK")
                    (result_type result_reg_class:$merge),
                    (op2_type op2_reg_class:$rs2),
                    (mask_type V0), GPR:$vl, sew, (XLenVT timm:$policy))>;
 
+class VPatUnaryMaskTA_E<string intrinsic_name,
+                        string inst,
+                        string kind,
+                        ValueType result_type,
+                        ValueType op2_type,
+                        ValueType mask_type,
+                        int log2sew,
+                        LMULInfo vlmul,
+                        int sew,
+                        VReg result_reg_class,
+                        VReg op2_reg_class> :
+  Pat<(result_type (!cast<Intrinsic>(intrinsic_name#"_mask")
+                   (result_type result_reg_class:$merge),
+                   (op2_type op2_reg_class:$rs2),
+                   (mask_type V0),
+                   VLOpFrag, (XLenVT timm:$policy))),
+                   (!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX#"_E"#sew#"_MASK")
+                   (result_type result_reg_class:$merge),
+                   (op2_type op2_reg_class:$rs2),
+                   (mask_type V0), GPR:$vl, log2sew, (XLenVT timm:$policy))>;
+
 class VPatMaskUnaryNoMask<string intrinsic_name,
                           string inst,
                           MTypeInfo mti> :
   Pat<(mti.Mask (!cast<Intrinsic>(intrinsic_name)
                 (mti.Mask VR:$rs2),
                 VLOpFrag)),
                 (!cast<Instruction>(inst#"_M_"#mti.BX)
                 (mti.Mask VR:$rs2),
Pat<(result_type (!cast<Intrinsic>(intrinsic)
                    (mask_type VR:$rs2),
                    VLOpFrag)),
                    (!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX)
                    (result_type result_reg_class:$merge),
                    (op1_type op1_reg_class:$rs1),
                    (mask_type VR:$rs2),
                    GPR:$vl, sew)>;
 
+class VPatUnaryAnyMask_E<string intrinsic,
+                         string inst,
+                         string kind,
+                         ValueType result_type,
+                         ValueType op1_type,
+                         ValueType mask_type,
+                         int log2sew,
+                         LMULInfo vlmul,
+                         int sew,
+                         VReg result_reg_class,
+                         VReg op1_reg_class> :
+  Pat<(result_type (!cast<Intrinsic>(intrinsic)
+                   (result_type result_reg_class:$merge),
+                   (op1_type op1_reg_class:$rs1),
+                   (mask_type VR:$rs2),
+                   VLOpFrag)),
+                   (!cast<Instruction>(inst#"_"#kind#"_"#vlmul.MX#"_E"#sew)
+                   (result_type result_reg_class:$merge),
+                   (op1_type op1_reg_class:$rs1),
+                   (mask_type VR:$rs2),
+                   GPR:$vl, log2sew)>;
+
 class VPatBinaryM<string intrinsic_name,
                   string inst,
                   ValueType result_type,
                   ValueType op1_type,
                   ValueType op2_type,
                   int sew,
                   VReg op1_reg_class,
                   DAGOperand op2_kind> :
multiclass VPatUnaryV_V_AnyMask<string intrinsic, string instruction,
   foreach vti = vtilist in {
     def : VPatUnaryAnyMask<intrinsic, instruction, "VM",
                            vti.Vector, vti.Vector, vti.Mask,
                            vti.Log2SEW, vti.LMul, vti.RegClass,
                            vti.RegClass>;
   }
 }
 
+multiclass VPatUnaryV_V_AnyMask_E<string intrinsic, string instruction,
+                                  list<VTypeInfo> vtilist> {
+  foreach vti = vtilist in {
+    def : VPatUnaryAnyMask_E<intrinsic, instruction, "VM",
+                           vti.Vector, vti.Vector, vti.Mask,
+                           vti.Log2SEW, vti.LMul, vti.SEW, vti.RegClass,
+                           vti.RegClass>;
+  }
+}
+
 multiclass VPatUnaryM_M<string intrinsic,
                          string inst>
 {
   foreach mti = AllMasks in {
     def : VPatMaskUnaryNoMask<intrinsic, inst, mti>;
     def : VPatMaskUnaryMask<intrinsic, inst, mti>;
   }
 }
def : VPatUnaryNoMaskTU<intrinsic, instruction, "V",
                             vti.Vector, vti.Vector,
                             vti.Log2SEW, vti.LMul, vti.RegClass, vti.RegClass>;
     def : VPatUnaryMaskTA<intrinsic, instruction, "V",
                           vti.Vector, vti.Vector, vti.Mask,
                           vti.Log2SEW, vti.LMul, vti.RegClass, vti.RegClass>;
   }
 }
 
+multiclass VPatUnaryV_V_E<string intrinsic, string instruction,
+                        list<VTypeInfo> vtilist> {
+  foreach vti = vtilist in {
+    def : VPatUnaryNoMask_E<intrinsic, instruction, "V",
+                            vti.Vector, vti.Vector,
+                            vti.Log2SEW, vti.LMul, vti.SEW, vti.RegClass>;
+    def : VPatUnaryNoMaskTU_E<intrinsic, instruction, "V",
+                              vti.Vector, vti.Vector,
+                              vti.Log2SEW, vti.LMul, vti.SEW,
+                              vti.RegClass, vti.RegClass>;
+    def : VPatUnaryMaskTA_E<intrinsic, instruction, "V",
+                            vti.Vector, vti.Vector, vti.Mask,
+                            vti.Log2SEW, vti.LMul, vti.SEW,
+                            vti.RegClass, vti.RegClass>;
+  }
+}
+
 multiclass VPatNullaryV<string intrinsic, string instruction>
 {
   foreach vti = AllIntegerVectors in {
     def : Pat<(vti.Vector (!cast<Intrinsic>(intrinsic)
                           (vti.Vector undef),
                           VLOpFrag)),
                           (!cast<Instruction>(instruction#"_V_" # vti.LMul.MX)
                           GPR:$vl, vti.Log2SEW)>;
multiclass VPatBinaryV_VV<string intrinsic, string instruction,
                           list<VTypeInfo> vtilist> {
   foreach vti = vtilist in
     defm : VPatBinaryTA<intrinsic, instruction # "_VV_" # vti.LMul.MX,
                         vti.Vector, vti.Vector, vti.Vector,vti.Mask,
                         vti.Log2SEW, vti.RegClass,
                         vti.RegClass, vti.RegClass>;
 }
 
-multiclass VPatBinaryV_VV_INT<string intrinsic, string instruction,
+multiclass VPatBinaryV_VV_E<string intrinsic, string instruction,
+                            list<VTypeInfo> vtilist> {
+  foreach vti = vtilist in
+    defm : VPatBinaryTA<intrinsic,
+                        instruction # "_VV_" # vti.LMul.MX # "_E" # vti.SEW,
+                        vti.Vector, vti.Vector, vti.Vector,vti.Mask,
+                        vti.Log2SEW, vti.RegClass,
+                        vti.RegClass, vti.RegClass>;
+}
+
+multiclass VPatBinaryV_VV_INT_E<string intrinsic, string instruction,
                           list<VTypeInfo> vtilist> {
   foreach vti = vtilist in {
     defvar ivti = GetIntVTypeInfo<vti>.Vti;
-    defm : VPatBinaryTA<intrinsic, instruction # "_VV_" # vti.LMul.MX,
+    defm : VPatBinaryTA<intrinsic,
+                        instruction # "_VV_" # vti.LMul.MX # "_E" # vti.SEW,
                         vti.Vector, vti.Vector, ivti.Vector, vti.Mask,
                         vti.Log2SEW, vti.RegClass,
                         vti.RegClass, vti.RegClass>;
   }
 }
 
-multiclass VPatBinaryV_VV_INT_EEW<string intrinsic, string instruction,
+multiclass VPatBinaryV_VV_INT_E_EEW<string intrinsic, string instruction,
                                   int eew, list<VTypeInfo> vtilist> {
   foreach vti = vtilist in {
     // emul = lmul * eew / sew
     defvar vlmul = vti.LMul;
     defvar octuple_lmul = vlmul.octuple;
     defvar octuple_emul = !srl(!mul(octuple_lmul, eew), vti.Log2SEW);
     if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then {
       defvar emul_str = octuple_to_str<octuple_emul>.ret;
       defvar ivti = !cast<VTypeInfo>("VI" # eew # emul_str);
-      defvar inst = instruction # "_VV_" # vti.LMul.MX # "_" # emul_str;
+      defvar inst = instruction # "_VV_" # vti.LMul.MX # "_E" # vti.SEW # "_" # emul_str;
       defm : VPatBinaryTA<intrinsic, inst,
                           vti.Vector, vti.Vector, ivti.Vector, vti.Mask,
                           vti.Log2SEW, vti.RegClass,
                           vti.RegClass, ivti.RegClass>;
     }
   }
 }
 
 multiclass VPatBinaryV_VX<string intrinsic, string instruction,
                           list<VTypeInfo> vtilist> {
   foreach vti = vtilist in {
     defvar kind = "V"#vti.ScalarSuffix;
     defm : VPatBinaryTA<intrinsic, instruction#"_"#kind#"_"#vti.LMul.MX,
                         vti.Vector, vti.Vector, vti.Scalar, vti.Mask,
                         vti.Log2SEW, vti.RegClass,
                         vti.RegClass, vti.ScalarRegClass>;
   }
 }
 
+multiclass VPatBinaryV_VX_E<string intrinsic, string instruction,
+                            list<VTypeInfo> vtilist> {
+  foreach vti = vtilist in {
+    defvar kind = "V"#vti.ScalarSuffix;
+    defm : VPatBinaryTA<intrinsic,
+                        instruction#"_"#kind#"_"#vti.LMul.MX#"_E"#vti.SEW,
+                        vti.Vector, vti.Vector, vti.Scalar, vti.Mask,
+                        vti.Log2SEW, vti.RegClass,
+                        vti.RegClass, vti.ScalarRegClass>;
+  }
+}
+
 multiclass VPatBinaryV_VX_INT<string intrinsic, string instruction,
                           list<VTypeInfo> vtilist> {
   foreach vti = vtilist in
     defm : VPatBinaryTA<intrinsic, instruction # "_VX_" # vti.LMul.MX,
                         vti.Vector, vti.Vector, XLenVT, vti.Mask,
                         vti.Log2SEW, vti.RegClass,
                         vti.RegClass, GPR>;
 }
: VPatBinaryV_VV<intrinsic, instruction, vtilist>,
       VPatBinaryV_VX<intrinsic, instruction, vtilist>,
       VPatBinaryV_VI<intrinsic, instruction, vtilist, ImmType>;
 
 multiclass VPatBinaryV_VV_VX<string intrinsic, string instruction,
                              list<VTypeInfo> vtilist>
     : VPatBinaryV_VV<intrinsic, instruction, vtilist>,
       VPatBinaryV_VX<intrinsic, instruction, vtilist>;
 
+multiclass VPatBinaryV_VV_VX_E<string intrinsic, string instruction,
+                               list<VTypeInfo> vtilist>
+    : VPatBinaryV_VV_E<intrinsic, instruction, vtilist>,
+      VPatBinaryV_VX_E<intrinsic, instruction, vtilist>;
+
 multiclass VPatBinaryV_VX_VI<string intrinsic, string instruction,
                              list<VTypeInfo> vtilist>
     : VPatBinaryV_VX<intrinsic, instruction, vtilist>,
       VPatBinaryV_VI<intrinsic, instruction, vtilist, simm5>;
 
 multiclass VPatBinaryW_VV_VX<string intrinsic, string instruction,
                              list<VTypeInfoToWide> vtilist>
     : VPatBinaryW_VV<intrinsic, instruction, vtilist>,

 
 multiclass VPatBinaryM_VX_VI<string intrinsic, string instruction,
                              list<VTypeInfo> vtilist>
     : VPatBinaryM_VX<intrinsic, instruction, vtilist>,
       VPatBinaryM_VI<intrinsic, instruction, vtilist>;
 
 multiclass VPatBinaryV_VV_VX_VI_INT<string intrinsic, string instruction,
                                     list<VTypeInfo> vtilist, Operand ImmType = simm5>
-    : VPatBinaryV_VV_INT<intrinsic#"_vv", instruction, vtilist>,
+    : VPatBinaryV_VV_INT_E<intrinsic#"_vv", instruction, vtilist>,
       VPatBinaryV_VX_INT<intrinsic#"_vx", instruction, vtilist>,
       VPatBinaryV_VI<intrinsic#"_vx", instruction, vtilist, ImmType>;
 
 multiclass VPatReductionV_VS<string intrinsic, string instruction, bit IsFloat = 0> {
   foreach vti = !if(IsFloat, NoGroupFloatVectors, NoGroupIntegerVectors) in
   {
     defvar vectorM1 = !cast<VTypeInfo>(!if(IsFloat, "VF", "VI") # vti.SEW # "M1");
     defm : VPatTernary<intrinsic, instruction, "VS",

 defm : VPatBinaryV_VV_VX<"int_riscv_vmul", "PseudoVMUL", AllIntegerVectors>;
 defm : VPatBinaryV_VV_VX<"int_riscv_vmulh", "PseudoVMULH", AllIntegerVectors>;
 defm : VPatBinaryV_VV_VX<"int_riscv_vmulhu", "PseudoVMULHU", AllIntegerVectors>;
 defm : VPatBinaryV_VV_VX<"int_riscv_vmulhsu", "PseudoVMULHSU", AllIntegerVectors>;
 
 //===----------------------------------------------------------------------===//
 // 11.11. Vector Integer Divide Instructions
 //===----------------------------------------------------------------------===//
-defm : VPatBinaryV_VV_VX<"int_riscv_vdivu", "PseudoVDIVU", AllIntegerVectors>;
-defm : VPatBinaryV_VV_VX<"int_riscv_vdiv", "PseudoVDIV", AllIntegerVectors>;
-defm : VPatBinaryV_VV_VX<"int_riscv_vremu", "PseudoVREMU", AllIntegerVectors>;
-defm : VPatBinaryV_VV_VX<"int_riscv_vrem", "PseudoVREM", AllIntegerVectors>;
+defm : VPatBinaryV_VV_VX_E<"int_riscv_vdivu", "PseudoVDIVU", AllIntegerVectors>;
+defm : VPatBinaryV_VV_VX_E<"int_riscv_vdiv", "PseudoVDIV", AllIntegerVectors>;
+defm : VPatBinaryV_VV_VX_E<"int_riscv_vremu", "PseudoVREMU", AllIntegerVectors>;
+defm : VPatBinaryV_VV_VX_E<"int_riscv_vrem", "PseudoVREM", AllIntegerVectors>;
 
 //===----------------------------------------------------------------------===//
 // 11.12. Vector Widening Integer Multiply Instructions
 //===----------------------------------------------------------------------===//
 defm : VPatBinaryW_VV_VX<"int_riscv_vwmul", "PseudoVWMUL", AllWidenableIntVectors>;
 defm : VPatBinaryW_VV_VX<"int_riscv_vwmulu", "PseudoVWMULU", AllWidenableIntVectors>;
 defm : VPatBinaryW_VV_VX<"int_riscv_vwmulsu", "PseudoVWMULSU", AllWidenableIntVectors>;
 

 defm : VPatBinaryW_VV_VX<"int_riscv_vfwsub", "PseudoVFWSUB", AllWidenableFloatVectors>;
 defm : VPatBinaryW_WV_WX<"int_riscv_vfwadd_w", "PseudoVFWADD", AllWidenableFloatVectors>;
 defm : VPatBinaryW_WV_WX<"int_riscv_vfwsub_w", "PseudoVFWSUB", AllWidenableFloatVectors>;
 
 //===----------------------------------------------------------------------===//
 // 13.4. Vector Single-Width Floating-Point Multiply/Divide Instructions
 //===----------------------------------------------------------------------===//
 defm : VPatBinaryV_VV_VX<"int_riscv_vfmul", "PseudoVFMUL", AllFloatVectors>;
-defm : VPatBinaryV_VV_VX<"int_riscv_vfdiv", "PseudoVFDIV", AllFloatVectors>;
-defm : VPatBinaryV_VX<"int_riscv_vfrdiv", "PseudoVFRDIV", AllFloatVectors>;
+defm : VPatBinaryV_VV_VX_E<"int_riscv_vfdiv", "PseudoVFDIV", AllFloatVectors>;
+defm : VPatBinaryV_VX_E<"int_riscv_vfrdiv", "PseudoVFRDIV", AllFloatVectors>;
 
 //===----------------------------------------------------------------------===//
 // 13.5. Vector Widening Floating-Point Multiply
 //===----------------------------------------------------------------------===//
 defm : VPatBinaryW_VV_VX<"int_riscv_vfwmul", "PseudoVFWMUL", AllWidenableFloatVectors>;
 
 //===----------------------------------------------------------------------===//
 // 13.6. Vector Single-Width Floating-Point Fused Multiply-Add Instructions

 defm : VPatTernaryW_VV_VX<"int_riscv_vfwmacc", "PseudoVFWMACC", AllWidenableFloatVectors>;
 defm : VPatTernaryW_VV_VX<"int_riscv_vfwnmacc", "PseudoVFWNMACC", AllWidenableFloatVectors>;
 defm : VPatTernaryW_VV_VX<"int_riscv_vfwmsac", "PseudoVFWMSAC", AllWidenableFloatVectors>;
 defm : VPatTernaryW_VV_VX<"int_riscv_vfwnmsac", "PseudoVFWNMSAC", AllWidenableFloatVectors>;
 
 //===----------------------------------------------------------------------===//
 // 13.8. Vector Floating-Point Square-Root Instruction
 //===----------------------------------------------------------------------===//
-defm : VPatUnaryV_V<"int_riscv_vfsqrt", "PseudoVFSQRT", AllFloatVectors>;
+defm : VPatUnaryV_V_E<"int_riscv_vfsqrt", "PseudoVFSQRT", AllFloatVectors>;
 
 //===----------------------------------------------------------------------===//
 // 13.9. Vector Floating-Point Reciprocal Square-Root Estimate Instruction
 //===----------------------------------------------------------------------===//
 defm : VPatUnaryV_V<"int_riscv_vfrsqrt7", "PseudoVFRSQRT7", AllFloatVectors>;
 
 //===----------------------------------------------------------------------===//
 // 13.10. Vector Floating-Point Reciprocal Estimate Instruction

 } // Predicates = [HasVInstructionsAnyF]
 
 //===----------------------------------------------------------------------===//
 // 16.4. Vector Register Gather Instructions
 //===----------------------------------------------------------------------===//
 let Predicates = [HasVInstructions] in {
   defm : VPatBinaryV_VV_VX_VI_INT<"int_riscv_vrgather", "PseudoVRGATHER",
                                   AllIntegerVectors, uimm5>;
-  defm : VPatBinaryV_VV_INT_EEW<"int_riscv_vrgatherei16_vv", "PseudoVRGATHEREI16",
+  defm : VPatBinaryV_VV_INT_E_EEW<"int_riscv_vrgatherei16_vv", "PseudoVRGATHEREI16",
                                 /* eew */ 16, AllIntegerVectors>;
 } // Predicates = [HasVInstructions]
 
 let Predicates = [HasVInstructionsAnyF] in {
   defm : VPatBinaryV_VV_VX_VI_INT<"int_riscv_vrgather", "PseudoVRGATHER",
                                   AllFloatVectors, uimm5>;
-  defm : VPatBinaryV_VV_INT_EEW<"int_riscv_vrgatherei16_vv", "PseudoVRGATHEREI16",
+  defm : VPatBinaryV_VV_INT_E_EEW<"int_riscv_vrgatherei16_vv", "PseudoVRGATHEREI16",
                                 /* eew */ 16, AllFloatVectors>;
 } // Predicates = [HasVInstructionsAnyF]
 
 //===----------------------------------------------------------------------===//
 // 16.5. Vector Compress Instruction
 //===----------------------------------------------------------------------===//
 let Predicates = [HasVInstructions] in {
-  defm : VPatUnaryV_V_AnyMask<"int_riscv_vcompress", "PseudoVCOMPRESS", AllIntegerVectors>;
+  defm : VPatUnaryV_V_AnyMask_E<"int_riscv_vcompress", "PseudoVCOMPRESS", AllIntegerVectors>;
 } // Predicates = [HasVInstructions]
 
 let Predicates = [HasVInstructionsAnyF] in {
-  defm : VPatUnaryV_V_AnyMask<"int_riscv_vcompress", "PseudoVCOMPRESS", AllFloatVectors>;
+  defm : VPatUnaryV_V_AnyMask_E<"int_riscv_vcompress", "PseudoVCOMPRESS", AllFloatVectors>;
 } // Predicates = [HasVInstructionsAnyF]
 
 // Include the non-intrinsic ISel patterns
 include "RISCVInstrInfoVVLPatterns.td"
 include "RISCVInstrInfoVSDPatterns.td"

llvm/lib/Target/RISCV/RISCVInstrInfoVSDPatterns.td

class VPatBinarySDNode_VV<SDNode vop,
     Pat<(result_type (vop
                      (op_type op_reg_class:$rs1),
                      (op_type op_reg_class:$rs2))),
         (!cast<Instruction>(instruction_name#"_VV_"# vlmul.MX)
                      op_reg_class:$rs1,
                      op_reg_class:$rs2,
                      avl, sew)>;
 
+class VPatBinarySDNode_VV_E<SDNode vop,
+                          string instruction_name,
+                          ValueType result_type,
+                          ValueType op_type,
+                          int log2sew,
+                          LMULInfo vlmul,
+                          int sew,
+                          OutPatFrag avl,
+                          VReg op_reg_class> :
+    Pat<(result_type (vop
+                     (op_type op_reg_class:$rs1),
+                     (op_type op_reg_class:$rs2))),
+        (!cast<Instruction>(instruction_name#"_VV_"# vlmul.MX#"_E"#sew)
+                     op_reg_class:$rs1,
+                     op_reg_class:$rs2,
+                     avl, log2sew)>;
+
 class VPatBinarySDNode_XI<SDNode vop,
                           string instruction_name,
                           string suffix,
                           ValueType result_type,
                           ValueType vop_type,
                           int sew,
                           LMULInfo vlmul,
                           OutPatFrag avl,
                           VReg vop_reg_class,
                           ComplexPattern SplatPatKind,
                           DAGOperand xop_kind> :
     Pat<(result_type (vop
                      (vop_type vop_reg_class:$rs1),
                      (vop_type (SplatPatKind xop_kind:$rs2)))),
         (!cast<Instruction>(instruction_name#_#suffix#_# vlmul.MX)
                      vop_reg_class:$rs1,
                      xop_kind:$rs2,
                      avl, sew)>;
 
+class VPatBinarySDNode_XI_E<SDNode vop,
+                          string instruction_name,
+                          string suffix,
+                          ValueType result_type,
+                          ValueType vop_type,
+                          int log2sew,
+                          LMULInfo vlmul,
+                          int sew,
+                          OutPatFrag avl,
+                          VReg vop_reg_class,
+                          ComplexPattern SplatPatKind,
+                          DAGOperand xop_kind> :
+    Pat<(result_type (vop
+                     (vop_type vop_reg_class:$rs1),
+                     (vop_type (SplatPatKind xop_kind:$rs2)))),
+        (!cast<Instruction>(instruction_name#_#suffix#_# vlmul.MX#"_E"#sew)
+                     vop_reg_class:$rs1,
+                     xop_kind:$rs2,
+                     avl, log2sew)>;
+
 multiclass VPatBinarySDNode_VV_VX<SDNode vop, string instruction_name> {
   foreach vti = AllIntegerVectors in {
     def : VPatBinarySDNode_VV<vop, instruction_name,
                               vti.Vector, vti.Vector, vti.Log2SEW,
                               vti.LMul, vti.AVL, vti.RegClass>;
     def : VPatBinarySDNode_XI<vop, instruction_name, "VX",
                               vti.Vector, vti.Vector, vti.Log2SEW,
                               vti.LMul, vti.AVL, vti.RegClass,
                               SplatPat, GPR>;
   }
 }
 
+multiclass VPatBinarySDNode_VV_VX_E<SDNode vop, string instruction_name> {
+  foreach vti = AllIntegerVectors in {
+    def : VPatBinarySDNode_VV_E<vop, instruction_name,
+                                vti.Vector, vti.Vector, vti.Log2SEW,
+                                vti.LMul, vti.SEW, vti.AVL, vti.RegClass>;
+    def : VPatBinarySDNode_XI_E<vop, instruction_name, "VX",
+                                vti.Vector, vti.Vector, vti.Log2SEW,
+                                vti.LMul, vti.SEW, vti.AVL, vti.RegClass,
+                                SplatPat, GPR>;
+  }
+}
+
 multiclass VPatBinarySDNode_VV_VX_VI<SDNode vop, string instruction_name,
                                      Operand ImmType = simm5>
     : VPatBinarySDNode_VV_VX<vop, instruction_name> {
   foreach vti = AllIntegerVectors in {
     def : VPatBinarySDNode_XI<vop, instruction_name, "VI",
                               vti.Vector, vti.Vector, vti.Log2SEW,
                               vti.LMul, vti.AVL, vti.RegClass,
                               !cast<ComplexPattern>(SplatPat#_#ImmType),
class VPatBinarySDNode_VF<SDNode vop,
                           DAGOperand xop_kind> :
     Pat<(result_type (vop (vop_type vop_reg_class:$rs1),
                           (vop_type (SplatFPOp xop_kind:$rs2)))),
         (!cast<Instruction>(instruction_name#"_"#vlmul.MX)
                      vop_reg_class:$rs1,
                      (xop_type xop_kind:$rs2),
                      avl, sew)>;
 
+class VPatBinarySDNode_VF_E<SDNode vop,
+                          string instruction_name,
+                          ValueType result_type,
+                          ValueType vop_type,
+                          ValueType xop_type,
+                          int log2sew,
+                          LMULInfo vlmul,
+                          int sew,
+                          OutPatFrag avl,
+                          VReg vop_reg_class,
+                          DAGOperand xop_kind> :
+    Pat<(result_type (vop (vop_type vop_reg_class:$rs1),
+                          (vop_type (SplatFPOp xop_kind:$rs2)))),
+        (!cast<Instruction>(instruction_name#"_"#vlmul.MX#"_E"#sew)
+                     vop_reg_class:$rs1,
+                     (xop_type xop_kind:$rs2),
+                     avl, log2sew)>;
+
 multiclass VPatBinaryFPSDNode_VV_VF<SDNode vop, string instruction_name> {
   foreach vti = AllFloatVectors in {
     def : VPatBinarySDNode_VV<vop, instruction_name,
                               vti.Vector, vti.Vector, vti.Log2SEW,
                               vti.LMul, vti.AVL, vti.RegClass>;
     def : VPatBinarySDNode_VF<vop, instruction_name#"_V"#vti.ScalarSuffix,
                               vti.Vector, vti.Vector, vti.Scalar,
                               vti.Log2SEW, vti.LMul, vti.AVL, vti.RegClass,
                               vti.ScalarRegClass>;
   }
 }
 
+multiclass VPatBinaryFPSDNode_VV_VF_E<SDNode vop, string instruction_name> {
+  foreach vti = AllFloatVectors in {
+    def : VPatBinarySDNode_VV_E<vop, instruction_name,
+                              vti.Vector, vti.Vector, vti.Log2SEW,
+                              vti.LMul, vti.SEW, vti.AVL, vti.RegClass>;
+    def : VPatBinarySDNode_VF_E<vop, instruction_name#"_V"#vti.ScalarSuffix,
+                              vti.Vector, vti.Vector, vti.Scalar,
+                              vti.Log2SEW, vti.LMul, vti.SEW, vti.AVL,
+                              vti.RegClass, vti.ScalarRegClass>;
+  }
+}
+
 multiclass VPatBinaryFPSDNode_R_VF<SDNode vop, string instruction_name> {
   foreach fvti = AllFloatVectors in
     def : Pat<(fvti.Vector (vop (fvti.Vector (SplatFPOp fvti.Scalar:$rs2)),
                                 (fvti.Vector fvti.RegClass:$rs1))),
               (!cast<Instruction>(instruction_name#"_V"#fvti.ScalarSuffix#"_"#fvti.LMul.MX)
                            fvti.RegClass:$rs1,
                            (fvti.Scalar fvti.ScalarRegClass:$rs2),
                            fvti.AVL, fvti.Log2SEW)>;
 }
 
+multiclass VPatBinaryFPSDNode_R_VF_E<SDNode vop, string instruction_name> {
+  foreach fvti = AllFloatVectors in
+    def : Pat<(fvti.Vector (vop (fvti.Vector (SplatFPOp fvti.Scalar:$rs2)),
+                                (fvti.Vector fvti.RegClass:$rs1))),
+              (!cast<Instruction>(instruction_name#"_V"#fvti.ScalarSuffix#"_"#fvti.LMul.MX#"_E"#fvti.SEW)
+                           fvti.RegClass:$rs1,
+                           (fvti.Scalar fvti.ScalarRegClass:$rs2),
+                           fvti.AVL, fvti.Log2SEW)>;
+}
+
 multiclass VPatIntegerSetCCSDNode_VV<string instruction_name,
                                      CondCode cc> {
   foreach vti = AllIntegerVectors in {
     defvar instruction = !cast<Instruction>(instruction_name#"_VV_"#vti.LMul.MX);
     def : Pat<(vti.Mask (setcc (vti.Vector vti.RegClass:$rs1),
                                (vti.Vector vti.RegClass:$rs2), cc)),
               (instruction vti.RegClass:$rs1, vti.RegClass:$rs2, vti.AVL,
               vti.Log2SEW)>;

 defm : VPatBinarySDNode_VV_VX<smax, "PseudoVMAX">;
 
 // 11.10. Vector Single-Width Integer Multiply Instructions
 defm : VPatBinarySDNode_VV_VX<mul, "PseudoVMUL">;
 defm : VPatBinarySDNode_VV_VX<mulhs, "PseudoVMULH">;
 defm : VPatBinarySDNode_VV_VX<mulhu, "PseudoVMULHU">;
 
 // 11.11. Vector Integer Divide Instructions
-defm : VPatBinarySDNode_VV_VX<udiv, "PseudoVDIVU">;
-defm : VPatBinarySDNode_VV_VX<sdiv, "PseudoVDIV">;
-defm : VPatBinarySDNode_VV_VX<urem, "PseudoVREMU">;
-defm : VPatBinarySDNode_VV_VX<srem, "PseudoVREM">;
+defm : VPatBinarySDNode_VV_VX_E<udiv, "PseudoVDIVU">;
+defm : VPatBinarySDNode_VV_VX_E<sdiv, "PseudoVDIV">;
+defm : VPatBinarySDNode_VV_VX_E<urem, "PseudoVREMU">;
+defm : VPatBinarySDNode_VV_VX_E<srem, "PseudoVREM">;
 
 // 11.12. Vector Widening Integer Multiply Instructions
 defm : VPatWidenBinarySDNode_VV_VX<mul, sext_oneuse, sext_oneuse,
                                    "PseudoVWMUL">;
 defm : VPatWidenBinarySDNode_VV_VX<mul, zext_oneuse, zext_oneuse,
                                    "PseudoVWMULU">;
 defm : VPatWidenBinarySDNode_VV_VX<mul, anyext_oneuse, anyext_oneuse,
                                    "PseudoVWMULU">;

 defm : VPatBinaryFPSDNode_R_VF<fsub, "PseudoVFRSUB">;
 
 // 13.3. Vector Widening Floating-Point Add/Subtract Instructions
 defm : VPatWidenBinaryFPSDNode_VV_VF_WV_WF<fadd, "PseudoVFWADD">;
 defm : VPatWidenBinaryFPSDNode_VV_VF_WV_WF<fsub, "PseudoVFWSUB">;
 
 // 13.4. Vector Single-Width Floating-Point Multiply/Divide Instructions
 defm : VPatBinaryFPSDNode_VV_VF<fmul, "PseudoVFMUL">;
-defm : VPatBinaryFPSDNode_VV_VF<fdiv, "PseudoVFDIV">;
-defm : VPatBinaryFPSDNode_R_VF<fdiv, "PseudoVFRDIV">;
+defm : VPatBinaryFPSDNode_VV_VF_E<fdiv, "PseudoVFDIV">;
+defm : VPatBinaryFPSDNode_R_VF_E<fdiv, "PseudoVFRDIV">;
 
 // 13.5. Vector Widening Floating-Point Multiply Instructions
 defm : VPatWidenBinaryFPSDNode_VV_VF<fmul, "PseudoVFWMUL">;
 
 // 13.6 Vector Single-Width Floating-Point Fused Multiply-Add Instructions.
 foreach fvti = AllFloatVectors in {
   // NOTE: We choose VFMADD because it has the most commuting freedom. So it
   // works best with how TwoAddressInstructionPass tries commuting.

 defm : VPatWidenFPMulAccSDNode_VV_VF<"PseudoVFWMACC">;
 defm : VPatWidenFPNegMulAccSDNode_VV_VF<"PseudoVFWNMACC">;
 defm : VPatWidenFPMulSacSDNode_VV_VF<"PseudoVFWMSAC">;
 defm : VPatWidenFPNegMulSacSDNode_VV_VF<"PseudoVFWNMSAC">;
 
 foreach vti = AllFloatVectors in {
   // 13.8. Vector Floating-Point Square-Root Instruction
   def : Pat<(fsqrt (vti.Vector vti.RegClass:$rs2)),
-            (!cast<Instruction>("PseudoVFSQRT_V_"# vti.LMul.MX)
+            (!cast<Instruction>("PseudoVFSQRT_V_"# vti.LMul.MX#"_E"#vti.SEW)
                  vti.RegClass:$rs2, vti.AVL, vti.Log2SEW)>;
 
   // 13.12. Vector Floating-Point Sign-Injection Instructions
   def : Pat<(fabs (vti.Vector vti.RegClass:$rs)),
             (!cast<Instruction>("PseudoVFSGNJX_VV_"# vti.LMul.MX)
                  vti.RegClass:$rs, vti.RegClass:$rs, vti.AVL, vti.Log2SEW)>;
   // Handle fneg with VFSGNJN using the same input for both operands.
   def : Pat<(fneg (vti.Vector vti.RegClass:$rs)),

llvm/lib/Target/RISCV/RISCVInstrInfoVVLPatterns.td

def : Pat<(result_type (vop
                          VLOpFrag)),
         (!cast<Instruction>(instruction_name#"_"#suffix#"_"# vlmul.MX#"_MASK")
                      result_reg_class:$merge,
                      op1_reg_class:$rs1,
                      op2_reg_class:$rs2,
                      (mask_type V0), GPR:$vl, sew, TAIL_AGNOSTIC)>;
 }
 
+multiclass VPatBinaryVL_V_E<SDNode vop,
+                          string instruction_name,
+                          string suffix,
+                          ValueType result_type,
+                          ValueType op1_type,
+                          ValueType op2_type,
+                          ValueType mask_type,
+                          int log2sew,
+                          LMULInfo vlmul,
+                          int sew,
+                          VReg result_reg_class,
+                          VReg op1_reg_class,
+                          VReg op2_reg_class> {
+  def : Pat<(result_type (vop
+                         (op1_type op1_reg_class:$rs1),
+                         (op2_type op2_reg_class:$rs2),
+                         (result_type result_reg_class:$merge),
+                         (mask_type V0),
+                         VLOpFrag)),
+        (!cast<Instruction>(instruction_name#"_"#suffix#"_"# vlmul.MX#"_E"# sew#"_MASK")
+                     result_reg_class:$merge,
+                     op1_reg_class:$rs1,
+                     op2_reg_class:$rs2,
+                     (mask_type V0), GPR:$vl, log2sew, TAIL_AGNOSTIC)>;
+}
+
 multiclass VPatTiedBinaryNoMaskVL_V<SDNode vop,
                                     string instruction_name,
                                     string suffix,
                                     ValueType result_type,
                                     ValueType op2_type,
                                     int sew,
                                     LMULInfo vlmul,
                                     VReg result_reg_class,
def : Pat<(result_type (vop
                      VLOpFrag)),
         (!cast<Instruction>(instruction_name#_#suffix#_# vlmul.MX#"_MASK")
                      result_reg_class:$merge,
                      vop_reg_class:$rs1,
                      xop_kind:$rs2,
                      (mask_type V0), GPR:$vl, sew, TAIL_AGNOSTIC)>;
 }
 
+multiclass VPatBinaryVL_XI_E<SDNode vop,
+                           string instruction_name,
+                           string suffix,
+                           ValueType result_type,
+                           ValueType vop1_type,
+                           ValueType vop2_type,
+                           ValueType mask_type,
+                           int log2sew,
+                           LMULInfo vlmul,
+                           int sew,
+                           VReg result_reg_class,
+                           VReg vop_reg_class,
+                           ComplexPattern SplatPatKind,
+                           DAGOperand xop_kind> {
+  def : Pat<(result_type (vop
+                     (vop1_type vop_reg_class:$rs1),
+                     (vop2_type (SplatPatKind (XLenVT xop_kind:$rs2))),
+                     (result_type result_reg_class:$merge),
+                     (mask_type V0),
+                     VLOpFrag)),
+        (!cast<Instruction>(instruction_name#_#suffix#_# vlmul.MX#"_E"# sew#"_MASK")
+                     result_reg_class:$merge,
+                     vop_reg_class:$rs1,
+                     xop_kind:$rs2,
+                     (mask_type V0), GPR:$vl, log2sew, TAIL_AGNOSTIC)>;
+}
+
 multiclass VPatBinaryVL_VV_VX<SDNode vop, string instruction_name> {
   foreach vti = AllIntegerVectors in {
     defm : VPatBinaryVL_V<vop, instruction_name, "VV",
                            vti.Vector, vti.Vector, vti.Vector, vti.Mask,
                            vti.Log2SEW, vti.LMul, vti.RegClass, vti.RegClass,
                            vti.RegClass>;
     defm : VPatBinaryVL_XI<vop, instruction_name, "VX",
                            vti.Vector, vti.Vector, vti.Vector, vti.Mask,
                            vti.Log2SEW, vti.LMul, vti.RegClass, vti.RegClass,
                            SplatPat, GPR>;
   }
 }
 
+multiclass VPatBinaryVL_VV_VX_E<SDNode vop, string instruction_name> {
+  foreach vti = AllIntegerVectors in {
+    defm : VPatBinaryVL_V_E<vop, instruction_name, "VV",
+                           vti.Vector, vti.Vector, vti.Vector, vti.Mask,
+                           vti.Log2SEW, vti.LMul, vti.SEW,
+                           vti.RegClass, vti.RegClass, vti.RegClass>;
+    defm : VPatBinaryVL_XI_E<vop, instruction_name, "VX",
+                           vti.Vector, vti.Vector, vti.Vector, vti.Mask,
+                           vti.Log2SEW, vti.LMul, vti.SEW,
+                           vti.RegClass, vti.RegClass, SplatPat, GPR>;
+  }
+}
+
 multiclass VPatBinaryVL_VV_VX_VI<SDNode vop, string instruction_name,
                                  Operand ImmType = simm5>
     : VPatBinaryVL_VV_VX<vop, instruction_name> {
   foreach vti = AllIntegerVectors in {
     defm : VPatBinaryVL_XI<vop, instruction_name, "VI",
                            vti.Vector, vti.Vector, vti.Vector, vti.Mask,
                            vti.Log2SEW, vti.LMul, vti.RegClass, vti.RegClass,
                            !cast<ComplexPattern>(SplatPat#_#ImmType),
def : Pat<(result_type (vop (vop_type vop_reg_class:$rs1),
                          VLOpFrag)),
         (!cast<Instruction>(instruction_name#"_"#vlmul.MX#"_MASK")
                      result_reg_class:$merge,
                      vop_reg_class:$rs1,
                      scalar_reg_class:$rs2,
                      (mask_type V0), GPR:$vl, sew, TAIL_AGNOSTIC)>;
 }
 
+multiclass VPatBinaryVL_VF_E<SDNode vop,
+                           string instruction_name,
+                           ValueType result_type,
+                           ValueType vop_type,
+                           ValueType mask_type,
+                           int log2sew,
+                           LMULInfo vlmul,
+                           int sew,
+                           VReg result_reg_class,
+                           VReg vop_reg_class,
+                           RegisterClass scalar_reg_class> {
+  def : Pat<(result_type (vop (vop_type vop_reg_class:$rs1),
+                         (vop_type (SplatFPOp scalar_reg_class:$rs2)),
+                         (result_type result_reg_class:$merge),
+                         (mask_type V0),
+                         VLOpFrag)),
+        (!cast<Instruction>(instruction_name#"_"#vlmul.MX#"_E"#sew#"_MASK")
+                     result_reg_class:$merge,
+                     vop_reg_class:$rs1,
+                     scalar_reg_class:$rs2,
+                     (mask_type V0), GPR:$vl, log2sew, TAIL_AGNOSTIC)>;
+}
+
 multiclass VPatBinaryFPVL_VV_VF<SDNode vop, string instruction_name> {
   foreach vti = AllFloatVectors in {
     defm : VPatBinaryVL_V<vop, instruction_name, "VV",
                           vti.Vector, vti.Vector, vti.Vector, vti.Mask,
                           vti.Log2SEW, vti.LMul, vti.RegClass, vti.RegClass,
                           vti.RegClass>;
     defm : VPatBinaryVL_VF<vop, instruction_name#"_V"#vti.ScalarSuffix,
                            vti.Vector, vti.Vector, vti.Mask, vti.Log2SEW,
                            vti.LMul, vti.RegClass, vti.RegClass,
                            vti.ScalarRegClass>;
   }
 }
 
+multiclass VPatBinaryFPVL_VV_VF_E<SDNode vop, string instruction_name> {
+  foreach vti = AllFloatVectors in {
+    defm : VPatBinaryVL_V_E<vop, instruction_name, "VV",
+                          vti.Vector, vti.Vector, vti.Vector, vti.Mask,
+                          vti.Log2SEW, vti.LMul, vti.SEW,
+                          vti.RegClass, vti.RegClass, vti.RegClass>;
+    defm : VPatBinaryVL_VF_E<vop, instruction_name#"_V"#vti.ScalarSuffix,
+                           vti.Vector, vti.Vector, vti.Mask, vti.Log2SEW,
+                           vti.LMul, vti.SEW, vti.RegClass, vti.RegClass,
+                           vti.ScalarRegClass>;
+  }
+}
+
 multiclass VPatBinaryFPVL_R_VF<SDNode vop, string instruction_name> {
   foreach fvti = AllFloatVectors in {
     def : Pat<(fvti.Vector (vop (SplatFPOp fvti.ScalarRegClass:$rs2),
                                 fvti.RegClass:$rs1,
                                 (fvti.Vector fvti.RegClass:$merge),
                                 (fvti.Mask V0),
                                 VLOpFrag)),
               (!cast<Instruction>(instruction_name#"_V"#fvti.ScalarSuffix#"_"#fvti.LMul.MX#"_MASK")
                            fvti.RegClass:$merge,
                            fvti.RegClass:$rs1, fvti.ScalarRegClass:$rs2,
                            (fvti.Mask V0), GPR:$vl, fvti.Log2SEW, TAIL_AGNOSTIC)>;
   }
 }
 
+multiclass VPatBinaryFPVL_R_VF_E<SDNode vop, string instruction_name> {
+  foreach fvti = AllFloatVectors in {
+    def : Pat<(fvti.Vector (vop (SplatFPOp fvti.ScalarRegClass:$rs2),
+                                fvti.RegClass:$rs1,
+                                (fvti.Vector fvti.RegClass:$merge),
+                                (fvti.Mask V0),
+                                VLOpFrag)),
+              (!cast<Instruction>(instruction_name#"_V"#fvti.ScalarSuffix#"_"#fvti.LMul.MX#"_E"#fvti.SEW#"_MASK")
+                           fvti.RegClass:$merge,
+                           fvti.RegClass:$rs1, fvti.ScalarRegClass:$rs2,
+                           (fvti.Mask V0), GPR:$vl, fvti.Log2SEW, TAIL_AGNOSTIC)>;
+  }
+}
+
 multiclass VPatIntegerSetCCVL_VV<VTypeInfo vti, string instruction_name,
                                  CondCode cc> {
   def : Pat<(vti.Mask (riscv_setcc_vl (vti.Vector vti.RegClass:$rs1),
                                       vti.RegClass:$rs2, cc,
                                       VR:$merge,
                                       (vti.Mask V0),
                                       VLOpFrag)),
             (!cast<Instruction>(instruction_name#"_VV_"#vti.LMul.MX#"_MASK")

 defm : VPatBinaryVL_VV_VX<riscv_smax_vl, "PseudoVMAX">;
 
 // 11.10. Vector Single-Width Integer Multiply Instructions
 defm : VPatBinaryVL_VV_VX<riscv_mul_vl, "PseudoVMUL">;
 defm : VPatBinaryVL_VV_VX<riscv_mulhs_vl, "PseudoVMULH">;
 defm : VPatBinaryVL_VV_VX<riscv_mulhu_vl, "PseudoVMULHU">;
 
 // 11.11. Vector Integer Divide Instructions
-defm : VPatBinaryVL_VV_VX<riscv_udiv_vl, "PseudoVDIVU">;
-defm : VPatBinaryVL_VV_VX<riscv_sdiv_vl, "PseudoVDIV">;
-defm : VPatBinaryVL_VV_VX<riscv_urem_vl, "PseudoVREMU">;
-defm : VPatBinaryVL_VV_VX<riscv_srem_vl, "PseudoVREM">;
+defm : VPatBinaryVL_VV_VX_E<riscv_udiv_vl, "PseudoVDIVU">;
+defm : VPatBinaryVL_VV_VX_E<riscv_sdiv_vl, "PseudoVDIV">;
+defm : VPatBinaryVL_VV_VX_E<riscv_urem_vl, "PseudoVREMU">;
+defm : VPatBinaryVL_VV_VX_E<riscv_srem_vl, "PseudoVREM">;
 
 // 11.12. Vector Widening Integer Multiply Instructions
 defm : VPatBinaryWVL_VV_VX<riscv_vwmul_vl, "PseudoVWMUL">;
 defm : VPatBinaryWVL_VV_VX<riscv_vwmulu_vl, "PseudoVWMULU">;
 defm : VPatBinaryWVL_VV_VX<riscv_vwmulsu_vl, "PseudoVWMULSU">;
 
 // 11.13 Vector Single-Width Integer Multiply-Add Instructions
 defm : VPatMultiplyAddVL_VV_VX<riscv_add_vl, "PseudoVMADD">;

 defm : VPatBinaryFPVL_R_VF<riscv_fsub_vl, "PseudoVFRSUB">;
 
 // 13.3. Vector Widening Floating-Point Add/Subtract Instructions
 defm : VPatWidenBinaryFPVL_VV_VF_WV_WF<riscv_fadd_vl, "PseudoVFWADD">;
 defm : VPatWidenBinaryFPVL_VV_VF_WV_WF<riscv_fsub_vl, "PseudoVFWSUB">;
 
 // 13.4. Vector Single-Width Floating-Point Multiply/Divide Instructions
 defm : VPatBinaryFPVL_VV_VF<riscv_fmul_vl, "PseudoVFMUL">;
-defm : VPatBinaryFPVL_VV_VF<riscv_fdiv_vl, "PseudoVFDIV">;
-defm : VPatBinaryFPVL_R_VF<riscv_fdiv_vl, "PseudoVFRDIV">;
+defm : VPatBinaryFPVL_VV_VF_E<riscv_fdiv_vl, "PseudoVFDIV">;
+defm : VPatBinaryFPVL_R_VF_E<riscv_fdiv_vl, "PseudoVFRDIV">;
 
 // 13.5. Vector Widening Floating-Point Multiply Instructions
 defm : VPatWidenBinaryFPVL_VV_VF<riscv_fmul_vl, riscv_fpextend_vl_oneuse, "PseudoVFWMUL">;
 
 // 13.6 Vector Single-Width Floating-Point Fused Multiply-Add Instructions.
 defm : VPatFPMulAddVL_VV_VF<riscv_vfmadd_vl,  "PseudoVFMADD">;
 defm : VPatFPMulAddVL_VV_VF<riscv_vfmsub_vl,  "PseudoVFMSUB">;
 defm : VPatFPMulAddVL_VV_VF<riscv_vfnmadd_vl, "PseudoVFNMADD">;

 
 defm : VPatFPSetCCVL_VV_VF_FV<SETLE,  "PseudoVMFLE", "PseudoVMFGE">;
 defm : VPatFPSetCCVL_VV_VF_FV<SETOLE, "PseudoVMFLE", "PseudoVMFGE">;
 
 foreach vti = AllFloatVectors in {
   // 13.8. Vector Floating-Point Square-Root Instruction
   def : Pat<(riscv_fsqrt_vl (vti.Vector vti.RegClass:$rs2), (vti.Mask V0),
                             VLOpFrag),
-            (!cast<Instruction>("PseudoVFSQRT_V_"# vti.LMul.MX #"_MASK")
+            (!cast<Instruction>("PseudoVFSQRT_V_"# vti.LMul.MX # "_E" # vti.SEW # "_MASK")
                  (vti.Vector (IMPLICIT_DEF)), vti.RegClass:$rs2,
                  (vti.Mask V0), GPR:$vl, vti.Log2SEW, TA_MA)>;
 
   // 13.12. Vector Floating-Point Sign-Injection Instructions
   def : Pat<(riscv_fabs_vl (vti.Vector vti.RegClass:$rs), (vti.Mask V0),
                            VLOpFrag),
             (!cast<Instruction>("PseudoVFSGNJX_VV_"# vti.LMul.MX #"_MASK")
                  (vti.Vector (IMPLICIT_DEF)), vti.RegClass:$rs,
def : Pat<(vti.Vector (riscv_vmv_s_x_vl (vti.Vector vti.RegClass:$merge),
                 vti.RegClass:$merge,
                 (vti.Scalar vti.ScalarRegClass:$rs1), GPR:$vl, vti.Log2SEW)>;
 
   def : Pat<(vti.Vector (riscv_vrgather_vv_vl vti.RegClass:$rs2,
                                               vti.RegClass:$rs1,
                                               vti.RegClass:$merge,
                                               (vti.Mask V0),
                                               VLOpFrag)),
-            (!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX#"_MASK")
+            (!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX#"_E"# vti.SEW#"_MASK")
                  vti.RegClass:$merge, vti.RegClass:$rs2, vti.RegClass:$rs1,
                  (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
   def : Pat<(vti.Vector (riscv_vrgather_vx_vl vti.RegClass:$rs2, GPR:$rs1,
                                               vti.RegClass:$merge,
                                               (vti.Mask V0),
                                               VLOpFrag)),
             (!cast<Instruction>("PseudoVRGATHER_VX_"# vti.LMul.MX#"_MASK")
                  vti.RegClass:$merge, vti.RegClass:$rs2, GPR:$rs1,
foreach vti = AllIntegerVectors in {
 
   // emul = lmul * 16 / sew
   defvar vlmul = vti.LMul;
   defvar octuple_lmul = vlmul.octuple;
   defvar octuple_emul = !srl(!mul(octuple_lmul, 16), vti.Log2SEW);
   if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then {
     defvar emul_str = octuple_to_str<octuple_emul>.ret;
     defvar ivti = !cast<VTypeInfo>("VI16" # emul_str);
-    defvar inst = "PseudoVRGATHEREI16_VV_" # vti.LMul.MX # "_" # emul_str;
+    defvar inst = "PseudoVRGATHEREI16_VV_" # vti.LMul.MX # "_E" # vti.SEW # "_" # emul_str;
 
     def : Pat<(vti.Vector
                (riscv_vrgatherei16_vv_vl vti.RegClass:$rs2,
                                          (ivti.Vector ivti.RegClass:$rs1),
                                          vti.RegClass:$merge,
                                          (vti.Mask V0),
                                          VLOpFrag)),
               (!cast<Instruction>(inst#"_MASK")
foreach vti = AllFloatVectors in {
   defvar ivti = GetIntVTypeInfo<vti>.Vti;
 
   def : Pat<(vti.Vector
              (riscv_vrgather_vv_vl vti.RegClass:$rs2,
                                    (ivti.Vector vti.RegClass:$rs1),
                                    vti.RegClass:$merge,
                                    (vti.Mask V0),
                                    VLOpFrag)),
-            (!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX#"_MASK")
+            (!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX#"_E"# vti.SEW#"_MASK")
                  vti.RegClass:$merge, vti.RegClass:$rs2, vti.RegClass:$rs1,
                  (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
   def : Pat<(vti.Vector (riscv_vrgather_vx_vl vti.RegClass:$rs2, GPR:$rs1,
                                               vti.RegClass:$merge,
                                               (vti.Mask V0),
                                               VLOpFrag)),
             (!cast<Instruction>("PseudoVRGATHER_VX_"# vti.LMul.MX#"_MASK")
                  vti.RegClass:$merge, vti.RegClass:$rs2, GPR:$rs1,
def : Pat<(vti.Vector
                  (vti.Mask V0), GPR:$vl, vti.Log2SEW, TAIL_AGNOSTIC)>;
 
   defvar vlmul = vti.LMul;
   defvar octuple_lmul = vlmul.octuple;
   defvar octuple_emul = !srl(!mul(octuple_lmul, 16), vti.Log2SEW);
   if !and(!ge(octuple_emul, 1), !le(octuple_emul, 64)) then {
     defvar emul_str = octuple_to_str<octuple_emul>.ret;
     defvar ivti = !cast<VTypeInfo>("VI16" # emul_str);
-    defvar inst = "PseudoVRGATHEREI16_VV_" # vti.LMul.MX # "_" # emul_str;
+    defvar inst = "PseudoVRGATHEREI16_VV_" # vti.LMul.MX # "_E" # vti.SEW # "_" # emul_str;
 
     def : Pat<(vti.Vector
                (riscv_vrgatherei16_vv_vl vti.RegClass:$rs2,
                                          (ivti.Vector ivti.RegClass:$rs1),
                                          vti.RegClass:$merge,
                                          (vti.Mask V0),
                                          VLOpFrag)),
               (!cast<Instruction>(inst#"_MASK")

llvm/lib/Target/RISCV/RISCVScheduleV.td

 defvar UpperBoundLMUL = "UpperBound";
 defvar SchedMxList = ["UpperBound", "M1", "M2", "M4", "M8", "MF2", "MF4", "MF8"];
 // Used for widening and narrowing instructions as it doesn't contain M8.
 defvar SchedMxListW = ["UpperBound", "MF8", "MF4", "MF2", "M1", "M2", "M4"];
 defvar SchedMxListFW = ["UpperBound", "MF4", "MF2", "M1", "M2", "M4"];
 // Used for widening floating-point Reduction as it doesn't contain MF8.
 defvar SchedMxListFWRed = ["UpperBound", "MF4", "MF2", "M1", "M2", "M4", "M8"];
 
+class SchedSEWSet<string mx> {
+  list<int> val = !cond(!eq(mx, "M1"):  [8, 16, 32, 64],
+                        !eq(mx, "M2"):  [8, 16, 32, 64],
+                        !eq(mx, "M4"):  [8, 16, 32, 64],
+                        !eq(mx, "M8"):  [8, 16, 32, 64],
+                        !eq(mx, "MF2"): [8, 16, 32],
+                        !eq(mx, "MF4"): [8, 16],
+                        !eq(mx, "MF8"): [8]);
+}
+
 // Creates SchedWrite for each (name, LMUL) pair for LMUL in SchedMxList
 multiclass LMULSchedWrites<string name> {
   foreach mx = SchedMxList in {
     def name # "_" # mx : SchedWrite;
   }
 }
 
+// Creates SchedWrite for each (name, LMUL, SEW) tuple
+multiclass LMULSEWSchedWrites<string name> {
+  foreach mx = SchedMxList in {
+    if !eq(mx, "UpperBound") then
+      def name # "_" # mx : SchedWrite;
+    else
+      foreach sew = SchedSEWSet<mx>.val in
+        def name # "_" # mx # "_E" # sew : SchedWrite;
+  }
+}
+
 // Creates SchedWrite for each (name, LMUL) pair for LMUL in SchedMxListW
 multiclass LMULSchedWritesW<string name> {
   foreach mx = SchedMxListW in {
     def name # "_" # mx : SchedWrite;
   }
 }
 
 // Creates SchedWrite for each (name, LMUL) pair for LMUL in SchedMxListFW

 
 // Creates SchedRead for each (name, LMUL) pair for LMUL in SchedMxList
 multiclass LMULSchedReads<string name> {
   foreach mx = SchedMxList in {
     def name # "_" # mx : SchedRead;
   }
 }
 
+// Creates SchedRead for each (name, LMUL) pair for LMUL in SchedMxList
+multiclass LMULSEWSchedReads<string name> {
+  foreach mx = SchedMxList in {
+    if !eq(mx, "UpperBound") then
+      def name # "_" # mx : SchedRead;
+    else
+      foreach sew = SchedSEWSet<mx>.val in
+        def name # "_" # mx # "_E" # sew : SchedRead;
+  }
+}
+
 // Creates SchedRead for each (name, LMUL) pair for LMUL in SchedMxListW
 multiclass LMULSchedReadsW<string name> {
   foreach mx = SchedMxListW in {
     def name # "_" # mx : SchedRead;
   }
 }
 
 // Creates SchedRead for each (name, LMUL) pair for LMUL in SchedMxListFW
 multiclass LMULSchedReadsFW<string name> {
   foreach mx = SchedMxListFW in {
     def name # "_" # mx : SchedRead;
   }
 }
 
 // Creates WriteRes for each (name, LMUL, resources) tuple for LMUL
 // in SchedMxList
 multiclass LMULWriteRes<string name, list<ProcResourceKind> resources> {
   foreach mx = SchedMxList in {
     def : WriteRes<!cast<SchedWrite>(name # "_" # mx), resources>;
   }
 }
 
 // Creates WriteRes for each (name, LMUL, resources) tuple for LMUL
+// in SchedMxList
+multiclass LMULSEWWriteRes<string name, list<ProcResourceKind> resources> {
+  foreach mx = SchedMxList in {
+    if !eq(mx, "UpperBound") then
+      def : WriteRes<!cast<SchedWrite>(name # "_" # mx), resources>;
+    else
+      foreach sew = SchedSEWSet<mx>.val in
+        def : WriteRes<!cast<SchedWrite>(name # "_" # mx # "_E" # sew), resources>;
+  }
+}
+
+// Creates WriteRes for each (name, LMUL, resources) tuple for LMUL
 // in SchedMxListW
 multiclass LMULWriteResW<string name, list<ProcResourceKind> resources> {
   foreach mx = SchedMxListW in {
     def : WriteRes<!cast<SchedWrite>(name # "_" # mx), resources>;
   }
 }
 
 // Creates WriteRes for each (name, LMUL, resources) tuple for LMUL

 // in SchedMxList
 multiclass LMULReadAdvance<string name, int val, list<SchedWrite> writes = []> {
   foreach mx = SchedMxList in {
     def : ReadAdvance<!cast<SchedRead>(name # "_" # mx), val, writes>;
   }
 }
 
 // Creates ReadAdvance for each (name, LMUL, val) tuple for LMUL
+// in SchedMxList
+multiclass LMULSEWReadAdvance<string name, int val, list<SchedWrite> writes = []> {
+  foreach mx = SchedMxList in {
+    if !eq(mx, "UpperBound") then
+      def : ReadAdvance<!cast<SchedRead>(name # "_" # mx), val, writes>;
+    else
+      foreach sew = SchedSEWSet<mx>.val in
+        def : ReadAdvance<!cast<SchedRead>(name # "_" # mx # "_E" # sew), val, writes>;
+  }
+}
+
+// Creates ReadAdvance for each (name, LMUL, val) tuple for LMUL
 // in SchedMxListW
 multiclass LMULReadAdvanceW<string name, int val, list<SchedWrite> writes = []> {
   foreach mx = SchedMxListW in {
     def : ReadAdvance<!cast<SchedRead>(name # "_" # mx), val, writes>;
   }
 }
 
 // Creates ReadAdvance for each (name, LMUL, val) tuple for LMUL
 // in SchedMxListFW
 multiclass LMULReadAdvanceFW<string name, int val, list<SchedWrite> writes = []> {
   foreach mx = SchedMxListFW in {
     def : ReadAdvance<!cast<SchedRead>(name # "_" # mx), val, writes>;
   }
 }
 
 // 3.6 Vector Byte Length vlenb
 def WriteRdVLENB      : SchedWrite;
 
 // 6. Configuration-Setting Instructions
 def WriteVSETVLI      : SchedWrite;
 def WriteVSETIVLI     : SchedWrite;
 def WriteVSETVL       : SchedWrite;
 
 // 7. Vector Loads and Stores
 // 7.4. Vector Unit-Stride Instructions
 defm "" : LMULSchedWrites<"WriteVLDE">;
 defm "" : LMULSchedWrites<"WriteVSTE">;
 // 7.4.1. Vector Unit-Strided Mask
 defm "" : LMULSchedWrites<"WriteVLDM">;

 // 11.9. Vector Integer Min/Max Instructions
 defm "" : LMULSchedWrites<"WriteVICmpV">;
 defm "" : LMULSchedWrites<"WriteVICmpX">;
 defm "" : LMULSchedWrites<"WriteVICmpI">;
 // 11.10. Vector Single-Width Integer Multiply Instructions
 defm "" : LMULSchedWrites<"WriteVIMulV">;
 defm "" : LMULSchedWrites<"WriteVIMulX">;
 // 11.11. Vector Integer Divide Instructions
-defm "" : LMULSchedWrites<"WriteVIDivV">;
-defm "" : LMULSchedWrites<"WriteVIDivX">;
+defm "" : LMULSEWSchedWrites<"WriteVIDivV">;
+defm "" : LMULSEWSchedWrites<"WriteVIDivX">;
 // 11.12. Vector Widening Integer Multiply Instructions
 defm "" : LMULSchedWritesW<"WriteVIWMulV">;
 defm "" : LMULSchedWritesW<"WriteVIWMulX">;
 // 11.13. Vector Single-Width Integer Multiply-Add Instructions
 defm "" : LMULSchedWrites<"WriteVIMulAddV">;
 defm "" : LMULSchedWrites<"WriteVIMulAddX">;
 // 11.14. Vector Widening Integer Multiply-Add Instructions
 defm "" : LMULSchedWritesW<"WriteVIWMulAddV">;

 defm "" : LMULSchedWrites<"WriteVFALUV">;
 defm "" : LMULSchedWrites<"WriteVFALUF">;
 // 13.3. Vector Widening Floating-Point Add/Subtract Instructions
 defm "" : LMULSchedWritesFW<"WriteVFWALUV">;
 defm "" : LMULSchedWritesFW<"WriteVFWALUF">;
 // 13.4. Vector Single-Width Floating-Point Multiply/Divide Instructions
 defm "" : LMULSchedWrites<"WriteVFMulV">;
 defm "" : LMULSchedWrites<"WriteVFMulF">;
-defm "" : LMULSchedWrites<"WriteVFDivV">;
-defm "" : LMULSchedWrites<"WriteVFDivF">;
+defm "" : LMULSEWSchedWrites<"WriteVFDivV">;
+defm "" : LMULSEWSchedWrites<"WriteVFDivF">;
 // 13.5. Vector Widening Floating-Point Multiply
 defm "" : LMULSchedWritesFW<"WriteVFWMulV">;
 defm "" : LMULSchedWritesFW<"WriteVFWMulF">;
 // 13.6. Vector Single-Width Floating-Point Fused Multiply-Add Instructions
 defm "" : LMULSchedWrites<"WriteVFMulAddV">;
 defm "" : LMULSchedWrites<"WriteVFMulAddF">;
 // 13.7. Vector Widening Floating-Point Fused Multiply-Add Instructions
 defm "" : LMULSchedWritesFW<"WriteVFWMulAddV">;
 defm "" : LMULSchedWritesFW<"WriteVFWMulAddF">;
 // 13.8. Vector Floating-Point Square-Root Instruction
-defm "" : LMULSchedWrites<"WriteVFSqrtV">;
+defm "" : LMULSEWSchedWrites<"WriteVFSqrtV">;
 // 13.9. Vector Floating-Point Reciprocal Square-Root Estimate Instruction
 // 13.10. Vector Floating-Point Reciprocal Estimate Instruction
 defm "" : LMULSchedWrites<"WriteVFRecpV">;
 // 13.11. Vector Floating-Point MIN/MAX Instructions
 // 13.13. Vector Floating-Point Compare Instructions
 defm "" : LMULSchedWrites<"WriteVFCmpV">;
 defm "" : LMULSchedWrites<"WriteVFCmpF">;
 // 13.12. Vector Floating-Point Sign-Injection Instructions

 defm "" : LMULSchedWrites<"WriteVFMovVF">;
 defm "" : LMULSchedWrites<"WriteVFMovFV">;
 // 16.3. Vector Slide Instructions
 defm "" : LMULSchedWrites<"WriteVISlideX">;
 defm "" : LMULSchedWrites<"WriteVISlideI">;
 defm "" : LMULSchedWrites<"WriteVISlide1X">;
 defm "" : LMULSchedWrites<"WriteVFSlide1F">;
 // 16.4. Vector Register Gather Instructions
-defm "" : LMULSchedWrites<"WriteVGatherV">;
+defm "" : LMULSEWSchedWrites<"WriteVGatherV">;
 defm "" : LMULSchedWrites<"WriteVGatherX">;
 defm "" : LMULSchedWrites<"WriteVGatherI">;
 // 16.5. Vector Compress Instruction
-defm "" : LMULSchedWrites<"WriteVCompressV">;
+defm "" : LMULSEWSchedWrites<"WriteVCompressV">;
 // 16.6. Whole Vector Register Move
 // These are already LMUL aware
 def WriteVMov1V       : SchedWrite;
 def WriteVMov2V       : SchedWrite;
 def WriteVMov4V       : SchedWrite;
 def WriteVMov8V       : SchedWrite;
 
 //===----------------------------------------------------------------------===//

 // 11.8. Vector Integer Comparison Instructions
 // 11.9. Vector Integer Min/Max Instructions
 defm "" : LMULSchedReads<"ReadVICmpV">;
 defm "" : LMULSchedReads<"ReadVICmpX">;
 // 11.10. Vector Single-Width Integer Multiply Instructions
 defm "" : LMULSchedReads<"ReadVIMulV">;
 defm "" : LMULSchedReads<"ReadVIMulX">;
 // 11.11. Vector Integer Divide Instructions
-defm "" : LMULSchedReads<"ReadVIDivV">;
-defm "" : LMULSchedReads<"ReadVIDivX">;
+defm "" : LMULSEWSchedReads<"ReadVIDivV">;
+defm "" : LMULSEWSchedReads<"ReadVIDivX">;
 // 11.12. Vector Widening Integer Multiply Instructions
 defm "" : LMULSchedReadsW<"ReadVIWMulV">;
 defm "" : LMULSchedReadsW<"ReadVIWMulX">;
 // 11.13. Vector Single-Width Integer Multiply-Add Instructions
 defm "" : LMULSchedReads<"ReadVIMulAddV">;
 defm "" : LMULSchedReads<"ReadVIMulAddX">;
 // 11.14. Vector Widening Integer Multiply-Add Instructions
 defm "" : LMULSchedReadsW<"ReadVIWMulAddV">;

 defm "" : LMULSchedReads<"ReadVFALUV">;
 defm "" : LMULSchedReads<"ReadVFALUF">;
 // 13.3. Vector Widening Floating-Point Add/Subtract Instructions
 defm "" : LMULSchedReadsFW<"ReadVFWALUV">;
 defm "" : LMULSchedReadsFW<"ReadVFWALUF">;
 // 13.4. Vector Single-Width Floating-Point Multiply/Divide Instructions
 defm "" : LMULSchedReads<"ReadVFMulV">;
 defm "" : LMULSchedReads<"ReadVFMulF">;
-defm "" : LMULSchedReads<"ReadVFDivV">;
-defm "" : LMULSchedReads<"ReadVFDivF">;
+defm "" : LMULSEWSchedReads<"ReadVFDivV">;
+defm "" : LMULSEWSchedReads<"ReadVFDivF">;
 // 13.5. Vector Widening Floating-Point Multiply
 defm "" : LMULSchedReadsFW<"ReadVFWMulV">;
 defm "" : LMULSchedReadsFW<"ReadVFWMulF">;
 // 13.6. Vector Single-Width Floating-Point Fused Multiply-Add Instructions
 defm "" : LMULSchedReads<"ReadVFMulAddV">;
 defm "" : LMULSchedReads<"ReadVFMulAddF">;
 // 13.7. Vector Widening Floating-Point Fused Multiply-Add Instructions
 defm "" : LMULSchedReadsFW<"ReadVFWMulAddV">;
 defm "" : LMULSchedReadsFW<"ReadVFWMulAddF">;
 // 13.8. Vector Floating-Point Square-Root Instruction
-defm "" : LMULSchedReads<"ReadVFSqrtV">;
+defm "" : LMULSEWSchedReads<"ReadVFSqrtV">;
 // 13.9. Vector Floating-Point Reciprocal Square-Root Estimate Instruction
 // 13.10. Vector Floating-Point Reciprocal Estimate Instruction
 defm "" : LMULSchedReads<"ReadVFRecpV">;
 // 13.11. Vector Floating-Point MIN/MAX Instructions
 // 13.13. Vector Floating-Point Compare Instructions
 defm "" : LMULSchedReads<"ReadVFCmpV">;
 defm "" : LMULSchedReads<"ReadVFCmpF">;
 // 13.12. Vector Floating-Point Sign-Injection Instructions

 defm "" : LMULSchedReads<"ReadVFMovFV">;
 defm "" : LMULSchedReads<"ReadVFMovFX">;
 // 16.3. Vector Slide Instructions
 defm "" : LMULSchedReads<"ReadVISlideV">;
 defm "" : LMULSchedReads<"ReadVISlideX">;
 defm "" : LMULSchedReads<"ReadVFSlideV">;
 defm "" : LMULSchedReads<"ReadVFSlideF">;
 // 16.4. Vector Register Gather Instructions
-defm "" : LMULSchedReads<"ReadVGatherV">;
+defm "" : LMULSEWSchedReads<"ReadVGatherV">;
 defm "" : LMULSchedReads<"ReadVGatherX">;
+defm "" : LMULSchedReads<"ReadVGatherI">;
 // 16.5. Vector Compress Instruction
-defm "" : LMULSchedReads<"ReadVCompressV">;
+defm "" : LMULSEWSchedReads<"ReadVCompressV">;
 // 16.6. Whole Vector Register Move
 // These are already LMUL aware
 def ReadVMov1V        : SchedRead;
 def ReadVMov2V        : SchedRead;
 def ReadVMov4V        : SchedRead;
 def ReadVMov8V        : SchedRead;
 
 // Others

 defm "" : LMULWriteResW<"WriteVNShiftV", []>;
 defm "" : LMULWriteResW<"WriteVNShiftX", []>;
 defm "" : LMULWriteResW<"WriteVNShiftI", []>;
 defm "" : LMULWriteRes<"WriteVICmpV", []>;
 defm "" : LMULWriteRes<"WriteVICmpX", []>;
 defm "" : LMULWriteRes<"WriteVICmpI", []>;
 defm "" : LMULWriteRes<"WriteVIMulV", []>;
 defm "" : LMULWriteRes<"WriteVIMulX", []>;
-defm "" : LMULWriteRes<"WriteVIDivV", []>;
-defm "" : LMULWriteRes<"WriteVIDivX", []>;
+defm "" : LMULSEWWriteRes<"WriteVIDivV", []>;
+defm "" : LMULSEWWriteRes<"WriteVIDivX", []>;
 defm "" : LMULWriteResW<"WriteVIWMulV", []>;
 defm "" : LMULWriteResW<"WriteVIWMulX", []>;
 defm "" : LMULWriteRes<"WriteVIMulAddV", []>;
 defm "" : LMULWriteRes<"WriteVIMulAddX", []>;
 defm "" : LMULWriteResW<"WriteVIWMulAddV", []>;
 defm "" : LMULWriteResW<"WriteVIWMulAddX", []>;
 defm "" : LMULWriteRes<"WriteVIMergeV", []>;
 defm "" : LMULWriteRes<"WriteVIMergeX", []>;

 
 // 13. Vector Floating-Point Instructions
 defm "" : LMULWriteRes<"WriteVFALUV", []>;
 defm "" : LMULWriteRes<"WriteVFALUF", []>;
 defm "" : LMULWriteResFW<"WriteVFWALUV", []>;
 defm "" : LMULWriteResFW<"WriteVFWALUF", []>;
 defm "" : LMULWriteRes<"WriteVFMulV", []>;
 defm "" : LMULWriteRes<"WriteVFMulF", []>;
-defm "" : LMULWriteRes<"WriteVFDivV", []>;
-defm "" : LMULWriteRes<"WriteVFDivF", []>;
+defm "" : LMULSEWWriteRes<"WriteVFDivV", []>;
+defm "" : LMULSEWWriteRes<"WriteVFDivF", []>;
 defm "" : LMULWriteResFW<"WriteVFWMulV", []>;
 defm "" : LMULWriteResFW<"WriteVFWMulF", []>;
 defm "" : LMULWriteRes<"WriteVFMulAddV", []>;
 defm "" : LMULWriteRes<"WriteVFMulAddF", []>;
 defm "" : LMULWriteResFW<"WriteVFWMulAddV", []>;
 defm "" : LMULWriteResFW<"WriteVFWMulAddF", []>;
-defm "" : LMULWriteRes<"WriteVFSqrtV", []>;
+defm "" : LMULSEWWriteRes<"WriteVFSqrtV", []>;
 defm "" : LMULWriteRes<"WriteVFRecpV", []>;
 defm "" : LMULWriteRes<"WriteVFCmpV", []>;
 defm "" : LMULWriteRes<"WriteVFCmpF", []>;
 defm "" : LMULWriteRes<"WriteVFSgnjV", []>;
 defm "" : LMULWriteRes<"WriteVFSgnjF", []>;
 defm "" : LMULWriteRes<"WriteVFClassV", []>;
 defm "" : LMULWriteRes<"WriteVFMergeV", []>;
 defm "" : LMULWriteRes<"WriteVFMovV", []>;

 defm "" : LMULWriteRes<"WriteVIMovVX", []>;
 defm "" : LMULWriteRes<"WriteVIMovXV", []>;
 defm "" : LMULWriteRes<"WriteVFMovVF", []>;
 defm "" : LMULWriteRes<"WriteVFMovFV", []>;
 defm "" : LMULWriteRes<"WriteVISlideX", []>;
 defm "" : LMULWriteRes<"WriteVISlideI", []>;
 defm "" : LMULWriteRes<"WriteVISlide1X", []>;
 defm "" : LMULWriteRes<"WriteVFSlide1F", []>;
-defm "" : LMULWriteRes<"WriteVGatherV", []>;
+defm "" : LMULSEWWriteRes<"WriteVGatherV", []>;
 defm "" : LMULWriteRes<"WriteVGatherX", []>;
 defm "" : LMULWriteRes<"WriteVGatherI", []>;
-defm "" : LMULWriteRes<"WriteVCompressV", []>;
+defm "" : LMULSEWWriteRes<"WriteVCompressV", []>;
 // These are already LMUL aware
 def : WriteRes<WriteVMov1V, []>;
 def : WriteRes<WriteVMov2V, []>;
 def : WriteRes<WriteVMov4V, []>;
 def : WriteRes<WriteVMov8V, []>;
 
 // 6. Configuration-Setting Instructions
 def : ReadAdvance<ReadVSETVLI, 0>;

 defm "" : LMULReadAdvance<"ReadVShiftV", 0>;
 defm "" : LMULReadAdvance<"ReadVShiftX", 0>;
 defm "" : LMULReadAdvanceW<"ReadVNShiftV", 0>;
 defm "" : LMULReadAdvanceW<"ReadVNShiftX", 0>;
 defm "" : LMULReadAdvance<"ReadVICmpV", 0>;
 defm "" : LMULReadAdvance<"ReadVICmpX", 0>;
 defm "" : LMULReadAdvance<"ReadVIMulV", 0>;
 defm "" : LMULReadAdvance<"ReadVIMulX", 0>;
-defm "" : LMULReadAdvance<"ReadVIDivV", 0>;
-defm "" : LMULReadAdvance<"ReadVIDivX", 0>;
+defm "" : LMULSEWReadAdvance<"ReadVIDivV", 0>;
+defm "" : LMULSEWReadAdvance<"ReadVIDivX", 0>;
 defm "" : LMULReadAdvanceW<"ReadVIWMulV", 0>;
 defm "" : LMULReadAdvanceW<"ReadVIWMulX", 0>;
 defm "" : LMULReadAdvance<"ReadVIMulAddV", 0>;
 defm "" : LMULReadAdvance<"ReadVIMulAddX", 0>;
 defm "" : LMULReadAdvanceW<"ReadVIWMulAddV", 0>;
 defm "" : LMULReadAdvanceW<"ReadVIWMulAddX", 0>;
 defm "" : LMULReadAdvance<"ReadVIMergeV", 0>;
 defm "" : LMULReadAdvance<"ReadVIMergeX", 0>;

 
 // 13. Vector Floating-Point Instructions
 defm "" : LMULReadAdvance<"ReadVFALUV", 0>;
 defm "" : LMULReadAdvance<"ReadVFALUF", 0>;
 defm "" : LMULReadAdvanceFW<"ReadVFWALUV", 0>;
 defm "" : LMULReadAdvanceFW<"ReadVFWALUF", 0>;
 defm "" : LMULReadAdvance<"ReadVFMulV", 0>;
 defm "" : LMULReadAdvance<"ReadVFMulF", 0>;
-defm "" : LMULReadAdvance<"ReadVFDivV", 0>;
-defm "" : LMULReadAdvance<"ReadVFDivF", 0>;
+defm "" : LMULSEWReadAdvance<"ReadVFDivV", 0>;
+defm "" : LMULSEWReadAdvance<"ReadVFDivF", 0>;
 defm "" : LMULReadAdvanceFW<"ReadVFWMulV", 0>;
 defm "" : LMULReadAdvanceFW<"ReadVFWMulF", 0>;
 defm "" : LMULReadAdvance<"ReadVFMulAddV", 0>;
 defm "" : LMULReadAdvance<"ReadVFMulAddF", 0>;
 defm "" : LMULReadAdvanceFW<"ReadVFWMulAddV", 0>;
 defm "" : LMULReadAdvanceFW<"ReadVFWMulAddF", 0>;
-defm "" : LMULReadAdvance<"ReadVFSqrtV", 0>;
+defm "" : LMULSEWReadAdvance<"ReadVFSqrtV", 0>;
 defm "" : LMULReadAdvance<"ReadVFRecpV", 0>;
 defm "" : LMULReadAdvance<"ReadVFCmpV", 0>;
 defm "" : LMULReadAdvance<"ReadVFCmpF", 0>;
 defm "" : LMULReadAdvance<"ReadVFSgnjV", 0>;
 defm "" : LMULReadAdvance<"ReadVFSgnjF", 0>;
 defm "" : LMULReadAdvance<"ReadVFClassV", 0>;
 defm "" : LMULReadAdvance<"ReadVFMergeV", 0>;
 defm "" : LMULReadAdvance<"ReadVFMergeF", 0>;

 defm "" : LMULReadAdvance<"ReadVIMovXX", 0>;
 defm "" : LMULReadAdvance<"ReadVFMovVF", 0>;
 defm "" : LMULReadAdvance<"ReadVFMovFV", 0>;
 defm "" : LMULReadAdvance<"ReadVFMovFX", 0>;
 defm "" : LMULReadAdvance<"ReadVISlideV", 0>;
 defm "" : LMULReadAdvance<"ReadVISlideX", 0>;
 defm "" : LMULReadAdvance<"ReadVFSlideV", 0>;
 defm "" : LMULReadAdvance<"ReadVFSlideF", 0>;
-defm "" : LMULReadAdvance<"ReadVGatherV", 0>;
+defm "" : LMULSEWReadAdvance<"ReadVGatherV", 0>;
 defm "" : LMULReadAdvance<"ReadVGatherX", 0>;
-defm "" : LMULReadAdvance<"ReadVCompressV", 0>;
+defm "" : LMULReadAdvance<"ReadVGatherI", 0>;
+defm "" : LMULSEWReadAdvance<"ReadVCompressV", 0>;
 // These are already LMUL aware
 def : ReadAdvance<ReadVMov1V, 0>;
 def : ReadAdvance<ReadVMov2V, 0>;
 def : ReadAdvance<ReadVMov4V, 0>;
 def : ReadAdvance<ReadVMov8V, 0>;
 
 // Others
 def : ReadAdvance<ReadVMask, 0>;
 
 } // Unsupported
 } // UnsupportedSchedV