[RISCV][CodeGen] Account for LMUL from VS2 for Vector Reduction Instructions

[monkchiang]

Summary

The Reduction instruction destination register LMUL is 1. But the source
register(vs2) has different LMUL(MF8 to M8). It's beneficial to know how
many registers are working on reduction instructions.
This patch creates separate SchedWrite for each relevant LMUL that from VS2.

[michaelmaitland]

Does it make sense to change the SchedReadss and ReadAdvance to be LMUL specific for this class of instuctions?

[michaelmaitland]

llvm/lib/Target/RISCV/RISCVInstrInfoV.td

728–729

Other LMUL specific SchedWrites use the suffix UpperBound. Is there a difference in meaning between UpperBound and From_UpperBound?

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

3303–3304

Other LMUL specific SchedWrites use the suffix MX. Is there a difference in meaning between MX and From_MX?

[michaelmaitland]

llvm/lib/Target/RISCV/RISCVScheduleV.td

755

VFW prefix should use LMULWriteResFW class.

[monkchiang]

In D141565#4048418, @michaelmaitland wrote:

Does it make sense to change the SchedReadss and ReadAdvance to be LMUL specific for this class of instuctions?

It is helpful to describe the forwarding behavior. In order to simplify the design,
I think that the description Latency and ResourceCycles are all done in WriteRes.

llvm/lib/Target/RISCV/RISCVInstrInfoV.td

728–729

The destunation LMUL of Reduction instructions is 1, I use
From_UpperBound to distinguish the source operand of reading: LMUL=MF8 to M8.
Do you suggest I use UpperBound , Read_UpperBound or something else?

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

3303–3304

The use of From_MX is just to be noticed that the LMUL of the source operand is different.
But it is the same for scheduling, I personally prefer to use MX .

[michaelmaitland]

The suffixes From_UpperBound and From_MX come from the naming of the LMULWriteRes in the ScheduleV file. I think it makes sense to differentiate these LMULWriteRes with some sort of suffix, because they represent behavior that is different than the other LMULWriteRes definitions. I think From is a good suffix choice. Perhaps a comment above the definition of these LMULWriteRess that elaborates on the meaning of the suffix would be a helpful reference.

[monkchiang]

Address Michael Maitland's comment. Thanks!

[michaelmaitland]

llvm/lib/Target/RISCV/RISCVScheduleV.td

755

I think we still want WriteVFWRedV_From to use LMULWriteResFW class since its a floating point widening operation. After that change, it LGTM.

[monkchiang]

Change LMULWriteRes to LMULWriteResFWRed.

The following link shows LMUL of vfwredosum and vfwredusum are between MF4 to M8.
So I add LMULWriteResFWRed for widening floating-point Reduction.
https://github.com/riscv-non-isa/rvv-intrinsic-doc/blob/master/auto-generated/intrinsic_funcs/09_vector_reduction_functions.md#vector-widening-floating-point-reduction-functions

[michaelmaitland]

LGTM

llvm/lib/Target/RISCV/RISCVInstrInfoV.td

 multiclass VNCVTF_FV_VS2<string opcodestr, bits<6> funct6, bits<5> vs1> {
   def "" : VALUVs2<funct6, vs1, OPFVV, opcodestr>,
            Sched<[WriteVFNCvtFToFV_UpperBound, ReadVFNCvtFToFV_UpperBound,
                   ReadVMask]>;
 }
 
 multiclass VRED_MV_V<string opcodestr, bits<6> funct6> {
   def _VS : VALUVV<funct6, OPMVV, opcodestr # ".vs">,
-            Sched<[WriteVIRedV, ReadVIRedV, ReadVIRedV0, ReadVMask]>;
+            Sched<[WriteVIRedV_From_UpperBound, ReadVIRedV, ReadVIRedV0,
+                   ReadVMask]>;
 }
 
 multiclass VWRED_IV_V<string opcodestr, bits<6> funct6> {
   def _VS : VALUVV<funct6, OPIVV, opcodestr # ".vs">,
-            Sched<[WriteVIWRedV, ReadVIWRedV, ReadVIWRedV0, ReadVMask]>;
+            Sched<[WriteVIWRedV_From_UpperBound, ReadVIWRedV, ReadVIWRedV0,
+                   ReadVMask]>;
 }
 
 multiclass VRED_FV_V<string opcodestr, bits<6> funct6> {
   def _VS : VALUVV<funct6, OPFVV, opcodestr # ".vs">,
-            Sched<[WriteVFRedV, ReadVFRedV, ReadVFRedV0, ReadVMask]>;
+            Sched<[WriteVFRedV_From_UpperBound, ReadVFRedV, ReadVFRedV0,
+                   ReadVMask]>;
 }
 
 multiclass VREDO_FV_V<string opcodestr, bits<6> funct6> {
   def _VS : VALUVV<funct6, OPFVV, opcodestr # ".vs">,
-            Sched<[WriteVFRedOV, ReadVFRedOV, ReadVFRedOV0, ReadVMask]>;
+            Sched<[WriteVFRedOV_From_UpperBound, ReadVFRedOV, ReadVFRedOV0,
+                   ReadVMask]>;
 }
 
 multiclass VWRED_FV_V<string opcodestr, bits<6> funct6> {
   def _VS : VALUVV<funct6, OPFVV, opcodestr # ".vs">,
-            Sched<[WriteVFWRedV, ReadVFWRedV, ReadVFWRedV0, ReadVMask]>;
+            Sched<[WriteVFWRedV_From_UpperBound, ReadVFWRedV, ReadVFWRedV0,
+                   ReadVMask]>;
 }
 
 multiclass VWREDO_FV_V<string opcodestr, bits<6> funct6> {
   def _VS : VALUVV<funct6, OPFVV, opcodestr # ".vs">,
-            Sched<[WriteVFWRedOV, ReadVFWRedOV, ReadVFWRedOV0, ReadVMask]>;
+            Sched<[WriteVFWRedOV_From_UpperBound, ReadVFWRedOV, ReadVFWRedOV0,
+                   ReadVMask]>;
 }
 
 multiclass VMALU_MV_Mask<string opcodestr, bits<6> funct6, string vm = "v"> {
   def M : VALUVVNoVm<funct6, OPMVV, opcodestr #"." #vm #"m">,
           Sched<[WriteVMALUV_UpperBound, ReadVMALUV_UpperBound,
                  ReadVMALUV_UpperBound]>;
 }
 

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

defm "" : VPseudoBinaryM_VX<m>,
               Sched<[WriteVICmpX_MX, ReadVICmpV_MX, ReadVICmpX_MX, ReadVMask]>;
     defm "" : VPseudoBinaryM_VI<m>,
               Sched<[WriteVICmpI_MX, ReadVICmpV_MX, ReadVMask]>;
   }
 }
 
 multiclass VPseudoVRED_VS {
   foreach m = MxList in {
+    defvar mx = m.MX;
+    defvar WriteVIRedV_From_MX = !cast<SchedWrite>("WriteVIRedV_From_" # mx);
     defm _VS : VPseudoTernary<V_M1.vrclass, m.vrclass, V_M1.vrclass, m>,
-               Sched<[WriteVIRedV, ReadVIRedV, ReadVIRedV, ReadVIRedV, ReadVMask]>;
+               Sched<[WriteVIRedV_From_MX, ReadVIRedV, ReadVIRedV, ReadVIRedV,
+                      ReadVMask]>;
   }
 }
 
 multiclass VPseudoVWRED_VS {
   foreach m = MxList in {
+    defvar mx = m.MX;
+    defvar WriteVIWRedV_From_MX = !cast<SchedWrite>("WriteVIWRedV_From_" # mx);
     defm _VS : VPseudoTernary<V_M1.vrclass, m.vrclass, V_M1.vrclass, m>,
-               Sched<[WriteVIWRedV, ReadVIWRedV, ReadVIWRedV, ReadVIWRedV, ReadVMask]>;
+               Sched<[WriteVIWRedV_From_MX, ReadVIWRedV, ReadVIWRedV,
+                      ReadVIWRedV, ReadVMask]>;
   }
 }
 
 multiclass VPseudoVFRED_VS {
   foreach m = MxListF in {
+    defvar mx = m.MX;
+    defvar WriteVFRedV_From_MX = !cast<SchedWrite>("WriteVFRedV_From_" # mx);
     defm _VS : VPseudoTernary<V_M1.vrclass, m.vrclass, V_M1.vrclass, m>,
-               Sched<[WriteVFRedV, ReadVFRedV, ReadVFRedV, ReadVFRedV, ReadVMask]>;
+               Sched<[WriteVFRedV_From_MX, ReadVFRedV, ReadVFRedV, ReadVFRedV,
+                      ReadVMask]>;
   }
 }
 
 multiclass VPseudoVFREDO_VS {
   foreach m = MxListF in {
+    defvar mx = m.MX;
+    defvar WriteVFRedOV_From_MX = !cast<SchedWrite>("WriteVFRedOV_From_" # mx);
     defm _VS : VPseudoTernary<V_M1.vrclass, m.vrclass, V_M1.vrclass, m>,
-               Sched<[WriteVFRedOV, ReadVFRedOV, ReadVFRedOV, ReadVFRedOV, ReadVMask]>;
+               Sched<[WriteVFRedOV_From_MX, ReadVFRedOV, ReadVFRedOV,
+                      ReadVFRedOV, ReadVMask]>;
   }
 }
 
 multiclass VPseudoVFWRED_VS {
   foreach m = MxListF in {
+    defvar mx = m.MX;
+    defvar WriteVFWRedV_From_MX = !cast<SchedWrite>("WriteVFWRedV_From_" # mx);
     defm _VS : VPseudoTernary<V_M1.vrclass, m.vrclass, V_M1.vrclass, m>,
-               Sched<[WriteVFWRedV, ReadVFWRedV, ReadVFWRedV, ReadVFWRedV, ReadVMask]>;
+               Sched<[WriteVFWRedV_From_MX, ReadVFWRedV, ReadVFWRedV,
+                      ReadVFWRedV, ReadVMask]>;
   }
 }
 
 multiclass VPseudoConversion<VReg RetClass,
                              VReg Op1Class,
                              LMULInfo MInfo,
                              string Constraint = ""> {
   let VLMul = MInfo.value in {

llvm/lib/Target/RISCV/RISCVScheduleV.td

 //===-- RISCVScheduleV.td - RISCV Scheduling Definitions V -*- tablegen -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 //===----------------------------------------------------------------------===//
 /// Define scheduler resources associated with def operands.
 
 defvar UpperBoundLMUL = "UpperBound";
 defvar SchedMxList = ["UpperBound", "M1", "M2", "M4", "M8", "MF2", "MF4", "MF8"];
 // Used for widening and narrowing instructions as it doesn't contain M8.
 defvar SchedMxListW = ["UpperBound", "MF8", "MF4", "MF2", "M1", "M2", "M4"];
 defvar SchedMxListFW = ["UpperBound", "MF4", "MF2", "M1", "M2", "M4"];
+// Used for widening floating-point Reduction as it doesn't contain MF8.
+defvar SchedMxListFWRed = ["UpperBound", "MF4", "MF2", "M1", "M2", "M4", "M8"];
 
 // Creates SchedWrite for each (name, LMUL) pair for LMUL in SchedMxList
 multiclass LMULSchedWrites<string name> {
   foreach mx = SchedMxList in {
     def name # "_" # mx : SchedWrite;
   }
 }
 
 // Creates SchedWrite for each (name, LMUL) pair for LMUL in SchedMxListW
 multiclass LMULSchedWritesW<string name> {
   foreach mx = SchedMxListW in {
     def name # "_" # mx : SchedWrite;
   }
 }
 
 // Creates SchedWrite for each (name, LMUL) pair for LMUL in SchedMxListFW
 multiclass LMULSchedWritesFW<string name> {
   foreach mx = SchedMxListFW in {
     def name # "_" # mx : SchedWrite;
   }
 }
 
+// Creates SchedWrite for each (name, LMUL) pair for LMUL in SchedMxListFWRed
+multiclass LMULSchedWritesFWRed<string name> {
+  foreach mx = SchedMxListFWRed in {
+    def name # "_" # mx : SchedWrite;
+  }
+}
+
 // Creates SchedRead for each (name, LMUL) pair for LMUL in SchedMxList
 multiclass LMULSchedReads<string name> {
   foreach mx = SchedMxList in {
     def name # "_" # mx : SchedRead;
   }
 }
 
 // Creates SchedRead for each (name, LMUL) pair for LMUL in SchedMxListW

 // Creates WriteRes for each (name, LMUL, resources) tuple for LMUL
 // in SchedMxListFW
 multiclass LMULWriteResFW<string name, list<ProcResourceKind> resources> {
   foreach mx = SchedMxListFW in {
     def : WriteRes<!cast<SchedWrite>(name # "_" # mx), resources>;
   }
 }
 
+// Creates WriteRes for each (name, LMUL, resources) tuple for LMUL
+// in SchedMxListFWRed
+multiclass LMULWriteResFWRed<string name, list<ProcResourceKind> resources> {
+  foreach mx = SchedMxListFWRed in {
+    def : WriteRes<!cast<SchedWrite>(name # "_" # mx), resources>;
+  }
+}
+
 // Creates ReadAdvance for each (name, LMUL, val) tuple for LMUL
 // in SchedMxList
 multiclass LMULReadAdvance<string name, int val, list<SchedWrite> writes = []> {
   foreach mx = SchedMxList in {
     def : ReadAdvance<!cast<SchedRead>(name # "_" # mx), val, writes>;
   }
 }
 

 defm "" : LMULSchedWritesFW<"WriteVFWCvtFToIV">;
 defm "" : LMULSchedWritesFW<"WriteVFWCvtFToFV">;
 // 13.19. Narrowing Floating-Point/Integer Type-Convert Instructions
 defm "" : LMULSchedWritesFW<"WriteVFNCvtIToFV">;
 defm "" : LMULSchedWritesW<"WriteVFNCvtFToIV">;
 defm "" : LMULSchedWritesFW<"WriteVFNCvtFToFV">;
 
 // 14. Vector Reduction Operations
+// The latency of reduction is determined by the size of the read resource.
+// The LMUL range of read resource(VS2) for reduction operantion is between
+// MF8 and M8. Use the _From suffix to indicate the number of the
+// LMUL from VS2.
 // 14.1. Vector Single-Width Integer Reduction Instructions
-def WriteVIRedV       : SchedWrite;
+defm "" : LMULSchedWrites<"WriteVIRedV_From">;
 // 14.2. Vector Widening Integer Reduction Instructions
-def WriteVIWRedV      : SchedWrite;
+defm "" : LMULSchedWrites<"WriteVIWRedV_From">;
 // 14.3. Vector Single-Width Floating-Point Reduction Instructions
-def WriteVFRedV       : SchedWrite;
-def WriteVFRedOV      : SchedWrite;
+defm "" : LMULSchedWrites<"WriteVFRedV_From">;
+defm "" : LMULSchedWrites<"WriteVFRedOV_From">;
 // 14.4. Vector Widening Floating-Point Reduction Instructions
-def WriteVFWRedV      : SchedWrite;
-def WriteVFWRedOV     : SchedWrite;
+defm "" : LMULSchedWritesFWRed<"WriteVFWRedV_From">;
+defm "" : LMULSchedWritesFWRed<"WriteVFWRedOV_From">;
 
 // 15. Vector Mask Instructions
 // 15.1. Vector Mask-Register Logical Instructions
 defm "" : LMULSchedWrites<"WriteVMALUV">;
 // 15.2. Vector Mask Population Count
 defm "" : LMULSchedWrites<"WriteVMPopV">;
 // 15.3. Vector Find-First-Set Mask Bit
 defm "" : LMULSchedWrites<"WriteVMFFSV">;

 defm "" : LMULWriteResW<"WriteVFWCvtIToFV", []>;
 defm "" : LMULWriteResFW<"WriteVFWCvtFToIV", []>;
 defm "" : LMULWriteResFW<"WriteVFWCvtFToFV", []>;
 defm "" : LMULWriteResFW<"WriteVFNCvtIToFV", []>;
 defm "" : LMULWriteResW<"WriteVFNCvtFToIV", []>;
 defm "" : LMULWriteResFW<"WriteVFNCvtFToFV", []>;
 
 // 14. Vector Reduction Operations
-def : WriteRes<WriteVIRedV, []>;
-def : WriteRes<WriteVIWRedV, []>;
-def : WriteRes<WriteVFRedV, []>;
-def : WriteRes<WriteVFRedOV, []>;
-def : WriteRes<WriteVFWRedV, []>;
-def : WriteRes<WriteVFWRedOV, []>;
+defm "" : LMULWriteRes<"WriteVIRedV_From", []>;
+defm "" : LMULWriteRes<"WriteVIWRedV_From", []>;
+defm "" : LMULWriteRes<"WriteVFRedV_From", []>;
+defm "" : LMULWriteRes<"WriteVFRedOV_From", []>;
+defm "" : LMULWriteResFWRed<"WriteVFWRedV_From", []>;
+defm "" : LMULWriteResFWRed<"WriteVFWRedOV_From", []>;
 
 // 15. Vector Mask Instructions
 defm "" : LMULWriteRes<"WriteVMALUV", []>;
 defm "" : LMULWriteRes<"WriteVMPopV", []>;
 defm "" : LMULWriteRes<"WriteVMFFSV", []>;
 defm "" : LMULWriteRes<"WriteVMSFSV", []>;
 defm "" : LMULWriteRes<"WriteVMIotV", []>;
 defm "" : LMULWriteRes<"WriteVMIdxV", []>;