[RISCV] Define vmclr.m/vmset.m intrinsics.

[khchen]

Summary

Define vmclr.m/vmset.m intrinsics and lower to vmxor.mm/vmxnor.mm.

Ideally all rvv pseudo instructions could be implemented in C header,
but those two instructions don't take an input, codegen can not guarantee
that the source register becomes the same as the destination.

I define two pseduo insts, one would expand to vsetvli + pseduo-v-inst,
then pseudo-v-inst would expand to corresponding v-inst.

[khchen]

update ceudoinstructions comments

[craig.topper]

llvm/lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

65

Drop the default and just pass value at both call sites. There's no strong reason for vmset to the be the default behavior so its better to make it explicit.

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

559

pseduo -> pseudo

llvm/test/CodeGen/RISCV/rvv/vmclr-rv32.ll

3

Can we drop -riscv-no-aliases so that we'll print vmset/vmclr?

[khchen]

1. Address craig.topper's comments.
2. Set BaseInstr as corresponding v-inst, so we only need to define one pseudo instruction.

[craig.topper]

LGTM with that change.

llvm/lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

240

Just use MBBI->getDebugLoc() here and drop MI.

[frasercrmck]

Out of curiosity, what would the pros/cons be between instead expanding these early in the ISel custom inserter vs. late in the expand-pseudos pass as you've done? I would imagine that the vmxor and vmxnor instructions will have better integration with the rest of the compiler (scheduling info, for instance) so we'd prefer to have those around. Otherwise we might have to add special-cases for these pseudos to treat them like their underlying instructions.

llvm/include/llvm/IR/IntrinsicsRISCV.td

407

nit: the class is called RISCVNullaryIntrinsic producing llvm_anyvector_ty but says it outputs "mask type". Is that too specific? The other intrinsic classes which output "mask type" are called e.g. RISCVMaskXXX. Should this be named e.g. RISCVMaskNullary(Intrinsic)? I don't think we expect other nullaries, do we?

llvm/lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

238

Opcode would be a more familiar casing for this variable name.

246

Is there a reason not to use BuildMI(..., DstReg) instead of this extra .addReg?

[craig.topper]

In D93849#2472514, @frasercrmck wrote:

Out of curiosity, what would the pros/cons be between instead expanding these early in the ISel custom inserter vs. late in the expand-pseudos pass as you've done? I would imagine that the vmxor and vmxnor instructions will have better integration with the rest of the compiler (scheduling info, for instance) so we'd prefer to have those around. Otherwise we might have to add special-cases for these pseudos to treat them like their underlying instructions.

If we expand them early there's no guarantee the register allocator would pick the same register for the inputs as the output. Our downstream repo and probably BSC's repo was expanding them in a custom inserter and putting the same vreg on the inputs and the outputs with the input operands marked with undef. Which seemed to work, but I'm surprised that isn't flagged as a violation of SSA by the machine verifier. Expanding after register allocation allows us to force the source registers explicitly.

I would imagine a high performance out of order processor would want to detect these instructions as a special case of xor/xnor and not read the data from the input register. Whether they would check that the inputs and the output are the same register or just the inputs are the same register I'm not sure. X86 CPUs for many generations have recognized xor eax, eax as special zero idiom that doesn't have a dependency on the previous value of eax. The X86 backend also uses pseudos and a post RA expansion. For scheduling, X86 has various WriteZero classes the pseudos are assigned to and I believe there is special code in most of the scheduler models to detect xor with same input post RA.

[khchen]

Address frasercrmck's comments, thanks!

I did this way jsut because I think PseduoInst should be expanded after RA
which can ensure source and dest use the same register(non-SSA). I didn't
consider the pros/cons yet.
Could you please share me is there any target also expand PsedoInst in the
ISel custom inserter? I'm wondering that in order to get more precise schedule info,
it means ideally any target would prefer expand pseudo as early as possible.
Maybe the another way is adding schedule info for those pseudo insts?

[frasercrmck]

In D93849#2472560, @craig.topper wrote:

If we expand them early there's no guarantee the register allocator would pick the same register for the inputs as the output. Our downstream repo and probably BSC's repo was expanding them in a custom inserter and putting the same vreg on the inputs and the outputs with the input operands marked with undef. Which seemed to work, but I'm surprised that isn't flagged as a violation of SSA by the machine verifier. Expanding after register allocation allows us to force the source registers explicitly.

I would imagine a high performance out of order processor would want to detect these instructions as a special case of xor/xnor and not read the data from the input register. Whether they would check that the inputs and the output are the same register or just the inputs are the same register I'm not sure. X86 CPUs for many generations have recognized xor eax, eax as special zero idiom that doesn't have a dependency on the previous value of eax. The X86 backend also uses pseudos and a post RA expansion. For scheduling, X86 has various WriteZero classes the pseudos are assigned to and I believe there is special code in most of the scheduler models to detect xor with same input post RA.

I similarly don't understand how the pre-RA undef inputs managed to "just work".

That's really interesting to learn about the X86 aliases/idioms, thanks. The processors I've worked on have been rather different so that might explain where I was coming from (see below).

In D93849#2472580, @khchen wrote:

I did this way jsut because I think PseduoInst should be expanded after RA
which can ensure source and dest use the same register(non-SSA). I didn't
consider the pros/cons yet.
Could you please share me is there any target also expand PsedoInst in the
ISel custom inserter? I'm wondering that in order to get more precise schedule info,
it means ideally any target would prefer expand pseudo as early as possible.
Maybe the another way is adding schedule info for those pseudo insts?

My previous (downstream) project had a complex auto-generated per-operand scheduling model with all sorts of bypasses, all to satisfy an exposed pipeline, so it was important for us to have information about the underlying instruction as accurate as possible as early as possible to avoid hundreds of special cases and copy/paste code in the rest of the compiler. So our general approach was to limit the number pseudos that survived past pre-RA scheduling. I can't point to an upstream target that does this. I haven't yet familiarised myself with the RISCV scheduling info so I can't be of much help about the best approach there. If Craig's right then it sounds like it's up to the RISC-V implementation about whether aliases could be "special" from the processor's point of view or whether they're just assembler syntactic-sugar.

I think Craig's comments help resolve the matter about where to do this expansion. So LGTM!

[khchen]

Those different chooses reason are interesting to learn, I really appreciate your information.

llvm/include/llvm/IR/IntrinsicsRISCV.td

let TargetPrefix = "riscv" in {
   // For mask unary operations with mask type in/out with mask
   // Output: (mask type output)
   // Input: (mask type maskedoff, mask type vector_in, mask, vl)
   class RISCVMaskUnaryMOutMask
         : Intrinsic<[llvm_anyint_ty],
                     [LLVMMatchType<0>, LLVMMatchType<0>,
                      LLVMMatchType<0>, llvm_anyint_ty],
                     [IntrNoMem]>, RISCVVIntrinsic;
+  // Output: (vector)
+  // Input: (vl)
+  class RISCVNullaryIntrinsic
+        : Intrinsic<[llvm_anyvector_ty],
+                    [llvm_anyint_ty],
+                    [IntrNoMem]>, RISCVVIntrinsic;
 
   multiclass RISCVUSLoad {
     def "int_riscv_" # NAME : RISCVUSLoad;
     def "int_riscv_" # NAME # "_mask" : RISCVUSLoadMask;
   }
   multiclass RISCVSLoad {
     def "int_riscv_" # NAME : RISCVSLoad;
     def "int_riscv_" # NAME # "_mask" : RISCVSLoadMask;
let TargetPrefix = "riscv" in {
   def int_riscv_vmand: RISCVBinaryAAANoMask;
   def int_riscv_vmnand: RISCVBinaryAAANoMask;
   def int_riscv_vmandnot: RISCVBinaryAAANoMask;
   def int_riscv_vmxor: RISCVBinaryAAANoMask;
   def int_riscv_vmor: RISCVBinaryAAANoMask;
   def int_riscv_vmnor: RISCVBinaryAAANoMask;
   def int_riscv_vmornot: RISCVBinaryAAANoMask;
   def int_riscv_vmxnor: RISCVBinaryAAANoMask;
+  def int_riscv_vmclr : RISCVNullaryIntrinsic;
+  def int_riscv_vmset : RISCVNullaryIntrinsic;
 
   defm vpopc : RISCVMaskUnarySOut;
   defm vfirst : RISCVMaskUnarySOut;
   defm vmsbf : RISCVMaskUnaryMOut;
   defm vmsof : RISCVMaskUnaryMOut;
   defm vmsif : RISCVMaskUnaryMOut;
 
   // Output: (vector)
   // Input: (mask type input, vl)
   def int_riscv_viota : Intrinsic<[llvm_anyvector_ty],
                                   [LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
                                    llvm_anyint_ty],
                                   [IntrNoMem]>, RISCVVIntrinsic;
   // Output: (vector)
   // Input: (maskedoff, mask type vector_in, mask, vl)
   def int_riscv_viota_mask : Intrinsic<[llvm_anyvector_ty],
                                        [LLVMMatchType<0>,
                                         LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
                                         LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
                                         llvm_anyint_ty],
                                        [IntrNoMem]>, RISCVVIntrinsic;
   // Output: (vector)
   // Input: (vl)
-  def int_riscv_vid : Intrinsic<[llvm_anyvector_ty],
-                                [llvm_anyint_ty],
-                                [IntrNoMem]>, RISCVVIntrinsic;
+  def int_riscv_vid : RISCVNullaryIntrinsic;
+
   // Output: (vector)
   // Input: (maskedoff, mask, vl)
   def int_riscv_vid_mask : Intrinsic<[llvm_anyvector_ty],
                                      [LLVMMatchType<0>,
                                       LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
                                       llvm_anyint_ty],
                                      [IntrNoMem]>, RISCVVIntrinsic;
 
 } // TargetPrefix = "riscv"

llvm/lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

bool expandLoadAddress(MachineBasicBlock &MBB,
                          MachineBasicBlock::iterator &NextMBBI);
   bool expandLoadTLSIEAddress(MachineBasicBlock &MBB,
                               MachineBasicBlock::iterator MBBI,
                               MachineBasicBlock::iterator &NextMBBI);
   bool expandLoadTLSGDAddress(MachineBasicBlock &MBB,
                               MachineBasicBlock::iterator MBBI,
                               MachineBasicBlock::iterator &NextMBBI);
   bool expandVSetVL(MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI);
+  bool expandVMSET_VMCLR(MachineBasicBlock &MBB,
+                         MachineBasicBlock::iterator MBBI, unsigned Opcode);
 };
 
 char RISCVExpandPseudo::ID = 0;
 
 bool RISCVExpandPseudo::runOnMachineFunction(MachineFunction &MF) {
   TII = static_cast<const RISCVInstrInfo *>(MF.getSubtarget().getInstrInfo());
   bool Modified = false;
   for (auto &MBB : MF)
     Modified |= expandMBB(MBB);
bool RISCVExpandPseudo::expandMI(MachineBasicBlock &MBB,
   case RISCV::PseudoLA:
     return expandLoadAddress(MBB, MBBI, NextMBBI);
   case RISCV::PseudoLA_TLS_IE:
     return expandLoadTLSIEAddress(MBB, MBBI, NextMBBI);
   case RISCV::PseudoLA_TLS_GD:
     return expandLoadTLSGDAddress(MBB, MBBI, NextMBBI);
   case RISCV::PseudoVSETVLI:
     return expandVSetVL(MBB, MBBI);
+  case RISCV::PseudoVMCLR_M_B1:
+  case RISCV::PseudoVMCLR_M_B2:
+  case RISCV::PseudoVMCLR_M_B4:
+  case RISCV::PseudoVMCLR_M_B8:
+  case RISCV::PseudoVMCLR_M_B16:
+  case RISCV::PseudoVMCLR_M_B32:
+  case RISCV::PseudoVMCLR_M_B64:
+    // vmclr.m vd => vmxor.mm vd, vd, vd
+    return expandVMSET_VMCLR(MBB, MBBI, RISCV::VMXOR_MM);
+  case RISCV::PseudoVMSET_M_B1:
+  case RISCV::PseudoVMSET_M_B2:
+  case RISCV::PseudoVMSET_M_B4:
+  case RISCV::PseudoVMSET_M_B8:
+  case RISCV::PseudoVMSET_M_B16:
+  case RISCV::PseudoVMSET_M_B32:
+  case RISCV::PseudoVMSET_M_B64:
+    // vmset.m vd => vmxnor.mm vd, vd, vd
+    return expandVMSET_VMCLR(MBB, MBBI, RISCV::VMXNOR_MM);
   }
 
   return false;
 }
 
 bool RISCVExpandPseudo::expandAuipcInstPair(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
     MachineBasicBlock::iterator &NextMBBI, unsigned FlagsHi,
BuildMI(MBB, MBBI, DL, Desc)
       .addReg(DstReg, RegState::Define | getDeadRegState(DstIsDead))
       .add(MBBI->getOperand(1))  // VL
       .add(MBBI->getOperand(2)); // VType
 
   MBBI->eraseFromParent(); // The pseudo instruction is gone now.
   return true;
 }
 
+bool RISCVExpandPseudo::expandVMSET_VMCLR(MachineBasicBlock &MBB,
+                                          MachineBasicBlock::iterator MBBI,
+                                          unsigned Opcode) {
+  DebugLoc DL = MBBI->getDebugLoc();
+  Register DstReg = MBBI->getOperand(0).getReg();
+  const MCInstrDesc &Desc = TII->get(Opcode);
+  BuildMI(MBB, MBBI, DL, Desc, DstReg)
+      .addReg(DstReg, RegState::Undef)
+      .addReg(DstReg, RegState::Undef);
+  MBBI->eraseFromParent(); // The pseudo instruction is gone now.
+  return true;
+}
+
 } // end of anonymous namespace
 
 INITIALIZE_PASS(RISCVExpandPseudo, "riscv-expand-pseudo",
                 RISCV_EXPAND_PSEUDO_NAME, false, false)
 namespace llvm {
 
 FunctionPass *createRISCVExpandPseudoPass() { return new RISCVExpandPseudo(); }
 
 } // end of namespace llvm

llvm/lib/Target/RISCV/RISCVInstrInfoVPseudos.td

class VPseudoNullaryMask<VReg RegClass>:
   let Constraints ="$rd = $merge";
   let Uses = [VL, VTYPE];
   let VLIndex = 3;
   let SEWIndex = 4;
   let MergeOpIndex = 1;
   let BaseInstr = !cast<Instruction>(PseudoToVInst<NAME>.VInst);
 }
 
+// Nullary for pseudo instructions. They are expanded in
+// RISCVExpandPseudoInsts pass.
+class VPseudoNullaryPseudoM<string BaseInst>
+       : Pseudo<(outs VR:$rd), (ins GPR:$vl, ixlenimm:$sew), []>,
+       RISCVVPseudo {
+  let mayLoad = 0;
+  let mayStore = 0;
+  let hasSideEffects = 0;
+  let usesCustomInserter = 1;
+  let Uses = [VL, VTYPE];
+  let VLIndex = 1;
+  let SEWIndex = 2;
+  // BaseInstr is not used in RISCVExpandPseudoInsts pass.
+  // Just fill a corresponding real v-inst to pass tablegen check.
+  let BaseInstr = !cast<Instruction>(BaseInst);
+}
+
 // RetClass could be GPR or VReg.
 class VPseudoUnaryNoMask<DAGOperand RetClass, VReg OpClass, string Constraint = ""> :
         Pseudo<(outs RetClass:$rd),
                (ins OpClass:$rs2, GPR:$vl, ixlenimm:$sew), []>,
         RISCVVPseudo {
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
   let usesCustomInserter = 1;
   let Constraints = Constraint;
multiclass VPseudoMaskNullaryV {
   foreach m = MxList.m in {
     let VLMul = m.value in {
       def "_V_" # m.MX : VPseudoNullaryNoMask<m.vrclass>;
       def "_V_" # m.MX # "_MASK" : VPseudoNullaryMask<m.vrclass>;
     }
   }
 }
 
+multiclass VPseudoNullaryPseudoM <string BaseInst> {
+  foreach mti = AllMasks in {
+    let VLMul = mti.LMul.value in {
+      def "_M_" # mti.BX : VPseudoNullaryPseudoM<BaseInst # "_MM">;
+    }
+  }
+}
+
 multiclass VPseudoUnaryV_M {
   defvar constraint = "@earlyclobber $rd";
   foreach m = MxList.m in {
     let VLMul = m.value in {
       def "_" # m.MX : VPseudoUnaryNoMask<m.vrclass, VR, constraint>;
       def "_" # m.MX # "_MASK" : VPseudoUnaryMask<m.vrclass, VR, constraint>;
     }
   }
def : Pat<(vti.Vector (!cast<Intrinsic>(intrinsic # "_mask")
                           (vti.Vector vti.RegClass:$merge),
                           (vti.Mask V0), (XLenVT GPR:$vl))),
                           (!cast<Instruction>(instruction#"_V_" # vti.LMul.MX # "_MASK")
                           vti.RegClass:$merge, (vti.Mask V0),
                           (NoX0 GPR:$vl), vti.SEW)>;
   }
 }
 
+multiclass VPatNullaryM<string intrinsic, string inst> {
+  foreach mti = AllMasks in
+    def : Pat<(mti.Mask (!cast<Intrinsic>(intrinsic)
+                        (XLenVT GPR:$vl))),
+                        (!cast<Instruction>(inst#"_M_"#mti.BX)
+                        (NoX0 GPR:$vl), mti.SEW)>;
+}
+
+
 multiclass VPatBinary<string intrinsic,
                       string inst,
                       string kind,
                       ValueType result_type,
                       ValueType op1_type,
                       ValueType op2_type,
                       ValueType mask_type,
                       int sew,

 defm PseudoVMNAND: VPseudoBinaryM_MM;
 defm PseudoVMANDNOT: VPseudoBinaryM_MM;
 defm PseudoVMXOR: VPseudoBinaryM_MM;
 defm PseudoVMOR: VPseudoBinaryM_MM;
 defm PseudoVMNOR: VPseudoBinaryM_MM;
 defm PseudoVMORNOT: VPseudoBinaryM_MM;
 defm PseudoVMXNOR: VPseudoBinaryM_MM;
 
+// Pseudo insturctions
+defm PseudoVMCLR : VPseudoNullaryPseudoM<"VMXOR">;
+defm PseudoVMSET : VPseudoNullaryPseudoM<"VMXNOR">;
+
 //===----------------------------------------------------------------------===//
 // 16.2. Vector mask population count vpopc
 //===----------------------------------------------------------------------===//
 
 defm PseudoVPOPC: VPseudoUnaryS_M;
 
 //===----------------------------------------------------------------------===//
 // 16.3. vfirst find-first-set mask bit

 defm "" : VPatBinaryM_MM<"int_riscv_vmnand", "PseudoVMNAND">;
 defm "" : VPatBinaryM_MM<"int_riscv_vmandnot", "PseudoVMANDNOT">;
 defm "" : VPatBinaryM_MM<"int_riscv_vmxor", "PseudoVMXOR">;
 defm "" : VPatBinaryM_MM<"int_riscv_vmor", "PseudoVMOR">;
 defm "" : VPatBinaryM_MM<"int_riscv_vmnor", "PseudoVMNOR">;
 defm "" : VPatBinaryM_MM<"int_riscv_vmornot", "PseudoVMORNOT">;
 defm "" : VPatBinaryM_MM<"int_riscv_vmxnor", "PseudoVMXNOR">;
 
+// pseudo instructions
+defm "" : VPatNullaryM<"int_riscv_vmclr", "PseudoVMCLR">;
+defm "" : VPatNullaryM<"int_riscv_vmset", "PseudoVMSET">;
+
 //===----------------------------------------------------------------------===//
 // 16.2. Vector mask population count vpopc
 //===----------------------------------------------------------------------===//
 defm "" : VPatUnaryS_M<"int_riscv_vpopc", "PseudoVPOPC">;
 
 //===----------------------------------------------------------------------===//
 // 16.3. vfirst find-first-set mask bit
 //===----------------------------------------------------------------------===//

llvm/test/CodeGen/RISCV/rvv/vmclr-rv32.ll

+; RUN: llc -mtriple=riscv32 -mattr=+experimental-v,+experimental-zfh -verify-machineinstrs \
+; RUN:   < %s | FileCheck %s
+declare <vscale x 1 x i1> @llvm.riscv.vmclr.nxv1i1(
+  i32);
+
+define <vscale x 1 x i1> @intrinsic_vmclr_m_pseudo_nxv1i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv1i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf8
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 1 x i1> @llvm.riscv.vmclr.nxv1i1(
+    i32 %0)
+
+  ret <vscale x 1 x i1> %a
+}
+
+declare <vscale x 2 x i1> @llvm.riscv.vmclr.nxv2i1(
+  i32);
+
+define <vscale x 2 x i1> @intrinsic_vmclr_m_pseudo_nxv2i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv2i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf4
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 2 x i1> @llvm.riscv.vmclr.nxv2i1(
+    i32 %0)
+
+  ret <vscale x 2 x i1> %a
+}
+
+declare <vscale x 4 x i1> @llvm.riscv.vmclr.nxv4i1(
+  i32);
+
+define <vscale x 4 x i1> @intrinsic_vmclr_m_pseudo_nxv4i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv4i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf2
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 4 x i1> @llvm.riscv.vmclr.nxv4i1(
+    i32 %0)
+
+  ret <vscale x 4 x i1> %a
+}
+
+declare <vscale x 8 x i1> @llvm.riscv.vmclr.nxv8i1(
+  i32);
+
+define <vscale x 8 x i1> @intrinsic_vmclr_m_pseudo_nxv8i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv8i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m1
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 8 x i1> @llvm.riscv.vmclr.nxv8i1(
+    i32 %0)
+
+  ret <vscale x 8 x i1> %a
+}
+
+declare <vscale x 16 x i1> @llvm.riscv.vmclr.nxv16i1(
+  i32);
+
+define <vscale x 16 x i1> @intrinsic_vmclr_m_pseudo_nxv16i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv16i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m2
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 16 x i1> @llvm.riscv.vmclr.nxv16i1(
+    i32 %0)
+
+  ret <vscale x 16 x i1> %a
+}
+
+declare <vscale x 32 x i1> @llvm.riscv.vmclr.nxv32i1(
+  i32);
+
+define <vscale x 32 x i1> @intrinsic_vmclr_m_pseudo_nxv32i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv32i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m4
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 32 x i1> @llvm.riscv.vmclr.nxv32i1(
+    i32 %0)
+
+  ret <vscale x 32 x i1> %a
+}
+
+declare <vscale x 64 x i1> @llvm.riscv.vmclr.nxv64i1(
+  i32);
+
+define <vscale x 64 x i1> @intrinsic_vmclr_m_pseudo_nxv64i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv64i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m8
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 64 x i1> @llvm.riscv.vmclr.nxv64i1(
+    i32 %0)
+
+  ret <vscale x 64 x i1> %a
+}

llvm/test/CodeGen/RISCV/rvv/vmclr-rv64.ll

+; RUN: llc -mtriple=riscv64 -mattr=+experimental-v,+experimental-zfh -verify-machineinstrs \
+; RUN:   < %s | FileCheck %s
+declare <vscale x 1 x i1> @llvm.riscv.vmclr.nxv1i1(
+  i64);
+
+define <vscale x 1 x i1> @intrinsic_vmclr_m_pseudo_nxv1i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv1i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf8
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 1 x i1> @llvm.riscv.vmclr.nxv1i1(
+    i64 %0)
+
+  ret <vscale x 1 x i1> %a
+}
+
+declare <vscale x 2 x i1> @llvm.riscv.vmclr.nxv2i1(
+  i64);
+
+define <vscale x 2 x i1> @intrinsic_vmclr_m_pseudo_nxv2i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv2i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf4
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 2 x i1> @llvm.riscv.vmclr.nxv2i1(
+    i64 %0)
+
+  ret <vscale x 2 x i1> %a
+}
+
+declare <vscale x 4 x i1> @llvm.riscv.vmclr.nxv4i1(
+  i64);
+
+define <vscale x 4 x i1> @intrinsic_vmclr_m_pseudo_nxv4i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv4i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf2
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 4 x i1> @llvm.riscv.vmclr.nxv4i1(
+    i64 %0)
+
+  ret <vscale x 4 x i1> %a
+}
+
+declare <vscale x 8 x i1> @llvm.riscv.vmclr.nxv8i1(
+  i64);
+
+define <vscale x 8 x i1> @intrinsic_vmclr_m_pseudo_nxv8i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv8i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m1
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 8 x i1> @llvm.riscv.vmclr.nxv8i1(
+    i64 %0)
+
+  ret <vscale x 8 x i1> %a
+}
+
+declare <vscale x 16 x i1> @llvm.riscv.vmclr.nxv16i1(
+  i64);
+
+define <vscale x 16 x i1> @intrinsic_vmclr_m_pseudo_nxv16i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv16i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m2
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 16 x i1> @llvm.riscv.vmclr.nxv16i1(
+    i64 %0)
+
+  ret <vscale x 16 x i1> %a
+}
+
+declare <vscale x 32 x i1> @llvm.riscv.vmclr.nxv32i1(
+  i64);
+
+define <vscale x 32 x i1> @intrinsic_vmclr_m_pseudo_nxv32i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv32i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m4
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 32 x i1> @llvm.riscv.vmclr.nxv32i1(
+    i64 %0)
+
+  ret <vscale x 32 x i1> %a
+}
+
+declare <vscale x 64 x i1> @llvm.riscv.vmclr.nxv64i1(
+  i64);
+
+define <vscale x 64 x i1> @intrinsic_vmclr_m_pseudo_nxv64i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmclr_m_pseudo_nxv64i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m8
+; CHECK:       vmclr.m {{v[0-9]+}}
+  %a = call <vscale x 64 x i1> @llvm.riscv.vmclr.nxv64i1(
+    i64 %0)
+
+  ret <vscale x 64 x i1> %a
+}

llvm/test/CodeGen/RISCV/rvv/vmset-rv32.ll

+; RUN: llc -mtriple=riscv32 -mattr=+experimental-v,+experimental-zfh -verify-machineinstrs \
+; RUN:   < %s | FileCheck %s
+declare <vscale x 1 x i1> @llvm.riscv.vmset.nxv1i1(
+  i32);
+
+define <vscale x 1 x i1> @intrinsic_vmset_m_pseudo_nxv1i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv1i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf8
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 1 x i1> @llvm.riscv.vmset.nxv1i1(
+    i32 %0)
+
+  ret <vscale x 1 x i1> %a
+}
+
+declare <vscale x 2 x i1> @llvm.riscv.vmset.nxv2i1(
+  i32);
+
+define <vscale x 2 x i1> @intrinsic_vmset_m_pseudo_nxv2i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv2i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf4
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 2 x i1> @llvm.riscv.vmset.nxv2i1(
+    i32 %0)
+
+  ret <vscale x 2 x i1> %a
+}
+
+declare <vscale x 4 x i1> @llvm.riscv.vmset.nxv4i1(
+  i32);
+
+define <vscale x 4 x i1> @intrinsic_vmset_m_pseudo_nxv4i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv4i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf2
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 4 x i1> @llvm.riscv.vmset.nxv4i1(
+    i32 %0)
+
+  ret <vscale x 4 x i1> %a
+}
+
+declare <vscale x 8 x i1> @llvm.riscv.vmset.nxv8i1(
+  i32);
+
+define <vscale x 8 x i1> @intrinsic_vmset_m_pseudo_nxv8i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv8i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m1
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 8 x i1> @llvm.riscv.vmset.nxv8i1(
+    i32 %0)
+
+  ret <vscale x 8 x i1> %a
+}
+
+declare <vscale x 16 x i1> @llvm.riscv.vmset.nxv16i1(
+  i32);
+
+define <vscale x 16 x i1> @intrinsic_vmset_m_pseudo_nxv16i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv16i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m2
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 16 x i1> @llvm.riscv.vmset.nxv16i1(
+    i32 %0)
+
+  ret <vscale x 16 x i1> %a
+}
+
+declare <vscale x 32 x i1> @llvm.riscv.vmset.nxv32i1(
+  i32);
+
+define <vscale x 32 x i1> @intrinsic_vmset_m_pseudo_nxv32i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv32i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m4
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 32 x i1> @llvm.riscv.vmset.nxv32i1(
+    i32 %0)
+
+  ret <vscale x 32 x i1> %a
+}
+
+declare <vscale x 64 x i1> @llvm.riscv.vmset.nxv64i1(
+  i32);
+
+define <vscale x 64 x i1> @intrinsic_vmset_m_pseudo_nxv64i1(i32 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv64i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m8
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 64 x i1> @llvm.riscv.vmset.nxv64i1(
+    i32 %0)
+
+  ret <vscale x 64 x i1> %a
+}

llvm/test/CodeGen/RISCV/rvv/vmset-rv64.ll

+; RUN: llc -mtriple=riscv64 -mattr=+experimental-v,+experimental-zfh -verify-machineinstrs \
+; RUN:   < %s | FileCheck %s
+declare <vscale x 1 x i1> @llvm.riscv.vmset.nxv1i1(
+  i64);
+
+define <vscale x 1 x i1> @intrinsic_vmset_m_pseudo_nxv1i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv1i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf8
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 1 x i1> @llvm.riscv.vmset.nxv1i1(
+    i64 %0)
+
+  ret <vscale x 1 x i1> %a
+}
+
+declare <vscale x 2 x i1> @llvm.riscv.vmset.nxv2i1(
+  i64);
+
+define <vscale x 2 x i1> @intrinsic_vmset_m_pseudo_nxv2i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv2i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf4
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 2 x i1> @llvm.riscv.vmset.nxv2i1(
+    i64 %0)
+
+  ret <vscale x 2 x i1> %a
+}
+
+declare <vscale x 4 x i1> @llvm.riscv.vmset.nxv4i1(
+  i64);
+
+define <vscale x 4 x i1> @intrinsic_vmset_m_pseudo_nxv4i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv4i1
+; CHECK:       vsetvli {{.*}}, a0, e8,mf2
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 4 x i1> @llvm.riscv.vmset.nxv4i1(
+    i64 %0)
+
+  ret <vscale x 4 x i1> %a
+}
+
+declare <vscale x 8 x i1> @llvm.riscv.vmset.nxv8i1(
+  i64);
+
+define <vscale x 8 x i1> @intrinsic_vmset_m_pseudo_nxv8i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv8i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m1
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 8 x i1> @llvm.riscv.vmset.nxv8i1(
+    i64 %0)
+
+  ret <vscale x 8 x i1> %a
+}
+
+declare <vscale x 16 x i1> @llvm.riscv.vmset.nxv16i1(
+  i64);
+
+define <vscale x 16 x i1> @intrinsic_vmset_m_pseudo_nxv16i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv16i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m2
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 16 x i1> @llvm.riscv.vmset.nxv16i1(
+    i64 %0)
+
+  ret <vscale x 16 x i1> %a
+}
+
+declare <vscale x 32 x i1> @llvm.riscv.vmset.nxv32i1(
+  i64);
+
+define <vscale x 32 x i1> @intrinsic_vmset_m_pseudo_nxv32i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv32i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m4
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 32 x i1> @llvm.riscv.vmset.nxv32i1(
+    i64 %0)
+
+  ret <vscale x 32 x i1> %a
+}
+
+declare <vscale x 64 x i1> @llvm.riscv.vmset.nxv64i1(
+  i64);
+
+define <vscale x 64 x i1> @intrinsic_vmset_m_pseudo_nxv64i1(i64 %0) nounwind {
+entry:
+; CHECK-LABEL: intrinsic_vmset_m_pseudo_nxv64i1
+; CHECK:       vsetvli {{.*}}, a0, e8,m8
+; CHECK:       vmset.m {{v[0-9]+}}
+  %a = call <vscale x 64 x i1> @llvm.riscv.vmset.nxv64i1(
+    i64 %0)
+
+  ret <vscale x 64 x i1> %a
+}