[RISCV][Clang] Add support for Zmmul extension

[ksyx]

Summary

This patch implements recently ratified extension Zmmul, a subextension of M (Integer Multiplication and Division) consisting only multiplication part of it.

[craig.topper]

Do we need to add this to RISCVTargetStreamer::emitTargetAttributes?

[jrtc27]

In D103313#2787316, @craig.topper wrote:

Do we need to add this to RISCVTargetStreamer::emitTargetAttributes?

Yes, that, but also RISCVAsmParser::parseDirectiveAttribute and ELFObjectFileBase::getRISCVFeatures, though it seems that last one has been a bit neglected.

clang/lib/Basic/Targets/RISCV.cpp

268–269

These are supposed to be (approximately) sorted in terms of their canonical ordering. I don't know where Zmmul is proposed to go relative to the other Zfoo's, but it belongs down somewhere in there, I guess alphabetical if we don't know any better, not up here with the single-letter extensions.

llvm/lib/Target/RISCV/RISCVSubtarget.h

48

Ditto

152

Ditto

[jrtc27]

In D103313#2787326, @jrtc27 wrote:

In D103313#2787316, @craig.topper wrote:

Do we need to add this to RISCVTargetStreamer::emitTargetAttributes?

Yes, that, but also RISCVAsmParser::parseDirectiveAttribute and ELFObjectFileBase::getRISCVFeatures, though it seems that last one has been a bit neglected.

(we really need to cut down on how many places parse/produce arch strings...)

[ksyx]

• Move down zmmul lines to fit the ordering
• Add Zmmul into parseDirectiveAttribute

[ksyx]

I have took a look into ELFObjectFile.cpp but I am not sure what work I need to do there since it seems other Z* extensions are not being handled there.

llvm/lib/Target/RISCV/RISCVSubtarget.h

48

The pattern here seems to be that Z* extensions is following its parent their parent extensions, should we follow it? (See B and V)

[ksyx]

llvm/lib/Target/RISCV/RISCVSubtarget.h

48

But for Zfh it is not following HasStdExtF 🤔

[Jim]

It should add arch string tests such as in clang/test/Driver/riscv-arch.c.

[ksyx]

add: zmmul arch string test

[JiahaoChenConor]

Hi, do you still continue working on this patch?

[ksyx]

In D103313#3359310, @JiahaoChenConor wrote:

Hi, do you still continue working on this patch?

In fact, not pretty sure what else I need to improve on this.

[craig.topper]

In D103313#3359628, @ksyx wrote:

In D103313#3359310, @JiahaoChenConor wrote:

Hi, do you still continue working on this patch?

In fact, not pretty sure what else I need to improve on this.

At minimum it probably needs to rebased because RISCVISAInfo.cpp has been added to cut down on some duplication.

[ksyx]

Rebase and adapt to new ISAInfo module.

[ksyx]

Fix patch apply failed

[ksyx]

The Zmmul extension has just ratified. Update patch to rebase to current main.

[craig.topper]

llvm/lib/Target/RISCV/RISCV.td

18

This will cause Zmmul to appear in the ELF attributes any time is M is enabled. Is that what we want?

llvm/test/CodeGen/RISCV/attributes.ll

86–88

Is this a backwards compatibility issue?

[ksyx]

Make Zmmul extension independent instead of implied by M extension for backward ELF attribute compatibility.

[ksyx]

ping :)

[craig.topper]

Need to update RISCVTargetLowering::decomposeMulByConstant.

[ksyx]

Rebase and update use of M extension existence check to include Zmmul whenever needed.

[craig.topper]

llvm/lib/Support/RISCVISAInfo.cpp

80

If it is ratified, is the version really 0.1?

[asb]

llvm/lib/Support/RISCVISAInfo.cpp

80

It looks like it should be 1.0 now https://github.com/riscv/riscv-isa-manual/commit/f518c259c008f926eba4aba67804f62531b6e94b

[ksyx]

Update zmmul version & rebase

clang/lib/Basic/Targets/RISCV.cpp

for (auto &Extension : ISAInfo->getExtensions()) {
     auto ExtName = Extension.first;
     auto ExtInfo = Extension.second;
     unsigned Version =
         (ExtInfo.MajorVersion * 1000000) + (ExtInfo.MinorVersion * 1000);
 
     Builder.defineMacro(Twine("__riscv_", ExtName), Twine(Version));
   }
 
-  if (ISAInfo->hasExtension("m")) {
+  if (ISAInfo->hasExtension("m") || ISAInfo->hasExtension("zmmul"))
     Builder.defineMacro("__riscv_mul");
+
+  if (ISAInfo->hasExtension("m")) {
     Builder.defineMacro("__riscv_div");
     Builder.defineMacro("__riscv_muldiv");
   }
 
   if (ISAInfo->hasExtension("a")) {
     Builder.defineMacro("__riscv_atomic");
     Builder.defineMacro("__GCC_HAVE_SYNC_COMPARE_AND_SWAP_1");
     Builder.defineMacro("__GCC_HAVE_SYNC_COMPARE_AND_SWAP_2");
if (ISAInfo->isSupportedExtensionFeature(Feature))
     return ISAInfo->hasExtension(Feature);
 
   return false;
 }
 
 /// Perform initialization based on the user configured set of features.
 bool RISCVTargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
                                            DiagnosticsEngine &Diags) {
   unsigned XLen = getTriple().isArch64Bit() ? 64 : 32;
   auto ParseResult = llvm::RISCVISAInfo::parseFeatures(XLen, Features);
   if (!ParseResult) {
     std::string Buffer;
     llvm::raw_string_ostream OutputErrMsg(Buffer);
     handleAllErrors(ParseResult.takeError(), [&](llvm::StringError &ErrMsg) {
       OutputErrMsg << ErrMsg.getMessage();
     });
     Diags.Report(diag::err_invalid_feature_combination) << OutputErrMsg.str();
     return false;

clang/test/Driver/riscv-arch.c

 // RUN: %clang --target=riscv32-unknown-elf -march=rv32izhinx1p0 -### %s \
 // RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-ZHINX-GOOD %s
 // RV32-ZHINX-GOOD: "-target-feature" "+zhinx"
 
 // RUN: %clang --target=riscv32-unknown-elf -march=rv32izhinx0p1 -### %s \
 // RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-ZHINX-BADVERS %s
 // RV32-ZHINX-BADVERS: error: invalid arch name 'rv32izhinx0p1'
 // RV32-ZHINX-BADVERS: unsupported version number 0.1 for extension 'zhinx'
+
+// RUN: %clang -target riscv32-unknown-elf -march=rv32i_zmmul2p0 -### %s \
+// RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-ZMMUL-BADVERS %s
+// RV32-ZMMUL-BADVERS: error: invalid arch name 'rv32i_zmmul2p0'
+// RV32-ZMMUL-BADVERS: unsupported version number 2.0 for extension
+
+// RUN: %clang -target riscv32-unknown-elf -march=rv32i_zmmul1p0 -### %s \
+// RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-ZMMUL-GOODVERS %s
+// RV32-ZMMUL-GOODVERS: "-target-feature" "+zmmul"

llvm/lib/Support/RISCVISAInfo.cpp

static const RISCVSupportedExtension SupportedExtensions[] = {
     {"zksed", RISCVExtensionVersion{1, 0}},
     {"zksh", RISCVExtensionVersion{1, 0}},
     {"zkr", RISCVExtensionVersion{1, 0}},
     {"zkn", RISCVExtensionVersion{1, 0}},
     {"zks", RISCVExtensionVersion{1, 0}},
     {"zkt", RISCVExtensionVersion{1, 0}},
     {"zk", RISCVExtensionVersion{1, 0}},
 
+    {"zmmul", RISCVExtensionVersion{1, 0}},
+
     {"v", RISCVExtensionVersion{1, 0}},
     {"zvl32b", RISCVExtensionVersion{1, 0}},
     {"zvl64b", RISCVExtensionVersion{1, 0}},
     {"zvl128b", RISCVExtensionVersion{1, 0}},
     {"zvl256b", RISCVExtensionVersion{1, 0}},
     {"zvl512b", RISCVExtensionVersion{1, 0}},
     {"zvl1024b", RISCVExtensionVersion{1, 0}},
     {"zvl2048b", RISCVExtensionVersion{1, 0}},

llvm/lib/Target/RISCV/RISCV.td

 
 //===----------------------------------------------------------------------===//
 // RISC-V subtarget features and instruction predicates.
 //===----------------------------------------------------------------------===//
 
 def FeatureStdExtM
     : SubtargetFeature<"m", "HasStdExtM", "true",
                        "'M' (Integer Multiplication and Division)">;
 def HasStdExtM : Predicate<"Subtarget->hasStdExtM()">,
                            AssemblerPredicate<(all_of FeatureStdExtM),
                            "'M' (Integer Multiplication and Division)">;
 
+def FeatureStdExtZmmul
+    : SubtargetFeature<"zmmul", "HasStdExtZmmul", "true",
+                       "'Zmmul' (Integer Multiplication)">;
+def HasStdExtZmmul : Predicate<"Subtarget->hasStdExtZmmul()">,
+                               AssemblerPredicate<(all_of FeatureStdExtZmmul),
+                               "'Zmmul' (Integer Multiplication)">;
+
+def HasStdExtMOrZmmul
+    : Predicate<"Subtarget->hasStdExtM() || Subtarget->hasStdExtZmmul()">,
+                AssemblerPredicate<(any_of FeatureStdExtM, FeatureStdExtZmmul),
+                                   "'M' (Integer Multiplication and Division) or "
+                                   "'Zmmul' (Integer Multiplication)">;
+
 def FeatureStdExtA
     : SubtargetFeature<"a", "HasStdExtA", "true",
                        "'A' (Atomic Instructions)">;
 def HasStdExtA : Predicate<"Subtarget->hasStdExtA()">,
                            AssemblerPredicate<(all_of FeatureStdExtA),
                            "'A' (Atomic Instructions)">;
 
 def FeatureStdExtF

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

setOperationAction({ISD::UADDO, ISD::USUBO, ISD::UADDSAT, ISD::USUBSAT},
                        MVT::i32, Custom);
   } else {
     setLibcallName(
         {RTLIB::SHL_I128, RTLIB::SRL_I128, RTLIB::SRA_I128, RTLIB::MUL_I128},
         nullptr);
     setLibcallName(RTLIB::MULO_I64, nullptr);
   }
 
-  if (!Subtarget.hasStdExtM()) {
-    setOperationAction({ISD::MUL, ISD::MULHS, ISD::MULHU, ISD::SDIV, ISD::UDIV,
-                        ISD::SREM, ISD::UREM},
-                       XLenVT, Expand);
+  if (!Subtarget.hasStdExtM() && !Subtarget.hasStdExtZmmul()) {
+    setOperationAction({ISD::MUL, ISD::MULHS, ISD::MULHU}, XLenVT, Expand);
   } else {
     if (Subtarget.is64Bit()) {
       setOperationAction(ISD::MUL, {MVT::i32, MVT::i128}, Custom);
+    } else {
+      setOperationAction(ISD::MUL, MVT::i64, Custom);
+    }
+  }
 
+  if (!Subtarget.hasStdExtM()) {
+    setOperationAction({ISD::SDIV, ISD::UDIV, ISD::SREM, ISD::UREM},
+                       XLenVT, Expand);
+  } else {
+    if (Subtarget.is64Bit()) {
       setOperationAction({ISD::SDIV, ISD::UDIV, ISD::UREM},
                           {MVT::i8, MVT::i16, MVT::i32}, Custom);
-    } else {
-      setOperationAction(ISD::MUL, MVT::i64, Custom);
     }
   }
 
   setOperationAction(
       {ISD::SDIVREM, ISD::UDIVREM, ISD::SMUL_LOHI, ISD::UMUL_LOHI}, XLenVT,
       Expand);
 
   setOperationAction({ISD::SHL_PARTS, ISD::SRL_PARTS, ISD::SRA_PARTS}, XLenVT,
if (Subtarget.is64Bit() && Type == MVT::i32)
     return true;
 
   return IsSigned;
 }
 
 bool RISCVTargetLowering::decomposeMulByConstant(LLVMContext &Context, EVT VT,
                                                  SDValue C) const {
   // Check integral scalar types.
+  const bool HasExtMOrZmmul =
+      Subtarget.hasStdExtM() || Subtarget.hasStdExtZmmul();
   if (VT.isScalarInteger()) {
     // Omit the optimization if the sub target has the M extension and the data
     // size exceeds XLen.
-    if (Subtarget.hasStdExtM() && VT.getSizeInBits() > Subtarget.getXLen())
+    if (HasExtMOrZmmul && VT.getSizeInBits() > Subtarget.getXLen())
       return false;
     if (auto *ConstNode = dyn_cast<ConstantSDNode>(C.getNode())) {
       // Break the MUL to a SLLI and an ADD/SUB.
       const APInt &Imm = ConstNode->getAPIntValue();
       if ((Imm + 1).isPowerOf2() || (Imm - 1).isPowerOf2() ||
           (1 - Imm).isPowerOf2() || (-1 - Imm).isPowerOf2())
         return true;
       // Optimize the MUL to (SH*ADD x, (SLLI x, bits)) if Imm is not simm12.
       if (Subtarget.hasStdExtZba() && !Imm.isSignedIntN(12) &&
           ((Imm - 2).isPowerOf2() || (Imm - 4).isPowerOf2() ||
            (Imm - 8).isPowerOf2()))
         return true;
       // Omit the following optimization if the sub target has the M extension
       // and the data size >= XLen.
-      if (Subtarget.hasStdExtM() && VT.getSizeInBits() >= Subtarget.getXLen())
+      if (HasExtMOrZmmul && VT.getSizeInBits() >= Subtarget.getXLen())
         return false;
       // Break the MUL to two SLLI instructions and an ADD/SUB, if Imm needs
       // a pair of LUI/ADDI.
       if (!Imm.isSignedIntN(12) && Imm.countTrailingZeros() < 12) {
         APInt ImmS = Imm.ashr(Imm.countTrailingZeros());
         if ((ImmS + 1).isPowerOf2() || (ImmS - 1).isPowerOf2() ||
             (1 - ImmS).isPowerOf2())
           return true;

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

BuildMI(MBB, II, DL, get(RISCV::SLLI), ScaledRegister)
         .setMIFlag(Flag);
     BuildMI(MBB, II, DL, get(RISCV::SUB), VL)
         .addReg(ScaledRegister, RegState::Kill)
         .addReg(VL, RegState::Kill)
         .setMIFlag(Flag);
   } else {
     Register N = MRI.createVirtualRegister(&RISCV::GPRRegClass);
     movImm(MBB, II, DL, N, NumOfVReg, Flag);
-    if (!STI.hasStdExtM())
+    if (!STI.hasStdExtM() && !STI.hasStdExtZmmul())
       MF.getFunction().getContext().diagnose(DiagnosticInfoUnsupported{
           MF.getFunction(),
-          "M-extension must be enabled to calculate the vscaled size/offset."});
+          "M- or Zmmul-extension must be enabled to calculate the vscaled size/"
+          "offset."});
     BuildMI(MBB, II, DL, get(RISCV::MUL), VL)
         .addReg(VL, RegState::Kill)
         .addReg(N, RegState::Kill)
         .setMIFlag(Flag);
   }
 
   return VL;
 }

llvm/lib/Target/RISCV/RISCVInstrInfoM.td

 def riscv_divw  : SDNode<"RISCVISD::DIVW",  SDT_RISCVIntBinOpW>;
 def riscv_divuw : SDNode<"RISCVISD::DIVUW", SDT_RISCVIntBinOpW>;
 def riscv_remuw : SDNode<"RISCVISD::REMUW", SDT_RISCVIntBinOpW>;
 
 //===----------------------------------------------------------------------===//
 // Instructions
 //===----------------------------------------------------------------------===//
 
-let Predicates = [HasStdExtM] in {
+let Predicates = [HasStdExtMOrZmmul] in {
 def MUL     : ALU_rr<0b0000001, 0b000, "mul", /*Commutable*/1>,
               Sched<[WriteIMul, ReadIMul, ReadIMul]>;
 def MULH    : ALU_rr<0b0000001, 0b001, "mulh", /*Commutable*/1>,
               Sched<[WriteIMul, ReadIMul, ReadIMul]>;
 def MULHSU  : ALU_rr<0b0000001, 0b010, "mulhsu">,
               Sched<[WriteIMul, ReadIMul, ReadIMul]>;
 def MULHU   : ALU_rr<0b0000001, 0b011, "mulhu", /*Commutable*/1>,
               Sched<[WriteIMul, ReadIMul, ReadIMul]>;
+} // Predicates = [HasStdExtMOrZmmul]
+
+let Predicates = [HasStdExtM] in {
 def DIV     : ALU_rr<0b0000001, 0b100, "div">,
               Sched<[WriteIDiv, ReadIDiv, ReadIDiv]>;
 def DIVU    : ALU_rr<0b0000001, 0b101, "divu">,
               Sched<[WriteIDiv, ReadIDiv, ReadIDiv]>;
 def REM     : ALU_rr<0b0000001, 0b110, "rem">,
               Sched<[WriteIDiv, ReadIDiv, ReadIDiv]>;
 def REMU    : ALU_rr<0b0000001, 0b111, "remu">,
               Sched<[WriteIDiv, ReadIDiv, ReadIDiv]>;
 } // Predicates = [HasStdExtM]
 
-let Predicates = [HasStdExtM, IsRV64] in {
+let Predicates = [HasStdExtMOrZmmul, IsRV64] in {
 def MULW    : ALUW_rr<0b0000001, 0b000, "mulw", /*Commutable*/1>,
               Sched<[WriteIMul32, ReadIMul32, ReadIMul32]>;
+} // Predicates = [HasStdExtMOrZmmul, IsRV64]
+
+let Predicates = [HasStdExtM, IsRV64] in {
 def DIVW    : ALUW_rr<0b0000001, 0b100, "divw">,
               Sched<[WriteIDiv32, ReadIDiv32, ReadIDiv32]>;
 def DIVUW   : ALUW_rr<0b0000001, 0b101, "divuw">,
               Sched<[WriteIDiv32, ReadIDiv32, ReadIDiv32]>;
 def REMW    : ALUW_rr<0b0000001, 0b110, "remw">,
               Sched<[WriteIDiv32, ReadIDiv32, ReadIDiv32]>;
 def REMUW   : ALUW_rr<0b0000001, 0b111, "remuw">,
               Sched<[WriteIDiv32, ReadIDiv32, ReadIDiv32]>;
 } // Predicates = [HasStdExtM, IsRV64]
 
 //===----------------------------------------------------------------------===//
 // Pseudo-instructions and codegen patterns
 //===----------------------------------------------------------------------===//
 
-let Predicates = [HasStdExtM] in {
+let Predicates = [HasStdExtMOrZmmul] in {
 def : PatGprGpr<mul, MUL>;
 def : PatGprGpr<mulhs, MULH>;
 def : PatGprGpr<mulhu, MULHU>;
 def : PatGprGpr<riscv_mulhsu, MULHSU>;
+} // Predicates = [HasStdExtMOrZmmul]
+
+let Predicates = [HasStdExtM] in {
 def : PatGprGpr<sdiv, DIV>;
 def : PatGprGpr<udiv, DIVU>;
 def : PatGprGpr<srem, REM>;
 def : PatGprGpr<urem, REMU>;
 } // Predicates = [HasStdExtM]
 
-let Predicates = [HasStdExtM, IsRV64] in {
 // Select W instructions if only the lower 32-bits of the result are used.
+let Predicates = [HasStdExtMOrZmmul, IsRV64] in
 def : PatGprGpr<binop_allwusers<mul>, MULW>;
 
+let Predicates = [HasStdExtM, IsRV64] in {
 def : PatGprGpr<riscv_divw, DIVW>;
 def : PatGprGpr<riscv_divuw, DIVUW>;
 def : PatGprGpr<riscv_remuw, REMUW>;
 
 // Handle the specific cases where using DIVU/REMU would be correct and result
 // in fewer instructions than emitting DIVUW/REMUW then zero-extending the
 // result.
 def : Pat<(and (riscv_divuw (assertzexti32 GPR:$rs1),
                             (assertzexti32 GPR:$rs2)), 0xffffffff),
           (DIVU GPR:$rs1, GPR:$rs2)>;
 def : Pat<(and (riscv_remuw (assertzexti32 GPR:$rs1),
                             (assertzexti32 GPR:$rs2)), 0xffffffff),
           (REMU GPR:$rs1, GPR:$rs2)>;
 
 // Although the sexti32 operands may not have originated from an i32 srem,
 // this pattern is safe as it is impossible for two sign extended inputs to
 // produce a result where res[63:32]=0 and res[31]=1.
 def : Pat<(srem (sexti32 (i64 GPR:$rs1)), (sexti32 (i64 GPR:$rs2))),
           (REMW GPR:$rs1, GPR:$rs2)>;
 } // Predicates = [HasStdExtM, IsRV64]
 
-let Predicates = [HasStdExtM, IsRV64, NotHasStdExtZba] in {
+let Predicates = [HasStdExtMOrZmmul, IsRV64, NotHasStdExtZba] in {
 // Special case for calculating the full 64-bit product of a 32x32 unsigned
 // multiply where the inputs aren't known to be zero extended. We can shift the
 // inputs left by 32 and use a MULHU. This saves two SRLIs needed to finish
 // zeroing the upper 32 bits.
 def : Pat<(i64 (mul (and GPR:$rs1, 0xffffffff), (and GPR:$rs2, 0xffffffff))),
           (MULHU (SLLI GPR:$rs1, 32), (SLLI GPR:$rs2, 32))>;
-} // Predicates = [HasStdExtM, IsRV64, NotHasStdExtZba]
+} // Predicates = [HasStdExtMOrZmmul, IsRV64, NotHasStdExtZba]

llvm/lib/Target/RISCV/RISCVSubtarget.h

public:
   };
 
 private:
   virtual void anchor();
 
   RISCVProcFamilyEnum RISCVProcFamily = Others;
 
   bool HasStdExtM = false;
   bool HasStdExtA = false;
   bool HasStdExtF = false;
   bool HasStdExtD = false;
   bool HasStdExtC = false;
   bool HasStdExtZihintpause = false;
   bool HasStdExtZba = false;
   bool HasStdExtZbb = false;
   bool HasStdExtZbc = false;
   bool HasStdExtZbe = false;
private:
   bool HasStdExtZkr = false;
   bool HasStdExtZkn = false;
   bool HasStdExtZks = false;
   bool HasStdExtZkt = false;
   bool HasStdExtZk = false;
   bool HasStdExtZicbom = false;
   bool HasStdExtZicboz = false;
   bool HasStdExtZicbop = false;
+  bool HasStdExtZmmul = false;
   bool HasRV64 = false;
   bool IsRV32E = false;
   bool EnableLinkerRelax = false;
   bool EnableRVCHintInstrs = true;
   bool EnableDefaultUnroll = true;
   bool EnableSaveRestore = false;
   bool EnableUnalignedScalarMem = false;
   bool HasLUIADDIFusion = false;
public:
 
   /// Returns RISCV processor family.
   /// Avoid this function! CPU specifics should be kept local to this class
   /// and preferably modeled with SubtargetFeatures or properties in
   /// initializeProperties().
   RISCVProcFamilyEnum getProcFamily() const { return RISCVProcFamily; }
 
   bool hasStdExtM() const { return HasStdExtM; }
   bool hasStdExtA() const { return HasStdExtA; }
   bool hasStdExtF() const { return HasStdExtF; }
   bool hasStdExtD() const { return HasStdExtD; }
   bool hasStdExtC() const { return HasStdExtC; }
   bool hasStdExtV() const { return HasStdExtV; }
   bool hasStdExtZihintpause() const { return HasStdExtZihintpause; }
   bool hasStdExtZba() const { return HasStdExtZba; }
   bool hasStdExtZbb() const { return HasStdExtZbb; }
   bool hasStdExtZbc() const { return HasStdExtZbc; }
public:
   bool hasStdExtZkne() const { return HasStdExtZkne; }
   bool hasStdExtZknh() const { return HasStdExtZknh; }
   bool hasStdExtZksed() const { return HasStdExtZksed; }
   bool hasStdExtZksh() const { return HasStdExtZksh; }
   bool hasStdExtZkr() const { return HasStdExtZkr; }
   bool hasStdExtZicbom() const { return HasStdExtZicbom; }
   bool hasStdExtZicboz() const { return HasStdExtZicboz; }
   bool hasStdExtZicbop() const { return HasStdExtZicbop; }
+  bool hasStdExtZmmul() const { return HasStdExtZmmul; }
   bool is64Bit() const { return HasRV64; }
   bool isRV32E() const { return IsRV32E; }
   bool enableLinkerRelax() const { return EnableLinkerRelax; }
   bool enableRVCHintInstrs() const { return EnableRVCHintInstrs; }
   bool enableDefaultUnroll() const { return EnableDefaultUnroll; }
   bool enableSaveRestore() const { return EnableSaveRestore; }
   bool enableUnalignedScalarMem() const { return EnableUnalignedScalarMem; }
   bool hasLUIADDIFusion() const { return HasLUIADDIFusion; }

llvm/test/CodeGen/RISCV/attributes.ll

 ;; Generate ELF attributes from llc.
 
 ; RUN: llc -mtriple=riscv32 -mattr=+m %s -o - | FileCheck --check-prefix=RV32M %s
+; RUN: llc -mtriple=riscv32 -mattr=+zmmul %s -o - | FileCheck --check-prefix=RV32ZMMUL %s
+; RUN: llc -mtriple=riscv32 -mattr=+m,+zmmul %s -o - | FileCheck --check-prefix=RV32MZMMUL %s
 ; RUN: llc -mtriple=riscv32 -mattr=+a %s -o - | FileCheck --check-prefix=RV32A %s
 ; RUN: llc -mtriple=riscv32 -mattr=+f %s -o - | FileCheck --check-prefix=RV32F %s
 ; RUN: llc -mtriple=riscv32 -mattr=+d %s -o - | FileCheck --check-prefix=RV32D %s
 ; RUN: llc -mtriple=riscv32 -mattr=+c %s -o - | FileCheck --check-prefix=RV32C %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zihintpause %s -o - | FileCheck --check-prefix=RV32ZIHINTPAUSE %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zfhmin %s -o - | FileCheck --check-prefix=RV32ZFHMIN %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zfh %s -o - | FileCheck --check-prefix=RV32ZFH %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zba %s -o - | FileCheck --check-prefix=RV32ZBA %s

 ; RUN: llc -mtriple=riscv32 -mattr=+zk %s -o - | FileCheck --check-prefix=RV32ZK %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zkn,+zkr,+zkt %s -o - | FileCheck --check-prefix=RV32COMBINEINTOZK %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zbkb,+zbkc,+zbkx,+zkne,+zknd,+zknh %s -o - | FileCheck --check-prefix=RV32COMBINEINTOZKN %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zbkb,+zbkc,+zbkx,+zksed,+zksh %s -o - | FileCheck --check-prefix=RV32COMBINEINTOZKS %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zicbom %s -o - | FileCheck --check-prefix=RV32ZICBOM %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zicboz %s -o - | FileCheck --check-prefix=RV32ZICBOZ %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zicbop %s -o - | FileCheck --check-prefix=RV32ZICBOP %s
 ; RUN: llc -mtriple=riscv64 -mattr=+m %s -o - | FileCheck --check-prefix=RV64M %s
+; RUN: llc -mtriple=riscv64 -mattr=+zmmul %s -o - | FileCheck --check-prefix=RV64ZMMUL %s
+; RUN: llc -mtriple=riscv64 -mattr=+m,+zmmul %s -o - | FileCheck --check-prefix=RV64MZMMUL %s
 ; RUN: llc -mtriple=riscv64 -mattr=+a %s -o - | FileCheck --check-prefix=RV64A %s
 ; RUN: llc -mtriple=riscv64 -mattr=+f %s -o - | FileCheck --check-prefix=RV64F %s
 ; RUN: llc -mtriple=riscv64 -mattr=+d %s -o - | FileCheck --check-prefix=RV64D %s
 ; RUN: llc -mtriple=riscv64 -mattr=+c %s -o - | FileCheck --check-prefix=RV64C %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zihintpause %s -o - | FileCheck --check-prefix=RV64ZIHINTPAUSE %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zfhmin %s -o - | FileCheck --check-prefix=RV64ZFHMIN %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zfh %s -o - | FileCheck --check-prefix=RV64ZFH %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zba %s -o - | FileCheck --check-prefix=RV64ZBA %s

 ; RUN: llc -mtriple=riscv64 -mattr=+zk %s -o - | FileCheck --check-prefix=RV64ZK %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zkn,+zkr,+zkt %s -o - | FileCheck --check-prefix=RV64COMBINEINTOZK %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zbkb,+zbkc,+zbkx,+zkne,+zknd,+zknh %s -o - | FileCheck --check-prefix=RV64COMBINEINTOZKN %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zbkb,+zbkc,+zbkx,+zksed,+zksh %s -o - | FileCheck --check-prefix=RV64COMBINEINTOZKS %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zicbom %s -o - | FileCheck --check-prefix=RV64ZICBOM %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zicboz %s -o - | FileCheck --check-prefix=RV64ZICBOZ %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zicbop %s -o - | FileCheck --check-prefix=RV64ZICBOP %s
 
 ; RV32M: .attribute 5, "rv32i2p0_m2p0"
+; RV32ZMMUL: .attribute 5, "rv32i2p0_zmmul1p0"
+; RV32MZMMUL: .attribute 5, "rv32i2p0_m2p0_zmmul1p0"
 ; RV32A: .attribute 5, "rv32i2p0_a2p0"
 ; RV32F: .attribute 5, "rv32i2p0_f2p0"
 ; RV32D: .attribute 5, "rv32i2p0_f2p0_d2p0"
 ; RV32C: .attribute 5, "rv32i2p0_c2p0"
 ; RV32ZIHINTPAUSE: .attribute 5, "rv32i2p0_zihintpause2p0"
 ; RV32ZFHMIN: .attribute 5, "rv32i2p0_f2p0_zfhmin1p0"
 ; RV32ZFH: .attribute 5, "rv32i2p0_f2p0_zfh1p0"
 ; RV32ZBA: .attribute 5, "rv32i2p0_zba1p0"

 ; RV32COMBINEINTOZK: .attribute 5, "rv32i2p0_zbkb1p0_zbkc1p0_zbkx1p0_zk1p0_zkn1p0_zknd1p0_zkne1p0_zknh1p0_zkr1p0_zkt1p0"
 ; RV32COMBINEINTOZKN: .attribute 5, "rv32i2p0_zbkb1p0_zbkc1p0_zbkx1p0_zkn1p0_zknd1p0_zkne1p0_zknh1p0"
 ; RV32COMBINEINTOZKS: .attribute 5, "rv32i2p0_zbkb1p0_zbkc1p0_zbkx1p0_zks1p0_zksed1p0_zksh1p0"
 ; RV32ZICBOM: .attribute 5, "rv32i2p0_zicbom1p0"
 ; RV32ZICBOZ: .attribute 5, "rv32i2p0_zicboz1p0"
 ; RV32ZICBOP: .attribute 5, "rv32i2p0_zicbop1p0"
 
 ; RV64M: .attribute 5, "rv64i2p0_m2p0"
+; RV64ZMMUL: .attribute 5, "rv64i2p0_zmmul1p0"
+; RV64MZMMUL: .attribute 5, "rv64i2p0_m2p0_zmmul1p0"
 ; RV64A: .attribute 5, "rv64i2p0_a2p0"
 ; RV64F: .attribute 5, "rv64i2p0_f2p0"
 ; RV64D: .attribute 5, "rv64i2p0_f2p0_d2p0"
 ; RV64C: .attribute 5, "rv64i2p0_c2p0"
 ; RV64ZIHINTPAUSE: .attribute 5, "rv64i2p0_zihintpause2p0"
 ; RV64ZFHMIN: .attribute 5, "rv64i2p0_f2p0_zfhmin1p0"
 ; RV64ZFH: .attribute 5, "rv64i2p0_f2p0_zfh1p0"
 ; RV64ZBA: .attribute 5, "rv64i2p0_zba1p0"

llvm/test/CodeGen/RISCV/zmmul.ll

+; RUN: llc -mtriple=riscv32 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-DIV %s
+; RUN: llc -mtriple=riscv64 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-DIV %s
+; RUN: llc -mtriple=riscv32 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-REM %s
+; RUN: llc -mtriple=riscv64 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-REM %s
+
+; RUN: llc -mtriple=riscv32 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-UDIV %s
+; RUN: llc -mtriple=riscv64 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-UDIV %s
+; RUN: llc -mtriple=riscv32 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-UREM %s
+; RUN: llc -mtriple=riscv64 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-UREM %s
+
+; RUN: llc -mtriple=riscv32 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | FileCheck -check-prefix=CHECK-MUL %s
+; RUN: llc -mtriple=riscv64 -mattr=+zmmul -verify-machineinstrs < %s \
+; RUN:  | FileCheck -check-prefix=CHECK-MUL %s
+
+; RUN: llc -mtriple=riscv32 -mattr=+m -verify-machineinstrs < %s \
+; RUN:  | FileCheck -check-prefixes=CHECK-MUL,CHECK-UDIV,CHECK-DIV,CHECK-UREM,CHECK-REM %s
+; RUN: llc -mtriple=riscv64 -mattr=+m -verify-machineinstrs < %s \
+; RUN:  | FileCheck -check-prefixes=CHECK-MUL,CHECK-UDIV,CHECK-DIV,CHECK-UREM,CHECK-REM %s
+
+define i32 @foo(i32 %a, i32 %b) {
+; CHECK-UDIV: divu{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %1 = udiv i32 %a, %b
+; CHECK-DIV: div{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %2 = sdiv i32 %a, %1
+; CHECK-MUL: mul{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %3 = mul i32 %b, %2
+; CHECK-UREM: remu{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %4 = urem i32 %3, %b
+; CHECK-REM: rem{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %5 = srem i32 %4, %a
+  ret i32 %5
+}

llvm/test/MC/RISCV/rv32i-invalid.s

 # Memory operand not formatted correctly
 lw a4, a5, 111 # CHECK: :[[@LINE]]:12: error: invalid operand for instruction
 
 # Too few operands
 ori a0, a1 # CHECK: :[[@LINE]]:1: error: too few operands for instruction
 xor s2, s2 # CHECK: :[[@LINE]]:1: error: too few operands for instruction
 
 # Instruction not in the base ISA
-mul a4, ra, s0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+div a4, ra, s0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
 amomaxu.w s5, s4, (s3) # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'A' (Atomic Instructions)
 fadd.s ft0, ft1, ft2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'F' (Single-Precision Floating-Point){{$}}
 fadd.h ft0, ft1, ft2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zfh' (Half-Precision Floating-Point)
 fadd.s a0, a1, a2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zfinx' (Float in Integer)
 fadd.d a0, a2, a4 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zdinx' (Double in Integer)
 fadd.h a0, a1, a2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zhinx' (Half Float in Integer)
 flh ft0, (a0) # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zfh' (Half-Precision Floating-Point) or 'Zfhmin' (Half-Precision Floating-Point Minimal){{$}}
 sh1add a0, a1, a2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zba' (Address Generation Instructions)

llvm/test/MC/RISCV/rv32zmmul-invaild.s

+# RUN: not llvm-mc %s -triple=riscv32 -mattr=+zmmul -riscv-no-aliases 2>&1 \
+# RUN:  | FileCheck -check-prefixes=CHECK-ERROR %s
+
+# CHECK-ERROR: 5:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+div s0, s0, s0
+
+# CHECK-ERROR: 8:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+divu gp, a0, a1
+
+# CHECK-ERROR: 11:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+rem s2, s2, s8
+
+# CHECK-ERROR: 14:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+remu x18, x18, x24

llvm/test/MC/RISCV/rv32zmmul-valid.s

+# RUN: llvm-mc %s -triple=riscv32 -mattr=+zmmul -riscv-no-aliases 2>&1 \
+# RUN:  | FileCheck -check-prefixes=CHECK-INST %s
+
+# CHECK-INST: mul a4, ra, s0
+mul a4, ra, s0
+
+# CHECK-INST: mulh ra, zero, zero
+mulh x1, x0, x0
+
+# CHECK-INST: mulhsu t0, t2, t1
+mulhsu t0, t2, t1
+
+# CHECK-INST: mulhu a5, a4, a3
+mulhu a5, a4, a3

llvm/test/MC/RISCV/rv64zmmul-invalid.s

+# RUN: not llvm-mc %s -triple=riscv64 -mattr=+zmmul -riscv-no-aliases 2>&1 \
+# RUN:  | FileCheck -check-prefixes=CHECK-ERROR %s
+
+# CHECK-ERROR: 5:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+divw tp, t0, t1
+
+# CHECK-ERROR: 8:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+divuw t2, s0, s2
+
+# CHECK-ERROR: 11:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+remw a0, a1, a2
+
+# CHECK-ERROR: 14:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+remuw a3, a4, a5

llvm/test/MC/RISCV/rv64zmmul-valid.s

+# RUN: llvm-mc %s -triple=riscv64 -mattr=+zmmul -riscv-no-aliases 2>&1 \
+# RUN:  | FileCheck -check-prefixes=CHECK-INST %s
+
+# CHECK-INST: mulw ra, sp, gp
+mulw ra, sp, gp