[RISCV][Clang] Add support for Zmmul extension

[ksyx]

Summary

This patch implements recently ratified extension Zmmul, a subextension of M (Integer Multiplication and Division) consisting only multiplication part of it.

[craig.topper]

Do we need to add this to RISCVTargetStreamer::emitTargetAttributes?

[jrtc27]

In D103313#2787316, @craig.topper wrote:

Do we need to add this to RISCVTargetStreamer::emitTargetAttributes?

Yes, that, but also RISCVAsmParser::parseDirectiveAttribute and ELFObjectFileBase::getRISCVFeatures, though it seems that last one has been a bit neglected.

clang/lib/Basic/Targets/RISCV.cpp

270–271

These are supposed to be (approximately) sorted in terms of their canonical ordering. I don't know where Zmmul is proposed to go relative to the other Zfoo's, but it belongs down somewhere in there, I guess alphabetical if we don't know any better, not up here with the single-letter extensions.

llvm/lib/Target/RISCV/RISCVSubtarget.h

64

Ditto

161

Ditto

[jrtc27]

In D103313#2787326, @jrtc27 wrote:

In D103313#2787316, @craig.topper wrote:

Do we need to add this to RISCVTargetStreamer::emitTargetAttributes?

Yes, that, but also RISCVAsmParser::parseDirectiveAttribute and ELFObjectFileBase::getRISCVFeatures, though it seems that last one has been a bit neglected.

(we really need to cut down on how many places parse/produce arch strings...)

[ksyx]

• Move down zmmul lines to fit the ordering
• Add Zmmul into parseDirectiveAttribute

[ksyx]

I have took a look into ELFObjectFile.cpp but I am not sure what work I need to do there since it seems other Z* extensions are not being handled there.

llvm/lib/Target/RISCV/RISCVSubtarget.h

64

The pattern here seems to be that Z* extensions is following its parent their parent extensions, should we follow it? (See B and V)

[ksyx]

llvm/lib/Target/RISCV/RISCVSubtarget.h

64

But for Zfh it is not following HasStdExtF 🤔

[Jim]

It should add arch string tests such as in clang/test/Driver/riscv-arch.c.

[ksyx]

add: zmmul arch string test

[JiahaoChenConor]

Hi, do you still continue working on this patch?

[ksyx]

In D103313#3359310, @JiahaoChenConor wrote:

Hi, do you still continue working on this patch?

In fact, not pretty sure what else I need to improve on this.

[craig.topper]

In D103313#3359628, @ksyx wrote:

In D103313#3359310, @JiahaoChenConor wrote:

Hi, do you still continue working on this patch?

In fact, not pretty sure what else I need to improve on this.

At minimum it probably needs to rebased because RISCVISAInfo.cpp has been added to cut down on some duplication.

[ksyx]

Rebase and adapt to new ISAInfo module.

[ksyx]

Fix patch apply failed

[ksyx]

The Zmmul extension has just ratified. Update patch to rebase to current main.

[craig.topper]

llvm/lib/Target/RISCV/RISCV.td

25

This will cause Zmmul to appear in the ELF attributes any time is M is enabled. Is that what we want?

llvm/test/CodeGen/RISCV/attributes.ll

70

Is this a backwards compatibility issue?

[ksyx]

Make Zmmul extension independent instead of implied by M extension for backward ELF attribute compatibility.

[ksyx]

ping :)

[craig.topper]

Need to update RISCVTargetLowering::decomposeMulByConstant.

[ksyx]

Rebase and update use of M extension existence check to include Zmmul whenever needed.

[craig.topper]

llvm/lib/Support/RISCVISAInfo.cpp

80

If it is ratified, is the version really 0.1?

[asb]

llvm/lib/Support/RISCVISAInfo.cpp

80

It looks like it should be 1.0 now https://github.com/riscv/riscv-isa-manual/commit/f518c259c008f926eba4aba67804f62531b6e94b

[ksyx]

Update zmmul version & rebase

clang/lib/Basic/Targets/RISCV.cpp

for (auto &Extension : ISAInfo->getExtensions()) {
     auto ExtName = Extension.first;
     auto ExtInfo = Extension.second;
     unsigned Version =
         (ExtInfo.MajorVersion * 1000000) + (ExtInfo.MinorVersion * 1000);
 
     Builder.defineMacro(Twine("__riscv_", ExtName), Twine(Version));
   }
 
-  if (ISAInfo->hasExtension("m")) {
+  if (ISAInfo->hasExtension("m") || ISAInfo->hasExtension("zmmul"))
     Builder.defineMacro("__riscv_mul");
+
+  if (ISAInfo->hasExtension("m")) {
     Builder.defineMacro("__riscv_div");
     Builder.defineMacro("__riscv_muldiv");
   }
 
   if (ISAInfo->hasExtension("a")) {
     Builder.defineMacro("__riscv_atomic");
     Builder.defineMacro("__GCC_HAVE_SYNC_COMPARE_AND_SWAP_1");
     Builder.defineMacro("__GCC_HAVE_SYNC_COMPARE_AND_SWAP_2");
if (ISAInfo->isSupportedExtensionFeature(Feature))
     return ISAInfo->hasExtension(Feature);
 
   return false;
 }
 
 /// Perform initialization based on the user configured set of features.
 bool RISCVTargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
                                            DiagnosticsEngine &Diags) {
   unsigned XLen = getTriple().isArch64Bit() ? 64 : 32;
   auto ParseResult = llvm::RISCVISAInfo::parseFeatures(XLen, Features);
   if (!ParseResult) {
     std::string Buffer;
     llvm::raw_string_ostream OutputErrMsg(Buffer);
     handleAllErrors(ParseResult.takeError(), [&](llvm::StringError &ErrMsg) {
       OutputErrMsg << ErrMsg.getMessage();
     });
     Diags.Report(diag::err_invalid_feature_combination) << OutputErrMsg.str();
     return false;

clang/test/Driver/riscv-arch.c

 
 // RUN: %clang -target riscv32-unknown-elf -march=rv32izkt1p0 -### %s \
 // RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-ZKT %s
 // RV32-ZKT: "-target-feature" "+zkt"
 
 // RUN: %clang -target riscv32-unknown-elf -march=rv32izk1p0 -### %s \
 // RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-ZK %s
 // RV32-ZK: "-target-feature" "+zk"
+// RUN: %clang -target riscv32-unknown-elf -march=rv32i_zmmul -### %s \
+// RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-EXPERIMENTAL-ZMMUL-NOFLAG %s
+// RV32-EXPERIMENTAL-ZMMUL-NOFLAG: error: invalid arch name 'rv32i_zmmul'
+// RV32-EXPERIMENTAL-ZMMUL-NOFLAG: requires '-menable-experimental-extensions'
+
+// RUN: %clang -target riscv32-unknown-elf -march=rv32i_zmmul -menable-experimental-extensions -### %s \
+// RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-EXPERIMENTAL-ZMMUL-NOVERS %s
+// RV32-EXPERIMENTAL-ZMMUL-NOVERS: error: invalid arch name 'rv32i_zmmul'
+// RV32-EXPERIMENTAL-ZMMUL-NOVERS: experimental extension requires explicit version number
+
+// RUN: %clang -target riscv32-unknown-elf -march=rv32i_zmmul2p0 -menable-experimental-extensions -### %s \
+// RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-EXPERIMENTAL-ZMMUL-BADVERS %s
+// RV32-EXPERIMENTAL-ZMMUL-BADVERS: error: invalid arch name 'rv32i_zmmul2p0'
+// RV32-EXPERIMENTAL-ZMMUL-BADVERS: unsupported version number 2.0 for experimental extension
+
+// RUN: %clang -target riscv32-unknown-elf -march=rv32i_zmmul0p1 -menable-experimental-extensions -### %s \
+// RUN: -fsyntax-only 2>&1 | FileCheck -check-prefix=RV32-EXPERIMENTAL-ZMMUL-GOODVERS %s
+// RV32-EXPERIMENTAL-ZMMUL-GOODVERS: "-target-feature" "+experimental-zmmul"

llvm/lib/Support/RISCVISAInfo.cpp

static const RISCVSupportedExtension SupportedExtensions[] = {
     {"zksed", RISCVExtensionVersion{1, 0}},
     {"zksh", RISCVExtensionVersion{1, 0}},
     {"zkr", RISCVExtensionVersion{1, 0}},
     {"zkn", RISCVExtensionVersion{1, 0}},
     {"zks", RISCVExtensionVersion{1, 0}},
     {"zkt", RISCVExtensionVersion{1, 0}},
     {"zk", RISCVExtensionVersion{1, 0}},
 
     {"v", RISCVExtensionVersion{1, 0}},
     {"zvl32b", RISCVExtensionVersion{1, 0}},
     {"zvl64b", RISCVExtensionVersion{1, 0}},
     {"zvl128b", RISCVExtensionVersion{1, 0}},
     {"zvl256b", RISCVExtensionVersion{1, 0}},
     {"zvl512b", RISCVExtensionVersion{1, 0}},
     {"zvl1024b", RISCVExtensionVersion{1, 0}},
     {"zvl2048b", RISCVExtensionVersion{1, 0}},
     {"zvl4096b", RISCVExtensionVersion{1, 0}},

 
 static const RISCVSupportedExtension SupportedExperimentalExtensions[] = {
     {"zbe", RISCVExtensionVersion{0, 93}},
     {"zbf", RISCVExtensionVersion{0, 93}},
     {"zbm", RISCVExtensionVersion{0, 93}},
     {"zbp", RISCVExtensionVersion{0, 93}},
     {"zbr", RISCVExtensionVersion{0, 93}},
     {"zbt", RISCVExtensionVersion{0, 93}},
+
+    {"zmmul", RISCVExtensionVersion{0, 1}},
 };
 
 static bool stripExperimentalPrefix(StringRef &Ext) {
   return Ext.consume_front("experimental-");
 }
 
 // This function finds the first character that doesn't belong to a version
 // (e.g. zbe0p93 is extension 'zbe' of version '0p93'). So the function will

llvm/lib/Target/RISCV/RISCV.td

 //===-- RISCV.td - Describe the RISCV Target Machine -------*- tablegen -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 include "llvm/Target/Target.td"
 
 //===----------------------------------------------------------------------===//
 // RISC-V subtarget features and instruction predicates.
 //===----------------------------------------------------------------------===//
 
+def FeatureStdExtZmmul
+    : SubtargetFeature<"experimental-zmmul", "HasStdExtZmmul", "true",
+                       "'Zmmul' (Integer Multiplication)">;
+def HasStdExtZmmul : Predicate<"Subtarget->hasStdExtZmmul()">,
+                               AssemblerPredicate<(all_of FeatureStdExtZmmul),
+                               "'Zmmul' (Integer Multiplication)">;
+
 def FeatureStdExtM
     : SubtargetFeature<"m", "HasStdExtM", "true",
-                       "'M' (Integer Multiplication and Division)">;
+                       "'M' (Integer Multiplication and Division)",
+                       [FeatureStdExtZmmul]>;
 def HasStdExtM : Predicate<"Subtarget->hasStdExtM()">,
                            AssemblerPredicate<(all_of FeatureStdExtM),
                            "'M' (Integer Multiplication and Division)">;
 
 def FeatureStdExtA
     : SubtargetFeature<"a", "HasStdExtA", "true",
                        "'A' (Atomic Instructions)">;
 def HasStdExtA : Predicate<"Subtarget->hasStdExtA()">,

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
   } else {
     setLibcallName(RTLIB::SHL_I128, nullptr);
     setLibcallName(RTLIB::SRL_I128, nullptr);
     setLibcallName(RTLIB::SRA_I128, nullptr);
     setLibcallName(RTLIB::MUL_I128, nullptr);
     setLibcallName(RTLIB::MULO_I64, nullptr);
   }
 
-  if (!Subtarget.hasStdExtM()) {
+  if (!Subtarget.hasStdExtZmmul()) {
     setOperationAction(ISD::MUL, XLenVT, Expand);
     setOperationAction(ISD::MULHS, XLenVT, Expand);
     setOperationAction(ISD::MULHU, XLenVT, Expand);
+  } else {
+    if (Subtarget.is64Bit()) {
+      setOperationAction(ISD::MUL, MVT::i32, Custom);
+      setOperationAction(ISD::MUL, MVT::i128, Custom);
+    } else {
+      setOperationAction(ISD::MUL, MVT::i64, Custom);
+    }
+  }
+
+  if (!Subtarget.hasStdExtM()) {
     setOperationAction(ISD::SDIV, XLenVT, Expand);
     setOperationAction(ISD::UDIV, XLenVT, Expand);
     setOperationAction(ISD::SREM, XLenVT, Expand);
     setOperationAction(ISD::UREM, XLenVT, Expand);
   } else {
     if (Subtarget.is64Bit()) {
-      setOperationAction(ISD::MUL, MVT::i32, Custom);
-      setOperationAction(ISD::MUL, MVT::i128, Custom);
-
       setOperationAction(ISD::SDIV, MVT::i8, Custom);
       setOperationAction(ISD::UDIV, MVT::i8, Custom);
       setOperationAction(ISD::UREM, MVT::i8, Custom);
       setOperationAction(ISD::SDIV, MVT::i16, Custom);
       setOperationAction(ISD::UDIV, MVT::i16, Custom);
       setOperationAction(ISD::UREM, MVT::i16, Custom);
       setOperationAction(ISD::SDIV, MVT::i32, Custom);
       setOperationAction(ISD::UDIV, MVT::i32, Custom);
       setOperationAction(ISD::UREM, MVT::i32, Custom);
-    } else {
-      setOperationAction(ISD::MUL, MVT::i64, Custom);
     }
   }
 
   setOperationAction(ISD::SDIVREM, XLenVT, Expand);
   setOperationAction(ISD::UDIVREM, XLenVT, Expand);
   setOperationAction(ISD::SMUL_LOHI, XLenVT, Expand);
   setOperationAction(ISD::UMUL_LOHI, XLenVT, Expand);
 

llvm/lib/Target/RISCV/RISCVInstrInfoM.td

 def riscv_divw  : SDNode<"RISCVISD::DIVW",  SDT_RISCVIntBinOpW>;
 def riscv_divuw : SDNode<"RISCVISD::DIVUW", SDT_RISCVIntBinOpW>;
 def riscv_remuw : SDNode<"RISCVISD::REMUW", SDT_RISCVIntBinOpW>;
 
 //===----------------------------------------------------------------------===//
 // Instructions
 //===----------------------------------------------------------------------===//
 
-let Predicates = [HasStdExtM] in {
+let Predicates = [HasStdExtZmmul] in {
 def MUL     : ALU_rr<0b0000001, 0b000, "mul">,
               Sched<[WriteIMul, ReadIMul, ReadIMul]>;
 def MULH    : ALU_rr<0b0000001, 0b001, "mulh">,
               Sched<[WriteIMul, ReadIMul, ReadIMul]>;
 def MULHSU  : ALU_rr<0b0000001, 0b010, "mulhsu">,
               Sched<[WriteIMul, ReadIMul, ReadIMul]>;
 def MULHU   : ALU_rr<0b0000001, 0b011, "mulhu">,
               Sched<[WriteIMul, ReadIMul, ReadIMul]>;
+} // Predicates = [HasStdExtZmmul]
+
+let Predicates = [HasStdExtM] in {
 def DIV     : ALU_rr<0b0000001, 0b100, "div">,
               Sched<[WriteIDiv, ReadIDiv, ReadIDiv]>;
 def DIVU    : ALU_rr<0b0000001, 0b101, "divu">,
               Sched<[WriteIDiv, ReadIDiv, ReadIDiv]>;
 def REM     : ALU_rr<0b0000001, 0b110, "rem">,
               Sched<[WriteIDiv, ReadIDiv, ReadIDiv]>;
 def REMU    : ALU_rr<0b0000001, 0b111, "remu">,
               Sched<[WriteIDiv, ReadIDiv, ReadIDiv]>;
 } // Predicates = [HasStdExtM]
 
-let Predicates = [HasStdExtM, IsRV64] in {
+let Predicates = [HasStdExtZmmul, IsRV64] in {
 def MULW    : ALUW_rr<0b0000001, 0b000, "mulw">,
               Sched<[WriteIMul32, ReadIMul32, ReadIMul32]>;
+} // Predicates = [HasStdExtZmmul, IsRV64]
+
+let Predicates = [HasStdExtM, IsRV64] in {
 def DIVW    : ALUW_rr<0b0000001, 0b100, "divw">,
               Sched<[WriteIDiv32, ReadIDiv32, ReadIDiv32]>;
 def DIVUW   : ALUW_rr<0b0000001, 0b101, "divuw">,
               Sched<[WriteIDiv32, ReadIDiv32, ReadIDiv32]>;
 def REMW    : ALUW_rr<0b0000001, 0b110, "remw">,
               Sched<[WriteIDiv32, ReadIDiv32, ReadIDiv32]>;
 def REMUW   : ALUW_rr<0b0000001, 0b111, "remuw">,
               Sched<[WriteIDiv32, ReadIDiv32, ReadIDiv32]>;
 } // Predicates = [HasStdExtM, IsRV64]
 
 //===----------------------------------------------------------------------===//
 // Pseudo-instructions and codegen patterns
 //===----------------------------------------------------------------------===//
 
-let Predicates = [HasStdExtM] in {
+let Predicates = [HasStdExtZmmul] in {
 def : PatGprGpr<mul, MUL>;
 def : PatGprGpr<mulhs, MULH>;
 def : PatGprGpr<mulhu, MULHU>;
 def : PatGprGpr<riscv_mulhsu, MULHSU>;
+} // Predicates = [HasStdExtZmmul]
+
+let Predicates = [HasStdExtM] in {
 def : PatGprGpr<sdiv, DIV>;
 def : PatGprGpr<udiv, DIVU>;
 def : PatGprGpr<srem, REM>;
 def : PatGprGpr<urem, REMU>;
 } // Predicates = [HasStdExtM]
 
-let Predicates = [HasStdExtM, IsRV64] in {
 // Select W instructions if only the lower 32-bits of the result are used.
+let Predicates = [HasStdExtZmmul, IsRV64] in
 def : PatGprGpr<binop_allwusers<mul>, MULW>;
 
+let Predicates = [HasStdExtM, IsRV64] in {
 def : PatGprGpr<riscv_divw, DIVW>;
 def : PatGprGpr<riscv_divuw, DIVUW>;
 def : PatGprGpr<riscv_remuw, REMUW>;
 
 // Handle the specific cases where using DIVU/REMU would be correct and result
 // in fewer instructions than emitting DIVUW/REMUW then zero-extending the
 // result.
 def : Pat<(and (riscv_divuw (assertzexti32 GPR:$rs1),
                             (assertzexti32 GPR:$rs2)), 0xffffffff),
           (DIVU GPR:$rs1, GPR:$rs2)>;
 def : Pat<(and (riscv_remuw (assertzexti32 GPR:$rs1),
                             (assertzexti32 GPR:$rs2)), 0xffffffff),
           (REMU GPR:$rs1, GPR:$rs2)>;
 
 // Although the sexti32 operands may not have originated from an i32 srem,
 // this pattern is safe as it is impossible for two sign extended inputs to
 // produce a result where res[63:32]=0 and res[31]=1.
 def : Pat<(srem (sexti32 (i64 GPR:$rs1)), (sexti32 (i64 GPR:$rs2))),
           (REMW GPR:$rs1, GPR:$rs2)>;
 } // Predicates = [HasStdExtM, IsRV64]
 
-let Predicates = [HasStdExtM, IsRV64, NotHasStdExtZba] in {
+let Predicates = [HasStdExtZmmul, IsRV64, NotHasStdExtZba] in {
 // Special case for calculating the full 64-bit product of a 32x32 unsigned
 // multiply where the inputs aren't known to be zero extended. We can shift the
 // inputs left by 32 and use a MULHU. This saves two SRLIs needed to finish
 // zeroing the upper 32 bits.
 def : Pat<(i64 (mul (and GPR:$rs1, 0xffffffff), (and GPR:$rs2, 0xffffffff))),
           (MULHU (SLLI GPR:$rs1, 32), (SLLI GPR:$rs2, 32))>;
-} // Predicates = [HasStdExtM, IsRV64, NotHasStdExtZba]
+} // Predicates = [HasStdExtZmmul, IsRV64, NotHasStdExtZba]

llvm/lib/Target/RISCV/RISCVSubtarget.h

public:
   };
 
 private:
   virtual void anchor();
 
   RISCVProcFamilyEnum RISCVProcFamily = Others;
 
   bool HasStdExtM = false;
   bool HasStdExtA = false;
   bool HasStdExtF = false;
   bool HasStdExtD = false;
   bool HasStdExtC = false;
   bool HasStdExtZihintpause = false;
   bool HasStdExtZba = false;
   bool HasStdExtZbb = false;
   bool HasStdExtZbc = false;
   bool HasStdExtZbe = false;
private:
   bool HasStdExtZknh = false;
   bool HasStdExtZksed = false;
   bool HasStdExtZksh = false;
   bool HasStdExtZkr = false;
   bool HasStdExtZkn = false;
   bool HasStdExtZks = false;
   bool HasStdExtZkt = false;
   bool HasStdExtZk = false;
+  bool HasStdExtZmmul = false;
   bool HasRV64 = false;
   bool IsRV32E = false;
   bool EnableLinkerRelax = false;
   bool EnableRVCHintInstrs = true;
   bool EnableSaveRestore = false;
   unsigned XLen = 32;
   ExtZvl ZvlLen = ExtZvl::NotSet;
   MVT XLenVT = MVT::i32;
public:
 
   /// Returns RISCV processor family.
   /// Avoid this function! CPU specifics should be kept local to this class
   /// and preferably modeled with SubtargetFeatures or properties in
   /// initializeProperties().
   RISCVProcFamilyEnum getProcFamily() const { return RISCVProcFamily; }
 
   bool hasStdExtM() const { return HasStdExtM; }
   bool hasStdExtA() const { return HasStdExtA; }
   bool hasStdExtF() const { return HasStdExtF; }
   bool hasStdExtD() const { return HasStdExtD; }
   bool hasStdExtC() const { return HasStdExtC; }
   bool hasStdExtV() const { return HasStdExtV; }
   bool hasStdExtZihintpause() const { return HasStdExtZihintpause; }
   bool hasStdExtZba() const { return HasStdExtZba; }
   bool hasStdExtZbb() const { return HasStdExtZbb; }
   bool hasStdExtZbc() const { return HasStdExtZbc; }
public:
   bool hasStdExtZbkc() const { return HasStdExtZbkc; }
   bool hasStdExtZbkx() const { return HasStdExtZbkx; }
   bool hasStdExtZknd() const { return HasStdExtZknd; }
   bool hasStdExtZkne() const { return HasStdExtZkne; }
   bool hasStdExtZknh() const { return HasStdExtZknh; }
   bool hasStdExtZksed() const { return HasStdExtZksed; }
   bool hasStdExtZksh() const { return HasStdExtZksh; }
   bool hasStdExtZkr() const { return HasStdExtZkr; }
+  bool hasStdExtZmmul() const { return HasStdExtZmmul; }
   bool is64Bit() const { return HasRV64; }
   bool isRV32E() const { return IsRV32E; }
   bool enableLinkerRelax() const { return EnableLinkerRelax; }
   bool enableRVCHintInstrs() const { return EnableRVCHintInstrs; }
   bool enableSaveRestore() const { return EnableSaveRestore; }
   MVT getXLenVT() const { return XLenVT; }
   unsigned getXLen() const { return XLen; }
   unsigned getFLen() const {

llvm/test/CodeGen/RISCV/attributes.ll

 ; RUN: llc -mtriple=riscv64 -mattr=+zksed %s -o - | FileCheck --check-prefix=RV64ZKSED %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zksh %s -o - | FileCheck --check-prefix=RV64ZKSH %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zkr %s -o - | FileCheck --check-prefix=RV64ZKR %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zkn %s -o - | FileCheck --check-prefix=RV64ZKN %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zks %s -o - | FileCheck --check-prefix=RV64ZKS %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zkt %s -o - | FileCheck --check-prefix=RV64ZKT %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zk %s -o - | FileCheck --check-prefix=RV64ZK %s
 
-; RV32M: .attribute 5, "rv32i2p0_m2p0"
+; RV32M: .attribute 5, "rv32i2p0_m2p0_zmmul0p1"
 ; RV32A: .attribute 5, "rv32i2p0_a2p0"
 ; RV32F: .attribute 5, "rv32i2p0_f2p0"
 ; RV32D: .attribute 5, "rv32i2p0_f2p0_d2p0"
 ; RV32C: .attribute 5, "rv32i2p0_c2p0"
 ; RV32ZIHINTPAUSE: .attribute 5, "rv32i2p0_zihintpause2p0"
 ; RV32ZFHMIN: .attribute 5, "rv32i2p0_f2p0_zfhmin1p0"
 ; RV32ZFH: .attribute 5, "rv32i2p0_f2p0_zfh1p0"
 ; RV32ZBA: .attribute 5, "rv32i2p0_zba1p0"

 ; RV32ZKSED: .attribute 5, "rv32i2p0_zksed1p0"
 ; RV32ZKSH: .attribute 5, "rv32i2p0_zksh1p0"
 ; RV32ZKR: .attribute 5, "rv32i2p0_zkr1p0"
 ; RV32ZKN: .attribute 5, "rv32i2p0_zbkb1p0_zbkc1p0_zbkx1p0_zkn1p0_zknd1p0_zkne1p0_zknh1p0"
 ; RV32ZKS: .attribute 5, "rv32i2p0_zbkb1p0_zbkc1p0_zbkx1p0_zks1p0_zksed1p0_zksh1p0"
 ; RV32ZKT: .attribute 5, "rv32i2p0_zkt1p0"
 ; RV32ZK: .attribute 5, "rv32i2p0_zbkb1p0_zbkc1p0_zbkx1p0_zk1p0_zkn1p0_zknd1p0_zkne1p0_zknh1p0_zkr1p0_zkt1p0"
 
-; RV64M: .attribute 5, "rv64i2p0_m2p0"
+; RV64M: .attribute 5, "rv64i2p0_m2p0_zmmul0p1"
 ; RV64A: .attribute 5, "rv64i2p0_a2p0"
 ; RV64F: .attribute 5, "rv64i2p0_f2p0"
 ; RV64D: .attribute 5, "rv64i2p0_f2p0_d2p0"
 ; RV64C: .attribute 5, "rv64i2p0_c2p0"
 ; RV64ZIHINTPAUSE: .attribute 5, "rv64i2p0_zihintpause2p0"
 ; RV64ZFHMIN: .attribute 5, "rv64i2p0_f2p0_zfhmin1p0"
 ; RV64ZFH: .attribute 5, "rv64i2p0_f2p0_zfh1p0"
 ; RV64ZBA: .attribute 5, "rv64i2p0_zba1p0"

llvm/test/CodeGen/RISCV/zmmul.ll

+; RUN: llc -mtriple=riscv32 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-DIV %s
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-DIV %s
+; RUN: llc -mtriple=riscv32 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-REM %s
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-REM %s
+
+; RUN: llc -mtriple=riscv32 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-UDIV %s
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-UDIV %s
+; RUN: llc -mtriple=riscv32 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-UREM %s
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | not FileCheck -check-prefix=CHECK-UREM %s
+
+; RUN: llc -mtriple=riscv32 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | FileCheck -check-prefix=CHECK-MUL %s
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-zmmul -verify-machineinstrs < %s \
+; RUN:  | FileCheck -check-prefix=CHECK-MUL %s
+
+; RUN: llc -mtriple=riscv32 -mattr=+m -verify-machineinstrs < %s \
+; RUN:  | FileCheck -check-prefixes=CHECK-MUL,CHECK-UDIV,CHECK-DIV,CHECK-UREM,CHECK-REM %s
+; RUN: llc -mtriple=riscv64 -mattr=+m -verify-machineinstrs < %s \
+; RUN:  | FileCheck -check-prefixes=CHECK-MUL,CHECK-UDIV,CHECK-DIV,CHECK-UREM,CHECK-REM %s
+
+define i32 @foo(i32 %a, i32 %b) {
+; CHECK-UDIV: divu{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %1 = udiv i32 %a, %b
+; CHECK-DIV: div{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %2 = sdiv i32 %a, %1
+; CHECK-MUL: mul{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %3 = mul i32 %b, %2
+; CHECK-UREM: remu{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %4 = urem i32 %3, %b
+; CHECK-REM: rem{{w?}} {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}, {{[as]}}{{[0-9]}}
+  %5 = srem i32 %4, %a
+  ret i32 %5
+}

llvm/test/MC/RISCV/rv32i-invalid.s

 # Memory operand not formatted correctly
 lw a4, a5, 111 # CHECK: :[[@LINE]]:12: error: invalid operand for instruction
 
 # Too few operands
 ori a0, a1 # CHECK: :[[@LINE]]:1: error: too few operands for instruction
 xor s2, s2 # CHECK: :[[@LINE]]:1: error: too few operands for instruction
 
 # Instruction not in the base ISA
-mul a4, ra, s0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+div a4, ra, s0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
 amomaxu.w s5, s4, (s3) # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'A' (Atomic Instructions)
 fadd.s ft0, ft1, ft2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'F' (Single-Precision Floating-Point){{$}}
 fadd.h ft0, ft1, ft2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zfh' (Half-Precision Floating-Point)
 fadd.s a0, a1, a2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zfinx' (Float in Integer)
 fadd.d a0, a2, a4 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zdinx' (Double in Integer)
 fadd.h a0, a1, a2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zhinx' (Half Float in Integer)
 flh ft0, (a0) # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zfh' (Half-Precision Floating-Point) or 'Zfhmin' (Half-Precision Floating-Point Minimal){{$}}
 sh1add a0, a1, a2 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'Zba' (Address Generation Instructions)

llvm/test/MC/RISCV/rv32zmmul-invaild.s

+# RUN: not llvm-mc %s -triple=riscv32 -mattr=+experimental-zmmul -riscv-no-aliases 2>&1 \
+# RUN:  | FileCheck -check-prefixes=CHECK-ERROR %s
+
+# CHECK-ERROR: 5:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+div s0, s0, s0
+
+# CHECK-ERROR: 8:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+divu gp, a0, a1
+
+# CHECK-ERROR: 11:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+rem s2, s2, s8
+
+# CHECK-ERROR: 14:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+remu x18, x18, x24

llvm/test/MC/RISCV/rv32zmmul-valid.s

+# RUN: llvm-mc %s -triple=riscv32 -mattr=+experimental-zmmul -riscv-no-aliases 2>&1 \
+# RUN:  | FileCheck -check-prefixes=CHECK-INST %s
+
+# CHECK-INST: mul a4, ra, s0
+mul a4, ra, s0
+
+# CHECK-INST: mulh ra, zero, zero
+mulh x1, x0, x0
+
+# CHECK-INST: mulhsu t0, t2, t1
+mulhsu t0, t2, t1
+
+# CHECK-INST: mulhu a5, a4, a3
+mulhu a5, a4, a3

llvm/test/MC/RISCV/rv64zmmul-invalid.s

+# RUN: not llvm-mc %s -triple=riscv64 -mattr=+experimental-zmmul -riscv-no-aliases 2>&1 \
+# RUN:  | FileCheck -check-prefixes=CHECK-ERROR %s
+
+# CHECK-ERROR: 5:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+divw tp, t0, t1
+
+# CHECK-ERROR: 8:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+divuw t2, s0, s2
+
+# CHECK-ERROR: 11:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+remw a0, a1, a2
+
+# CHECK-ERROR: 14:1: error: instruction requires the following: 'M' (Integer Multiplication and Division)
+remuw a3, a4, a5

llvm/test/MC/RISCV/rv64zmmul-valid.s

+# RUN: llvm-mc %s -triple=riscv64 -mattr=+experimental-zmmul -riscv-no-aliases 2>&1 \
+# RUN:  | FileCheck -check-prefixes=CHECK-INST %s
+
+# CHECK-INST: mulw ra, sp, gp
+mulw ra, sp, gp