[RISCV] Add vendor-defined XTheadFMemIdx (FP Indexed Memory Operations) extension

[mtsamis]

Summary

The vendor-defined XTHeadFMemIdx (no comparable standard extension exists
at the time of writing) extension adds indexed load/store instructions
for floating-point registers.

It is supported by the C9xx cores (e.g., found in the wild in the
Allwinner D1) by Alibaba T-Head.

The current (as of this commit) public documentation for this
extension is available at:

https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf

Support for these instructions has already landed in GNU Binutils:

https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=f511f80fa3fcaf6bcbe727fb902b8bd5ec8f9c20

Depends on D144249

[craig.topper]

LGTM

llvm/docs/RISCVUsage.rst

``XTHeadBb``
   LLVM implements `the THeadBb (basic bit-manipulation) vendor-defined instructions specified in <https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf>`_  by T-HEAD of Alibaba.  Instructions are prefixed with `th.` as described in the specification.
 
 ``XTHeadBs``
   LLVM implements `the THeadBs (single-bit operations) vendor-defined instructions specified in <https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf>`_  by T-HEAD of Alibaba.  Instructions are prefixed with `th.` as described in the specification.
 
 ``XTHeadCmo``
   LLVM implements `the THeadCmo (cache management operations) vendor-defined instructions specified in <https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf>`_  by T-HEAD of Alibaba.  Instructions are prefixed with `th.` as described in the specification.
 
+``XTHeadFMemIdx``
+  LLVM implements `the THeadFMemIdx (indexed memory operations for floating point) vendor-defined instructions specified in <https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf>`_  by T-HEAD of Alibaba.  Instructions are prefixed with `th.` as described in the specification.
+
 ``XTheadMac``
   LLVM implements `the XTheadMac (multiply-accumulate instructions) vendor-defined instructions specified in <https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf>`_  by T-HEAD of Alibaba.  Instructions are prefixed with `th.` as described in the specification.
 
 ``XTHeadMemIdx``
   LLVM implements `the THeadMemIdx (indexed memory operations) vendor-defined instructions specified in <https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf>`_  by T-HEAD of Alibaba.  Instructions are prefixed with `th.` as described in the specification.
 
 ``XTHeadMemPair``
   LLVM implements `the THeadMemPair (two-GPR memory operations) vendor-defined instructions specified in <https://github.com/T-head-Semi/thead-extension-spec/releases/download/2.2.2/xthead-2023-01-30-2.2.2.pdf>`_  by T-HEAD of Alibaba.  Instructions are prefixed with `th.` as described in the specification.

llvm/docs/ReleaseNotes.rst

 * Adds support for the vendor-defined XTHeadMac (multiply-accumulate instructions) extension.
 * Added support for the vendor-defined XTHeadMemPair (two-GPR memory operations)
   extension disassembler/assembler.
 * Added support for the vendor-defined XTHeadMemIdx (indexed memory operations)
   extension disassembler/assembler.
 * Support for the now-ratified Zawrs extension is no longer experimental.
 * Adds support for the vendor-defined XTHeadCmo (cache management operations) extension.
 * Adds support for the vendor-defined XTHeadSync (multi-core synchronization instructions) extension.
+* Added support for the vendor-defined XTHeadFMemIdx (indexed memory operations for floating point) extension.
 
 Changes to the WebAssembly Backend
 ----------------------------------
 
 * ...
 
 Changes to the Windows Target
 -----------------------------

llvm/lib/Support/RISCVISAInfo.cpp

static const RISCVSupportedExtension SupportedExtensions[] = {
     {"svpbmt", RISCVExtensionVersion{1, 0}},
     {"svinval", RISCVExtensionVersion{1, 0}},
 
     // vendor-defined ('X') extensions
     {"xtheadba", RISCVExtensionVersion{1, 0}},
     {"xtheadbb", RISCVExtensionVersion{1, 0}},
     {"xtheadbs", RISCVExtensionVersion{1, 0}},
     {"xtheadcmo", RISCVExtensionVersion{1, 0}},
+    {"xtheadfmemidx", RISCVExtensionVersion{1, 0}},
     {"xtheadmac", RISCVExtensionVersion{1, 0}},
     {"xtheadmemidx", RISCVExtensionVersion{1, 0}},
     {"xtheadmempair", RISCVExtensionVersion{1, 0}},
     {"xtheadsync", RISCVExtensionVersion{1, 0}},
     {"xtheadvdot", RISCVExtensionVersion{1, 0}},
     {"xventanacondops", RISCVExtensionVersion{1, 0}},
 };
 

llvm/lib/Target/RISCV/Disassembler/RISCVDisassembler.cpp

if ((Bytes[0] & 0x3) == 0x3) {
     }
     if (STI.hasFeature(RISCV::FeatureVendorXTHeadCmo)) {
       LLVM_DEBUG(dbgs() << "Trying XTHeadCmo custom opcode table:\n");
       Result = decodeInstruction(DecoderTableTHeadCmo32, MI, Insn, Address,
                                  this, STI);
       if (Result != MCDisassembler::Fail)
         return Result;
     }
+    if (STI.hasFeature(RISCV::FeatureVendorXTHeadFMemIdx)) {
+      LLVM_DEBUG(dbgs() << "Trying XTHeadFMemIdx custom opcode table:\n");
+      Result = decodeInstruction(DecoderTableTHeadFMemIdx32, MI, Insn, Address,
+                                 this, STI);
+      if (Result != MCDisassembler::Fail)
+        return Result;
+    }
     if (STI.hasFeature(RISCV::FeatureVendorXTHeadMac)) {
       LLVM_DEBUG(dbgs() << "Trying XTHeadMac custom opcode table:\n");
       Result = decodeInstruction(DecoderTableTHeadMac32, MI, Insn, Address,
                                  this, STI);
       if (Result != MCDisassembler::Fail)
         return Result;
     }
     if (STI.hasFeature(RISCV::FeatureVendorXTHeadMemIdx)) {

llvm/lib/Target/RISCV/RISCVFeatures.td

 
 def FeatureVendorXTHeadCmo
     : SubtargetFeature<"xtheadcmo", "HasVendorXTHeadCmo", "true",
                        "'xtheadcmo' (T-Head cache management instructions)">;
 def HasVendorXTHeadCmo : Predicate<"Subtarget->hasVendorXTHeadCmo()">,
                                    AssemblerPredicate<(all_of FeatureVendorXTHeadCmo),
                                    "'xtheadcmo' (T-Head cache management instructions)">;
 
+def FeatureVendorXTHeadFMemIdx
+    : SubtargetFeature<"xtheadfmemidx", "HasVendorXTHeadFMemIdx", "true",
+                       "'xtheadfmemidx' (T-Head FP Indexed Memory Operations)",
+                       [FeatureStdExtF]>;
+def HasVendorXTHeadFMemIdx : Predicate<"Subtarget->hasVendorXTHeadFMemIdx()">,
+                                       AssemblerPredicate<(all_of FeatureVendorXTHeadFMemIdx),
+                                       "'xtheadfmemidx' (T-Head FP Indexed Memory Operations)">;
+
 def FeatureVendorXTHeadMac
     : SubtargetFeature<"xtheadmac", "HasVendorXTHeadMac", "true",
                        "'xtheadmac' (T-Head Multiply-Accumulate Instructions)">;
 def HasVendorXTHeadMac : Predicate<"Subtarget->hasVendorXTHeadMac()">,
                                    AssemblerPredicate<(all_of FeatureVendorXTHeadMac),
                                    "'xtheadmac' (T-Head Multiply-Accumulate Instructions)">;
 
 def FeatureVendorXTHeadMemIdx

llvm/lib/Target/RISCV/RISCVInstrInfoXTHead.td

 let hasSideEffects = 1, mayLoad = 0, mayStore = 0 in
 class THCacheInst_void<bits<5> funct5, string opcodestr>
     : RVInstR<0b0000000, 0, OPC_CUSTOM_0, (outs), (ins), opcodestr, ""> {
   let rd = 0;
   let rs1 = 0;
   let rs2 = funct5;
 }
 
-let Predicates = [HasVendorXTHeadMemIdx], DecoderNamespace = "THeadMemIdx",
-  hasSideEffects = 0, mayLoad = 1, mayStore = 0 in {
+let hasSideEffects = 0, mayLoad = 1, mayStore = 0 in {
 class THLoadIndexed<RegisterClass Ty, bits<5> funct5, string opcodestr>
     : RVInstR<!shl(funct5, 2), !if(!eq(Ty, GPR), 0b100, 0b110), OPC_CUSTOM_0,
               (outs Ty:$rd), (ins GPR:$rs1, GPR:$rs2, uimm2:$uimm2),
               opcodestr, "$rd, $rs1, $rs2, $uimm2"> {
   bits<2> uimm2;
   let Inst{26-25} = uimm2;
 }
 
 class THLoadUpdate<bits<5> funct5, string opcodestr>
     : RVInstI<0b100, OPC_CUSTOM_0, (outs GPR:$rd, GPR:$rs1_wb),
               (ins GPR:$rs1, simm5:$simm5, uimm2:$uimm2),
               opcodestr, "$rd, (${rs1}), $simm5, $uimm2"> {
   bits<5> simm5;
   bits<2> uimm2;
   let imm12{11-7} = funct5;
   let imm12{6-5} = uimm2;
   let imm12{4-0} = simm5;
   let Constraints = "@earlyclobber $rd, $rs1_wb = $rs1";
 }
 }
 
-let Predicates = [HasVendorXTHeadMemIdx], DecoderNamespace = "THeadMemIdx",
-  hasSideEffects = 0, mayLoad = 0, mayStore = 1 in {
+let hasSideEffects = 0, mayLoad = 0, mayStore = 1 in {
 class THStoreIndexed<RegisterClass StTy, bits<5> funct5, string opcodestr>
     : RVInstR<!shl(funct5, 2), !if(!eq(StTy, GPR), 0b101, 0b111), OPC_CUSTOM_0,
               (outs), (ins StTy:$rd, GPR:$rs1, GPR:$rs2, uimm2:$uimm2),
               opcodestr, "$rd, $rs1, $rs2, $uimm2"> {
   bits<2> uimm2;
   let Inst{26-25} = uimm2;
 }
 

 
 let Predicates = [HasVendorXTHeadMemPair, IsRV64] in {
 def TH_LDD  : THLoadPair<0b11111, "th.ldd">,
               Sched<[WriteLDD, WriteLDD, ReadMemBase]>;
 def TH_SDD  : THStorePair<0b11111, "th.sdd">,
               Sched<[WriteSTD, WriteSTD, ReadStoreData, ReadMemBase]>;
 }
 
-let Predicates = [HasVendorXTHeadMemIdx] in {
+let Predicates = [HasVendorXTHeadMemIdx], DecoderNamespace = "THeadMemIdx" in {
 // T-Head Load/Store + Update instructions.
 def TH_LBIA : THLoadUpdate<0b00011, "th.lbia">,
   Sched<[WriteLDB, ReadMemBase]>;
 def TH_LBIB : THLoadUpdate<0b00001, "th.lbib">,
   Sched<[WriteLDB, ReadMemBase]>;
 def TH_LBUIA : THLoadUpdate<0b10011, "th.lbuia">,
    Sched<[WriteLDB, ReadMemBase]>;
 def TH_LBUIB : THLoadUpdate<0b10001, "th.lbuib">,
def TH_SURH : THStoreIndexed<GPR, 0b00110, "th.surh">,
   Sched<[WriteLDB, ReadMemBase]>;
 
 def TH_SRW : THStoreIndexed<GPR, 0b01000, "th.srw">,
   Sched<[WriteSTW, ReadStoreData, ReadMemBase]>;
 def TH_SURW : THStoreIndexed<GPR, 0b01010, "th.surw">,
   Sched<[WriteLDB, ReadMemBase]>;
 }
 
-let Predicates = [HasVendorXTHeadMemIdx, IsRV64] in {
+let Predicates = [HasVendorXTHeadMemIdx, IsRV64], DecoderNamespace = "THeadMemIdx" in {
 // T-Head Load/Store + Update instructions.
 def TH_LWUIA : THLoadUpdate<0b11011, "th.lwuia">,
   Sched<[WriteLDH, ReadMemBase]>;
 def TH_LWUIB : THLoadUpdate<0b11001, "th.lwuib">,
   Sched<[WriteLDH, ReadMemBase]>;
 
 def TH_LDIA : THLoadUpdate<0b01111, "th.ldia">,
   Sched<[WriteLDW, ReadMemBase]>;
def TH_LURD : THLoadIndexed<GPR, 0b01110, "th.lurd">,
   Sched<[WriteLDB, ReadMemBase]>;
 
 def TH_SRD : THStoreIndexed<GPR, 0b01100, "th.srd">,
   Sched<[WriteSTW, ReadStoreData, ReadMemBase]>;
 def TH_SURD : THStoreIndexed<GPR, 0b01110, "th.surd">,
   Sched<[WriteLDB, ReadMemBase]>;
 }
 
+// T-Head Load/Store Indexed instructions for floating point registers.
+
+let Predicates = [HasVendorXTHeadFMemIdx, HasStdExtF],
+    DecoderNamespace = "THeadFMemIdx" in {
+def TH_FLRW : THLoadIndexed<FPR32, 0b01000, "th.flrw">,
+              Sched<[WriteFLD32, ReadFMemBase]>;
+def TH_FSRW : THStoreIndexed<FPR32, 0b01000, "th.fsrw">,
+              Sched<[WriteFST32, ReadFStoreData, ReadFMemBase]>;
+}
+
+let Predicates = [HasVendorXTHeadFMemIdx, HasStdExtD],
+    DecoderNamespace = "THeadFMemIdx" in {
+def TH_FLRD : THLoadIndexed<FPR64, 0b01100, "th.flrd">,
+              Sched<[WriteFLD64, ReadFMemBase]>;
+def TH_FSRD : THStoreIndexed<FPR64, 0b01100, "th.fsrd">,
+              Sched<[WriteFST64, ReadFStoreData, ReadFMemBase]>;
+}
+
+let Predicates = [HasVendorXTHeadFMemIdx, HasStdExtF, IsRV64],
+    DecoderNamespace = "THeadFMemIdx" in {
+def TH_FLURW : THLoadIndexed<FPR32, 0b01010, "th.flurw">,
+               Sched<[WriteFLD32, ReadFMemBase]>;
+def TH_FSURW : THStoreIndexed<FPR32, 0b01010, "th.fsurw">,
+               Sched<[WriteFST32, ReadFStoreData, ReadFMemBase]>;
+}
+
+let Predicates = [HasVendorXTHeadFMemIdx, HasStdExtD, IsRV64],
+    DecoderNamespace = "THeadFMemIdx" in {
+def TH_FLURD : THLoadIndexed<FPR64, 0b01110, "th.flurd">,
+               Sched<[WriteFLD64, ReadFMemBase]>;
+def TH_FSURD : THStoreIndexed<FPR64, 0b01110, "th.fsurd">,
+               Sched<[WriteFST64, ReadFStoreData, ReadFMemBase]>;
+}
+
 let Predicates = [HasVendorXTHeadVdot],
     Constraints = "@earlyclobber $vd",
     RVVConstraint = WidenV in {
 defm THVdotVMAQA      : THVdotVMAQA<"th.vmaqa",     0b100000>;
 defm THVdotVMAQAU     : THVdotVMAQA<"th.vmaqau",    0b100010>;
 defm THVdotVMAQASU    : THVdotVMAQA<"th.vmaqasu",   0b100100>;
 defm THVdotVMAQAUS    : THVdotVMAQA_VX<"th.vmaqaus",0b100110>;
 }

 defm : StZextIdxPat<truncstorei8, TH_SURB, GPR>;
 defm : StZextIdxPat<truncstorei16, TH_SURH, GPR>;
 defm : StIdxPat<truncstorei32, TH_SRW, GPR>;
 defm : StZextIdxPat<truncstorei32, TH_SURW, GPR>;
 defm : StIdxPat<store, TH_SRD, GPR>;
 defm : StZextIdxPat<store, TH_SURD, GPR>;
 }
 
+let Predicates = [HasVendorXTHeadFMemIdx, HasStdExtF, IsRV32] in {
+defm : LdIdxPat<load, TH_FLRW, f32>;
+defm : StIdxPat<store, TH_FSRW, FPR32, f32>;
+}
+
+let Predicates = [HasVendorXTHeadFMemIdx, HasStdExtD, IsRV32] in {
+defm : LdIdxPat<load, TH_FLRD, f64>;
+defm : StIdxPat<store, TH_FSRD, FPR64, f64>;
+}
+
+let Predicates = [HasVendorXTHeadFMemIdx, HasStdExtF, IsRV64] in {
+defm : LdIdxPat<load, TH_FLRW, f32>;
+defm : LdZextIdxPat<load, TH_FLURW, f32>;
+defm : StIdxPat<store, TH_FSRW, FPR32, f32>;
+defm : StZextIdxPat<store, TH_FSURW, FPR32, f32>;
+}
+
+let Predicates = [HasVendorXTHeadFMemIdx, HasStdExtD, IsRV64] in {
+defm : LdIdxPat<load, TH_FLRD, f64>;
+defm : LdZextIdxPat<load, TH_FLURD, f64>;
+defm : StIdxPat<store, TH_FSRD, FPR64, f64>;
+defm : StZextIdxPat<store, TH_FSURD, FPR64, f64>;
+}
+
 def simm5shl2 : ComplexPattern<XLenVT, 2, "selectSimm5Shl2">;
 
 multiclass StoreUpdatePat<PatFrag st, Instruction Inst, ValueType vt = XLenVT> {
 def : Pat<(st (vt GPR:$rd), GPR:$rs1, (simm5shl2 simm5:$simm5, uimm2:$uimm2)),
           (Inst GPR:$rd, GPR:$rs1, simm5:$simm5, uimm2:$uimm2)>;
 }
 
 let Predicates = [HasVendorXTHeadMemIdx] in {

llvm/test/CodeGen/RISCV/attributes.ll

 ; RUN: llc -mtriple=riscv32 -mattr=+zbkb,+zbkc,+zbkx,+zksed,+zksh %s -o - | FileCheck --check-prefixes=CHECK,RV32COMBINEINTOZKS %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zicbom %s -o - | FileCheck --check-prefixes=CHECK,RV32ZICBOM %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zicboz %s -o - | FileCheck --check-prefixes=CHECK,RV32ZICBOZ %s
 ; RUN: llc -mtriple=riscv32 -mattr=+zicbop %s -o - | FileCheck --check-prefixes=CHECK,RV32ZICBOP %s
 ; RUN: llc -mtriple=riscv32 -mattr=+svnapot %s -o - | FileCheck --check-prefixes=CHECK,RV32SVNAPOT %s
 ; RUN: llc -mtriple=riscv32 -mattr=+svpbmt %s -o - | FileCheck --check-prefixes=CHECK,RV32SVPBMT %s
 ; RUN: llc -mtriple=riscv32 -mattr=+svinval %s -o - | FileCheck --check-prefixes=CHECK,RV32SVINVAL %s
 ; RUN: llc -mtriple=riscv32 -mattr=+xtheadcmo %s -o - | FileCheck --check-prefix=RV32XTHEADCMO %s
+; RUN: llc -mtriple=riscv32 -mattr=+xtheadfmemidx %s -o - | FileCheck --check-prefix=RV32XTHEADFMEMIDX %s
 ; RUN: llc -mtriple=riscv32 -mattr=+xtheadmac %s -o - | FileCheck --check-prefixes=CHECK,RV32XTHEADMAC %s
 ; RUN: llc -mtriple=riscv32 -mattr=+xtheadmemidx %s -o - | FileCheck --check-prefix=RV32XTHEADMEMIDX %s
 ; RUN: llc -mtriple=riscv32 -mattr=+xtheadmempair %s -o - | FileCheck --check-prefix=RV32XTHEADMEMPAIR %s
 ; RUN: llc -mtriple=riscv32 -mattr=+xtheadsync %s -o - | FileCheck --check-prefix=RV32XTHEADSYNC %s
 ; RUN: llc -mtriple=riscv32 -mattr=+experimental-zca %s -o - | FileCheck --check-prefixes=CHECK,RV32ZCA %s
 ; RUN: llc -mtriple=riscv32 -mattr=+experimental-zcb %s -o - | FileCheck --check-prefixes=CHECK,RV32ZCB %s
 ; RUN: llc -mtriple=riscv32 -mattr=+experimental-zcd %s -o - | FileCheck --check-prefixes=CHECK,RV32ZCD %s
 ; RUN: llc -mtriple=riscv32 -mattr=+experimental-zcf %s -o - | FileCheck --check-prefixes=CHECK,RV32ZCF %s

 ; RUN: llc -mtriple=riscv64 -mattr=+svnapot %s -o - | FileCheck --check-prefixes=CHECK,RV64SVNAPOT %s
 ; RUN: llc -mtriple=riscv64 -mattr=+svpbmt %s -o - | FileCheck --check-prefixes=CHECK,RV64SVPBMT %s
 ; RUN: llc -mtriple=riscv64 -mattr=+svinval %s -o - | FileCheck --check-prefixes=CHECK,RV64SVINVAL %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xventanacondops %s -o - | FileCheck --check-prefixes=CHECK,RV64XVENTANACONDOPS %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xtheadba %s -o - | FileCheck --check-prefixes=CHECK,RV64XTHEADBA %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xtheadbb %s -o - | FileCheck --check-prefixes=CHECK,RV64XTHEADBB %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xtheadbs %s -o - | FileCheck --check-prefixes=CHECK,RV64XTHEADBS %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xtheadcmo %s -o - | FileCheck --check-prefix=RV64XTHEADCMO %s
+; RUN: llc -mtriple=riscv64 -mattr=+xtheadfmemidx %s -o - | FileCheck --check-prefix=RV64XTHEADFMEMIDX %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xtheadmac %s -o - | FileCheck --check-prefixes=CHECK,RV64XTHEADMAC %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xtheadmemidx %s -o - | FileCheck --check-prefix=RV64XTHEADMEMIDX %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xtheadmempair %s -o - | FileCheck --check-prefix=RV64XTHEADMEMPAIR %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xtheadsync %s -o - | FileCheck --check-prefix=RV64XTHEADSYNC %s
 ; RUN: llc -mtriple=riscv64 -mattr=+xtheadvdot %s -o - | FileCheck --check-prefixes=CHECK,RV64XTHEADVDOT %s
 ; RUN: llc -mtriple=riscv64 -mattr=+zawrs %s -o - | FileCheck --check-prefixes=CHECK,RV64ZAWRS %s
 ; RUN: llc -mtriple=riscv64 -mattr=+experimental-ztso %s -o - | FileCheck --check-prefixes=CHECK,RV64ZTSO %s
 ; RUN: llc -mtriple=riscv64 -mattr=+experimental-zca %s -o - | FileCheck --check-prefixes=CHECK,RV64ZCA %s

 ; RV32COMBINEINTOZKS: .attribute 5, "rv32i2p0_zbkb1p0_zbkc1p0_zbkx1p0_zks1p0_zksed1p0_zksh1p0"
 ; RV32ZICBOM: .attribute 5, "rv32i2p0_zicbom1p0"
 ; RV32ZICBOZ: .attribute 5, "rv32i2p0_zicboz1p0"
 ; RV32ZICBOP: .attribute 5, "rv32i2p0_zicbop1p0"
 ; RV32SVNAPOT: .attribute 5, "rv32i2p0_svnapot1p0"
 ; RV32SVPBMT: .attribute 5, "rv32i2p0_svpbmt1p0"
 ; RV32SVINVAL: .attribute 5, "rv32i2p0_svinval1p0"
 ; RV32XTHEADCMO: .attribute 5, "rv32i2p0_xtheadcmo1p0"
+; RV32XTHEADFMEMIDX: .attribute 5, "rv32i2p0_f2p0_xtheadfmemidx1p0"
 ; RV32XTHEADMAC: .attribute 5, "rv32i2p0_xtheadmac1p0"
 ; RV32XTHEADMEMIDX: .attribute 5, "rv32i2p0_xtheadmemidx1p0"
 ; RV32XTHEADMEMPAIR: .attribute 5, "rv32i2p0_xtheadmempair1p0"
 ; RV32XTHEADSYNC: .attribute 5, "rv32i2p0_xtheadsync1p0"
 ; RV32ZCA: .attribute 5, "rv32i2p0_zca1p0"
 ; RV32ZCB: .attribute 5, "rv32i2p0_zca1p0_zcb1p0"
 ; RV32ZCD: .attribute 5, "rv32i2p0_zcd1p0"
 ; RV32ZCF: .attribute 5, "rv32i2p0_zcf1p0"

 ; RV64SVNAPOT: .attribute 5, "rv64i2p0_svnapot1p0"
 ; RV64SVPBMT: .attribute 5, "rv64i2p0_svpbmt1p0"
 ; RV64SVINVAL: .attribute 5, "rv64i2p0_svinval1p0"
 ; RV64XVENTANACONDOPS: .attribute 5, "rv64i2p0_xventanacondops1p0"
 ; RV64XTHEADBA: .attribute 5, "rv64i2p0_xtheadba1p0"
 ; RV64XTHEADBB: .attribute 5, "rv64i2p0_xtheadbb1p0"
 ; RV64XTHEADBS: .attribute 5, "rv64i2p0_xtheadbs1p0"
 ; RV64XTHEADCMO: .attribute 5, "rv64i2p0_xtheadcmo1p0"
+; RV64XTHEADFMEMIDX: .attribute 5, "rv64i2p0_f2p0_xtheadfmemidx1p0"
 ; RV64XTHEADMAC: .attribute 5, "rv64i2p0_xtheadmac1p0"
 ; RV64XTHEADMEMIDX: .attribute 5, "rv64i2p0_xtheadmemidx1p0"
 ; RV64XTHEADMEMPAIR: .attribute 5, "rv64i2p0_xtheadmempair1p0"
 ; RV64XTHEADSYNC: .attribute 5, "rv64i2p0_xtheadsync1p0"
 ; RV64XTHEADVDOT: .attribute 5, "rv64i2p0_f2p0_d2p0_v1p0_zve32f1p0_zve32x1p0_zve64d1p0_zve64f1p0_zve64x1p0_zvl128b1p0_zvl32b1p0_zvl64b1p0_xtheadvdot1p0"
 ; RV64ZTSO: .attribute 5, "rv64i2p0_ztso0p1"
 ; RV64ZCA: .attribute 5, "rv64i2p0_zca1p0"
 ; RV64ZCB: .attribute 5, "rv64i2p0_zca1p0_zcb1p0"

llvm/test/CodeGen/RISCV/xtheadfmemidx.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -mattr=+d -mattr=+xtheadfmemidx -mattr=+m -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32XTHEADMEMIDX
+; RUN: llc -mtriple=riscv64 -mattr=+d -mattr=+xtheadfmemidx -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64XTHEADFMEMIDX
+
+define float @flrw(float* %a, i64 %b) {
+; RV32XTHEADMEMIDX-LABEL: flrw:
+; RV32XTHEADMEMIDX:       # %bb.0:
+; RV32XTHEADMEMIDX-NEXT:    th.flrw ft0, a0, a1, 2
+; RV32XTHEADMEMIDX-NEXT:    fadd.s fa0, ft0, ft0
+; RV32XTHEADMEMIDX-NEXT:    ret
+;
+; RV64XTHEADFMEMIDX-LABEL: flrw:
+; RV64XTHEADFMEMIDX:       # %bb.0:
+; RV64XTHEADFMEMIDX-NEXT:    th.flrw ft0, a0, a1, 2
+; RV64XTHEADFMEMIDX-NEXT:    fadd.s fa0, ft0, ft0
+; RV64XTHEADFMEMIDX-NEXT:    ret
+  %1 = getelementptr float, float* %a, i64 %b
+  %2 = load float, float* %1, align 4
+  %3 = fadd float %2, %2
+  ret float %3
+}
+
+define float @flurw(float* %a, i32 %b) {
+; RV32XTHEADMEMIDX-LABEL: flurw:
+; RV32XTHEADMEMIDX:       # %bb.0:
+; RV32XTHEADMEMIDX-NEXT:    th.flrw ft0, a0, a1, 2
+; RV32XTHEADMEMIDX-NEXT:    fadd.s fa0, ft0, ft0
+; RV32XTHEADMEMIDX-NEXT:    ret
+;
+; RV64XTHEADFMEMIDX-LABEL: flurw:
+; RV64XTHEADFMEMIDX:       # %bb.0:
+; RV64XTHEADFMEMIDX-NEXT:    th.flurw ft0, a0, a1, 2
+; RV64XTHEADFMEMIDX-NEXT:    fadd.s fa0, ft0, ft0
+; RV64XTHEADFMEMIDX-NEXT:    ret
+  %1 = zext i32 %b to i64
+  %2 = getelementptr float, float* %a, i64 %1
+  %3 = load float, float* %2, align 4
+  %4 = fadd float %3, %3
+  ret float %4
+}
+
+define void @fsrw(float* %a, i64 %b, float %c) {
+; RV32XTHEADMEMIDX-LABEL: fsrw:
+; RV32XTHEADMEMIDX:       # %bb.0:
+; RV32XTHEADMEMIDX-NEXT:    fadd.s ft0, fa0, fa0
+; RV32XTHEADMEMIDX-NEXT:    th.fsrw ft0, a0, a1, 2
+; RV32XTHEADMEMIDX-NEXT:    ret
+;
+; RV64XTHEADFMEMIDX-LABEL: fsrw:
+; RV64XTHEADFMEMIDX:       # %bb.0:
+; RV64XTHEADFMEMIDX-NEXT:    fadd.s ft0, fa0, fa0
+; RV64XTHEADFMEMIDX-NEXT:    th.fsrw ft0, a0, a1, 2
+; RV64XTHEADFMEMIDX-NEXT:    ret
+  %1 = fadd float %c, %c
+  %2 = getelementptr float, float* %a, i64 %b
+  store float %1, float* %2, align 4
+  ret void
+}
+
+define void @fsurw(float* %a, i32 %b, float %c) {
+; RV32XTHEADMEMIDX-LABEL: fsurw:
+; RV32XTHEADMEMIDX:       # %bb.0:
+; RV32XTHEADMEMIDX-NEXT:    fadd.s ft0, fa0, fa0
+; RV32XTHEADMEMIDX-NEXT:    th.fsrw ft0, a0, a1, 2
+; RV32XTHEADMEMIDX-NEXT:    ret
+;
+; RV64XTHEADFMEMIDX-LABEL: fsurw:
+; RV64XTHEADFMEMIDX:       # %bb.0:
+; RV64XTHEADFMEMIDX-NEXT:    fadd.s ft0, fa0, fa0
+; RV64XTHEADFMEMIDX-NEXT:    th.fsurw ft0, a0, a1, 2
+; RV64XTHEADFMEMIDX-NEXT:    ret
+  %1 = zext i32 %b to i64
+  %2 = fadd float %c, %c
+  %3 = getelementptr float, float* %a, i64 %1
+  store float %2, float* %3, align 4
+  ret void
+}
+
+define double @flrd(double* %a, i64 %b) {
+; RV32XTHEADMEMIDX-LABEL: flrd:
+; RV32XTHEADMEMIDX:       # %bb.0:
+; RV32XTHEADMEMIDX-NEXT:    th.flrd ft0, a0, a1, 3
+; RV32XTHEADMEMIDX-NEXT:    fadd.d fa0, ft0, ft0
+; RV32XTHEADMEMIDX-NEXT:    ret
+;
+; RV64XTHEADFMEMIDX-LABEL: flrd:
+; RV64XTHEADFMEMIDX:       # %bb.0:
+; RV64XTHEADFMEMIDX-NEXT:    th.flrd ft0, a0, a1, 3
+; RV64XTHEADFMEMIDX-NEXT:    fadd.d fa0, ft0, ft0
+; RV64XTHEADFMEMIDX-NEXT:    ret
+  %1 = getelementptr double, double* %a, i64 %b
+  %2 = load double, double* %1, align 8
+  %3 = fadd double %2, %2
+  ret double %3
+}
+
+define double @flurd(double* %a, i32 %b) {
+; RV32XTHEADMEMIDX-LABEL: flurd:
+; RV32XTHEADMEMIDX:       # %bb.0:
+; RV32XTHEADMEMIDX-NEXT:    th.flrd ft0, a0, a1, 3
+; RV32XTHEADMEMIDX-NEXT:    fadd.d fa0, ft0, ft0
+; RV32XTHEADMEMIDX-NEXT:    ret
+;
+; RV64XTHEADFMEMIDX-LABEL: flurd:
+; RV64XTHEADFMEMIDX:       # %bb.0:
+; RV64XTHEADFMEMIDX-NEXT:    th.flurd ft0, a0, a1, 3
+; RV64XTHEADFMEMIDX-NEXT:    fadd.d fa0, ft0, ft0
+; RV64XTHEADFMEMIDX-NEXT:    ret
+  %1 = zext i32 %b to i64
+  %2 = getelementptr double, double* %a, i64 %1
+  %3 = load double, double* %2, align 8
+  %4 = fadd double %3, %3
+  ret double %4
+}
+
+define void @fsrd(double* %a, i64 %b, double %c) {
+; RV32XTHEADMEMIDX-LABEL: fsrd:
+; RV32XTHEADMEMIDX:       # %bb.0:
+; RV32XTHEADMEMIDX-NEXT:    fadd.d ft0, fa0, fa0
+; RV32XTHEADMEMIDX-NEXT:    th.fsrd ft0, a0, a1, 3
+; RV32XTHEADMEMIDX-NEXT:    ret
+;
+; RV64XTHEADFMEMIDX-LABEL: fsrd:
+; RV64XTHEADFMEMIDX:       # %bb.0:
+; RV64XTHEADFMEMIDX-NEXT:    fadd.d ft0, fa0, fa0
+; RV64XTHEADFMEMIDX-NEXT:    th.fsrd ft0, a0, a1, 3
+; RV64XTHEADFMEMIDX-NEXT:    ret
+  %1 = fadd double %c, %c
+  %2 = getelementptr double, double* %a, i64 %b
+  store double %1, double* %2, align 8
+  ret void
+}
+
+define void @fsurd(double* %a, i32 %b, double %c) {
+; RV32XTHEADMEMIDX-LABEL: fsurd:
+; RV32XTHEADMEMIDX:       # %bb.0:
+; RV32XTHEADMEMIDX-NEXT:    fadd.d ft0, fa0, fa0
+; RV32XTHEADMEMIDX-NEXT:    th.fsrd ft0, a0, a1, 3
+; RV32XTHEADMEMIDX-NEXT:    ret
+;
+; RV64XTHEADFMEMIDX-LABEL: fsurd:
+; RV64XTHEADFMEMIDX:       # %bb.0:
+; RV64XTHEADFMEMIDX-NEXT:    fadd.d ft0, fa0, fa0
+; RV64XTHEADFMEMIDX-NEXT:    th.fsurd ft0, a0, a1, 3
+; RV64XTHEADFMEMIDX-NEXT:    ret
+  %1 = zext i32 %b to i64
+  %2 = fadd double %c, %c
+  %3 = getelementptr double, double* %a, i64 %1
+  store double %2, double* %3, align 8
+  ret void
+}

llvm/test/MC/RISCV/rv32xtheadfmemidx-d-invalid.s

+# RUN: not llvm-mc -triple riscv32 -mattr=+d -mattr=+xtheadfmemidx < %s 2>&1 | FileCheck %s
+
+th.flurd fa0, a1, a2, 0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: RV64I Base Instruction Set{{$}}
+th.flurw fa0, a1, a2, 0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: RV64I Base Instruction Set{{$}}
+th.fsurd fa0, a1, a2, 0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: RV64I Base Instruction Set{{$}}
+th.fsurw fa0, a1, a2, 0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: RV64I Base Instruction Set{{$}}

llvm/test/MC/RISCV/rv32xtheadfmemidx-f-invalid.s

+# RUN: not llvm-mc -triple riscv32 -mattr=+f -mattr=+xtheadfmemidx < %s 2>&1 | FileCheck %s
+
+th.flrd fa0, a1, a2, 0  # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'D' (Double-Precision Floating-Point){{$}}
+th.fsrd fa0, a1, a2, 0  # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'D' (Double-Precision Floating-Point){{$}}
+th.flurd fa0, a1, a2, 0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'D' (Double-Precision Floating-Point), RV64I Base Instruction Set{{$}}
+th.flurw fa0, a1, a2, 0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: RV64I Base Instruction Set{{$}}
+th.fsurd fa0, a1, a2, 0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: 'D' (Double-Precision Floating-Point), RV64I Base Instruction Set{{$}}
+th.fsurw fa0, a1, a2, 0 # CHECK: :[[@LINE]]:1: error: instruction requires the following: RV64I Base Instruction Set{{$}}

llvm/test/MC/RISCV/rv32xtheadfmemidx-valid.s

+# RUN: llvm-mc %s -triple=riscv32 -mattr=+d -mattr=+xtheadfmemidx -show-encoding \
+# RUN:     | FileCheck -check-prefixes=CHECK-ASM,CHECK-ASM-AND-OBJ %s
+# RUN: llvm-mc -filetype=obj -triple=riscv32 -mattr=+d -mattr=+xtheadfmemidx < %s \
+# RUN:     | llvm-objdump --mattr=+d --mattr=+xtheadfmemidx -d -r - \
+# RUN:     | FileCheck --check-prefix=CHECK-ASM-AND-OBJ %s
+
+# CHECK-ASM-AND-OBJ: th.flrd fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x60]
+th.flrd fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.flrd fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x66]
+th.flrd fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.flrw fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x40]
+th.flrw fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.flrw fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x46]
+th.flrw fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.fsrd fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x60]
+th.fsrd fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.fsrd fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x66]
+th.fsrd fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.fsrw fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x40]
+th.fsrw fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.fsrw fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x46]
+th.fsrw fa0, a1, a2, 3

llvm/test/MC/RISCV/rv64xtheadfmemidx-invalid.s

+# RUN: not llvm-mc -triple riscv32 -mattr=+d -mattr=+xtheadfmemidx < %s 2>&1 | FileCheck %s
+# RUN: not llvm-mc -triple riscv64 -mattr=+d -mattr=+xtheadfmemidx < %s 2>&1 | FileCheck %s
+
+th.flrd fa0, a1, a2, 5     # CHECK: :[[@LINE]]:22: error: immediate must be an integer in the range [0, 3]
+th.flrd a0, a1, a2, 3      # CHECK: :[[@LINE]]:9: error: invalid operand for instruction
+th.flrw 0(fa0), a1, a2, 0  # CHECK: :[[@LINE]]:9: error: invalid operand for instruction
+th.flrw fa0, 4(a1), a2, 3  # CHECK: :[[@LINE]]:14: error: invalid operand for instruction
+th.fsrd fa0, a1, -1(a2), 0 # CHECK: :[[@LINE]]:18: error: invalid operand for instruction
+th.fsrd fa0, a1, a2, -3    # CHECK: :[[@LINE]]:22: error: immediate must be an integer in the range [0, 3]

llvm/test/MC/RISCV/rv64xtheadfmemidx-valid.s

+# RUN: llvm-mc %s -triple=riscv64 -mattr=+d -mattr=+xtheadfmemidx -show-encoding \
+# RUN:     | FileCheck -check-prefixes=CHECK-ASM,CHECK-ASM-AND-OBJ %s
+# RUN: llvm-mc -filetype=obj -triple=riscv64 -mattr=+d -mattr=+xtheadfmemidx < %s \
+# RUN:     | llvm-objdump --mattr=+d --mattr=+xtheadfmemidx -d -r - \
+# RUN:     | FileCheck --check-prefix=CHECK-ASM-AND-OBJ %s
+
+# CHECK-ASM-AND-OBJ: th.flrd fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x60]
+th.flrd fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.flrd fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x66]
+th.flrd fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.flrw fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x40]
+th.flrw fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.flrw fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x46]
+th.flrw fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.flurd fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x70]
+th.flurd fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.flurd fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x76]
+th.flurd fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.flurw fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x50]
+th.flurw fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.flurw fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xe5,0xc5,0x56]
+th.flurw fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.fsrd fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x60]
+th.fsrd fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.fsrd fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x66]
+th.fsrd fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.fsrw fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x40]
+th.fsrw fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.fsrw fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x46]
+th.fsrw fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.fsurd fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x70]
+th.fsurd fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.fsurd fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x76]
+th.fsurd fa0, a1, a2, 3
+
+# CHECK-ASM-AND-OBJ: th.fsurw fa0, a1, a2, 0
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x50]
+th.fsurw fa0, a1, a2, 0
+
+# CHECK-ASM-AND-OBJ: th.fsurw fa0, a1, a2, 3
+# CHECK-ASM: encoding: [0x0b,0xf5,0xc5,0x56]
+th.fsurw fa0, a1, a2, 3