[AMDGPU][MC][GFX11] Add partial NSA format for image sample instructions

[mbrkusanin]

Summary

Image sample instructions that need more than 5 VGPRs for VAddr can use
partial NSA for NSA encoding format. VGPRs that can not fit into the
encoding are sequential after the last one.
This patch adds assembly and disassembly parts.

[mbrkusanin]

Same could be done for GFX10, however differences between GFX10.1 and GFX10.3 cause inconveniences.
MaxNSA size for 10.1 is 5 and for 10.3 is 13 so _V6, _V7,... opcodes for GFX10 already exist where every vaddr is a VGPR_32.
We would need new versions exclusive for non-10.3.

Not sure if that is the best approach. We would end up with opcodes like:
IMAGE_SAMPLE_D_V1_V6_nsa_gfx10
IMAGE_SAMPLE_D_V1_V6_partial_nsa_gfx10
or
IMAGE_SAMPLE_D_V1_V6_nsa_gfx1030
IMAGE_SAMPLE_D_V1_V6_nsa_gfx1010
or
IMAGE_SAMPLE_D_V1_V6_nsa_navi2_gfx10
IMAGE_SAMPLE_D_V1_V6_nsa_navi1_gfx10

[foad]

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

910–916

959

"more than"

971

986–987

[mbrkusanin]

[arsenm]

llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

3695

Should try to avoid raw generation checks and have some feature for this

[mbrkusanin]

• new feature FeaturePartialNSAEncoding
• NSAMaxSize is no longer a tablegen feature

[foad]

llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

10

I think the idea is that the assembler should not include any of the "codegen" parts of the backend. If getNSAMaxSize needs to be shared by codegen and the assembler, it should be moved into AMDGPUBaseInfo - that is why AMDGPUBaseInfo exists.

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

910–915

This should check !hasPartialNSAEncoding?

llvm/lib/Target/AMDGPU/GCNSubtarget.h

940

No else after return.

[mbrkusanin]

llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

10

Moved to AMDGPUBaseInfo.

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

910–915

Sure, it can.
Also changed isGFX10Plus() to STI.hasFeature(AMDGPU::FeatureNSAEncoding) below.

[foad]

LGTM but I'd really like someone familiar with MIMGInstructions.td to take a look too.

llvm/lib/Target/AMDGPU/GCNSubtarget.h

137

Remove NSAMaxSize field?

llvm/lib/Target/AMDGPU/MIMGInstructions.td

1068

"... more VGPRs than ..."

[foad]

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

1950

I'm not sure if it's allowed to call into GCNSubtarget from this code - does anyone else know?

[arsenm]

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

1950

No, GCNSubtarget is from codegen.

[foad]

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

1950

I guess it happens to work if you something defined in GCNSubtarget.h. But really I don't think we should be including GCNSubtarget.h here in the first place.

[foad]

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

1950

D144564

[mbrkusanin]

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

1950

Should I duplicate the code from getNSAMaxSize() here or have GCNSubtarget.h call AMDGPUBaseInfo version?

[foad]

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

1950

Have GCNSubtarget.h call AMDGPUBaseInfo version. You will have to use IsaVersion for the generation checks. There are lots of examples in AMDGPUBaseInfo.cpp.

[mbrkusanin]

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

1949–1954

or

if (isGFX10(STI))
  return hasGFX10_3Insts(STI) ? 13 : 5;
if (isGFX11(STI))
  return 5;
return 0;

[foad]

LGTM, thanks!

Hopefully @critson has taken a look and is happy too :)

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

1949–1954

Either way is fine. But it should probably be Version.Minor >= 3 just in case someone invents GFX 10.4.

llvm/lib/Target/AMDGPU/AMDGPU.td

 >;
 
 def FeatureNSAEncoding : SubtargetFeature<"nsa-encoding",
   "HasNSAEncoding",
   "true",
   "Support NSA encoding for image instructions"
 >;
 
+def FeaturePartialNSAEncoding : SubtargetFeature<"partial-nsa-encoding",
+  "HasPartialNSAEncoding",
+  "true",
+  "Support partial NSA encoding for image instructions"
+>;
+
 def FeatureImageInsts : SubtargetFeature<"image-insts",
   "HasImageInsts",
   "true",
   "Support image instructions"
 >;
 
 def FeatureExtendedImageInsts : SubtargetFeature<"extended-image-insts",
   "HasExtendedImageInsts",

 >;
 
 def FeatureGFX11FullVGPRs : SubtargetFeature<"gfx11-full-vgprs",
   "HasGFX11FullVGPRs",
   "true",
   "GFX11 with 50% more physical VGPRs and 50% larger allocation granule than GFX10"
 >;
 
-class SubtargetFeatureNSAMaxSize <int Value> : SubtargetFeature <
-  "nsa-max-size-"#Value,
-  "NSAMaxSize",
-  !cast<string>(Value),
-  "The maximum non-sequential address size in VGPRs."
->;
-
-def FeatureNSAMaxSize5 : SubtargetFeatureNSAMaxSize<5>;
-def FeatureNSAMaxSize13 : SubtargetFeatureNSAMaxSize<13>;
 
 def FeatureVOPD : SubtargetFeature<"vopd",
   "HasVOPDInsts",
   "true",
   "Has VOPD dual issue wave32 instructions"
 >;
 
 def FeatureVALUTransUseHazard : SubtargetFeature<"valu-trans-use-hazard",

 }
 
 def FeatureISAVersion10_1_0 : FeatureSet<
   !listconcat(FeatureGroup.GFX10_1_Bugs,
     [FeatureGFX10,
      FeatureLDSBankCount32,
      FeatureDLInsts,
      FeatureNSAEncoding,
-     FeatureNSAMaxSize5,
      FeatureWavefrontSize32,
      FeatureScalarStores,
      FeatureScalarAtomics,
      FeatureScalarFlatScratchInsts,
      FeatureGetWaveIdInst,
      FeatureMadMacF32Insts,
      FeatureDsSrc2Insts,
      FeatureLdsMisalignedBug,
      FeatureSupportsXNACK,
      FeatureBackOffBarrier])>;
 
 def FeatureISAVersion10_1_1 : FeatureSet<
   !listconcat(FeatureGroup.GFX10_1_Bugs,
     [FeatureGFX10,
      FeatureLDSBankCount32,
      FeatureDLInsts,
      FeatureDot1Insts,
      FeatureDot2Insts,
      FeatureDot5Insts,
      FeatureDot6Insts,
      FeatureDot7Insts,
      FeatureDot10Insts,
      FeatureNSAEncoding,
-     FeatureNSAMaxSize5,
      FeatureWavefrontSize32,
      FeatureScalarStores,
      FeatureScalarAtomics,
      FeatureScalarFlatScratchInsts,
      FeatureGetWaveIdInst,
      FeatureMadMacF32Insts,
      FeatureDsSrc2Insts,
      FeatureLdsMisalignedBug,
      FeatureSupportsXNACK,
      FeatureBackOffBarrier])>;
 
 def FeatureISAVersion10_1_2 : FeatureSet<
   !listconcat(FeatureGroup.GFX10_1_Bugs,
     [FeatureGFX10,
      FeatureLDSBankCount32,
      FeatureDLInsts,
      FeatureDot1Insts,
      FeatureDot2Insts,
      FeatureDot5Insts,
      FeatureDot6Insts,
      FeatureDot7Insts,
      FeatureDot10Insts,
      FeatureNSAEncoding,
-     FeatureNSAMaxSize5,
      FeatureWavefrontSize32,
      FeatureScalarStores,
      FeatureScalarAtomics,
      FeatureScalarFlatScratchInsts,
      FeatureGetWaveIdInst,
      FeatureMadMacF32Insts,
      FeatureDsSrc2Insts,
      FeatureLdsMisalignedBug,
      FeatureSupportsXNACK,
      FeatureBackOffBarrier])>;
 
 def FeatureISAVersion10_1_3 : FeatureSet<
   !listconcat(FeatureGroup.GFX10_1_Bugs,
     [FeatureGFX10,
      FeatureGFX10_AEncoding,
      FeatureLDSBankCount32,
      FeatureDLInsts,
      FeatureNSAEncoding,
-     FeatureNSAMaxSize5,
      FeatureWavefrontSize32,
      FeatureScalarStores,
      FeatureScalarAtomics,
      FeatureScalarFlatScratchInsts,
      FeatureGetWaveIdInst,
      FeatureMadMacF32Insts,
      FeatureDsSrc2Insts,
      FeatureLdsMisalignedBug,
def FeatureISAVersion10_3_0 : FeatureSet<
    FeatureDLInsts,
    FeatureDot1Insts,
    FeatureDot2Insts,
    FeatureDot5Insts,
    FeatureDot6Insts,
    FeatureDot7Insts,
    FeatureDot10Insts,
    FeatureNSAEncoding,
-   FeatureNSAMaxSize13,
    FeatureWavefrontSize32,
    FeatureShaderCyclesRegister,
    FeatureBackOffBarrier]>;
 
 def FeatureISAVersion11_Common : FeatureSet<
   [FeatureGFX11,
    FeatureLDSBankCount32,
    FeatureDLInsts,
    FeatureDot5Insts,
    FeatureDot7Insts,
    FeatureDot8Insts,
    FeatureDot9Insts,
    FeatureDot10Insts,
    FeatureNSAEncoding,
-   FeatureNSAMaxSize5,
+   FeaturePartialNSAEncoding,
    FeatureWavefrontSize32,
    FeatureShaderCyclesRegister,
    FeatureArchitectedFlatScratch,
    FeatureAtomicFaddRtnInsts,
    FeatureAtomicFaddNoRtnInsts,
    FeatureFlatAtomicFaddF32Inst,
    FeatureImageInsts,
    FeaturePackedTID,

llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

 //===- AMDGPUAsmParser.cpp - Parse SI asm to MCInst instructions ----------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "AMDKernelCodeT.h"
 #include "MCTargetDesc/AMDGPUMCTargetDesc.h"
 #include "MCTargetDesc/AMDGPUTargetStreamer.h"
 #include "SIDefines.h"
 #include "SIInstrInfo.h"
 #include "SIRegisterInfo.h"
 #include "TargetInfo/AMDGPUTargetInfo.h"
 #include "Utils/AMDGPUAsmUtils.h"
 #include "Utils/AMDGPUBaseInfo.h"
 #include "Utils/AMDKernelCodeTUtils.h"
public:
   }
 
   bool hasSGPR104_SGPR105() const { return isGFX10Plus(); }
 
   bool hasIntClamp() const {
     return getFeatureBits()[AMDGPU::FeatureIntClamp];
   }
 
+  bool hasPartialNSAEncoding() const {
+    return getFeatureBits()[AMDGPU::FeaturePartialNSAEncoding];
+  }
+
+  unsigned getNSAMaxSize() const {
+    return AMDGPU::getNSAMaxSize(getSTI());
+  }
+
   AMDGPUTargetStreamer &getTargetStreamer() {
     MCTargetStreamer &TS = *getParser().getStreamer().getTargetStreamer();
     return static_cast<AMDGPUTargetStreamer &>(TS);
   }
 
   const MCRegisterInfo *getMRI() const {
     // We need this const_cast because for some reason getContext() is not const
     // in MCAsmParser.
bool AMDGPUAsmParser::validateMIMGAddrSize(const MCInst &Inst) {
   unsigned ActualAddrSize =
       IsNSA ? SrsrcIdx - VAddr0Idx
             : AMDGPU::getRegOperandSize(getMRI(), Desc, VAddr0Idx) / 4;
   bool IsA16 = (A16Idx != -1 && Inst.getOperand(A16Idx).getImm());
 
   unsigned ExpectedAddrSize =
       AMDGPU::getAddrSizeMIMGOp(BaseOpcode, DimInfo, IsA16, hasG16());
 
-  if (!IsNSA) {
+  if (IsNSA) {
+    if (hasPartialNSAEncoding() && ExpectedAddrSize > getNSAMaxSize()) {
+      int VAddrLastIdx = SrsrcIdx - 1;
+      unsigned VAddrLastSize =
+          AMDGPU::getRegOperandSize(getMRI(), Desc, VAddrLastIdx) / 4;
+
+      return VAddrLastIdx - VAddr0Idx + VAddrLastSize == ExpectedAddrSize;
+    }
+  } else {
     if (ExpectedAddrSize > 12)
       ExpectedAddrSize = 16;
 
     // Allow oversized 8 VGPR vaddr when only 5/6/7 VGPRs are required.
     // This provides backward compatibility for assembly created
     // before 160b/192b/224b types were directly supported.
     if (ActualAddrSize == 8 && (ExpectedAddrSize >= 5 && ExpectedAddrSize <= 7))
       return true;

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

DecodeStatus AMDGPUDisassembler::convertMIMGInst(MCInst &MI) const {
 
   int VDstIdx = AMDGPU::getNamedOperandIdx(MI.getOpcode(),
                                            AMDGPU::OpName::vdst);
 
   int VDataIdx = AMDGPU::getNamedOperandIdx(MI.getOpcode(),
                                             AMDGPU::OpName::vdata);
   int VAddr0Idx =
       AMDGPU::getNamedOperandIdx(MI.getOpcode(), AMDGPU::OpName::vaddr0);
+  int RsrcIdx =
+      AMDGPU::getNamedOperandIdx(MI.getOpcode(), AMDGPU::OpName::srsrc);
   int DMaskIdx = AMDGPU::getNamedOperandIdx(MI.getOpcode(),
                                             AMDGPU::OpName::dmask);
 
   int TFEIdx   = AMDGPU::getNamedOperandIdx(MI.getOpcode(),
                                             AMDGPU::OpName::tfe);
   int D16Idx   = AMDGPU::getNamedOperandIdx(MI.getOpcode(),
                                             AMDGPU::OpName::d16);
 
   const AMDGPU::MIMGInfo *Info = AMDGPU::getMIMGInfo(MI.getOpcode());
   const AMDGPU::MIMGBaseOpcodeInfo *BaseOpcode =
       AMDGPU::getMIMGBaseOpcodeInfo(Info->BaseOpcode);
 
   assert(VDataIdx != -1);
   if (BaseOpcode->BVH) {
     // Add A16 operand for intersect_ray instructions
     if (AMDGPU::hasNamedOperand(MI.getOpcode(), AMDGPU::OpName::a16))
       addOperand(MI, MCOperand::createImm(1));
     return MCDisassembler::Success;
   }
 
   bool IsAtomic = (VDstIdx != -1);
   bool IsGather4 = MCII->get(MI.getOpcode()).TSFlags & SIInstrFlags::Gather4;
   bool IsNSA = false;
+  bool IsPartialNSA = false;
   unsigned AddrSize = Info->VAddrDwords;
 
   if (isGFX10Plus()) {
     unsigned DimIdx =
         AMDGPU::getNamedOperandIdx(MI.getOpcode(), AMDGPU::OpName::dim);
     int A16Idx =
         AMDGPU::getNamedOperandIdx(MI.getOpcode(), AMDGPU::OpName::a16);
     const AMDGPU::MIMGDimInfo *Dim =
         AMDGPU::getMIMGDimInfoByEncoding(MI.getOperand(DimIdx).getImm());
     const bool IsA16 = (A16Idx != -1 && MI.getOperand(A16Idx).getImm());
 
     AddrSize =
         AMDGPU::getAddrSizeMIMGOp(BaseOpcode, Dim, IsA16, AMDGPU::hasG16(STI));
 
     IsNSA = Info->MIMGEncoding == AMDGPU::MIMGEncGfx10NSA ||
             Info->MIMGEncoding == AMDGPU::MIMGEncGfx11NSA;
     if (!IsNSA) {
       if (AddrSize > 12)
         AddrSize = 16;
     } else {
       if (AddrSize > Info->VAddrDwords) {
-        // The NSA encoding does not contain enough operands for the combination
-        // of base opcode / dimension. Should this be an error?
+        if (!STI.hasFeature(AMDGPU::FeaturePartialNSAEncoding)) {
+          // The NSA encoding does not contain enough operands for the
+          // combination of base opcode / dimension. Should this be an error?
           return MCDisassembler::Success;
         }
+        IsPartialNSA = true;
+      }
     }
   }
 
   unsigned DMask = MI.getOperand(DMaskIdx).getImm() & 0xf;
   unsigned DstSize = IsGather4 ? 4 : std::max(llvm::popcount(DMask), 1);
 
   bool D16 = D16Idx >= 0 && MI.getOperand(D16Idx).getImm();
   if (D16 && AMDGPU::hasPackedD16(STI)) {
NewVdata = MRI.getMatchingSuperReg(Vdata0, AMDGPU::sub0,
                                        &MRI.getRegClass(DataRCID));
     if (NewVdata == AMDGPU::NoRegister) {
       // It's possible to encode this such that the low register + enabled
       // components exceeds the register count.
       return MCDisassembler::Success;
     }
   }
 
-  // If not using NSA on GFX10+, widen address register to correct size.
-  unsigned NewVAddr0 = AMDGPU::NoRegister;
-  if (isGFX10Plus() && !IsNSA && AddrSize != Info->VAddrDwords) {
-    unsigned VAddr0 = MI.getOperand(VAddr0Idx).getReg();
-    unsigned VAddrSub0 = MRI.getSubReg(VAddr0, AMDGPU::sub0);
-    VAddr0 = (VAddrSub0 != 0) ? VAddrSub0 : VAddr0;
+  // If not using NSA on GFX10+, widen vaddr0 address register to correct size.
+  // If using partial NSA on GFX11+ widen last address register.
+  int VAddrSAIdx = IsPartialNSA ? (RsrcIdx - 1) : VAddr0Idx;
+  unsigned NewVAddrSA = AMDGPU::NoRegister;
+  if (STI.hasFeature(AMDGPU::FeatureNSAEncoding) && (!IsNSA || IsPartialNSA) &&
+      AddrSize != Info->VAddrDwords) {
+    unsigned VAddrSA = MI.getOperand(VAddrSAIdx).getReg();
+    unsigned VAddrSubSA = MRI.getSubReg(VAddrSA, AMDGPU::sub0);
+    VAddrSA = VAddrSubSA ? VAddrSubSA : VAddrSA;
 
-    auto AddrRCID = MCII->get(NewOpcode).operands()[VAddr0Idx].RegClass;
-    NewVAddr0 = MRI.getMatchingSuperReg(VAddr0, AMDGPU::sub0,
+    auto AddrRCID = MCII->get(NewOpcode).operands()[VAddrSAIdx].RegClass;
+    NewVAddrSA = MRI.getMatchingSuperReg(VAddrSA, AMDGPU::sub0,
                                         &MRI.getRegClass(AddrRCID));
-    if (NewVAddr0 == AMDGPU::NoRegister)
+    if (!NewVAddrSA)
       return MCDisassembler::Success;
   }
 
   MI.setOpcode(NewOpcode);
 
   if (NewVdata != AMDGPU::NoRegister) {
     MI.getOperand(VDataIdx) = MCOperand::createReg(NewVdata);
 
     if (IsAtomic) {
       // Atomic operations have an additional operand (a copy of data)
       MI.getOperand(VDstIdx) = MCOperand::createReg(NewVdata);
     }
   }
 
-  if (NewVAddr0 != AMDGPU::NoRegister) {
-    MI.getOperand(VAddr0Idx) = MCOperand::createReg(NewVAddr0);
+  if (NewVAddrSA) {
+    MI.getOperand(VAddrSAIdx) = MCOperand::createReg(NewVAddrSA);
   } else if (IsNSA) {
     assert(AddrSize <= Info->VAddrDwords);
     MI.erase(MI.begin() + VAddr0Idx + AddrSize,
              MI.begin() + VAddr0Idx + Info->VAddrDwords);
   }
 
   return MCDisassembler::Success;
 }

llvm/lib/Target/AMDGPU/GCNSubtarget.h

 #define LLVM_LIB_TARGET_AMDGPU_GCNSUBTARGET_H
 
 #include "AMDGPUCallLowering.h"
 #include "AMDGPURegisterBankInfo.h"
 #include "AMDGPUSubtarget.h"
 #include "SIFrameLowering.h"
 #include "SIISelLowering.h"
 #include "SIInstrInfo.h"
+#include "Utils/AMDGPUBaseInfo.h"
 #include "llvm/CodeGen/SelectionDAGTargetInfo.h"
 
 #define GET_SUBTARGETINFO_HEADER
 #include "AMDGPUGenSubtargetInfo.inc"
 
 namespace llvm {
 
 class GCNTargetMachine;
protected:
   bool Has64BitDPP = false;
   bool HasPackedFP32Ops = false;
   bool HasImageInsts = false;
   bool HasExtendedImageInsts = false;
   bool HasR128A16 = false;
   bool HasA16 = false;
   bool HasG16 = false;
   bool HasNSAEncoding = false;
-  unsigned NSAMaxSize = 0;
+  bool HasPartialNSAEncoding = false;
   bool GFX10_AEncoding = false;
   bool GFX10_BEncoding = false;
   bool HasDLInsts = false;
   bool HasFmacF64Inst = false;
   bool HasDot1Insts = false;
   bool HasDot2Insts = false;
   bool HasDot3Insts = false;
   bool HasDot4Insts = false;
public:
   bool hasImageStoreD16Bug() const { return HasImageStoreD16Bug; }
 
   bool hasImageGather4D16Bug() const { return HasImageGather4D16Bug; }
 
   bool hasMADIntraFwdBug() const { return HasMADIntraFwdBug; }
 
   bool hasNSAEncoding() const { return HasNSAEncoding; }
 
-  unsigned getNSAMaxSize() const { return NSAMaxSize; }
+  bool hasPartialNSAEncoding() const { return HasPartialNSAEncoding; }
+
+  unsigned getNSAMaxSize() const { return AMDGPU::getNSAMaxSize(*this); }
 
   bool hasGFX10_AEncoding() const {
     return GFX10_AEncoding;
   }
 
   bool hasGFX10_BEncoding() const {
     return GFX10_BEncoding;
   }
 
   bool hasGFX10_3Insts() const {
     return GFX10_3Insts;

llvm/lib/Target/AMDGPU/MIMGInstructions.td

def MIMGInfoTable : GenericTable {
   let PrimaryKeyName = "getMIMGOpcodeHelper";
 }
 
 def getMIMGInfo : SearchIndex {
   let Table = MIMGInfoTable;
   let Key = ["Opcode"];
 }
 
+class NSAHelper {
+  dag AddrIns;
+  string AddrAsm;
+  int NSA;
+}
+
 // This class used to use !foldl to memoize the AddrAsmNames list.
 // It turned out that that was much slower than using !filter.
 class MIMGNSAHelper<int num_addrs,
-                    list<RegisterClass> addr_types=!listsplat(VGPR_32, num_addrs)> {
+                    list<RegisterClass> addr_types=!listsplat(VGPR_32, num_addrs)>
+  : NSAHelper<> {
   list<string> AddrAsmNames =
     !foreach(i, !filter(i, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
                         !lt(i, num_addrs)), "vaddr" # i);
-  dag AddrIns = !dag(ins, addr_types, AddrAsmNames);
-  string AddrAsm = "[$" # !interleave(AddrAsmNames, ", $") # "]";
+  let AddrIns = !dag(ins, addr_types, AddrAsmNames);
+  let AddrAsm = "[$" # !interleave(AddrAsmNames, ", $") # "]";
 
-  int NSA = !if(!le(num_addrs, 1), ?,
+  let NSA = !if(!le(num_addrs, 1), ?,
             !if(!le(num_addrs, 5), 1,
             !if(!le(num_addrs, 9), 2,
             !if(!le(num_addrs, 13), 3, ?))));
 }
 
+class PartialNSAHelper<int num_addrs, int max_addr, RegisterClass LastAddrRC>
+  : NSAHelper<> {
+
+  list<RegisterClass> addr_types =
+    !if(!ge(num_addrs, max_addr),
+      !listconcat(!listsplat(VGPR_32, !sub(max_addr, 1)), [LastAddrRC]),
+      !listsplat(VGPR_32, num_addrs));
+
+  int VAddrCount = !if(!gt(num_addrs, max_addr), max_addr, num_addrs);
+  list<string> AddrAsmNames =
+    !foreach(i, !filter(i, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
+             !lt(i, VAddrCount)), "vaddr" # i);
+
+  let AddrIns = !dag(ins, addr_types, AddrAsmNames);
+  let AddrAsm = "[$" # !interleave(AddrAsmNames, ", $") # "]";
+  let NSA = 1;
+}
+
 // Base class of all pre-gfx10 MIMG instructions.
 class MIMG_gfx6789<bits<8> op, dag outs, string dns = "">
   : MIMG<outs, dns>, MIMGe_gfx6789<op> {
   let SubtargetPredicate = isGFX6GFX7GFX8GFX9NotGFX90A;
   let AssemblerPredicate = isGFX6GFX7GFX8GFX9NotGFX90A;
 
   let MIMGEncoding = MIMGEncGfx6;
   let VAddrOperands = 1;
class MIMG_gfx11<int op, dag outs, string dns = "">
 
   let d16 = !if(BaseOpcode.HasD16, ?, 0);
   let nsa = 0;
 }
 
 // Base class for all NSA MIMG instructions.
 // Note that 1-dword addresses always use non-NSA variants.
 class MIMG_nsa_gfx11<int op, dag outs, int num_addrs, string dns="",
-                     list<RegisterClass> addr_types=[]>
+                     list<RegisterClass> addr_types=[],
+                     RegisterClass LastAddrRC = VGPR_32>
   : MIMG<outs, dns>, MIMGe_gfx11<op> {
   let SubtargetPredicate = isGFX11Plus;
   let AssemblerPredicate = isGFX11Plus;
 
   let MIMGEncoding = MIMGEncGfx11NSA;
   let VAddrOperands = num_addrs;
 
-  MIMGNSAHelper nsah = !if(!empty(addr_types),
-                           MIMGNSAHelper<num_addrs>,
-                           MIMGNSAHelper<num_addrs, addr_types>);
+  NSAHelper nsah = !if(!empty(addr_types),
+                        PartialNSAHelper<num_addrs, 5, LastAddrRC>,
   dag AddrIns = nsah.AddrIns;
   string AddrAsm = nsah.AddrAsm;
 
   let d16 = !if(BaseOpcode.HasD16, ?, 0);
   let nsa = nsah.NSA;
 }
 
 class MIMG_NoSampler_Helper <mimgopc op, string asm,
let InOperandList = !con((ins AddrRC:$vaddr0, SReg_256:$srsrc, SReg_128:$ssamp,
                            !if(BaseOpcode.HasD16, (ins D16:$d16), (ins)));
   let AsmString = opcode#" $vdata, $vaddr0, $srsrc, $ssamp$dmask$dim$unorm"
                     #"$cpol$r128$a16$tfe$lwe"
                     #!if(BaseOpcode.HasD16, "$d16", "");
 }
 
 class MIMG_Sampler_nsa_gfx11<mimgopc op, string opcode,
                              RegisterClass DataRC, int num_addrs,
-                             string dns="">
-  : MIMG_nsa_gfx11<op.GFX11, (outs DataRC:$vdata), num_addrs, dns> {
+                             RegisterClass LastVAddrSize, string dns="">
+  : MIMG_nsa_gfx11<op.GFX11, (outs DataRC:$vdata), num_addrs, dns, [],
+                   LastVAddrSize> {
   let InOperandList = !con(AddrIns,
                            (ins SReg_256:$srsrc, SReg_128:$ssamp, DMask:$dmask,
                                 Dim:$dim, UNorm:$unorm, CPol:$cpol,
                                 R128A16:$r128, A16:$a16, TFE:$tfe, LWE:$lwe),
                            !if(BaseOpcode.HasD16, (ins D16:$d16), (ins)));
   let AsmString = opcode#" $vdata, "#AddrAsm#", $srsrc, $ssamp$dmask$dim$unorm"
                     #"$cpol$r128$a16$tfe$lwe"
                     #!if(BaseOpcode.HasD16, "$d16", "");
 }
 
-class MIMGAddrSize<int dw, bit enable_disasm> {
+class MIMGAddrSize<int dw, bit enable_disasm, int AddrDW = dw> {
   int NumWords = dw;
 
-  RegisterClass RegClass = !if(!le(NumWords, 0), ?,
-                           !if(!eq(NumWords, 1), VGPR_32,
-                           !if(!eq(NumWords, 2), VReg_64,
-                           !if(!eq(NumWords, 3), VReg_96,
-                           !if(!eq(NumWords, 4), VReg_128,
-                           !if(!eq(NumWords, 5), VReg_160,
-                           !if(!eq(NumWords, 6), VReg_192,
-                           !if(!eq(NumWords, 7), VReg_224,
-                           !if(!le(NumWords, 8), VReg_256,
-                           !if(!le(NumWords, 9), VReg_288,
-                           !if(!le(NumWords, 10), VReg_320,
-                           !if(!le(NumWords, 11), VReg_352,
-                           !if(!le(NumWords, 12), VReg_384,
-                           !if(!le(NumWords, 16), VReg_512, ?))))))))))))));
+  RegisterClass RegClass = !if(!le(AddrDW, 0), ?,
+                           !if(!eq(AddrDW, 1), VGPR_32,
+                           !if(!eq(AddrDW, 2), VReg_64,
+                           !if(!eq(AddrDW, 3), VReg_96,
+                           !if(!eq(AddrDW, 4), VReg_128,
+                           !if(!eq(AddrDW, 5), VReg_160,
+                           !if(!eq(AddrDW, 6), VReg_192,
+                           !if(!eq(AddrDW, 7), VReg_224,
+                           !if(!eq(AddrDW, 8), VReg_256,
+                           !if(!eq(AddrDW, 9), VReg_288,
+                           !if(!eq(AddrDW, 10), VReg_320,
+                           !if(!eq(AddrDW, 11), VReg_352,
+                           !if(!eq(AddrDW, 12), VReg_384,
+                           !if(!le(AddrDW, 16), VReg_512, ?))))))))))))));
 
   // Whether the instruction variant with this vaddr size should be enabled for
   // the auto-generated disassembler.
   bit Disassemble = enable_disasm;
 }
 
+// Returns the MIMGAddrSize with the size of last VAddr for partial NSA
+class LastVAddrSize <int dw, int max_idx, bit enable_disasm>
+  : MIMGAddrSize<dw, enable_disasm,
+                 !if(!gt(dw, max_idx), !sub(dw, max_idx), 0)>;
+
 // Return whether x is in lst.
 class isIntInList<int x, list<int> lst> {
   bit ret = !foldl(0, lst, lhs, y, !or(lhs, !eq(x, y)));
 }
 
 // Return whether a value inside the range [min, max] (endpoints inclusive)
 // is in the given list.
 class isRangeInList<int min, int max, list<int> lst> {
   bit ret = !foldl(0, lst, lhs, y, !or(lhs, !and(!le(min, y), !le(y, max))));
 }
 
 class MIMGAddrSizes_dw_range<list<int> range> {
   int Min = !head(range);
   int Max = !if(!empty(!tail(range)), Min, !head(!tail(range)));
 }
 
-class MIMG_Sampler_AddrSizes<AMDGPUSampleVariant sample, bit isG16> {
+class MIMG_Sampler_AddrSizes<AMDGPUSampleVariant sample, bit isG16,
+                             int nsa_max_addr = 5> {
   // List of all possible numbers of address words, taking all combinations of
   // A16 and image dimension into account (note: no MSAA, since this is for
   // sample/gather ops).
   list<int> AllNumAddrWords =
     !foreach(dw, !if(sample.Gradients,
                      !if(isG16,
                          !if(!eq(sample.LodOrClamp, ""),
                              [2, 3, 4, 5, 6, 7],
class MIMG_Sampler_AddrSizes<AMDGPUSampleVariant sample, bit isG16,
   // the exact variant based on the sample variant and the image dimension.
   list<MIMGAddrSize> NSAInstrs =
     !foldl([]<MIMGAddrSize>, [[12, 11, 10], [9, 8, 7, 6], [5, 4, 3, 2]], prev, nsa_group,
            !listconcat(prev,
                        !foldl([]<MIMGAddrSize>, nsa_group, lhs, dw,
                               !if(isIntInList<dw, AllNumAddrWords>.ret,
                                   !listconcat(lhs, [MIMGAddrSize<dw, !empty(lhs)>]),
                                   lhs))));
+
+  // In NSA format if there is a requirement for more VGPRs than the format
+  // supports, then the rest are sequential after the last one. Generate
+  // machine instructions for all possible number of words. The disassembler
+  // defaults to the largest number of arguments but no larger than max nsa
+  // size. List is generated with the register class needed for last vaddr since
+  // it is the only one that could have a register other than VGPR32.
+  int EnableDisasmNum = !foldl(!head(AllNumAddrWords), !tail(AllNumAddrWords),
+                               acc, var, !if(!le(var, nsa_max_addr), var, acc));
+  list<LastVAddrSize> PartialNSAInstrs =
+        !foldl([]<LastVAddrSize>, [12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2], lhs, dw,
+               !if(isIntInList<dw, AllNumAddrWords>.ret,
+                   !listconcat(lhs, [LastVAddrSize<dw, !sub(nsa_max_addr, 1),
+                                                   !eq(dw, EnableDisasmNum)>]),
+                   lhs));
 }
 
 multiclass MIMG_Sampler_Src_Helper <mimgopc op, string asm,
                                     AMDGPUSampleVariant sample, RegisterClass dst_rc,
                                     bit enableDisasm = 0,
                                     bit ExtendedImageInst = 1, bit isG16 = 0> {
   foreach addr = MIMG_Sampler_AddrSizes<sample, isG16>.MachineInstrs in {
     let VAddrDwords = addr.NumWords in {
multiclass MIMG_Sampler_Src_Helper <mimgopc op, string asm,
 
   foreach addr = MIMG_Sampler_AddrSizes<sample, isG16>.NSAInstrs in {
     let VAddrDwords = addr.NumWords in {
       if op.HAS_GFX10M then {
         def _V # addr.NumWords # _nsa_gfx10
           : MIMG_Sampler_nsa_gfx10<op, asm, dst_rc, addr.NumWords,
                                    !if(!and(enableDisasm, addr.Disassemble), "AMDGPU", "")>;
       }
-      if !and(op.HAS_GFX11, !le(addr.NumWords, 5)) then {
+    }
+  }
+
+  foreach addr = MIMG_Sampler_AddrSizes<sample, isG16, 5/*MaxNSASize*/>.PartialNSAInstrs in {
+    let VAddrDwords = addr.NumWords in {
+      if op.HAS_GFX11 then {
         def _V # addr.NumWords # _nsa_gfx11
-          : MIMG_Sampler_nsa_gfx11<op, asm, dst_rc, addr.NumWords,
+          : MIMG_Sampler_nsa_gfx11<op, asm, dst_rc, addr.NumWords, addr.RegClass,
                                    !if(!and(enableDisasm, addr.Disassemble), "AMDGPU", "")>;
       }
     }
   }
 }
 
 class MIMG_Sampler_BaseOpcode<AMDGPUSampleVariant sample>
   : MIMGBaseOpcode {

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h

 }
 
 bool hasXNACK(const MCSubtargetInfo &STI);
 bool hasSRAMECC(const MCSubtargetInfo &STI);
 bool hasMIMG_R128(const MCSubtargetInfo &STI);
 bool hasA16(const MCSubtargetInfo &STI);
 bool hasG16(const MCSubtargetInfo &STI);
 bool hasPackedD16(const MCSubtargetInfo &STI);
+unsigned getNSAMaxSize(const MCSubtargetInfo &STI);
 
 bool isSI(const MCSubtargetInfo &STI);
 bool isCI(const MCSubtargetInfo &STI);
 bool isVI(const MCSubtargetInfo &STI);
 bool isGFX9(const MCSubtargetInfo &STI);
 bool isGFX9_GFX10(const MCSubtargetInfo &STI);
 bool isGFX8_GFX9_GFX10(const MCSubtargetInfo &STI);
 bool isGFX8Plus(const MCSubtargetInfo &STI);

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

bool hasG16(const MCSubtargetInfo &STI) {
   return STI.hasFeature(AMDGPU::FeatureG16);
 }
 
 bool hasPackedD16(const MCSubtargetInfo &STI) {
   return !STI.hasFeature(AMDGPU::FeatureUnpackedD16VMem) && !isCI(STI) &&
          !isSI(STI);
 }
 
+unsigned getNSAMaxSize(const MCSubtargetInfo &STI) {
+  auto Version = getIsaVersion(STI.getCPU());
+  if (Version.Major == 10)
+    return Version.Minor >= 3 ? 13 : 5;
+  if (Version.Major == 11)
+    return 5;
+  return 0;
+}
+
 bool isSI(const MCSubtargetInfo &STI) {
   return STI.hasFeature(AMDGPU::FeatureSouthernIslands);
 }
 
 bool isCI(const MCSubtargetInfo &STI) {
   return STI.hasFeature(AMDGPU::FeatureSeaIslands);
 }
 

llvm/test/MC/AMDGPU/gfx11_asm_mimg_features.s

 // GFX11: image_sample_l_o v[64:66], [v32, v16, v0, v2, v1], s[4:11], s[100:103] dmask:0x7 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x07,0x9c,0xf0,0x20,0x40,0x01,0x64,0x10,0x00,0x02,0x01]
 
 image_sample_lz_o v[64:66], [v32, v16, v0, v2], s[4:11], s[100:103] dmask:0x7 dim:SQ_RSRC_IMG_3D
 // GFX11: image_sample_lz_o v[64:66], [v32, v16, v0, v2], s[4:11], s[100:103] dmask:0x7 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x07,0xa4,0xf0,0x20,0x40,0x01,0x64,0x10,0x00,0x02,0x00]
 
 image_sample_c_lz_o v[64:66], [v32, v0, v16], s[4:11], s[100:103] dmask:0x7 dim:SQ_RSRC_IMG_1D
 // GFX11: image_sample_c_lz_o v[64:66], [v32, v0, v16], s[4:11], s[100:103] dmask:0x7 dim:SQ_RSRC_IMG_1D ; encoding: [0x01,0x07,0xb8,0xf0,0x20,0x40,0x01,0x64,0x00,0x10,0x00,0x00]
 
+image_sample_d v64, [v32, v16, v8, v0, v[36:37]], s[4:11], s[4:7] dmask:0x8 dim:SQ_RSRC_IMG_2D
+// GFX11: image_sample_d v64, [v32, v16, v8, v0, v[36:37]], s[4:11], s[4:7] dmask:0x8 dim:SQ_RSRC_IMG_2D ; encoding: [0x05,0x08,0x70,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_d v[64:65], [v32, v16, v8, v0, v[36:38]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE
+// GFX11: image_sample_d v[64:65], [v32, v16, v8, v0, v[36:38]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0d,0x0c,0x70,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_d_o v[64:65], [v32, v16, v8, v0, v[36:39]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE
+// GFX11: image_sample_d_o v[64:65], [v32, v16, v8, v0, v[36:39]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0d,0x0c,0x98,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_d v[64:65], [v32, v16, v8, v0, v[36:40]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D
+// GFX11: image_sample_d v[64:65], [v32, v16, v8, v0, v[36:40]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0x70,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_d_o v[64:65], [v32, v16, v8, v0, v[36:41]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D
+// GFX11: image_sample_d_o v[64:65], [v32, v16, v8, v0, v[36:41]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0x98,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_c_d_o v[64:65], [v32, v16, v8, v0, v[36:42]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D
+// GFX11: image_sample_c_d_o v[64:65], [v32, v16, v8, v0, v[36:42]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0xac,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_c_d_cl_o v[64:65], [v32, v16, v8, v0, v[36:43]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D
+// GFX11: image_sample_c_d_cl_o v[64:65], [v32, v16, v8, v0, v[36:43]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0x28,0xf1,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_d_o_g16 v[64:66], [v32, v16, v8, v0, v36], s[4:11], s[4:7] dmask:0xb dim:SQ_RSRC_IMG_1D_ARRAY
+// GFX11: image_sample_d_o_g16 v[64:66], [v32, v16, v8, v0, v36], s[4:11], s[4:7] dmask:0xb dim:SQ_RSRC_IMG_1D_ARRAY ; encoding: [0x11,0x0b,0xec,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:37]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE
+// GFX11: image_sample_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:37]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0d,0x0c,0xec,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_c_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:38]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE
+// GFX11: image_sample_c_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:38]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0d,0x0c,0xf0,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:39]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D
+// GFX11: image_sample_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:39]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0xec,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
+image_sample_c_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:40]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D
+// GFX11: image_sample_c_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:40]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0xf0,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+
 image_gather4 v[64:67], v32, s[4:11], s[100:103] dmask:0x1 dim:SQ_RSRC_IMG_1D
 // GFX11: image_gather4 v[64:67], v32, s[4:11], s[100:103] dmask:0x1 dim:SQ_RSRC_IMG_1D ; encoding: [0x00,0x01,0xbc,0xf0,0x20,0x40,0x01,0x64]
 
 image_gather4_cl v[64:67], v[32:35], s[4:11], s[100:103] dmask:0x2 dim:SQ_RSRC_IMG_CUBE
 // GFX11: image_gather4_cl v[64:67], v[32:35], s[4:11], s[100:103] dmask:0x2 dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0c,0x02,0x80,0xf1,0x20,0x40,0x01,0x64]
 
 image_gather4_l v[64:67], [v32, v0, v4], s[4:11], s[100:103] dmask:0x4 dim:SQ_RSRC_IMG_1D_ARRAY
 // GFX11: image_gather4_l v[64:67], [v32, v0, v4], s[4:11], s[100:103] dmask:0x4 dim:SQ_RSRC_IMG_1D_ARRAY ; encoding: [0x11,0x04,0xc0,0xf0,0x20,0x40,0x01,0x64,0x00,0x04,0x00,0x00]

llvm/test/MC/Disassembler/AMDGPU/gfx11_dasm_mimg_features.txt

 0x09,0x07,0x9c,0xf0,0x20,0x40,0x01,0x64,0x10,0x00,0x02,0x01
 
 # GFX11: image_sample_lz_o v[64:66], [v32, v16, v0, v2], s[4:11], s[100:103] dmask:0x7 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x07,0xa4,0xf0,0x20,0x40,0x01,0x64,0x10,0x00,0x02,0x00]
 0x09,0x07,0xa4,0xf0,0x20,0x40,0x01,0x64,0x10,0x00,0x02,0x00
 
 # GFX11: image_sample_c_lz_o v[64:66], [v32, v0, v16], s[4:11], s[100:103] dmask:0x7 dim:SQ_RSRC_IMG_1D ; encoding: [0x01,0x07,0xb8,0xf0,0x20,0x40,0x01,0x64,0x00,0x10,0x00,0x00]
 0x01,0x07,0xb8,0xf0,0x20,0x40,0x01,0x64,0x00,0x10,0x00,0x00
 
+# GFX11: image_sample_d v64, [v32, v16, v8, v0, v[36:37]], s[4:11], s[4:7] dmask:0x8 dim:SQ_RSRC_IMG_2D ; encoding: [0x05,0x08,0x70,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x05,0x08,0x70,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_d v[64:65], [v32, v16, v8, v0, v[36:38]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0d,0x0c,0x70,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x0d,0x0c,0x70,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_d_o v[64:65], [v32, v16, v8, v0, v[36:39]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0d,0x0c,0x98,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x0d,0x0c,0x98,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_d v[64:65], [v32, v16, v8, v0, v[36:40]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0x70,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x09,0x03,0x70,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_d_o v[64:65], [v32, v16, v8, v0, v[36:41]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0x98,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x09,0x03,0x98,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_c_d_o v[64:65], [v32, v16, v8, v0, v[36:42]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0xac,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x09,0x03,0xac,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_c_d_cl_o v[64:65], [v32, v16, v8, v0, v[36:43]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0x28,0xf1,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x09,0x03,0x28,0xf1,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_d_o_g16 v[64:66], [v32, v16, v8, v0, v36], s[4:11], s[4:7] dmask:0xb dim:SQ_RSRC_IMG_1D_ARRAY ; encoding: [0x11,0x0b,0xec,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x11,0x0b,0xec,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:37]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0d,0x0c,0xec,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x0d,0x0c,0xec,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_c_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:38]], s[4:11], s[4:7] dmask:0xc dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0d,0x0c,0xf0,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x0d,0x0c,0xf0,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:39]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0xec,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x09,0x03,0xec,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
+# GFX11: image_sample_c_d_o_g16 v[64:65], [v32, v16, v8, v0, v[36:40]], s[4:11], s[4:7] dmask:0x3 dim:SQ_RSRC_IMG_3D ; encoding: [0x09,0x03,0xf0,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24]
+0x09,0x03,0xf0,0xf0,0x20,0x40,0x01,0x04,0x10,0x08,0x00,0x24
+
 # GFX11: image_gather4 v[64:67], v32, s[4:11], s[100:103] dmask:0x1 dim:SQ_RSRC_IMG_1D ; encoding: [0x00,0x01,0xbc,0xf0,0x20,0x40,0x01,0x64]
 0x00,0x01,0xbc,0xf0,0x20,0x40,0x01,0x64
 
 # GFX11: image_gather4_cl v[64:67], v[32:35], s[4:11], s[100:103] dmask:0x2 dim:SQ_RSRC_IMG_CUBE ; encoding: [0x0c,0x02,0x80,0xf1,0x20,0x40,0x01,0x64]
 0x0c,0x02,0x80,0xf1,0x20,0x40,0x01,0x64
 
 # GFX11: image_gather4_l v[64:67], [v32, v0, v4], s[4:11], s[100:103] dmask:0x4 dim:SQ_RSRC_IMG_1D_ARRAY ; encoding: [0x11,0x04,0xc0,0xf0,0x20,0x40,0x01,0x64,0x00,0x04,0x00,0x00]
 0x11,0x04,0xc0,0xf0,0x20,0x40,0x01,0x64,0x00,0x04,0x00,0x00