[AMDGPU] Enable FLAT LDS DMA on gfx9/10 before gfx940

[rampitec]

Summary

We always had global and scratch loads to LDS in the gfx9,
but did not handle it. These were available via the 'lds'
encoding bit. In gfx940 this bit was reused as 'svs' which
resulted in new '_lds' opcodes effectively pushing this
bit into the opcode, but functionally it is the same. These
instructions are also available on gfx10.

[rampitec]

A potentially better alternative is to use gfx940 names with _LDS_ in the mnemonic instead of a modifier. This is logically a different opcode anyway. The only downside it is not compatible with the documentation and sp3. But then it was not implemented before and therefore not used, so there shall be no compatibility problem on practice. Well, it will also be different from MUBUF. Given the difference in both semantics and addressing mode I personally would prefer it to be different opcodes. At a pseudo level it is certainly easier to have separate ops for this.

Preferences?

[rampitec]

One more thing to note: it is already incompatible with sp3 because we prohibit unused vdst, while sp3 enforces it.

[arsenm]

In D125126#3498084, @rampitec wrote:

A potentially better alternative is to use gfx940 names with _LDS_ in the mnemonic instead of a modifier. This is logically a different opcode anyway. The only downside it is not compatible with the documentation and sp3. But then it was not implemented before and therefore not used, so there shall be no compatibility problem on practice. Well, it will also be different from MUBUF. Given the difference in both semantics and addressing mode I personally would prefer it to be different opcodes. At a pseudo level it is certainly easier to have separate ops for this.

Preferences?

It's probably better to have separate opcodes. In general I think the way we try to force all of these subtarget changes onto the same generic pseudos is more trouble than it's worth. It requires more and more code to verify and make use of the features, and it would be cleaner to move towards separate instruction definitions per subtarget

[rampitec]

In D125126#3498596, @arsenm wrote:

In D125126#3498084, @rampitec wrote:

A potentially better alternative is to use gfx940 names with _LDS_ in the mnemonic instead of a modifier. This is logically a different opcode anyway. The only downside it is not compatible with the documentation and sp3. But then it was not implemented before and therefore not used, so there shall be no compatibility problem on practice. Well, it will also be different from MUBUF. Given the difference in both semantics and addressing mode I personally would prefer it to be different opcodes. At a pseudo level it is certainly easier to have separate ops for this.

Preferences?

It's probably better to have separate opcodes. In general I think the way we try to force all of these subtarget changes onto the same generic pseudos is more trouble than it's worth. It requires more and more code to verify and make use of the features, and it would be cleaner to move towards separate instruction definitions per subtarget

I remember your idea about switchable instruction tables per subtarget, but this is not really that. This is more about the asm syntax compatibility: to make all gfx9/gfx10 the same for these instructions, or to follow the spec which was amended for gfx940. It was amended purely due to encoding considerations, but semantically it is still the same instructions and do exactly the same. So I believe using same pseudos it warranted here.

What I've heard on today's meeting we are leaning towards spec compatibility, so then the patch does exactly that.

[rampitec]

ping

[rampitec]

ping

llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

private:
   bool validateMFMA(const MCInst &Inst, const OperandVector &Operands);
   bool validateAGPRLdSt(const MCInst &Inst) const;
   bool validateVGPRAlign(const MCInst &Inst) const;
   bool validateBLGP(const MCInst &Inst, const OperandVector &Operands);
   bool validateGWS(const MCInst &Inst, const OperandVector &Operands);
   bool validateDivScale(const MCInst &Inst);
   bool validateCoherencyBits(const MCInst &Inst, const OperandVector &Operands,
                              const SMLoc &IDLoc);
+  bool validateFlatLdsDMA(const MCInst &Inst, const OperandVector &Operands,
+                          const SMLoc &IDLoc);
   Optional<StringRef> validateLdsDirect(const MCInst &Inst);
   unsigned getConstantBusLimit(unsigned Opcode) const;
   bool usesConstantBus(const MCInst &Inst, unsigned OpIdx);
   bool isInlineConstant(const MCInst &Inst, unsigned OpIdx) const;
   unsigned findImplicitSGPRReadInVOP(const MCInst &Inst) const;
 
   bool isSupportedMnemo(StringRef Mnemo,
                         const FeatureBitset &FBS);
if (CPol & CPol::GLC) {
                           : "instruction must not use glc");
       return false;
     }
   }
 
   return true;
 }
 
+bool AMDGPUAsmParser::validateFlatLdsDMA(const MCInst &Inst,
+                                         const OperandVector &Operands,
+                                         const SMLoc &IDLoc) {
+  if (isGFX940())
+    return true;
+
+  uint64_t TSFlags = MII.get(Inst.getOpcode()).TSFlags;
+  if ((TSFlags & (SIInstrFlags::VALU | SIInstrFlags::FLAT)) !=
+      (SIInstrFlags::VALU | SIInstrFlags::FLAT))
+    return true;
+  // This is FLAT LDS DMA.
+
+  SMLoc S = getImmLoc(AMDGPUOperand::ImmTyLDS, Operands);
+  StringRef CStr(S.getPointer());
+  if (!CStr.startswith("lds")) {
+    // This is incorrectly selected LDS DMA version of a FLAT load opcode.
+    // And LDS version should have 'lds' modifier, but it follows optional
+    // operands so its absense is ignored by the matcher.
+    Error(IDLoc, "invalid operands for instruction");
+    return false;
+  }
+
+  return true;
+}
+
 bool AMDGPUAsmParser::validateInstruction(const MCInst &Inst,
                                           const SMLoc &IDLoc,
                                           const OperandVector &Operands) {
   if (auto ErrMsg = validateLdsDirect(Inst)) {
     Error(getRegLoc(LDS_DIRECT, Operands), *ErrMsg);
     return false;
   }
   if (!validateSOPLiteral(Inst)) {
bool AMDGPUAsmParser::validateInstruction(const MCInst &Inst,
   if (!validateDivScale(Inst)) {
     Error(IDLoc, "ABS not allowed in VOP3B instructions");
     return false;
   }
   if (!validateCoherencyBits(Inst, Operands, IDLoc)) {
     return false;
   }
 
+  if (!validateFlatLdsDMA(Inst, Operands, IDLoc)) {
+    return false;
+  }
+
   return true;
 }
 
 static std::string AMDGPUMnemonicSpellCheck(StringRef S,
                                             const FeatureBitset &FBS,
                                             unsigned VariantID = 0);
 
 static bool AMDGPUCheckMnemonic(StringRef Mnemonic,

llvm/lib/Target/AMDGPU/FLATInstructions.td

class FLAT_Pseudo<string opName, dag outs, dag ins,
   bits<1> has_data = 1;
   bits<1> has_glc  = 1;
   bits<1> glcValue = 0;
   bits<1> has_dlc  = 1;
   bits<1> dlcValue = 0;
   bits<1> has_sccb  = 1;
   bits<1> sccbValue = 0;
   bits<1> has_sve  = 0; // Scratch VGPR Enable
+  bits<1> lds = 0;
   bits<1> sve = 0;
 
   let SubtargetPredicate = !if(is_flat_global, HasFlatGlobalInsts,
     !if(is_flat_scratch, HasFlatScratchInsts, HasFlatAddressSpace));
 
   // TODO: M0 if it could possibly access LDS (before gfx9? only)?
   let Uses = !if(is_flat_global, [EXEC], [EXEC, FLAT_SCR]);
 
class FLAT_Real <bits<7> op, FLAT_Pseudo ps> :
   bits<8> vaddr;
   bits<10> vdata;
   bits<7> saddr;
   bits<10> vdst;
 
   bits<5> cpol;
 
   // Only valid on gfx9
-  bits<1> lds = 0; // XXX - What does this actually do?
+  bits<1> lds = ps.lds; // LDS DMA for global and scratch
 
   // Segment, 00=flat, 01=scratch, 10=global, 11=reserved
   bits<2> seg = !if(ps.is_flat_global, 0b10,
                   !if(ps.is_flat_scratch, 0b01, 0));
 
   // Signed offset. Highest bit ignored for flat and treated as 12-bit
   // unsigned for flat accesses.
   bits<13> offset;
class FLAT_Global_Load_LDS_Pseudo <string opName, bit EnableSaddr = 0> : FLAT_Pseudo<
   opName,
   (outs ),
   !con(
       !if(EnableSaddr, (ins SReg_64:$saddr, VGPR_32:$vaddr), (ins VReg_64:$vaddr)),
       (ins flat_offset:$offset, CPol_0:$cpol)),
   " $vaddr"#!if(EnableSaddr, ", $saddr", ", off")#"$offset$cpol"> {
   let LGKM_CNT = 1;
   let is_flat_global = 1;
+  let lds = 1;
   let has_data = 0;
   let has_vdst = 0;
   let mayLoad = 1;
   let mayStore = 1;
   let has_saddr = 1;
   let enabled_saddr = EnableSaddr;
   let VALU = 1;
   let PseudoInstr = opName#!if(EnableSaddr, "_SADDR", "");
!if(EnableSaddr,
       (ins SReg_32_XEXEC_HI:$saddr, flat_offset:$offset, CPol:$cpol),
       !if(EnableVaddr,
         (ins VGPR_32:$vaddr, flat_offset:$offset, CPol:$cpol),
         (ins flat_offset:$offset, CPol:$cpol)))),
   " "#!if(EnableVaddr, "$vaddr, ", "off, ")#!if(EnableSaddr, "$saddr", "off")#"$offset$cpol"> {
 
   let LGKM_CNT = 1;
   let is_flat_scratch = 1;
+  let lds = 1;
   let has_data = 0;
   let has_vdst = 0;
   let mayLoad = 1;
   let mayStore = 1;
   let has_saddr = 1;
   let enabled_saddr = EnableSaddr;
   let has_vaddr = EnableVaddr;
   let has_sve = EnableSVE;

 
 defm GLOBAL_ATOMIC_DEC_X2 : FLAT_Global_Atomic_Pseudo <"global_atomic_dec_x2",
                               VReg_64, i64>;
 
 let SubtargetPredicate = HasGFX10_BEncoding in
 defm GLOBAL_ATOMIC_CSUB : FLAT_Global_Atomic_Pseudo_RTN <"global_atomic_csub",
                               VGPR_32, i32>;
 
-let SubtargetPredicate = isGFX940Plus in {
-
 defm GLOBAL_LOAD_LDS_UBYTE  : FLAT_Global_Load_LDS_Pseudo <"global_load_lds_ubyte">;
 defm GLOBAL_LOAD_LDS_SBYTE  : FLAT_Global_Load_LDS_Pseudo <"global_load_lds_sbyte">;
 defm GLOBAL_LOAD_LDS_USHORT : FLAT_Global_Load_LDS_Pseudo <"global_load_lds_ushort">;
 defm GLOBAL_LOAD_LDS_SSHORT : FLAT_Global_Load_LDS_Pseudo <"global_load_lds_sshort">;
 defm GLOBAL_LOAD_LDS_DWORD  : FLAT_Global_Load_LDS_Pseudo <"global_load_lds_dword">;
 
-} // End let SubtargetPredicate = isGFX940Plus
 } // End is_flat_global = 1
 
 
 
 let SubtargetPredicate = HasFlatScratchInsts in {
 defm SCRATCH_LOAD_UBYTE    : FLAT_Scratch_Load_Pseudo <"scratch_load_ubyte", VGPR_32>;
 defm SCRATCH_LOAD_SBYTE    : FLAT_Scratch_Load_Pseudo <"scratch_load_sbyte", VGPR_32>;
 defm SCRATCH_LOAD_USHORT   : FLAT_Scratch_Load_Pseudo <"scratch_load_ushort", VGPR_32>;

 defm SCRATCH_STORE_DWORD   : FLAT_Scratch_Store_Pseudo <"scratch_store_dword", VGPR_32>;
 defm SCRATCH_STORE_DWORDX2 : FLAT_Scratch_Store_Pseudo <"scratch_store_dwordx2", VReg_64>;
 defm SCRATCH_STORE_DWORDX3 : FLAT_Scratch_Store_Pseudo <"scratch_store_dwordx3", VReg_96>;
 defm SCRATCH_STORE_DWORDX4 : FLAT_Scratch_Store_Pseudo <"scratch_store_dwordx4", VReg_128>;
 
 defm SCRATCH_STORE_BYTE_D16_HI : FLAT_Scratch_Store_Pseudo <"scratch_store_byte_d16_hi", VGPR_32>;
 defm SCRATCH_STORE_SHORT_D16_HI : FLAT_Scratch_Store_Pseudo <"scratch_store_short_d16_hi", VGPR_32>;
 
-let SubtargetPredicate = isGFX940Plus in {
-
 defm SCRATCH_LOAD_LDS_UBYTE  : FLAT_Scratch_Load_LDS_Pseudo <"scratch_load_lds_ubyte">;
 defm SCRATCH_LOAD_LDS_SBYTE  : FLAT_Scratch_Load_LDS_Pseudo <"scratch_load_lds_sbyte">;
 defm SCRATCH_LOAD_LDS_USHORT : FLAT_Scratch_Load_LDS_Pseudo <"scratch_load_lds_ushort">;
 defm SCRATCH_LOAD_LDS_SSHORT : FLAT_Scratch_Load_LDS_Pseudo <"scratch_load_lds_sshort">;
 defm SCRATCH_LOAD_LDS_DWORD  : FLAT_Scratch_Load_LDS_Pseudo <"scratch_load_lds_dword">;
 
-} // End let SubtargetPredicate = isGFX940Plus
-
 } // End SubtargetPredicate = HasFlatScratchInsts
 
 let SubtargetPredicate = isGFX10Plus, is_flat_global = 1 in {
   defm GLOBAL_ATOMIC_FCMPSWAP :
     FLAT_Global_Atomic_Pseudo<"global_atomic_fcmpswap", VGPR_32, f32, v2f32, VReg_64>;
   defm GLOBAL_ATOMIC_FMIN :
     FLAT_Global_Atomic_Pseudo<"global_atomic_fmin", VGPR_32, f32>;
   defm GLOBAL_ATOMIC_FMAX :
multiclass FLAT_Real_AllAddr_SVE_vi<bits<7> op> {
   }
   let AssemblerPredicate = isGFX940Plus, SubtargetPredicate = isGFX940Plus in {
     def _VE_gfx940  : FLAT_Real_gfx940<op, !cast<FLAT_Pseudo>(NAME)>;
     def _SVS_gfx940 : FLAT_Real_gfx940<op, !cast<FLAT_Pseudo>(NAME#"_SVS")>;
     def _ST_gfx940  : FLAT_Real_gfx940<op, !cast<FLAT_Pseudo>(NAME#"_ST")>;
   }
 }
 
+multiclass FLAT_Real_AllAddr_LDS<bits<7> op, bits<7> pre_gfx940_op,
+  string pre_gfx940_name = !subst("_lds", "", !cast<FLAT_Pseudo>(NAME).PseudoInstr),
+  bit has_sccb = !cast<FLAT_Pseudo>(NAME).has_sccb> {
+
+  let OtherPredicates = [isGFX8GFX9NotGFX940] in {
+    def _vi : FLAT_Real_vi<pre_gfx940_op, !cast<FLAT_Pseudo>(NAME), has_sccb> {
+      let AsmString = pre_gfx940_name # !cast<FLAT_Pseudo>(NAME).AsmOperands # " lds";
+    }
+    def _SADDR_vi : FLAT_Real_vi<pre_gfx940_op, !cast<FLAT_Pseudo>(NAME#"_SADDR"), has_sccb> {
+      let AsmString = pre_gfx940_name # !cast<FLAT_Pseudo>(NAME#"_SADDR").AsmOperands # " lds";
+    }
+  }
+
+  let SubtargetPredicate = isGFX940Plus in {
+    def _gfx940 : FLAT_Real_gfx940<op, !cast<FLAT_Pseudo>(NAME)>;
+    def _SADDR_gfx940 : FLAT_Real_gfx940<op, !cast<FLAT_Pseudo>(NAME#"_SADDR")>;
+  }
+}
+
+multiclass FLAT_Real_AllAddr_SVE_LDS<bits<7> op, bits<7> pre_gfx940_op> {
+  defm "" : FLAT_Real_AllAddr_LDS<op, pre_gfx940_op>;
+  let SubtargetPredicate = isGFX940Plus in {
+    def _SVS_gfx940 : FLAT_Real_gfx940<op, !cast<FLAT_Pseudo>(NAME#"_SVS")>;
+    def _ST_gfx940  : FLAT_Real_gfx940<op, !cast<FLAT_Pseudo>(NAME#"_ST")>;
+  }
+}
+
 def FLAT_LOAD_UBYTE_vi         : FLAT_Real_vi <0x10, FLAT_LOAD_UBYTE>;
 def FLAT_LOAD_SBYTE_vi         : FLAT_Real_vi <0x11, FLAT_LOAD_SBYTE>;
 def FLAT_LOAD_USHORT_vi        : FLAT_Real_vi <0x12, FLAT_LOAD_USHORT>;
 def FLAT_LOAD_SSHORT_vi        : FLAT_Real_vi <0x13, FLAT_LOAD_SSHORT>;
 def FLAT_LOAD_DWORD_vi         : FLAT_Real_vi <0x14, FLAT_LOAD_DWORD>;
 def FLAT_LOAD_DWORDX2_vi       : FLAT_Real_vi <0x15, FLAT_LOAD_DWORDX2>;
 def FLAT_LOAD_DWORDX4_vi       : FLAT_Real_vi <0x17, FLAT_LOAD_DWORDX4>;
 def FLAT_LOAD_DWORDX3_vi       : FLAT_Real_vi <0x16, FLAT_LOAD_DWORDX3>;

 defm GLOBAL_STORE_BYTE_D16_HI : FLAT_Real_AllAddr_vi <0x19>;
 defm GLOBAL_STORE_SHORT : FLAT_Real_AllAddr_vi <0x1a>;
 defm GLOBAL_STORE_SHORT_D16_HI : FLAT_Real_AllAddr_vi <0x1b>;
 defm GLOBAL_STORE_DWORD : FLAT_Real_AllAddr_vi <0x1c>;
 defm GLOBAL_STORE_DWORDX2 : FLAT_Real_AllAddr_vi <0x1d>;
 defm GLOBAL_STORE_DWORDX3 : FLAT_Real_AllAddr_vi <0x1e>;
 defm GLOBAL_STORE_DWORDX4 : FLAT_Real_AllAddr_vi <0x1f>;
 
-let AssemblerPredicate = isGFX940Plus in {
-defm GLOBAL_LOAD_LDS_UBYTE  : FLAT_Real_AllAddr_vi <0x026>;
-defm GLOBAL_LOAD_LDS_SBYTE  : FLAT_Real_AllAddr_vi <0x027>;
-defm GLOBAL_LOAD_LDS_USHORT : FLAT_Real_AllAddr_vi <0x028>;
-defm GLOBAL_LOAD_LDS_SSHORT : FLAT_Real_AllAddr_vi <0x029>;
-defm GLOBAL_LOAD_LDS_DWORD  : FLAT_Real_AllAddr_vi <0x02a>;
-} // End let AssemblerPredicate = isGFX940Plus
+defm GLOBAL_LOAD_LDS_UBYTE  : FLAT_Real_AllAddr_LDS <0x026, 0x10>;
+defm GLOBAL_LOAD_LDS_SBYTE  : FLAT_Real_AllAddr_LDS <0x027, 0x11>;
+defm GLOBAL_LOAD_LDS_USHORT : FLAT_Real_AllAddr_LDS <0x028, 0x12>;
+defm GLOBAL_LOAD_LDS_SSHORT : FLAT_Real_AllAddr_LDS <0x029, 0x13>;
+defm GLOBAL_LOAD_LDS_DWORD  : FLAT_Real_AllAddr_LDS <0x02a, 0x14>;
 
 defm GLOBAL_ATOMIC_SWAP       : FLAT_Global_Real_Atomics_vi <0x40>;
 defm GLOBAL_ATOMIC_CMPSWAP    : FLAT_Global_Real_Atomics_vi <0x41>;
 defm GLOBAL_ATOMIC_ADD        : FLAT_Global_Real_Atomics_vi <0x42>;
 defm GLOBAL_ATOMIC_SUB        : FLAT_Global_Real_Atomics_vi <0x43>;
 defm GLOBAL_ATOMIC_SMIN       : FLAT_Global_Real_Atomics_vi <0x44>;
 defm GLOBAL_ATOMIC_UMIN       : FLAT_Global_Real_Atomics_vi <0x45>;
 defm GLOBAL_ATOMIC_SMAX       : FLAT_Global_Real_Atomics_vi <0x46>;

 defm GLOBAL_ATOMIC_SMAX_X2    : FLAT_Global_Real_Atomics_vi <0x66>;
 defm GLOBAL_ATOMIC_UMAX_X2    : FLAT_Global_Real_Atomics_vi <0x67>;
 defm GLOBAL_ATOMIC_AND_X2     : FLAT_Global_Real_Atomics_vi <0x68>;
 defm GLOBAL_ATOMIC_OR_X2      : FLAT_Global_Real_Atomics_vi <0x69>;
 defm GLOBAL_ATOMIC_XOR_X2     : FLAT_Global_Real_Atomics_vi <0x6a>;
 defm GLOBAL_ATOMIC_INC_X2     : FLAT_Global_Real_Atomics_vi <0x6b>;
 defm GLOBAL_ATOMIC_DEC_X2     : FLAT_Global_Real_Atomics_vi <0x6c>;
 
-let AssemblerPredicate = isGFX940Plus in {
-defm SCRATCH_LOAD_LDS_UBYTE  : FLAT_Real_AllAddr_SVE_vi <0x026>;
-defm SCRATCH_LOAD_LDS_SBYTE  : FLAT_Real_AllAddr_SVE_vi <0x027>;
-defm SCRATCH_LOAD_LDS_USHORT : FLAT_Real_AllAddr_SVE_vi <0x028>;
-defm SCRATCH_LOAD_LDS_SSHORT : FLAT_Real_AllAddr_SVE_vi <0x029>;
-defm SCRATCH_LOAD_LDS_DWORD  : FLAT_Real_AllAddr_SVE_vi <0x02a>;
-} // End let AssemblerPredicate = isGFX940Plus
+defm SCRATCH_LOAD_LDS_UBYTE  : FLAT_Real_AllAddr_SVE_LDS <0x026, 0x10>;
+defm SCRATCH_LOAD_LDS_SBYTE  : FLAT_Real_AllAddr_SVE_LDS <0x027, 0x11>;
+defm SCRATCH_LOAD_LDS_USHORT : FLAT_Real_AllAddr_SVE_LDS <0x028, 0x12>;
+defm SCRATCH_LOAD_LDS_SSHORT : FLAT_Real_AllAddr_SVE_LDS <0x029, 0x13>;
+defm SCRATCH_LOAD_LDS_DWORD  : FLAT_Real_AllAddr_SVE_LDS <0x02a, 0x14>;
 
 defm SCRATCH_LOAD_UBYTE         : FLAT_Real_AllAddr_SVE_vi <0x10>;
 defm SCRATCH_LOAD_SBYTE         : FLAT_Real_AllAddr_SVE_vi <0x11>;
 defm SCRATCH_LOAD_USHORT        : FLAT_Real_AllAddr_SVE_vi <0x12>;
 defm SCRATCH_LOAD_SSHORT        : FLAT_Real_AllAddr_SVE_vi <0x13>;
 defm SCRATCH_LOAD_DWORD         : FLAT_Real_AllAddr_SVE_vi <0x14>;
 defm SCRATCH_LOAD_DWORDX2       : FLAT_Real_AllAddr_SVE_vi <0x15>;
 defm SCRATCH_LOAD_DWORDX3       : FLAT_Real_AllAddr_SVE_vi <0x16>;
multiclass FLAT_Real_GlblAtomics_RTN_gfx10<bits<7> op> :
   FLAT_Real_RTN_gfx10<op>,
   FLAT_Real_SADDR_RTN_gfx10<op>;
 
 multiclass FLAT_Real_ScratchAllAddr_gfx10<bits<7> op> :
   FLAT_Real_Base_gfx10<op>,
   FLAT_Real_SADDR_gfx10<op>,
   FLAT_Real_ST_gfx10<op>;
 
+multiclass FLAT_Real_AllAddr_LDS_gfx10<bits<7> op,
+  string opname = !subst("_lds", "", !cast<FLAT_Pseudo>(NAME).PseudoInstr)> {
+  let AsmString = opname # !cast<FLAT_Pseudo>(NAME).AsmOperands # " lds" in
+  defm "" : FLAT_Real_Base_gfx10<op>;
+
+  let AsmString = opname # !cast<FLAT_Pseudo>(NAME#"_SADDR").AsmOperands # " lds" in
+  defm "" : FLAT_Real_SADDR_gfx10<op>;
+}
+
+multiclass FLAT_Real_ScratchAllAddr_LDS_gfx10<bits<7> op,
+  string opname = !subst("_lds", "", !cast<FLAT_Pseudo>(NAME).PseudoInstr)> {
+  defm "" : FLAT_Real_AllAddr_LDS_gfx10<op>;
+
+  let AsmString = opname # !cast<FLAT_Pseudo>(NAME#"_ST").AsmOperands # " lds" in
+  defm "" : FLAT_Real_ST_gfx10<op>;
+}
+
 // ENC_FLAT.
 defm FLAT_LOAD_UBYTE            : FLAT_Real_Base_gfx10<0x008>;
 defm FLAT_LOAD_SBYTE            : FLAT_Real_Base_gfx10<0x009>;
 defm FLAT_LOAD_USHORT           : FLAT_Real_Base_gfx10<0x00a>;
 defm FLAT_LOAD_SSHORT           : FLAT_Real_Base_gfx10<0x00b>;
 defm FLAT_LOAD_DWORD            : FLAT_Real_Base_gfx10<0x00c>;
 defm FLAT_LOAD_DWORDX2          : FLAT_Real_Base_gfx10<0x00d>;
 defm FLAT_LOAD_DWORDX4          : FLAT_Real_Base_gfx10<0x00e>;

 defm GLOBAL_ATOMIC_INC_X2       : FLAT_Real_GlblAtomics_gfx10<0x05c>;
 defm GLOBAL_ATOMIC_DEC_X2       : FLAT_Real_GlblAtomics_gfx10<0x05d>;
 defm GLOBAL_ATOMIC_FCMPSWAP_X2  : FLAT_Real_GlblAtomics_gfx10<0x05e>;
 defm GLOBAL_ATOMIC_FMIN_X2      : FLAT_Real_GlblAtomics_gfx10<0x05f>;
 defm GLOBAL_ATOMIC_FMAX_X2      : FLAT_Real_GlblAtomics_gfx10<0x060>;
 defm GLOBAL_LOAD_DWORD_ADDTID   : FLAT_Real_AllAddr_gfx10<0x016>;
 defm GLOBAL_STORE_DWORD_ADDTID  : FLAT_Real_AllAddr_gfx10<0x017>;
 
+defm GLOBAL_LOAD_LDS_UBYTE      : FLAT_Real_AllAddr_LDS_gfx10 <0x008>;
+defm GLOBAL_LOAD_LDS_SBYTE      : FLAT_Real_AllAddr_LDS_gfx10 <0x009>;
+defm GLOBAL_LOAD_LDS_USHORT     : FLAT_Real_AllAddr_LDS_gfx10 <0x00a>;
+defm GLOBAL_LOAD_LDS_SSHORT     : FLAT_Real_AllAddr_LDS_gfx10 <0x00b>;
+defm GLOBAL_LOAD_LDS_DWORD      : FLAT_Real_AllAddr_LDS_gfx10 <0x00c>;
+
 // ENC_FLAT_SCRATCH.
 defm SCRATCH_LOAD_UBYTE         : FLAT_Real_ScratchAllAddr_gfx10<0x008>;
 defm SCRATCH_LOAD_SBYTE         : FLAT_Real_ScratchAllAddr_gfx10<0x009>;
 defm SCRATCH_LOAD_USHORT        : FLAT_Real_ScratchAllAddr_gfx10<0x00a>;
 defm SCRATCH_LOAD_SSHORT        : FLAT_Real_ScratchAllAddr_gfx10<0x00b>;
 defm SCRATCH_LOAD_DWORD         : FLAT_Real_ScratchAllAddr_gfx10<0x00c>;
 defm SCRATCH_LOAD_DWORDX2       : FLAT_Real_ScratchAllAddr_gfx10<0x00d>;
 defm SCRATCH_LOAD_DWORDX4       : FLAT_Real_ScratchAllAddr_gfx10<0x00e>;
 defm SCRATCH_LOAD_DWORDX3       : FLAT_Real_ScratchAllAddr_gfx10<0x00f>;
 defm SCRATCH_STORE_BYTE         : FLAT_Real_ScratchAllAddr_gfx10<0x018>;
 defm SCRATCH_STORE_BYTE_D16_HI  : FLAT_Real_ScratchAllAddr_gfx10<0x019>;
 defm SCRATCH_STORE_SHORT        : FLAT_Real_ScratchAllAddr_gfx10<0x01a>;
 defm SCRATCH_STORE_SHORT_D16_HI : FLAT_Real_ScratchAllAddr_gfx10<0x01b>;
 defm SCRATCH_STORE_DWORD        : FLAT_Real_ScratchAllAddr_gfx10<0x01c>;
 defm SCRATCH_STORE_DWORDX2      : FLAT_Real_ScratchAllAddr_gfx10<0x01d>;
 defm SCRATCH_STORE_DWORDX4      : FLAT_Real_ScratchAllAddr_gfx10<0x01e>;
 defm SCRATCH_STORE_DWORDX3      : FLAT_Real_ScratchAllAddr_gfx10<0x01f>;
 defm SCRATCH_LOAD_UBYTE_D16     : FLAT_Real_ScratchAllAddr_gfx10<0x020>;
 defm SCRATCH_LOAD_UBYTE_D16_HI  : FLAT_Real_ScratchAllAddr_gfx10<0x021>;
 defm SCRATCH_LOAD_SBYTE_D16     : FLAT_Real_ScratchAllAddr_gfx10<0x022>;
 defm SCRATCH_LOAD_SBYTE_D16_HI  : FLAT_Real_ScratchAllAddr_gfx10<0x023>;
 defm SCRATCH_LOAD_SHORT_D16     : FLAT_Real_ScratchAllAddr_gfx10<0x024>;
 defm SCRATCH_LOAD_SHORT_D16_HI  : FLAT_Real_ScratchAllAddr_gfx10<0x025>;
+
+defm SCRATCH_LOAD_LDS_UBYTE     : FLAT_Real_ScratchAllAddr_LDS_gfx10 <0x008>;
+defm SCRATCH_LOAD_LDS_SBYTE     : FLAT_Real_ScratchAllAddr_LDS_gfx10 <0x009>;
+defm SCRATCH_LOAD_LDS_USHORT    : FLAT_Real_ScratchAllAddr_LDS_gfx10 <0x00a>;
+defm SCRATCH_LOAD_LDS_SSHORT    : FLAT_Real_ScratchAllAddr_LDS_gfx10 <0x00b>;
+defm SCRATCH_LOAD_LDS_DWORD     : FLAT_Real_ScratchAllAddr_LDS_gfx10 <0x00c>;

llvm/test/MC/AMDGPU/gfx1030_new.s

 s_waitcnt_depctr depctr_hold_cnt(1) depctr_sa_sdst(1) depctr_va_vdst(15) depctr_va_sdst(7) depctr_va_ssrc(1) depctr_va_vcc(1) depctr_vm_vsrc(7)
 // GFX10: encoding: [0x9f,0xff,0xa3,0xbf]
 
 s_waitcnt_depctr depctr_hold_cnt(1) & depctr_sa_sdst(1) & depctr_va_vdst(1) & depctr_va_sdst(1) & depctr_va_ssrc(1) & depctr_va_vcc(1) & depctr_vm_vsrc(1)
 // GFX10: encoding: [0x87,0x13,0xa3,0xbf]
 
 s_waitcnt_depctr depctr_hold_cnt(1), depctr_sa_sdst(1), depctr_va_vdst(14), depctr_va_sdst(6), depctr_va_ssrc(1), depctr_va_vcc(1), depctr_vm_vsrc(6)
 // GFX10: encoding: [0x9b,0xed,0xa3,0xbf]
+
+scratch_load_dword off, off offset:1024 lds
+// GFX10: [0x00,0x64,0x30,0xdc,0x00,0x00,0x7f,0x00]
+
+scratch_load_ubyte off, off offset:1024 lds
+// GFX10: [0x00,0x64,0x20,0xdc,0x00,0x00,0x7f,0x00]
+
+scratch_load_sbyte off, off offset:1024 lds
+// GFX10: [0x00,0x64,0x24,0xdc,0x00,0x00,0x7f,0x00]
+
+scratch_load_ushort off, off offset:1024 lds
+// GFX10: [0x00,0x64,0x28,0xdc,0x00,0x00,0x7f,0x00]
+
+scratch_load_sshort off, off offset:1024 lds
+// GFX10: [0x00,0x64,0x2c,0xdc,0x00,0x00,0x7f,0x00]

llvm/test/MC/AMDGPU/gfx10_asm_flat.s

 // GFX10: [0x00,0x90,0x58,0xdd,0x01,0x02,0x7d,0x00]
 
 global_atomic_xor v[1:2], v2, off dlc
 // GFX10: [0x00,0x90,0xec,0xdc,0x01,0x02,0x7d,0x00]
 
 global_atomic_xor_x2 v[1:2], v[2:3], off dlc
 // GFX10: [0x00,0x90,0x6c,0xdd,0x01,0x02,0x7d,0x00]
 
+global_load_dword v2, s[4:5] offset:1024 lds
+// GFX10: [0x00,0xa4,0x30,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_dword v2, s[4:5] offset:1024 lds
+// GFX10: [0x00,0xa4,0x30,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_ubyte v2, s[4:5] offset:1024 lds
+// GFX10: [0x00,0xa4,0x20,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_sbyte v2, s[4:5] offset:1024 lds
+// GFX10: [0x00,0xa4,0x24,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_ushort v2, s[4:5] offset:1024 lds
+// GFX10: [0x00,0xa4,0x28,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_sshort v2, s[4:5] offset:1024 lds
+// GFX10: [0x00,0xa4,0x2c,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_dword v[2:3], off offset:1024 lds
+// GFX10: [0x00,0xa4,0x30,0xdc,0x02,0x00,0x7d,0x00]
+
+global_load_ubyte v[2:3], off offset:1024 lds
+// GFX10: [0x00,0xa4,0x20,0xdc,0x02,0x00,0x7d,0x00]
+
+global_load_sbyte v[2:3], off offset:1024 lds
+// GFX10: [0x00,0xa4,0x24,0xdc,0x02,0x00,0x7d,0x00]
+
+global_load_ushort v[2:3], off offset:1024 lds
+// GFX10: [0x00,0xa4,0x28,0xdc,0x02,0x00,0x7d,0x00]
+
+global_load_sshort v[2:3], off offset:1024 lds
+// GFX10: [0x00,0xa4,0x2c,0xdc,0x02,0x00,0x7d,0x00]
+
+scratch_load_dword v2, off offset:1024 lds
+// GFX10: [0x00,0x64,0x30,0xdc,0x02,0x00,0x7d,0x00]
+
+scratch_load_ubyte v2, off offset:1024 lds
+// GFX10: [0x00,0x64,0x20,0xdc,0x02,0x00,0x7d,0x00]
+
+scratch_load_sbyte v2, off offset:1024 lds
+// GFX10: [0x00,0x64,0x24,0xdc,0x02,0x00,0x7d,0x00]
+
+scratch_load_ushort v2, off offset:1024 lds
+// GFX10: [0x00,0x64,0x28,0xdc,0x02,0x00,0x7d,0x00]
+
+scratch_load_sshort v2, off offset:1024 lds
+// GFX10: [0x00,0x64,0x2c,0xdc,0x02,0x00,0x7d,0x00]
+
+scratch_load_dword off, s4 offset:1024 lds
+// GFX10: [0x00,0x64,0x30,0xdc,0x00,0x00,0x04,0x00]
+
+scratch_load_ubyte off, s4 offset:1024 lds
+// GFX10: [0x00,0x64,0x20,0xdc,0x00,0x00,0x04,0x00]
+
+scratch_load_sbyte off, s4 offset:1024 lds
+// GFX10: [0x00,0x64,0x24,0xdc,0x00,0x00,0x04,0x00]
+
+scratch_load_ushort off, s4 offset:1024 lds
+// GFX10: [0x00,0x64,0x28,0xdc,0x00,0x00,0x04,0x00]
+
+scratch_load_sshort off, s4 offset:1024 lds
+// GFX10: [0x00,0x64,0x2c,0xdc,0x00,0x00,0x04,0x00]
+
 //===----------------------------------------------------------------------===//
 // Also see flat-gfx10.s, flat-global.s, flat-scratch-instructions.s.
 //===----------------------------------------------------------------------===//

llvm/test/MC/AMDGPU/gfx8_unsupported.s

 // CHECK: error: instruction not supported on this GPU
 
 v_xnor_b32_sdwa v255, v1, v2 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
 // CHECK: error: instruction not supported on this GPU
 
 v_xor3_b32 v255, v1, v2, v3
 // CHECK: error: instruction not supported on this GPU
 
+global_load_lds_dword v[2:3], off
+// CHECK: error: instruction not supported on this GPU
+
+global_load_dword v[2:3], off lds
+// CHECK: error: instruction not supported on this GPU
+
+scratch_load_dword v2, off lds
+// CHECK: error: instruction not supported on this GPU
+
+scratch_load_dword off, s2 lds
+// CHECK: error: instruction not supported on this GPU
+
 //===----------------------------------------------------------------------===//
 // Unsupported e32 variants.
 //===----------------------------------------------------------------------===//
 
 v_cvt_pkrtz_f16_f32_e32 v255, v1, v2
 // CHECK: error: e32 variant of this instruction is not supported
 
 //===----------------------------------------------------------------------===//

llvm/test/MC/AMDGPU/gfx9-asm-err.s

 v_subrev_u16_e64 v5, v1, 0.5
 // GFX9ERR: error: literal operands are not supported
 
 v_subrev_u16_e64 v5, v1, -4.0
 // GFX9ERR: error: literal operands are not supported
 
 v_cvt_u32_f64 v5, v[0:1] quad_perm:[0,2,1,1] row_mask:0xf bank_mask:0xf
 // GFX9ERR: error: not a valid operand.
+
+global_load_lds_dword v[2:3], off
+// GFX9ERR: error: instruction not supported on this GPU
+
+global_load_dword v[2:3], off
+// GFX9ERR: error: invalid operands for instruction
+
+scratch_load_dword v2, off, offset:256
+// GFX9ERR: error: invalid operands for instruction

llvm/test/MC/AMDGPU/gfx90a_err.s

 ds_gws_barrier a255 offset:4 gds
 // GFX90A: error: vgpr must be even aligned
 
 ds_ordered_count v5, v1 offset:65535 gds
 // GFX90A: error: instruction not supported on this GPU
 
 exp pos0 v3, v2, v1, v0
 // GFX90A: error: instruction not supported on this GPU
+
+global_load_lds_dword v[2:3], off
+// GFX90A: error: instruction not supported on this GPU
+
+scratch_load_lds_dword v2, off
+// GFX90A: error: instruction not supported on this GPU

llvm/test/MC/AMDGPU/gfx940_err.s

 s_getreg_b32 s1, hwreg(HW_REG_POPS_PACKER)
 // GFX940: error: specified hardware register is not supported on this GPU
 
 ds_ordered_count v5, v1 offset:65535 gds
 // GFX940: error: instruction not supported on this GPU
 
 exp pos0 v3, v2, v1, v0
 // GFX940: error: instruction not supported on this GPU
+
+global_load_dword v[2:3], off lds
+// GFX940: error: operands are not valid for this GPU or mode
+
+scratch_load_dword v2, off lds
+// GFX940: error: operands are not valid for this GPU or mode

llvm/test/MC/AMDGPU/gfx9_asm_flat.s

 scratch_load_short_d16_hi v5, off, s2 offset:-4096
 // CHECK: [0x00,0x50,0x94,0xdc,0x00,0x00,0x02,0x05]
 
 scratch_load_short_d16_hi v5, off, s2 offset:-1 glc
 // CHECK: [0xff,0x5f,0x95,0xdc,0x00,0x00,0x02,0x05]
 
 scratch_load_short_d16_hi v5, off, s2 offset:-1 slc
 // CHECK: [0xff,0x5f,0x96,0xdc,0x00,0x00,0x02,0x05]
+
+global_load_dword v[2:3], off lds
+// CHECK: [0x00,0xa0,0x50,0xdc,0x02,0x00,0x7f,0x00]
+
+global_load_dword v[2:3], off offset:2048 lds
+// CHECK: [0x00,0xa8,0x50,0xdc,0x02,0x00,0x7f,0x00]
+
+global_load_ubyte v[2:3], off offset:2048 lds
+// CHECK: [0x00,0xa8,0x40,0xdc,0x02,0x00,0x7f,0x00]
+
+global_load_sbyte v[2:3], off offset:2048 lds
+// CHECK: [0x00,0xa8,0x44,0xdc,0x02,0x00,0x7f,0x00]
+
+global_load_ushort v[2:3], off offset:2048 lds
+// CHECK: [0x00,0xa8,0x48,0xdc,0x02,0x00,0x7f,0x00]
+
+global_load_sshort v[2:3], off offset:2048 lds
+// CHECK: [0x00,0xa8,0x4c,0xdc,0x02,0x00,0x7f,0x00]
+
+global_load_dword v2, s[4:5] offset:2048 lds
+// CHECK: [0x00,0xa8,0x50,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_ubyte v2, s[4:5] offset:2048 lds
+// CHECK: [0x00,0xa8,0x40,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_sbyte v2, s[4:5] offset:2048 lds
+// CHECK: [0x00,0xa8,0x44,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_ushort v2, s[4:5] offset:2048 lds
+// CHECK: [0x00,0xa8,0x48,0xdc,0x02,0x00,0x04,0x00]
+
+global_load_sshort v2, s[4:5] offset:2048 lds
+// CHECK: [0x00,0xa8,0x4c,0xdc,0x02,0x00,0x04,0x00]
+
+scratch_load_dword v2, off lds
+// CHECK: [0x00,0x60,0x50,0xdc,0x02,0x00,0x7f,0x00]
+
+scratch_load_dword v2, off offset:2048 lds
+// CHECK: [0x00,0x68,0x50,0xdc,0x02,0x00,0x7f,0x00]
+
+scratch_load_dword v2, off offset:2048 lds
+// CHECK: [0x00,0x68,0x50,0xdc,0x02,0x00,0x7f,0x00]
+
+scratch_load_ubyte v2, off offset:2048 lds
+// CHECK: [0x00,0x68,0x40,0xdc,0x02,0x00,0x7f,0x00]
+
+scratch_load_sbyte v2, off offset:2048 lds
+// CHECK: [0x00,0x68,0x44,0xdc,0x02,0x00,0x7f,0x00]
+
+scratch_load_ushort v2, off offset:2048 lds
+// CHECK: [0x00,0x68,0x48,0xdc,0x02,0x00,0x7f,0x00]
+
+scratch_load_sshort v2, off offset:2048 lds
+// CHECK: [0x00,0x68,0x4c,0xdc,0x02,0x00,0x7f,0x00]
+
+scratch_load_dword off, s4 offset:2048 lds
+// CHECK: [0x00,0x68,0x50,0xdc,0x00,0x00,0x04,0x00]
+
+scratch_load_ubyte off, s4 offset:2048 lds
+// CHECK: [0x00,0x68,0x40,0xdc,0x00,0x00,0x04,0x00]
+
+scratch_load_sbyte off, s4 offset:2048 lds
+// CHECK: [0x00,0x68,0x44,0xdc,0x00,0x00,0x04,0x00]
+
+scratch_load_ushort off, s4 offset:2048 lds
+// CHECK: [0x00,0x68,0x48,0xdc,0x00,0x00,0x04,0x00]
+
+scratch_load_sshort off, s4 offset:2048 lds
+// CHECK: [0x00,0x68,0x4c,0xdc,0x00,0x00,0x04,0x00]

llvm/test/MC/Disassembler/AMDGPU/gfx1030_dasm_new.txt

 # GFX10: s_waitcnt_depctr depctr_hold_cnt(0) depctr_sa_sdst(0) depctr_va_vdst(0) depctr_va_sdst(0) depctr_va_ssrc(0) depctr_va_vcc(0) depctr_vm_vsrc(0) ; encoding: [0x00,0x00,0xa3,0xbf]
 0x00,0x00,0xa3,0xbf
 
 # GFX10: s_waitcnt_depctr depctr_hold_cnt(1) depctr_sa_sdst(1) depctr_va_vdst(15) depctr_va_sdst(7) depctr_va_ssrc(1) depctr_va_vcc(1) depctr_vm_vsrc(7) ; encoding: [0x9f,0xff,0xa3,0xbf]
 0x9f,0xff,0xa3,0xbf
 
 # GFX10: s_waitcnt_depctr depctr_va_vdst(1) depctr_va_sdst(1) depctr_vm_vsrc(1)  ; encoding: [0x87,0x13,0xa3,0xbf]
 0x87,0x13,0xa3,0xbf
+
+# GFX10: scratch_load_dword off, off offset:1024 lds ; encoding: [0x00,0x64,0x30,0xdc,0x00,0x00,0x7f,0x00]
+0x00,0x64,0x30,0xdc,0x00,0x00,0x7f,0x00
+
+# GFX10: scratch_load_ubyte off, off offset:1024 lds ; encoding: [0x00,0x64,0x20,0xdc,0x00,0x00,0x7f,0x00]
+0x00,0x64,0x20,0xdc,0x00,0x00,0x7f,0x00
+
+# GFX10: scratch_load_sbyte off, off offset:1024 lds ; encoding: [0x00,0x64,0x24,0xdc,0x00,0x00,0x7f,0x00]
+0x00,0x64,0x24,0xdc,0x00,0x00,0x7f,0x00
+
+# GFX10: scratch_load_ushort off, off offset:1024 lds ; encoding: [0x00,0x64,0x28,0xdc,0x00,0x00,0x7f,0x00]
+0x00,0x64,0x28,0xdc,0x00,0x00,0x7f,0x00
+
+# GFX10: scratch_load_sshort off, off offset:1024 lds ; encoding: [0x00,0x64,0x2c,0xdc,0x00,0x00,0x7f,0x00]
+0x00,0x64,0x2c,0xdc,0x00,0x00,0x7f,0x00

llvm/test/MC/Disassembler/AMDGPU/gfx10_dasm_all.txt

 # GFX10: s_waitcnt_depctr depctr_sa_sdst(0) depctr_va_vdst(0) depctr_va_sdst(0) depctr_va_ssrc(0) depctr_va_vcc(0) depctr_vm_vsrc(0) ; encoding: [0x00,0x00,0xa3,0xbf]
 0x00,0x00,0xa3,0xbf
 
 # GFX10: s_waitcnt_depctr depctr_va_vdst(1) depctr_va_sdst(1) depctr_vm_vsrc(1) ; encoding: [0x07,0x13,0xa3,0xbf]
 0x07,0x13,0xa3,0xbf
 
 # GFX10: s_waitcnt_depctr depctr_va_vdst(14) depctr_va_sdst(6) depctr_vm_vsrc(6) ; encoding: [0x1b,0xed,0xa3,0xbf]
 0x1b,0xed,0xa3,0xbf
+
+# GFX10: global_load_dword v2, s[4:5] offset:1024 lds ; encoding: [0x00,0xa4,0x30,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa4,0x30,0xdc,0x02,0x00,0x04,0x00
+
+# GFX10: global_load_dword v2, s[4:5] offset:1024 lds ; encoding: [0x00,0xa4,0x30,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa4,0x30,0xdc,0x02,0x00,0x04,0x00
+
+# GFX10: global_load_ubyte v2, s[4:5] offset:1024 lds ; encoding: [0x00,0xa4,0x20,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa4,0x20,0xdc,0x02,0x00,0x04,0x00
+
+# GFX10: global_load_sbyte v2, s[4:5] offset:1024 lds ; encoding: [0x00,0xa4,0x24,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa4,0x24,0xdc,0x02,0x00,0x04,0x00
+
+# GFX10: global_load_ushort v2, s[4:5] offset:1024 lds ; encoding: [0x00,0xa4,0x28,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa4,0x28,0xdc,0x02,0x00,0x04,0x00
+
+# GFX10: global_load_sshort v2, s[4:5] offset:1024 lds ; encoding: [0x00,0xa4,0x2c,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa4,0x2c,0xdc,0x02,0x00,0x04,0x00
+
+# GFX10: global_load_dword v[2:3], off offset:1024 lds ; encoding: [0x00,0xa4,0x30,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0xa4,0x30,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: global_load_ubyte v[2:3], off offset:1024 lds ; encoding: [0x00,0xa4,0x20,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0xa4,0x20,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: global_load_sbyte v[2:3], off offset:1024 lds ; encoding: [0x00,0xa4,0x24,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0xa4,0x24,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: global_load_ushort v[2:3], off offset:1024 lds ; encoding: [0x00,0xa4,0x28,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0xa4,0x28,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: global_load_sshort v[2:3], off offset:1024 lds ; encoding: [0x00,0xa4,0x2c,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0xa4,0x2c,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: scratch_load_dword v2, off offset:1024 lds ; encoding: [0x00,0x64,0x30,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0x64,0x30,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: scratch_load_ubyte v2, off offset:1024 lds ; encoding: [0x00,0x64,0x20,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0x64,0x20,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: scratch_load_sbyte v2, off offset:1024 lds ; encoding: [0x00,0x64,0x24,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0x64,0x24,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: scratch_load_ushort v2, off offset:1024 lds ; encoding: [0x00,0x64,0x28,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0x64,0x28,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: scratch_load_sshort v2, off offset:1024 lds ; encoding: [0x00,0x64,0x2c,0xdc,0x02,0x00,0x7d,0x00]
+0x00,0x64,0x2c,0xdc,0x02,0x00,0x7d,0x00
+
+# GFX10: scratch_load_dword off, s4 offset:1024 lds ; encoding: [0x00,0x64,0x30,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x64,0x30,0xdc,0x00,0x00,0x04,0x00
+
+# GFX10: scratch_load_ubyte off, s4 offset:1024 lds ; encoding: [0x00,0x64,0x20,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x64,0x20,0xdc,0x00,0x00,0x04,0x00
+
+# GFX10: scratch_load_sbyte off, s4 offset:1024 lds ; encoding: [0x00,0x64,0x24,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x64,0x24,0xdc,0x00,0x00,0x04,0x00
+
+# GFX10: scratch_load_ushort off, s4 offset:1024 lds ; encoding: [0x00,0x64,0x28,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x64,0x28,0xdc,0x00,0x00,0x04,0x00
+
+# GFX10: scratch_load_sshort off, s4 offset:1024 lds ; encoding: [0x00,0x64,0x2c,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x64,0x2c,0xdc,0x00,0x00,0x04,0x00

llvm/test/MC/Disassembler/AMDGPU/gfx9_dasm_all.txt

 # CHECK: v_cmpx_t_u32_sdwa s[6:7], v1, v2 src0_sel:DWORD src1_sel:WORD_0 ; encoding: [0xf9,0x04,0xbe,0x7d,0x01,0x86,0x06,0x04]
 0xf9,0x04,0xbe,0x7d,0x01,0x86,0x06,0x04
 
 # CHECK: v_cmpx_t_u32_sdwa s[6:7], v1, v2 src0_sel:DWORD src1_sel:WORD_1 ; encoding: [0xf9,0x04,0xbe,0x7d,0x01,0x86,0x06,0x05]
 0xf9,0x04,0xbe,0x7d,0x01,0x86,0x06,0x05
 
 # CHECK: v_cmpx_t_u32_sdwa s[6:7], v1, sext(v2) src0_sel:DWORD src1_sel:DWORD ; encoding: [0xf9,0x04,0xbe,0x7d,0x01,0x86,0x06,0x0e]
 0xf9,0x04,0xbe,0x7d,0x01,0x86,0x06,0x0e
+
+# CHECK: global_load_dword v[2:3], off lds       ; encoding: [0x00,0xa0,0x50,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0xa0,0x50,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: global_load_dword v[2:3], off offset:2048 lds ; encoding: [0x00,0xa8,0x50,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0xa8,0x50,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: global_load_ubyte v[2:3], off offset:2048 lds ; encoding: [0x00,0xa8,0x40,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0xa8,0x40,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: global_load_sbyte v[2:3], off offset:2048 lds ; encoding: [0x00,0xa8,0x44,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0xa8,0x44,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: global_load_ushort v[2:3], off offset:2048 lds ; encoding: [0x00,0xa8,0x48,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0xa8,0x48,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: global_load_sshort v[2:3], off offset:2048 lds ; encoding: [0x00,0xa8,0x4c,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0xa8,0x4c,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: global_load_dword v2, s[4:5] offset:2048 lds ; encoding: [0x00,0xa8,0x50,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa8,0x50,0xdc,0x02,0x00,0x04,0x00
+
+# CHECK: global_load_ubyte v2, s[4:5] offset:2048 lds ; encoding: [0x00,0xa8,0x40,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa8,0x40,0xdc,0x02,0x00,0x04,0x00
+
+# CHECK: global_load_sbyte v2, s[4:5] offset:2048 lds ; encoding: [0x00,0xa8,0x44,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa8,0x44,0xdc,0x02,0x00,0x04,0x00
+
+# CHECK: global_load_ushort v2, s[4:5] offset:2048 lds ; encoding: [0x00,0xa8,0x48,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa8,0x48,0xdc,0x02,0x00,0x04,0x00
+
+# CHECK: global_load_sshort v2, s[4:5] offset:2048 lds ; encoding: [0x00,0xa8,0x4c,0xdc,0x02,0x00,0x04,0x00]
+0x00,0xa8,0x4c,0xdc,0x02,0x00,0x04,0x00
+
+# CHECK: scratch_load_dword v2, off lds          ; encoding: [0x00,0x60,0x50,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0x60,0x50,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: scratch_load_dword v2, off offset:2048 lds ; encoding: [0x00,0x68,0x50,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0x68,0x50,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: scratch_load_dword v2, off offset:2048 lds ; encoding: [0x00,0x68,0x50,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0x68,0x50,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: scratch_load_ubyte v2, off offset:2048 lds ; encoding: [0x00,0x68,0x40,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0x68,0x40,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: scratch_load_sbyte v2, off offset:2048 lds ; encoding: [0x00,0x68,0x44,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0x68,0x44,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: scratch_load_ushort v2, off offset:2048 lds ; encoding: [0x00,0x68,0x48,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0x68,0x48,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: scratch_load_sshort v2, off offset:2048 lds ; encoding: [0x00,0x68,0x4c,0xdc,0x02,0x00,0x7f,0x00]
+0x00,0x68,0x4c,0xdc,0x02,0x00,0x7f,0x00
+
+# CHECK: scratch_load_dword off, s4 offset:2048 lds ; encoding: [0x00,0x68,0x50,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x68,0x50,0xdc,0x00,0x00,0x04,0x00
+
+# CHECK: scratch_load_ubyte off, s4 offset:2048 lds ; encoding: [0x00,0x68,0x40,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x68,0x40,0xdc,0x00,0x00,0x04,0x00
+
+# CHECK: scratch_load_sbyte off, s4 offset:2048 lds ; encoding: [0x00,0x68,0x44,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x68,0x44,0xdc,0x00,0x00,0x04,0x00
+
+# CHECK: scratch_load_ushort off, s4 offset:2048 lds ; encoding: [0x00,0x68,0x48,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x68,0x48,0xdc,0x00,0x00,0x04,0x00
+
+# CHECK: scratch_load_sshort off, s4 offset:2048 lds ; encoding: [0x00,0x68,0x4c,0xdc,0x00,0x00,0x04,0x00]
+0x00,0x68,0x4c,0xdc,0x00,0x00,0x04,0x00