[AMDGPU] gfx11 vop3dpp instructions

[Joe_Nash]

Summary

gfx11 adds the ability to use dpp modifiers on vop3 instructions.
This patch adds machine code layer support for that. The MCCodeEmitter
is changed to use APInt instead of uint64_t to support these wider
instructions.

Patch 16/N for upstreaming of AMDGPU gfx11 architecture

Depends on D126475

[rampitec]

Not all cases checked for assembly also present in disasm test.

llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

5694–5698

Technically that is _e96, but OK for now.

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

444

break on new line.

llvm/lib/Target/AMDGPU/VOP2Instructions.td

460

Indent is off.

[Joe_Nash]

Added disasm tests. Fixed noted justification issues. Disabled incomplete dot dpp instructions.

[Joe_Nash]

In D126483#3540612, @rampitec wrote:

Not all cases checked for assembly also present in disasm test.

Fixed

[Jake-Egan]

Hi, there's several failures on the AIX bot https://lab.llvm.org/buildbot/#/builders/214/builds/1687/steps/6/logs/stdio. Could you take a look and revert if it takes too long?

[Joe_Nash]

In D126483#3561366, @Jake-Egan wrote:

Hi, there's several failures on the AIX bot https://lab.llvm.org/buildbot/#/builders/214/builds/1687/steps/6/logs/stdio. Could you take a look and revert if it takes too long?

Hi, thanks for raising the issue. However, I can’t reproduce the failures locally, and other buildbots such as https://lab.llvm.org/buildbot/#/builders/104/builds/8086 are passing. Can you provide any other info that would help track the issue down? Or perhaps the bot is in a bad state?

[hubert.reinterpretcast]

In D126483#3561485, @Joe_Nash wrote:

In D126483#3561366, @Jake-Egan wrote:

Hi, there's several failures on the AIX bot https://lab.llvm.org/buildbot/#/builders/214/builds/1687/steps/6/logs/stdio. Could you take a look and revert if it takes too long?

Hi, thanks for raising the issue. However, I can’t reproduce the failures locally, and other buildbots such as https://lab.llvm.org/buildbot/#/builders/104/builds/8086 are passing. Can you provide any other info that would help track the issue down? Or perhaps the bot is in a bad state?

It's telling that the failures also occur on the big endian PPC Linux bot:
https://lab.llvm.org/buildbot/#/builders/100/builds/16288

So, the info would be to look for code that is unintentionally endianness-sensitive.

[Joe_Nash]

In D126483#3561612, @hubert.reinterpretcast wrote:

In D126483#3561485, @Joe_Nash wrote:

In D126483#3561366, @Jake-Egan wrote:

Hi, there's several failures on the AIX bot https://lab.llvm.org/buildbot/#/builders/214/builds/1687/steps/6/logs/stdio. Could you take a look and revert if it takes too long?

Hi, thanks for raising the issue. However, I can’t reproduce the failures locally, and other buildbots such as https://lab.llvm.org/buildbot/#/builders/104/builds/8086 are passing. Can you provide any other info that would help track the issue down? Or perhaps the bot is in a bad state?

It's telling that the failures also occur on the big endian PPC Linux bot:
https://lab.llvm.org/buildbot/#/builders/100/builds/16288

So, the info would be to look for code that is unintentionally endianness-sensitive.

Thanks! That sounds very reasonable. I don't expect this to be easy to find, so I will revert this and the dependent D126917

[nemanjai]

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

394

These conversions being little endian seem like a possible/likely source of the issues on big endian systems.

[Joe_Nash]

In light of D127195, I'm going to try relanding this.

llvm/lib/Target/AMDGPU/AMDGPU.td

def AMDGPUAsmVariants {
   string VOP3 = "VOP3";
   int VOP3_ID = 1;
   string SDWA = "SDWA";
   int SDWA_ID = 2;
   string SDWA9 = "SDWA9";
   int SDWA9_ID = 3;
   string DPP = "DPP";
   int DPP_ID = 4;
+  string VOP3_DPP = "VOP3_DPP";
+  int VOP3_DPP_ID = 5;
   string Disable = "Disable";
-  int Disable_ID = 5;
+  int Disable_ID = 6;
 }
 
 def DefaultAMDGPUAsmParserVariant : AsmParserVariant {
   let Variant = AMDGPUAsmVariants.Default_ID;
   let Name = AMDGPUAsmVariants.Default;
 }
 
 def VOP3AsmParserVariant : AsmParserVariant {
def SDWA9AsmParserVariant : AsmParserVariant {
   let Name = AMDGPUAsmVariants.SDWA9;
 }
 
 def DPPAsmParserVariant : AsmParserVariant {
   let Variant = AMDGPUAsmVariants.DPP_ID;
   let Name = AMDGPUAsmVariants.DPP;
 }
 
+def VOP3_DPPAsmParserVariant : AsmParserVariant {
+  let Variant = AMDGPUAsmVariants.VOP3_DPP_ID;
+  let Name = AMDGPUAsmVariants.VOP3_DPP;
+}
+
 def AMDGPU : Target {
   // Pull in Instruction Info:
   let InstructionSet = AMDGPUInstrInfo;
   let AssemblyParsers = [AMDGPUAsmParser];
   let AssemblyParserVariants = [DefaultAMDGPUAsmParserVariant,
                                 VOP3AsmParserVariant,
                                 SDWAAsmParserVariant,
                                 SDWA9AsmParserVariant,
-                                DPPAsmParserVariant];
+                                DPPAsmParserVariant,
+                                VOP3_DPPAsmParserVariant];
   let AssemblyWriters = [AMDGPUAsmWriter];
   let AllowRegisterRenaming = 1;
 }
 
 // Dummy Instruction itineraries for pseudo instructions
 def ALU_NULL : FuncUnit;
 def NullALU : InstrItinClass;
 

llvm/lib/Target/AMDGPU/AMDGPUInstructions.td

class AMDGPUInst <dag outs, dag ins, string asm = "",
   let AsmString = asm;
   let Pattern = pattern;
   let Itinerary = NullALU;
 
   // SoftFail is a field the disassembler can use to provide a way for
   // instructions to not match without killing the whole decode process. It is
   // mainly used for ARM, but Tablegen expects this field to exist or it fails
   // to build the decode table.
-  field bits<64> SoftFail = 0;
+  field bits<96> SoftFail = 0;
 
   let DecoderNamespace = Namespace;
 
   let TSFlags{63} = isRegisterLoad;
   let TSFlags{62} = isRegisterStore;
 }
 
 class AMDGPUShaderInst <dag outs, dag ins, string asm = "",

llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

 #include "llvm/ADT/SmallBitVector.h"
 #include "llvm/ADT/StringSet.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/BinaryFormat/ELF.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
+#include "llvm/MC/MCInstrDesc.h"
 #include "llvm/MC/MCParser/MCAsmLexer.h"
 #include "llvm/MC/MCParser/MCAsmParser.h"
 #include "llvm/MC/MCParser/MCParsedAsmOperand.h"
 #include "llvm/MC/MCParser/MCTargetAsmParser.h"
 #include "llvm/MC/MCSymbol.h"
 #include "llvm/MC/TargetRegistry.h"
 #include "llvm/Support/AMDGPUMetadata.h"
 #include "llvm/Support/AMDHSAKernelDescriptor.h"
enum ImmTy {
     ImmTyOffset0,
     ImmTyOffset1,
     ImmTyCPol,
     ImmTySWZ,
     ImmTyTFE,
     ImmTyD16,
     ImmTyClampSI,
     ImmTyOModSI,
-    ImmTyDPP8,
-    ImmTyDppCtrl,
-    ImmTyDppRowMask,
-    ImmTyDppBankMask,
-    ImmTyDppBoundCtrl,
-    ImmTyDppFi,
     ImmTySdwaDstSel,
     ImmTySdwaSrc0Sel,
     ImmTySdwaSrc1Sel,
     ImmTySdwaDstUnused,
     ImmTyDMask,
     ImmTyDim,
     ImmTyUNorm,
     ImmTyDA,
enum ImmTy {
     ImmTySendMsg,
     ImmTyInterpSlot,
     ImmTyInterpAttr,
     ImmTyAttrChan,
     ImmTyOpSel,
     ImmTyOpSelHi,
     ImmTyNegLo,
     ImmTyNegHi,
+    ImmTyDPP8,
+    ImmTyDppCtrl,
+    ImmTyDppRowMask,
+    ImmTyDppBankMask,
+    ImmTyDppBoundCtrl,
+    ImmTyDppFi,
     ImmTySwizzle,
     ImmTyGprIdxMode,
     ImmTyHigh,
     ImmTyBLGP,
     ImmTyCBSZ,
     ImmTyABID,
     ImmTyEndpgm,
     ImmTyWaitVDST,
public:
   bool isRegOrImmWithInt16InputMods() const {
     return isRegOrImmWithInputMods(AMDGPU::VS_32RegClassID, MVT::i16);
   }
 
   bool isRegOrImmWithInt32InputMods() const {
     return isRegOrImmWithInputMods(AMDGPU::VS_32RegClassID, MVT::i32);
   }
 
+  bool isRegOrInlineImmWithInt16InputMods() const {
+    return isRegOrInline(AMDGPU::VS_32RegClassID, MVT::i16);
+  }
+
+  bool isRegOrInlineImmWithInt32InputMods() const {
+    return isRegOrInline(AMDGPU::VS_32RegClassID, MVT::i32);
+  }
+
   bool isRegOrImmWithInt64InputMods() const {
     return isRegOrImmWithInputMods(AMDGPU::VS_64RegClassID, MVT::i64);
   }
 
   bool isRegOrImmWithFP16InputMods() const {
     return isRegOrImmWithInputMods(AMDGPU::VS_32RegClassID, MVT::f16);
   }
 
   bool isRegOrImmWithFP32InputMods() const {
     return isRegOrImmWithInputMods(AMDGPU::VS_32RegClassID, MVT::f32);
   }
 
   bool isRegOrImmWithFP64InputMods() const {
     return isRegOrImmWithInputMods(AMDGPU::VS_64RegClassID, MVT::f64);
   }
 
+  bool isRegOrInlineImmWithFP16InputMods() const {
+    return isRegOrInline(AMDGPU::VS_32RegClassID, MVT::f16);
+  }
+
+  bool isRegOrInlineImmWithFP32InputMods() const {
+    return isRegOrInline(AMDGPU::VS_32RegClassID, MVT::f32);
+  }
+
+
   bool isVReg() const {
     return isRegClass(AMDGPU::VGPR_32RegClassID) ||
            isRegClass(AMDGPU::VReg_64RegClassID) ||
            isRegClass(AMDGPU::VReg_96RegClassID) ||
            isRegClass(AMDGPU::VReg_128RegClassID) ||
            isRegClass(AMDGPU::VReg_160RegClassID) ||
            isRegClass(AMDGPU::VReg_192RegClassID) ||
            isRegClass(AMDGPU::VReg_256RegClassID) ||
public:
   int64_t parseDPPCtrlSel(StringRef Ctrl);
   int64_t parseDPPCtrlPerm();
   AMDGPUOperand::Ptr defaultRowMask() const;
   AMDGPUOperand::Ptr defaultBankMask() const;
   AMDGPUOperand::Ptr defaultBoundCtrl() const;
   AMDGPUOperand::Ptr defaultFI() const;
   void cvtDPP(MCInst &Inst, const OperandVector &Operands, bool IsDPP8 = false);
   void cvtDPP8(MCInst &Inst, const OperandVector &Operands) { cvtDPP(Inst, Operands, true); }
+  void cvtVOP3DPP(MCInst &Inst, const OperandVector &Operands, bool IsDPP8 = false);
+  void cvtVOP3DPP8(MCInst &Inst, const OperandVector &Operands) { cvtVOP3DPP(Inst, Operands, true); }
 
   OperandMatchResultTy parseSDWASel(OperandVector &Operands, StringRef Prefix,
                                     AMDGPUOperand::ImmTy Type);
   OperandMatchResultTy parseSDWADstUnused(OperandVector &Operands);
   void cvtSdwaVOP1(MCInst &Inst, const OperandVector &Operands);
   void cvtSdwaVOP2(MCInst &Inst, const OperandVector &Operands);
   void cvtSdwaVOP2b(MCInst &Inst, const OperandVector &Operands);
   void cvtSdwaVOP2e(MCInst &Inst, const OperandVector &Operands);
unsigned AMDGPUAsmParser::checkTargetMatchPredicate(MCInst &Inst) {
   }
 
   return Match_Success;
 }
 
 static ArrayRef<unsigned> getAllVariants() {
   static const unsigned Variants[] = {
     AMDGPUAsmVariants::DEFAULT, AMDGPUAsmVariants::VOP3,
-    AMDGPUAsmVariants::SDWA, AMDGPUAsmVariants::SDWA9, AMDGPUAsmVariants::DPP
+    AMDGPUAsmVariants::SDWA, AMDGPUAsmVariants::SDWA9,
+    AMDGPUAsmVariants::DPP, AMDGPUAsmVariants::VOP3_DPP
   };
 
   return makeArrayRef(Variants);
 }
 
 // What asm variants we should check
 ArrayRef<unsigned> AMDGPUAsmParser::getMatchedVariants() const {
+  if (isForcedDPP() && isForcedVOP3()) {
+    static const unsigned Variants[] = {AMDGPUAsmVariants::VOP3_DPP};
+    return makeArrayRef(Variants);
+  }
   if (getForcedEncodingSize() == 32) {
     static const unsigned Variants[] = {AMDGPUAsmVariants::DEFAULT};
     return makeArrayRef(Variants);
   }
 
   if (isForcedVOP3()) {
     static const unsigned Variants[] = {AMDGPUAsmVariants::VOP3};
     return makeArrayRef(Variants);
if (isForcedDPP()) {
     static const unsigned Variants[] = {AMDGPUAsmVariants::DPP};
     return makeArrayRef(Variants);
   }
 
   return getAllVariants();
 }
 
 StringRef AMDGPUAsmParser::getMatchedVariantName() const {
+  if (isForcedDPP() && isForcedVOP3())
+    return "e64_dpp";
+
   if (getForcedEncodingSize() == 32)
     return "e32";
 
   if (isForcedVOP3())
     return "e64";
 
   if (isForcedSDWA())
     return "sdwa";

 }
 
 StringRef AMDGPUAsmParser::parseMnemonicSuffix(StringRef Name) {
   // Clear any forced encodings from the previous instruction.
   setForcedEncodingSize(0);
   setForcedDPP(false);
   setForcedSDWA(false);
 
-  if (Name.endswith("_e64")) {
+  if (Name.endswith("_e64_dpp")) {
+    setForcedDPP(true);
+    setForcedEncodingSize(64);
+    return Name.substr(0, Name.size() - 8);
+  } else if (Name.endswith("_e64")) {
     setForcedEncodingSize(64);
     return Name.substr(0, Name.size() - 4);
   } else if (Name.endswith("_e32")) {
     setForcedEncodingSize(32);
     return Name.substr(0, Name.size() - 4);
   } else if (Name.endswith("_dpp")) {
     setForcedDPP(true);
     return Name.substr(0, Name.size() - 4);
static const OptionalOperand AMDGPUOptionalOperandTable[] = {
   {"unorm",   AMDGPUOperand::ImmTyUNorm, true, nullptr},
   {"da",      AMDGPUOperand::ImmTyDA,    true, nullptr},
   {"r128",    AMDGPUOperand::ImmTyR128A16,  true, nullptr},
   {"a16",     AMDGPUOperand::ImmTyA16,  true, nullptr},
   {"lwe",     AMDGPUOperand::ImmTyLWE,   true, nullptr},
   {"d16",     AMDGPUOperand::ImmTyD16,   true, nullptr},
   {"dmask",   AMDGPUOperand::ImmTyDMask, false, nullptr},
   {"dim",     AMDGPUOperand::ImmTyDim,   false, nullptr},
-  {"row_mask",   AMDGPUOperand::ImmTyDppRowMask, false, nullptr},
-  {"bank_mask",  AMDGPUOperand::ImmTyDppBankMask, false, nullptr},
-  {"bound_ctrl", AMDGPUOperand::ImmTyDppBoundCtrl, false, ConvertBoundCtrl},
-  {"fi",         AMDGPUOperand::ImmTyDppFi, false, nullptr},
   {"dst_sel",    AMDGPUOperand::ImmTySdwaDstSel, false, nullptr},
   {"src0_sel",   AMDGPUOperand::ImmTySdwaSrc0Sel, false, nullptr},
   {"src1_sel",   AMDGPUOperand::ImmTySdwaSrc1Sel, false, nullptr},
   {"dst_unused", AMDGPUOperand::ImmTySdwaDstUnused, false, nullptr},
   {"compr", AMDGPUOperand::ImmTyExpCompr, true, nullptr },
   {"vm", AMDGPUOperand::ImmTyExpVM, true, nullptr},
   {"op_sel", AMDGPUOperand::ImmTyOpSel, false, nullptr},
   {"op_sel_hi", AMDGPUOperand::ImmTyOpSelHi, false, nullptr},
   {"neg_lo", AMDGPUOperand::ImmTyNegLo, false, nullptr},
   {"neg_hi", AMDGPUOperand::ImmTyNegHi, false, nullptr},
+  {"dpp8",     AMDGPUOperand::ImmTyDPP8, false, nullptr},
+  {"dpp_ctrl", AMDGPUOperand::ImmTyDppCtrl, false, nullptr},
+  {"row_mask",   AMDGPUOperand::ImmTyDppRowMask, false, nullptr},
+  {"bank_mask",  AMDGPUOperand::ImmTyDppBankMask, false, nullptr},
+  {"bound_ctrl", AMDGPUOperand::ImmTyDppBoundCtrl, false, ConvertBoundCtrl},
+  {"fi",   AMDGPUOperand::ImmTyDppFi, false, nullptr},
   {"blgp", AMDGPUOperand::ImmTyBLGP, false, nullptr},
   {"cbsz", AMDGPUOperand::ImmTyCBSZ, false, nullptr},
   {"abid", AMDGPUOperand::ImmTyABID, false, nullptr},
   {"wait_vdst", AMDGPUOperand::ImmTyWaitVDST, false, nullptr},
   {"wait_exp", AMDGPUOperand::ImmTyWaitEXP, false, nullptr}
 };
 
 void AMDGPUAsmParser::onBeginOfFile() {
if (Op.IsBit) {
                Op.Type == AMDGPUOperand::ImmTyNegLo ||
                Op.Type == AMDGPUOperand::ImmTyNegHi) {
       res = parseOperandArrayWithPrefix(Op.Name, Operands, Op.Type,
                                         Op.ConvertResult);
     } else if (Op.Type == AMDGPUOperand::ImmTyDim) {
       res = parseDim(Operands);
     } else if (Op.Type == AMDGPUOperand::ImmTyCPol) {
       res = parseCPol(Operands);
+    } else if (Op.Type == AMDGPUOperand::ImmTyDPP8) {
+      res = parseDPP8(Operands);
+    } else if (Op.Type == AMDGPUOperand::ImmTyDppCtrl) {
+      res = parseDPPCtrl(Operands);
     } else {
       res = parseIntWithPrefix(Op.Name, Operands, Op.Type, Op.ConvertResult);
       if (Op.Type == AMDGPUOperand::ImmTyBLGP && res == MatchOperand_NoMatch) {
         res = parseOperandArrayWithPrefix("neg", Operands,
                                           AMDGPUOperand::ImmTyBLGP,
                                           nullptr);
       }
     }

 AMDGPUOperand::Ptr AMDGPUAsmParser::defaultBoundCtrl() const {
   return AMDGPUOperand::CreateImm(this, 0, SMLoc(), AMDGPUOperand::ImmTyDppBoundCtrl);
 }
 
 AMDGPUOperand::Ptr AMDGPUAsmParser::defaultFI() const {
   return AMDGPUOperand::CreateImm(this, 0, SMLoc(), AMDGPUOperand::ImmTyDppFi);
 }
 
+void AMDGPUAsmParser::cvtVOP3DPP(MCInst &Inst, const OperandVector &Operands, bool IsDPP8) {
+  OptionalImmIndexMap OptionalIdx;
+  unsigned Opc = Inst.getOpcode();
+  bool HasModifiers = AMDGPU::getNamedOperandIdx(Opc, AMDGPU::OpName::src0_modifiers) != -1;
+  unsigned I = 1;
+  const MCInstrDesc &Desc = MII.get(Inst.getOpcode());
+  for (unsigned J = 0; J < Desc.getNumDefs(); ++J) {
+    ((AMDGPUOperand &)*Operands[I++]).addRegOperands(Inst, 1);
+  }
+
+  int Fi = 0;
+  for (unsigned E = Operands.size(); I != E; ++I) {
+    auto TiedTo = Desc.getOperandConstraint(Inst.getNumOperands(),
+                                            MCOI::TIED_TO);
+    if (TiedTo != -1) {
+      assert((unsigned)TiedTo < Inst.getNumOperands());
+      // handle tied old or src2 for MAC instructions
+      Inst.addOperand(Inst.getOperand(TiedTo));
+    }
+    AMDGPUOperand &Op = ((AMDGPUOperand &)*Operands[I]);
+    // Add the register arguments
+    if (IsDPP8 && Op.isFI()) {
+      Fi = Op.getImm();
+    } else if (HasModifiers &&
+               isRegOrImmWithInputMods(Desc, Inst.getNumOperands())) {
+      Op.addRegOrImmWithFPInputModsOperands(Inst, 2);
+    } else if (Op.isReg()) {
+      Op.addRegOperands(Inst, 1);
+    } else if (Op.isImm() &&
+               Desc.OpInfo[Inst.getNumOperands()].RegClass != -1) {
+      assert(!HasModifiers && "Case should be unreachable with modifiers");
+      assert(!Op.IsImmKindLiteral() && "Cannot use literal with DPP");
+      Op.addImmOperands(Inst, 1);
+    } else if (Op.isImm()) {
+      OptionalIdx[Op.getImmTy()] = I;
+    } else {
+      llvm_unreachable("unhandled operand type");
+    }
+  }
+  if (AMDGPU::getNamedOperandIdx(Opc, AMDGPU::OpName::clamp) != -1) {
+    addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyClampSI);
+  }
+  if (AMDGPU::getNamedOperandIdx(Opc, AMDGPU::OpName::omod) != -1) {
+    addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyOModSI);
+  }
+  if (AMDGPU::getNamedOperandIdx(Opc, AMDGPU::OpName::op_sel) != -1) {
+    addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyOpSel);
+  }
+
+  if (IsDPP8) {
+    addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyDPP8);
+    using namespace llvm::AMDGPU::DPP;
+    Inst.addOperand(MCOperand::createImm(Fi? DPP8_FI_1 : DPP8_FI_0));
+  } else {
+    addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyDppCtrl, 0xe4);
+    addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyDppRowMask, 0xf);
+    addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyDppBankMask, 0xf);
+    addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyDppBoundCtrl);
+    if (AMDGPU::getNamedOperandIdx(Inst.getOpcode(), AMDGPU::OpName::fi) != -1) {
+      addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyDppFi);
+    }
+  }
+}
+
 void AMDGPUAsmParser::cvtDPP(MCInst &Inst, const OperandVector &Operands, bool IsDPP8) {
   OptionalImmIndexMap OptionalIdx;
 
   unsigned Opc = Inst.getOpcode();
   bool HasModifiers =
       AMDGPU::getNamedOperandIdx(Opc, AMDGPU::OpName::src0_modifiers) != -1;
   unsigned I = 1;
   const MCInstrDesc &Desc = MII.get(Inst.getOpcode());

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.h

 ///
 /// This file contains declaration for AMDGPU ISA disassembler
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_AMDGPU_DISASSEMBLER_AMDGPUDISASSEMBLER_H
 #define LLVM_LIB_TARGET_AMDGPU_DISASSEMBLER_AMDGPUDISASSEMBLER_H
 
+#include "llvm/ADT/APInt.h"
 #include "llvm/MC/MCDisassembler/MCDisassembler.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/Support/DataExtractor.h"
 #include <memory>
 
 namespace llvm {
 
 class MCInst;
 class MCOperand;
 class MCSubtargetInfo;
 class Twine;
 
+// Exposes an interface expected by autogenerated code in
+// FixedLenDecoderEmitter
+class DecoderUInt128 {
+private:
+  uint64_t Lo = 0;
+  uint64_t Hi = 0;
+
+public:
+  DecoderUInt128() = default;
+  DecoderUInt128(uint64_t Lo, uint64_t Hi = 0) : Lo(Lo), Hi(Hi) {}
+  operator bool() const { return Lo || Hi; }
+  void insertBits(uint64_t SubBits, unsigned BitPosition, unsigned NumBits) {
+    assert(NumBits && NumBits <= 64);
+    assert(SubBits >> 1 >> (NumBits - 1) == 0);
+    assert(BitPosition < 128);
+    if (BitPosition < 64) {
+      Lo |= SubBits << BitPosition;
+      Hi |= SubBits >> 1 >> (63 - BitPosition);
+    } else {
+      Hi |= SubBits << (BitPosition - 64);
+    }
+  }
+  uint64_t extractBitsAsZExtValue(unsigned NumBits,
+                                  unsigned BitPosition) const {
+    assert(NumBits && NumBits <= 64);
+    assert(BitPosition < 128);
+    uint64_t Val;
+    if (BitPosition < 64)
+      Val = Lo >> BitPosition | Hi << 1 << (63 - BitPosition);
+    else
+      Val = Hi >> (BitPosition - 64);
+    return Val & ((uint64_t(2) << (NumBits - 1)) - 1);
+  }
+  DecoderUInt128 operator&(const DecoderUInt128 &RHS) const {
+    return DecoderUInt128(Lo & RHS.Lo, Hi & RHS.Hi);
+  }
+  DecoderUInt128 operator&(const uint64_t &RHS) const {
+    return *this & DecoderUInt128(RHS);
+  }
+  DecoderUInt128 operator~() const { return DecoderUInt128(~Lo, ~Hi); }
+  bool operator==(const DecoderUInt128 &RHS) {
+    return Lo == RHS.Lo && Hi == RHS.Hi;
+  }
+  bool operator!=(const DecoderUInt128 &RHS) {
+    return Lo != RHS.Lo || Hi != RHS.Hi;
+  }
+  bool operator!=(const int &RHS) {
+    return *this != DecoderUInt128(RHS);
+  }
+  friend raw_ostream &operator<<(raw_ostream &OS, const DecoderUInt128 &RHS) {
+    return OS << APInt(128, {RHS.Lo, RHS.Hi});
+  }
+};
+
 //===----------------------------------------------------------------------===//
 // AMDGPUDisassembler
 //===----------------------------------------------------------------------===//
 
 class AMDGPUDisassembler : public MCDisassembler {
 private:
   std::unique_ptr<MCInstrInfo const> const MCII;
   const MCRegisterInfo &MRI;

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

 
 template <typename T> static inline T eatBytes(ArrayRef<uint8_t>& Bytes) {
   assert(Bytes.size() >= sizeof(T));
   const auto Res = support::endian::read<T, support::endianness::little>(Bytes.data());
   Bytes = Bytes.slice(sizeof(T));
   return Res;
 }
 
+static inline DecoderUInt128 eat12Bytes(ArrayRef<uint8_t> &Bytes) {
+  assert(Bytes.size() >= 12);
+  uint64_t Lo = support::endian::read<uint64_t, support::endianness::little>(
+      Bytes.data());
+  Bytes = Bytes.slice(8);
+  uint64_t Hi = support::endian::read<uint32_t, support::endianness::little>(
+      Bytes.data());
+  Bytes = Bytes.slice(4);
+  return DecoderUInt128(Lo, Hi);
+}
+
 // The disassembler is greedy, so we need to check FI operand value to
 // not parse a dpp if the correct literal is not set. For dpp16 the
 // autogenerated decoder checks the dpp literal
 static bool isValidDPP8(const MCInst &MI) {
   using namespace llvm::AMDGPU::DPP;
   int FiIdx = AMDGPU::getNamedOperandIdx(MI.getOpcode(), AMDGPU::OpName::fi);
   assert(FiIdx != -1);
   if ((unsigned)FiIdx >= MI.getNumOperands())
DecodeStatus AMDGPUDisassembler::getInstruction(MCInst &MI, uint64_t &Size,
 
   DecodeStatus Res = MCDisassembler::Fail;
   do {
     // ToDo: better to switch encoding length using some bit predicate
     // but it is unknown yet, so try all we can
 
     // Try to decode DPP and SDWA first to solve conflict with VOP1 and VOP2
     // encodings
+    if (isGFX11Plus() && Bytes.size() >= 12 ) {
+      DecoderUInt128 DecW = eat12Bytes(Bytes);
+      Res = tryDecodeInst(DecoderTableDPP8GFX1196, MI, DecW,
+                                          Address);
+      if (Res && convertDPP8Inst(MI) == MCDisassembler::Success)
+        break;
+      MI = MCInst(); // clear
+      Res = tryDecodeInst(DecoderTableDPPGFX1196, MI, DecW,
+                                          Address);
+      if (Res)
+        break;
+    }
+    // Reinitialize Bytes
+    Bytes = Bytes_.slice(0, MaxInstBytesNum);
+
     if (Bytes.size() >= 8) {
       const uint64_t QW = eatBytes<uint64_t>(Bytes);
 
       if (STI.getFeatureBits()[AMDGPU::FeatureGFX10_BEncoding]) {
         Res = tryDecodeInst(DecoderTableGFX10_B64, MI, QW, Address);
         if (Res) {
           if (AMDGPU::getNamedOperandIdx(MI.getOpcode(), AMDGPU::OpName::dpp8)
               == -1)

llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUInstPrinter.cpp

#endif
   if (!Keep16BitSuffixes)
     if (!RegName.consume_back(".l"))
       RegName.consume_back(".h");
 
   O << RegName;
 }
 
 void AMDGPUInstPrinter::printVOPDst(const MCInst *MI, unsigned OpNo,
-                                    const MCSubtargetInfo &STI,
-                                    raw_ostream &O) {
+                                    const MCSubtargetInfo &STI, raw_ostream &O) {
   auto Opcode = MI->getOpcode();
   auto Flags = MII.get(Opcode).TSFlags;
-
   if (OpNo == 0) {
-    if (Flags & SIInstrFlags::VOP3) {
+    if (Flags & SIInstrFlags::VOP3 && Flags & SIInstrFlags::DPP)
+      O << "_e64_dpp";
+    else if (Flags & SIInstrFlags::VOP3) {
       if (!getVOP3IsSingle(Opcode))
         O << "_e64";
-    } else if (Flags & SIInstrFlags::DPP) {
+    } else if (Flags & SIInstrFlags::DPP)
       O << "_dpp";
-    } else if (Flags & SIInstrFlags::SDWA) {
+    else if (Flags & SIInstrFlags::SDWA)
       O << "_sdwa";
-    } else if (((Flags & SIInstrFlags::VOP1) && !getVOP1IsSingle(Opcode)) ||
-               ((Flags & SIInstrFlags::VOP2) && !getVOP2IsSingle(Opcode))) {
+    else if (((Flags & SIInstrFlags::VOP1) && !getVOP1IsSingle(Opcode)) ||
+             ((Flags & SIInstrFlags::VOP2) && !getVOP2IsSingle(Opcode)))
       O << "_e32";
-    }
     O << " ";
   }
 
   printOperand(MI, OpNo, STI, O);
 
   // Print default vcc/vcc_lo operand.
   switch (Opcode) {
   default: break;

llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUMCCodeEmitter.h

 //===-- AMDGPUCodeEmitter.h - AMDGPU Code Emitter interface -----*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 /// \file
 /// CodeEmitter interface for SI codegen.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_AMDGPU_MCTARGETDESC_AMDGPUMCCODEEMITTER_H
 #define LLVM_LIB_TARGET_AMDGPU_MCTARGETDESC_AMDGPUMCCODEEMITTER_H
 
+#include "llvm/ADT/APInt.h"
 #include "llvm/MC/MCCodeEmitter.h"
 #include <cstdint>
 
 namespace llvm {
 
 class MCInst;
 class MCInstrInfo;
 class MCOperand;
 class MCSubtargetInfo;
 class FeatureBitset;
 
 class AMDGPUMCCodeEmitter : public MCCodeEmitter {
   virtual void anchor();
 
 protected:
   const MCInstrInfo &MCII;
 
   AMDGPUMCCodeEmitter(const MCInstrInfo &mcii) : MCII(mcii) {}
 
 public:
-
-  uint64_t getBinaryCodeForInstr(const MCInst &MI,
-                                 SmallVectorImpl<MCFixup> &Fixups,
-                                 const MCSubtargetInfo &STI) const;
+  void getBinaryCodeForInstr(const MCInst &MI, SmallVectorImpl<MCFixup> &Fixups,
+                             APInt &Inst, APInt &Scratch,
 
-  virtual uint64_t getMachineOpValue(const MCInst &MI, const MCOperand &MO,
-                                     SmallVectorImpl<MCFixup> &Fixups,
-                                     const MCSubtargetInfo &STI) const {
-    return 0;
-  }
+  virtual void getMachineOpValue(const MCInst &MI, const MCOperand &MO,
+                                 APInt &Op, SmallVectorImpl<MCFixup> &Fixups,
+                                 const MCSubtargetInfo &STI) const = 0;
 
-  virtual unsigned getSOPPBrEncoding(const MCInst &MI, unsigned OpNo,
-                                     SmallVectorImpl<MCFixup> &Fixups,
-                                     const MCSubtargetInfo &STI) const {
-    return 0;
-  }
+  virtual void getSOPPBrEncoding(const MCInst &MI, unsigned OpNo, APInt &Op,
+                                 const MCSubtargetInfo &STI) const = 0;
 
-  virtual unsigned getSMEMOffsetEncoding(const MCInst &MI, unsigned OpNo,
-                                         SmallVectorImpl<MCFixup> &Fixups,
-                                         const MCSubtargetInfo &STI) const {
-    return 0;
-  }
+  virtual void getSMEMOffsetEncoding(const MCInst &MI, unsigned OpNo, APInt &Op,
+                                     const MCSubtargetInfo &STI) const = 0;
 
-  virtual unsigned getSDWASrcEncoding(const MCInst &MI, unsigned OpNo,
-                                      SmallVectorImpl<MCFixup> &Fixups,
-                                      const MCSubtargetInfo &STI) const {
-    return 0;
-  }
+  virtual void getSDWASrcEncoding(const MCInst &MI, unsigned OpNo, APInt &Op,
+                                  const MCSubtargetInfo &STI) const = 0;
 
-  virtual unsigned getSDWAVopcDstEncoding(const MCInst &MI, unsigned OpNo,
-                                          SmallVectorImpl<MCFixup> &Fixups,
-                                          const MCSubtargetInfo &STI) const {
-    return 0;
-  }
+  virtual void getSDWAVopcDstEncoding(const MCInst &MI, unsigned OpNo,
+                                      APInt &Op,
+                                      const MCSubtargetInfo &STI) const = 0;
 
-  virtual unsigned getAVOperandEncoding(const MCInst &MI, unsigned OpNo,
-                                        SmallVectorImpl<MCFixup> &Fixups,
-                                        const MCSubtargetInfo &STI) const {
-    return 0;
-  }
+  virtual void getAVOperandEncoding(const MCInst &MI, unsigned OpNo, APInt &Op,
+                                    const MCSubtargetInfo &STI) const = 0;
 
 protected:
   FeatureBitset computeAvailableFeatures(const FeatureBitset &FB) const;
   void
   verifyInstructionPredicates(const MCInst &MI,
                               const FeatureBitset &AvailableFeatures) const;
 };
 
 } // End namespace llvm
 
 #endif

llvm/lib/Target/AMDGPU/MCTargetDesc/SIMCCodeEmitter.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "MCTargetDesc/AMDGPUFixupKinds.h"
 #include "MCTargetDesc/AMDGPUMCCodeEmitter.h"
 #include "MCTargetDesc/AMDGPUMCTargetDesc.h"
 #include "SIDefines.h"
 #include "Utils/AMDGPUBaseInfo.h"
+#include "llvm/ADT/APInt.h"
+#include "llvm/MC/MCCodeEmitter.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/Support/Casting.h"
 
public:
   SIMCCodeEmitter &operator=(const SIMCCodeEmitter &) = delete;
 
   /// Encode the instruction and write it to the OS.
   void encodeInstruction(const MCInst &MI, raw_ostream &OS,
                          SmallVectorImpl<MCFixup> &Fixups,
                          const MCSubtargetInfo &STI) const override;
 
   /// \returns the encoding for an MCOperand.
-  uint64_t getMachineOpValue(const MCInst &MI, const MCOperand &MO,
-                             SmallVectorImpl<MCFixup> &Fixups,
-                             const MCSubtargetInfo &STI) const override;
+  void getMachineOpValue(const MCInst &MI, const MCOperand &MO, APInt &Op,
 
   /// Use a fixup to encode the simm16 field for SOPP branch
   ///        instructions.
-  unsigned getSOPPBrEncoding(const MCInst &MI, unsigned OpNo,
-                             SmallVectorImpl<MCFixup> &Fixups,
-                             const MCSubtargetInfo &STI) const override;
+  void getSOPPBrEncoding(const MCInst &MI, unsigned OpNo, APInt &Op,
 
-  unsigned getSMEMOffsetEncoding(const MCInst &MI, unsigned OpNo,
-                                 SmallVectorImpl<MCFixup> &Fixups,
-                                 const MCSubtargetInfo &STI) const override;
+  void getSMEMOffsetEncoding(const MCInst &MI, unsigned OpNo, APInt &Op,
 
-  unsigned getSDWASrcEncoding(const MCInst &MI, unsigned OpNo,
-                              SmallVectorImpl<MCFixup> &Fixups,
-                              const MCSubtargetInfo &STI) const override;
+  void getSDWASrcEncoding(const MCInst &MI, unsigned OpNo, APInt &Op,
 
-  unsigned getSDWAVopcDstEncoding(const MCInst &MI, unsigned OpNo,
-                                  SmallVectorImpl<MCFixup> &Fixups,
-                                  const MCSubtargetInfo &STI) const override;
+  void getSDWAVopcDstEncoding(const MCInst &MI, unsigned OpNo, APInt &Op,
 
-  unsigned getAVOperandEncoding(const MCInst &MI, unsigned OpNo,
-                                SmallVectorImpl<MCFixup> &Fixups,
-                                const MCSubtargetInfo &STI) const override;
+  void getAVOperandEncoding(const MCInst &MI, unsigned OpNo, APInt &Op,
 
 private:
   uint64_t getImplicitOpSelHiEncoding(int Opcode) const;
+  void getMachineOpValueCommon(const MCInst &MI, const MCOperand &MO,
+                               unsigned OpNo, APInt &Op,
+                               SmallVectorImpl<MCFixup> &Fixups,
+                               const MCSubtargetInfo &STI) const;
 };
 
 } // end anonymous namespace
 
 MCCodeEmitter *llvm::createSIMCCodeEmitter(const MCInstrInfo &MCII,
                                            MCContext &Ctx) {
   return new SIMCCodeEmitter(MCII, Ctx);
 }

 
 void SIMCCodeEmitter::encodeInstruction(const MCInst &MI, raw_ostream &OS,
                                        SmallVectorImpl<MCFixup> &Fixups,
                                        const MCSubtargetInfo &STI) const {
   verifyInstructionPredicates(MI,
                               computeAvailableFeatures(STI.getFeatureBits()));
 
   int Opcode = MI.getOpcode();
-  uint64_t Encoding = getBinaryCodeForInstr(MI, Fixups, STI);
-  const MCInstrDesc &Desc = MCII.get(Opcode);
+  APInt Encoding, Scratch;
+  getBinaryCodeForInstr(MI, Fixups, Encoding, Scratch,  STI);
+  const MCInstrDesc &Desc = MCII.get(MI.getOpcode());
   unsigned bytes = Desc.getSize();
 
   // Set unused op_sel_hi bits to 1 for VOP3P and MAI instructions.
   // Note that accvgpr_read/write are MAI, have src0, but do not use op_sel.
   if ((Desc.TSFlags & SIInstrFlags::VOP3P) ||
       Opcode == AMDGPU::V_ACCVGPR_READ_B32_vi ||
       Opcode == AMDGPU::V_ACCVGPR_WRITE_B32_vi) {
     Encoding |= getImplicitOpSelHiEncoding(Opcode);
   }
 
   for (unsigned i = 0; i < bytes; i++) {
-    OS.write((uint8_t) ((Encoding >> (8 * i)) & 0xff));
+    OS.write((uint8_t)Encoding.extractBitsAsZExtValue(8, 8 * i));
   }
 
   // NSA encoding.
   if (AMDGPU::isGFX10Plus(STI) && Desc.TSFlags & SIInstrFlags::MIMG) {
     int vaddr0 = AMDGPU::getNamedOperandIdx(MI.getOpcode(),
                                             AMDGPU::OpName::vaddr0);
     int srsrc = AMDGPU::getNamedOperandIdx(MI.getOpcode(),
                                            AMDGPU::OpName::srsrc);
     assert(vaddr0 >= 0 && srsrc > vaddr0);
     unsigned NumExtraAddrs = srsrc - vaddr0 - 1;
     unsigned NumPadding = (-NumExtraAddrs) & 3;
 
-    for (unsigned i = 0; i < NumExtraAddrs; ++i)
-      OS.write((uint8_t)getMachineOpValue(MI, MI.getOperand(vaddr0 + 1 + i),
-                                          Fixups, STI));
+    for (unsigned i = 0; i < NumExtraAddrs; ++i) {
+      getMachineOpValue(MI, MI.getOperand(vaddr0 + 1 + i), Encoding, Fixups,
+                        STI);
+      OS.write((uint8_t)Encoding.getLimitedValue());
+    }
     for (unsigned i = 0; i < NumPadding; ++i)
       OS.write(0);
   }
 
   if ((bytes > 8 && STI.getFeatureBits()[AMDGPU::FeatureVOP3Literal]) ||
       (bytes > 4 && !STI.getFeatureBits()[AMDGPU::FeatureVOP3Literal]))
     return;
 
for (unsigned j = 0; j < 4; j++) {
       OS.write((uint8_t) ((Imm >> (8 * j)) & 0xff));
     }
 
     // Only one literal value allowed
     break;
   }
 }
 
-unsigned SIMCCodeEmitter::getSOPPBrEncoding(const MCInst &MI, unsigned OpNo,
-                                            SmallVectorImpl<MCFixup> &Fixups,
-                                            const MCSubtargetInfo &STI) const {
+void SIMCCodeEmitter::getSOPPBrEncoding(const MCInst &MI, unsigned OpNo,
+                                        APInt &Op,
   const MCOperand &MO = MI.getOperand(OpNo);
 
   if (MO.isExpr()) {
     const MCExpr *Expr = MO.getExpr();
     MCFixupKind Kind = (MCFixupKind)AMDGPU::fixup_si_sopp_br;
     Fixups.push_back(MCFixup::create(0, Expr, Kind, MI.getLoc()));
-    return 0;
+    Op = APInt::getNullValue(96);
+  } else {
+    getMachineOpValue(MI, MO, Op, Fixups, STI);
   }
-
-  return getMachineOpValue(MI, MO, Fixups, STI);
 }
 
-unsigned SIMCCodeEmitter::getSMEMOffsetEncoding(const MCInst &MI, unsigned OpNo,
-                                                SmallVectorImpl<MCFixup> &Fixups,
-                                                const MCSubtargetInfo &STI) const {
+void SIMCCodeEmitter::getSMEMOffsetEncoding(const MCInst &MI, unsigned OpNo,
+                                            APInt &Op,
   auto Offset = MI.getOperand(OpNo).getImm();
   // VI only supports 20-bit unsigned offsets.
   assert(!AMDGPU::isVI(STI) || isUInt<20>(Offset));
-  return Offset;
+  Op = Offset;
 }
 
-unsigned
-SIMCCodeEmitter::getSDWASrcEncoding(const MCInst &MI, unsigned OpNo,
+void SIMCCodeEmitter::getSDWASrcEncoding(const MCInst &MI, unsigned OpNo,
+                                         APInt &Op,
                                          SmallVectorImpl<MCFixup> &Fixups,
                                          const MCSubtargetInfo &STI) const {
   using namespace AMDGPU::SDWA;
 
   uint64_t RegEnc = 0;
 
   const MCOperand &MO = MI.getOperand(OpNo);
 
   if (MO.isReg()) {
     unsigned Reg = MO.getReg();
     RegEnc |= MRI.getEncodingValue(Reg);
     RegEnc &= SDWA9EncValues::SRC_VGPR_MASK;
     if (AMDGPU::isSGPR(AMDGPU::mc2PseudoReg(Reg), &MRI)) {
       RegEnc |= SDWA9EncValues::SRC_SGPR_MASK;
     }
-    return RegEnc;
+    Op = RegEnc;
+    return;
   } else {
     const MCInstrDesc &Desc = MCII.get(MI.getOpcode());
     uint32_t Enc = getLitEncoding(MO, Desc.OpInfo[OpNo], STI);
     if (Enc != ~0U && Enc != 255) {
-      return Enc | SDWA9EncValues::SRC_SGPR_MASK;
+      Op = Enc | SDWA9EncValues::SRC_SGPR_MASK;
+      return;
     }
   }
 
   llvm_unreachable("Unsupported operand kind");
-  return 0;
 }
 
-unsigned
-SIMCCodeEmitter::getSDWAVopcDstEncoding(const MCInst &MI, unsigned OpNo,
+void SIMCCodeEmitter::getSDWAVopcDstEncoding(const MCInst &MI, unsigned OpNo,
+                                             APInt &Op,
                                              SmallVectorImpl<MCFixup> &Fixups,
                                              const MCSubtargetInfo &STI) const {
   using namespace AMDGPU::SDWA;
 
   uint64_t RegEnc = 0;
 
   const MCOperand &MO = MI.getOperand(OpNo);
 
   unsigned Reg = MO.getReg();
   if (Reg != AMDGPU::VCC && Reg != AMDGPU::VCC_LO) {
     RegEnc |= MRI.getEncodingValue(Reg);
     RegEnc &= SDWA9EncValues::VOPC_DST_SGPR_MASK;
     RegEnc |= SDWA9EncValues::VOPC_DST_VCC_MASK;
   }
-  return RegEnc;
+  Op = RegEnc;
 }
 
-unsigned
-SIMCCodeEmitter::getAVOperandEncoding(const MCInst &MI, unsigned OpNo,
+void SIMCCodeEmitter::getAVOperandEncoding(const MCInst &MI, unsigned OpNo,
+                                           APInt &Op,
                                            SmallVectorImpl<MCFixup> &Fixups,
                                            const MCSubtargetInfo &STI) const {
   unsigned Reg = MI.getOperand(OpNo).getReg();
   uint64_t Enc = MRI.getEncodingValue(Reg);
 
   // VGPR and AGPR have the same encoding, but SrcA and SrcB operands of mfma
   // instructions use acc[0:1] modifier bits to distinguish. These bits are
   // encoded as a virtual 9th bit of the register for these operands.
   if (MRI.getRegClass(AMDGPU::AGPR_32RegClassID).contains(Reg) ||
       MRI.getRegClass(AMDGPU::AReg_64RegClassID).contains(Reg) ||
       MRI.getRegClass(AMDGPU::AReg_96RegClassID).contains(Reg) ||
       MRI.getRegClass(AMDGPU::AReg_128RegClassID).contains(Reg) ||
       MRI.getRegClass(AMDGPU::AReg_160RegClassID).contains(Reg) ||
       MRI.getRegClass(AMDGPU::AReg_192RegClassID).contains(Reg) ||
       MRI.getRegClass(AMDGPU::AReg_224RegClassID).contains(Reg) ||
       MRI.getRegClass(AMDGPU::AReg_256RegClassID).contains(Reg) ||
       MRI.getRegClass(AMDGPU::AReg_512RegClassID).contains(Reg) ||
       MRI.getRegClass(AMDGPU::AGPR_LO16RegClassID).contains(Reg))
     Enc |= 512;
 
-  return Enc;
+  Op = Enc;
 }
 
 static bool needsPCRel(const MCExpr *Expr) {
   switch (Expr->getKind()) {
   case MCExpr::SymbolRef: {
     auto *SE = cast<MCSymbolRefExpr>(Expr);
     MCSymbolRefExpr::VariantKind Kind = SE->getKind();
     return Kind != MCSymbolRefExpr::VK_AMDGPU_ABS32_LO &&
case MCExpr::Unary:
     return needsPCRel(cast<MCUnaryExpr>(Expr)->getSubExpr());
   case MCExpr::Target:
   case MCExpr::Constant:
     return false;
   }
   llvm_unreachable("invalid kind");
 }
 
-uint64_t SIMCCodeEmitter::getMachineOpValue(const MCInst &MI,
-                                            const MCOperand &MO,
+void SIMCCodeEmitter::getMachineOpValue(const MCInst &MI,
+                                        const MCOperand &MO, APInt &Op,
                                         SmallVectorImpl<MCFixup> &Fixups,
                                         const MCSubtargetInfo &STI) const {
-  if (MO.isReg())
-    return MRI.getEncodingValue(MO.getReg());
+  if (MO.isReg()){
+    Op = MRI.getEncodingValue(MO.getReg());
+    return;
+  }
+  unsigned OpNo = &MO - MI.begin();
+  getMachineOpValueCommon(MI, MO, OpNo, Op, Fixups, STI);
+}
+
+void SIMCCodeEmitter::getMachineOpValueCommon(
+    const MCInst &MI, const MCOperand &MO, unsigned OpNo, APInt &Op,
+    SmallVectorImpl<MCFixup> &Fixups, const MCSubtargetInfo &STI) const {
 
   if (MO.isExpr() && MO.getExpr()->getKind() != MCExpr::Constant) {
     // FIXME: If this is expression is PCRel or not should not depend on what
     // the expression looks like. Given that this is just a general expression,
     // it should probably be FK_Data_4 and whatever is producing
     //
     //    s_add_u32 s2, s2, (extern_const_addrspace+16
     //
     // And expecting a PCRel should instead produce
     //
     // .Ltmp1:
     //   s_add_u32 s2, s2, (extern_const_addrspace+16)-.Ltmp1
     MCFixupKind Kind;
     if (needsPCRel(MO.getExpr()))
       Kind = FK_PCRel_4;
     else
       Kind = FK_Data_4;
 
     const MCInstrDesc &Desc = MCII.get(MI.getOpcode());
     uint32_t Offset = Desc.getSize();
     assert(Offset == 4 || Offset == 8);
 
-    Fixups.push_back(
-      MCFixup::create(Offset, MO.getExpr(), Kind, MI.getLoc()));
-  }
-
-  // Figure out the operand number, needed for isSrcOperand check
-  unsigned OpNo = 0;
-  for (unsigned e = MI.getNumOperands(); OpNo < e; ++OpNo) {
-    if (&MO == &MI.getOperand(OpNo))
-      break;
+    Fixups.push_back(MCFixup::create(Offset, MO.getExpr(), Kind, MI.getLoc()));
   }
 
   const MCInstrDesc &Desc = MCII.get(MI.getOpcode());
   if (AMDGPU::isSISrcOperand(Desc, OpNo)) {
     uint32_t Enc = getLitEncoding(MO, Desc.OpInfo[OpNo], STI);
-    if (Enc != ~0U)
-      return Enc;
-
-  } else if (MO.isImm())
-    return MO.getImm();
+    if (Enc != ~0U) {
+      Op = Enc;
+      return;
+    }
+  } else if (MO.isImm()) {
+    Op = MO.getImm();
+    return;
+  }
 
   llvm_unreachable("Encoding of this operand type is not supported yet.");
-  return 0;
 }
 
 #define ENABLE_INSTR_PREDICATE_VERIFIER
 #include "AMDGPUGenMCCodeEmitter.inc"

llvm/lib/Target/AMDGPU/SIDefines.h

 } // namespace AMDGPU
 
 namespace AMDGPUAsmVariants {
   enum : unsigned {
     DEFAULT = 0,
     VOP3 = 1,
     SDWA = 2,
     SDWA9 = 3,
-    DPP = 4
+    DPP = 4,
+    VOP3_DPP = 5
   };
-}
+} // namespace AMDGPUAsmVariants
 
 namespace AMDGPU {
 namespace EncValues { // Encoding values of enum9/8/7 operands
 
 enum : unsigned {
   SGPR_MIN = 0,
   SGPR_MAX_SI = 101,
   SGPR_MAX_GFX10 = 105,

llvm/lib/Target/AMDGPU/SIInstrFormats.td

class Enc32 {
   int Size = 4;
 }
 
 class Enc64 {
   field bits<64> Inst;
   int Size = 8;
 }
 
+class Enc96 {
+  field bits<96> Inst;
+  int Size = 12;
+}
+
 def CPolBit {
   int GLC = 0;
   int SLC = 1;
   int DLC = 2;
   int SCC = 4;
 }
 
 class VOPDstOperand <RegisterClass rc> : RegisterOperand <rc, "printVOPDst">;

llvm/lib/Target/AMDGPU/SIInstrInfo.td

 def exp_compr : NamedOperandBit<"ExpCompr", NamedMatchClass<"ExpCompr">>;
 def exp_vm : NamedOperandBit<"ExpVM", NamedMatchClass<"ExpVM">>;
 
 def FORMAT : NamedOperandU8<"FORMAT", NamedMatchClass<"FORMAT", 0>>;
 
 def DMask : NamedOperandU16<"DMask", NamedMatchClass<"DMask">>;
 def Dim : NamedOperandU8<"Dim", NamedMatchClass<"Dim", 0>>;
 
-def dpp8 : NamedOperandU32<"DPP8", NamedMatchClass<"DPP8", 0>>;
-
-def dpp_ctrl : NamedOperandU32<"DPPCtrl", NamedMatchClass<"DPPCtrl", 0>>;
-def row_mask : NamedOperandU32<"RowMask", NamedMatchClass<"RowMask">>;
-def bank_mask : NamedOperandU32<"BankMask", NamedMatchClass<"BankMask">>;
-def bound_ctrl : NamedOperandBit<"BoundCtrl", NamedMatchClass<"BoundCtrl">>;
-def FI : NamedOperandU32<"FI", NamedMatchClass<"FI">>;
-
 def dst_sel : NamedOperandU32<"SDWADstSel", NamedMatchClass<"SDWADstSel">>;
 def src0_sel : NamedOperandU32<"SDWASrc0Sel", NamedMatchClass<"SDWASrc0Sel">>;
 def src1_sel : NamedOperandU32<"SDWASrc1Sel", NamedMatchClass<"SDWASrc1Sel">>;
 def dst_unused : NamedOperandU32<"SDWADstUnused", NamedMatchClass<"SDWADstUnused">>;
 
 def op_sel0 : NamedOperandU32Default0<"OpSel", NamedMatchClass<"OpSel">>;
 def op_sel_hi0 : NamedOperandU32Default0<"OpSelHi", NamedMatchClass<"OpSelHi">>;
 def neg_lo0 : NamedOperandU32Default0<"NegLo", NamedMatchClass<"NegLo">>;
 def neg_hi0 : NamedOperandU32Default0<"NegHi", NamedMatchClass<"NegHi">>;
 
+def dpp8 : NamedOperandU32<"DPP8", NamedMatchClass<"DPP8", 0>>;
+def dpp_ctrl : NamedOperandU32<"DPPCtrl", NamedMatchClass<"DPPCtrl", 0>>;
+
+def row_mask : NamedOperandU32<"RowMask", NamedMatchClass<"RowMask">>;
+def bank_mask : NamedOperandU32<"BankMask", NamedMatchClass<"BankMask">>;
+def bound_ctrl : NamedOperandBit<"BoundCtrl", NamedMatchClass<"BoundCtrl">>;
+def FI : NamedOperandU32<"FI", NamedMatchClass<"FI">>;
+
 def blgp : NamedOperandU32<"BLGP", NamedMatchClass<"BLGP">>;
 def cbsz : NamedOperandU32<"CBSZ", NamedMatchClass<"CBSZ">>;
 def abid : NamedOperandU32<"ABID", NamedMatchClass<"ABID">>;
 
 def hwreg : NamedOperandU32<"Hwreg", NamedMatchClass<"Hwreg", 0>>;
 
 def exp_tgt : NamedOperandU32<"ExpTgt", NamedMatchClass<"ExpTgt", 0>> {
 

 def f16kimm : kimmOperand<i16>;
 
 class FPInputModsMatchClass <int opSize> : AsmOperandClass {
   let Name = "RegOrImmWithFP"#opSize#"InputMods";
   let ParserMethod = "parseRegOrImmWithFPInputMods";
   let PredicateMethod = "isRegOrImmWithFP"#opSize#"InputMods";
 }
 
+class FPVCSrcInputModsMatchClass <int opSize> : FPInputModsMatchClass <opSize> {
+  let Name = "RegOrInlineImmWithFP"#opSize#"InputMods";
+  let PredicateMethod = "isRegOrInlineImmWithFP"#opSize#"InputMods";
+}
+
 def FP16InputModsMatchClass : FPInputModsMatchClass<16>;
 def FP32InputModsMatchClass : FPInputModsMatchClass<32>;
 def FP64InputModsMatchClass : FPInputModsMatchClass<64>;
 
+def FP16VCSrcInputModsMatchClass : FPVCSrcInputModsMatchClass<16>;
+def FP32VCSrcInputModsMatchClass : FPVCSrcInputModsMatchClass<32>;
+
 class InputMods <AsmOperandClass matchClass> : Operand <i32> {
   let OperandNamespace = "AMDGPU";
   let OperandType = "OPERAND_INPUT_MODS";
   let ParserMatchClass = matchClass;
 }
 
 class FPInputMods <FPInputModsMatchClass matchClass> : InputMods <matchClass> {
   let PrintMethod = "printOperandAndFPInputMods";
 }
 
 def FP16InputMods : FPInputMods<FP16InputModsMatchClass>;
 def FP32InputMods : FPInputMods<FP32InputModsMatchClass>;
 def FP64InputMods : FPInputMods<FP64InputModsMatchClass>;
 
+def FP16VCSrcInputMods : FPInputMods<FP16VCSrcInputModsMatchClass>;
+def FP32VCSrcInputMods : FPInputMods<FP32VCSrcInputModsMatchClass>;
+
 class IntInputModsMatchClass <int opSize> : AsmOperandClass {
   let Name = "RegOrImmWithInt"#opSize#"InputMods";
   let ParserMethod = "parseRegOrImmWithIntInputMods";
   let PredicateMethod = "isRegOrImmWithInt"#opSize#"InputMods";
 }
+class IntVCSrcInputModsMatchClass <int opSize> : IntInputModsMatchClass <opSize> {
+  let Name = "RegOrInlineImmWithInt"#opSize#"InputMods";
+  let PredicateMethod = "isRegOrInlineImmWithInt"#opSize#"InputMods";
+}
 def Int32InputModsMatchClass : IntInputModsMatchClass<32>;
 def Int64InputModsMatchClass : IntInputModsMatchClass<64>;
+def Int32VCSrcInputModsMatchClass : IntVCSrcInputModsMatchClass<32>;
 
 class IntInputMods <IntInputModsMatchClass matchClass> : InputMods <matchClass> {
   let PrintMethod = "printOperandAndIntInputMods";
 }
 def Int32InputMods : IntInputMods<Int32InputModsMatchClass>;
 def Int64InputMods : IntInputMods<Int64InputModsMatchClass>;
+def Int32VCSrcInputMods : IntInputMods<Int32VCSrcInputModsMatchClass>;
 
 class OpSelModsMatchClass : AsmOperandClass {
   let Name = "OpSelMods";
   let ParserMethod = "parseRegOrImm";
   let PredicateMethod = "isRegOrImm";
 }
 
 def IntOpSelModsMatchClass : OpSelModsMatchClass;
!if(!eq(VT.Size, 64),
                  )
               )
            )
         )
      )
   );
 }
 
+// Src2 of VOP3 DPP instructions cannot be a literal
+class getVOP3DPPSrcForVT<ValueType VT> {
+  bit isFP = isFloatType<VT>.ret;
+  RegisterOperand ret =
+      !if (!eq(VT.Value, i1.Value), SSrc_i1,
+           !if (isFP,
+                !if (!eq(VT.Value, f16.Value), VCSrc_f16,
+                     !if (!eq(VT.Value, v2f16.Value), VCSrc_v2f16, VCSrc_f32)),
+                !if (!eq(VT.Value, i16.Value), VCSrc_b16,
+                     !if (!eq(VT.Value, v2i16.Value), VCSrc_v2b16,
+                          VCSrc_b32))));
+}
+
 // Float or packed int
 class isModifierType<ValueType SrcVT> {
   bit ret = !or(!eq(SrcVT.Value, f16.Value),
                 !eq(SrcVT.Value, f32.Value),
                 !eq(SrcVT.Value, f64.Value),
                 !eq(SrcVT.Value, v2f16.Value),
                 !eq(SrcVT.Value, v2i16.Value),
                 !eq(SrcVT.Value, v2f32.Value),

 }
 
 // Return type of input modifiers operand specified input operand for DPP
 class getSrcModDPP <ValueType VT> {
   bit isFP = isFloatType<VT>.ret;
   Operand ret = !if(isFP, FPVRegInputMods, IntVRegInputMods);
 }
 
+// Return type of input modifiers operand for specified input operand for DPP
+class getSrcModVOP3DPP <ValueType VT, bit EnableF32SrcMods> {
+  bit isFP = isFloatType<VT>.ret;
+  bit isPacked = isPackedType<VT>.ret;
+  Operand ret =
+      !if (isFP,
+           !if (!eq(VT.Value, f16.Value), FP16VCSrcInputMods,
+                FP32VCSrcInputMods),
+           !if (EnableF32SrcMods, FP32VCSrcInputMods, Int32VCSrcInputMods));
+}
+
 // Return type of input modifiers operand specified input operand for SDWA
 class getSrcModSDWA <ValueType VT> {
   Operand ret = !if(!eq(VT.Value, f16.Value), FP16SDWAInputMods,
                 !if(!eq(VT.Value, f32.Value), FP32SDWAInputMods,
                 !if(!eq(VT.Value, i16.Value), Int16SDWAInputMods,
                 Int32SDWAInputMods)));
 }
 
class getInsVOP3OpSel <RegisterOperand Src0RC, RegisterOperand Src1RC,
                        Operand Src0Mod, Operand Src1Mod, Operand Src2Mod> {
   dag ret = getInsVOP3Base<Src0RC, Src1RC,
                     Src2RC, NumSrcArgs,
                     HasClamp, 1/*HasModifiers*/, 1/*HasSrc2Mods*/, HasOMod,
                     Src0Mod, Src1Mod, Src2Mod, 1/*HasOpSel*/, 0>.ret;
 }
 
 class getInsDPPBase <RegisterOperand OldRC, RegisterClass Src0RC, RegisterClass Src1RC,
-                 int NumSrcArgs, bit HasModifiers,
-                 Operand Src0Mod, Operand Src1Mod> {
+                 RegisterClass Src2RC, int NumSrcArgs, bit HasModifiers,
+                 Operand Src0Mod, Operand Src1Mod, Operand Src2Mod> {
 
   dag ret = !if (!eq(NumSrcArgs, 0),
                 // VOP1 without input operands (V_NOP)
                 (ins ),
             !if (!eq(NumSrcArgs, 1),
               !if (HasModifiers,
                 // VOP1_DPP with modifiers
                 (ins OldRC:$old, Src0Mod:$src0_modifiers,
                      Src0RC:$src0)
               /* else */,
                 // VOP1_DPP without modifiers
                 (ins OldRC:$old, Src0RC:$src0)
               /* endif */),
+            !if (!eq(NumSrcArgs, 2),
               !if (HasModifiers,
                 // VOP2_DPP with modifiers
                 (ins OldRC:$old,
                      Src0Mod:$src0_modifiers, Src0RC:$src0,
                      Src1Mod:$src1_modifiers, Src1RC:$src1)
               /* else */,
                 // VOP2_DPP without modifiers
                 (ins OldRC:$old,
                      Src0RC:$src0, Src1RC:$src1)
-                )));
+                )
+              /* NumSrcArgs == 3, VOP3 */,
+              !if (HasModifiers,
+                // VOP3_DPP with modifiers
+                (ins OldRC:$old,
+                     Src0Mod:$src0_modifiers, Src0RC:$src0,
+                     Src1Mod:$src1_modifiers, Src1RC:$src1,
+                     Src2Mod:$src2_modifiers, Src2RC:$src2)
+              /* else */,
+                // VOP3_DPP without modifiers
+                (ins OldRC:$old,
+                     Src0RC:$src0, Src1RC:$src1,
+                     Src2RC:$src2)
+                )
+             /* endif */)));
 }
 
 class getInsDPP <RegisterOperand OldRC, RegisterClass Src0RC, RegisterClass Src1RC,
-                 int NumSrcArgs, bit HasModifiers,
-                 Operand Src0Mod, Operand Src1Mod> {
-  dag ret = !con(getInsDPPBase<OldRC, Src0RC, Src1RC, NumSrcArgs,
-                               HasModifiers, Src0Mod, Src1Mod>.ret,
+                 RegisterClass Src2RC, int NumSrcArgs, bit HasModifiers,
+                 Operand Src0Mod, Operand Src1Mod, Operand Src2Mod> {
+  dag ret = !con(getInsDPPBase<OldRC, Src0RC, Src1RC, Src2RC, NumSrcArgs,
+                           HasModifiers, Src0Mod, Src1Mod, Src2Mod>.ret,
                  (ins dpp_ctrl:$dpp_ctrl, row_mask:$row_mask,
                      bank_mask:$bank_mask, bound_ctrl:$bound_ctrl));
 }
 
 class getInsDPP16 <RegisterOperand OldRC, RegisterClass Src0RC, RegisterClass Src1RC,
-                 int NumSrcArgs, bit HasModifiers,
-                 Operand Src0Mod, Operand Src1Mod> {
-  dag ret = !con(getInsDPP<OldRC, Src0RC, Src1RC, NumSrcArgs,
-                           HasModifiers, Src0Mod, Src1Mod>.ret,
+                 RegisterClass Src2RC, int NumSrcArgs, bit HasModifiers,
+                 Operand Src0Mod, Operand Src1Mod, Operand Src2Mod> {
+  dag ret = !con(getInsDPP<OldRC, Src0RC, Src1RC, Src2RC, NumSrcArgs,
+                           HasModifiers, Src0Mod, Src1Mod, Src2Mod>.ret,
                  (ins FI:$fi));
 }
 
 class getInsDPP8 <RegisterOperand OldRC, RegisterClass Src0RC, RegisterClass Src1RC,
-                 int NumSrcArgs, bit HasModifiers,
-                 Operand Src0Mod, Operand Src1Mod> {
-  dag ret = !con(getInsDPPBase<OldRC, Src0RC, Src1RC, NumSrcArgs,
-                               HasModifiers, Src0Mod, Src1Mod>.ret,
+                 RegisterClass Src2RC, int NumSrcArgs, bit HasModifiers,
+                 Operand Src0Mod, Operand Src1Mod, Operand Src2Mod> {
+  dag ret = !con(getInsDPPBase<OldRC, Src0RC, Src1RC, Src2RC, NumSrcArgs,
+                           HasModifiers, Src0Mod, Src1Mod, Src2Mod>.ret,
                  (ins dpp8:$dpp8, FI:$fi));
 }
 
+class getInsVOP3DPPBase<dag VOP3Base, RegisterOperand OldRC, int NumSrcArgs> {
+  dag old = ( ins OldRC:$old );
+  dag base = VOP3Base;
+  dag ret =  !con(
+                !if(!ne(NumSrcArgs, 0), old, (ins)),
+                base
+              );
+}
+
+class getInsVOP3DPP<dag VOP3Base, RegisterOperand OldRC, int NumSrcArgs> {
+  dag ret = !con(getInsVOP3DPPBase<VOP3Base,OldRC,NumSrcArgs>.ret,
+                 (ins dpp_ctrl:$dpp_ctrl, row_mask:$row_mask,
+                     bank_mask:$bank_mask, bound_ctrl:$bound_ctrl));
+}
+
+class getInsVOP3DPP16<dag VOP3Base, RegisterOperand OldRC, int NumSrcArgs> {
+  dag ret = !con(getInsVOP3DPP<VOP3Base,OldRC,NumSrcArgs>.ret,
+                 (ins FI:$fi));
+}
+
+class getInsVOP3DPP8<dag VOP3Base, RegisterOperand OldRC, int NumSrcArgs> {
+  dag ret = !con(getInsVOP3DPPBase<VOP3Base,OldRC,NumSrcArgs>.ret,
+                 (ins dpp8:$dpp8, FI:$fi));
+}
 
 // Ins for SDWA
 class getInsSDWA <RegisterOperand Src0RC, RegisterOperand Src1RC, int NumSrcArgs,
                   bit HasSDWAOMod, Operand Src0Mod, Operand Src1Mod,
                   ValueType DstVT> {
 
   dag ret = !if(!eq(NumSrcArgs, 0),
                // VOP1 without input operands (V_NOP)
class getAsmDPP <bit HasDst, int NumSrcArgs, bit HasModifiers, ValueType DstVT = i32> {
   string ret = dst#args#" $dpp_ctrl$row_mask$bank_mask$bound_ctrl";
 }
 
 class getAsmDPP16 <bit HasDst, int NumSrcArgs, bit HasModifiers, ValueType DstVT = i32> {
   string ret = getAsmDPP<HasDst, NumSrcArgs, HasModifiers, DstVT>.ret#"$fi";
 }
 
 class getAsmDPP8 <bit HasDst, int NumSrcArgs, bit HasModifiers, ValueType DstVT = i32>
-  : getAsmDPP<HasDst, NumSrcArgs, HasModifiers, DstVT> {
+  : getAsmDPP<HasDst, NumSrcArgs, HasModifiers, DstVT>{
   let ret = dst#args#" $dpp8$fi";
 }
 
+class getAsmVOP3DPPBase <int NumSrcArgs, bit HasDst, bit HasClamp,
+                       bit HasOpSel, bit HasOMod, bit IsVOP3P,
+                       bit HasModifiers, bit Src0HasMods,
+                       bit Src1HasMods, bit Src2HasMods, ValueType DstVT = i32> {
+  string dst = !if(HasDst,
+                   !if(!eq(DstVT.Size, 1),
+                       "$sdst",
+                       "$vdst"),
+                    ""); // use $sdst for VOPC
+  string isrc0 = !if(!eq(NumSrcArgs, 1), "$src0", "$src0,");
+  string isrc1 = !if(!eq(NumSrcArgs, 1), "",
+                     !if(!eq(NumSrcArgs, 2), " $src1",
+                                             " $src1,"));
+  string isrc2 = !if(!eq(NumSrcArgs, 3), " $src2", "");
+
+  string fsrc0 = !if(!eq(NumSrcArgs, 1), "$src0_modifiers", "$src0_modifiers,");
+  string fsrc1 = !if(!eq(NumSrcArgs, 1), "",
+                     !if(!eq(NumSrcArgs, 2), " $src1_modifiers",
+                                             " $src1_modifiers,"));
+  string fsrc2 = !if(!eq(NumSrcArgs, 3), " $src2_modifiers", "");
+
+  string src0 = !if(Src0HasMods, fsrc0, isrc0);
+  string src1 = !if(Src1HasMods, fsrc1, isrc1);
+  string src2 = !if(Src2HasMods, fsrc2, isrc2);
+  string opsel = !if(HasOpSel, "$op_sel", "");
+  string 3PMods = !if(IsVOP3P,
+                      !if(HasOpSel, "$op_sel_hi", "")
+                        #!if(HasModifiers, "$neg_lo$neg_hi", ""),
+                      "");
+  string clamp = !if(HasClamp, "$clamp", "");
+  string omod = !if(HasOMod, "$omod", "");
+
+  string ret = dst#", "#src0#src1#src2#opsel#3PMods#clamp#omod;
+
+}
+
+class getAsmVOP3DPP<string base> {
+  string ret = base # " $dpp_ctrl$row_mask$bank_mask$bound_ctrl";
+}
+
+class getAsmVOP3DPP16<string base> {
+  string ret = getAsmVOP3DPP<base>.ret # "$fi";
+}
+
+class getAsmVOP3DPP8<string base> {
+  string ret = base # " $dpp8$fi";
+}
+
 
 class getAsmSDWA <bit HasDst, int NumSrcArgs, ValueType DstVT = i32> {
   string dst = !if(HasDst,
                    !if(!eq(DstVT.Size, 1),
                        " vcc", // use vcc token as dst for VOPC instructions
                        "$vdst"),
                     "");
   string src0 = "$src0_modifiers";

 class BitOr<bit a, bit b> {
   bit ret = !if(a, 1, !if(b, 1, 0));
 }
 
 class BitAnd<bit a, bit b> {
   bit ret = !if(a, !if(b, 1, 0), 0);
 }
 
+class getHasVOP3DPP <ValueType DstVT = i32, ValueType Src0VT = i32,
+                 ValueType Src1VT = i32, ValueType Src2VT = i32> {
+  bit ret =    !if(!eq(DstVT.Size, 64),
+                    0, // 64-bit dst No DPP for 64-bit operands
+                    !if(!eq(Src0VT.Size, 64),
+                        0, // 64-bit src0
+                        !if(!eq(Src1VT.Size, 64),
+                            0, // 64-bit src1
+                            !if(!eq(Src2VT.Size, 64),
+                                0, // 64-bit src2
+                                1
+                            )
+                        )
+                    )
+                );
+}
+
+
 def PatGenMode {
   int NoPattern = 0;
   int Pattern   = 1;
 }
 
 class VOPProfile <list<ValueType> _ArgVT, bit _EnableF32SrcMods = 0,
                   bit _EnableClamp = 0> {
 
class VOPProfile <list<ValueType> _ArgVT, bit _EnableF32SrcMods = 0,
   field RegisterOperand DstRCSDWA = getSDWADstForVT<DstVT>.ret;
   field RegisterOperand Src0RC32 = getVOPSrc0ForVT<Src0VT>.ret;
   field RegisterOperand Src1RC32 = RegisterOperand<getVregSrcForVT<Src1VT>.ret>;
   field RegisterOperand Src0RC64 = getVOP3SrcForVT<Src0VT>.ret;
   field RegisterOperand Src1RC64 = getVOP3SrcForVT<Src1VT>.ret;
   field RegisterOperand Src2RC64 = getVOP3SrcForVT<Src2VT>.ret;
   field RegisterClass Src0DPP = getVregSrcForVT<Src0VT>.ret;
   field RegisterClass Src1DPP = getVregSrcForVT<Src1VT>.ret;
+  field RegisterClass Src2DPP = getVregSrcForVT<Src2VT>.ret;
+  field RegisterOperand Src0VOP3DPP = VGPRSrc_32;
+  field RegisterOperand Src1VOP3DPP = VGPRSrc_32;
+  field RegisterOperand Src2VOP3DPP = getVOP3DPPSrcForVT<Src2VT>.ret;
   field RegisterOperand Src0SDWA = getSDWASrcForVT<Src0VT>.ret;
   field RegisterOperand Src1SDWA = getSDWASrcForVT<Src0VT>.ret;
   field Operand Src0Mod = getSrcMod<Src0VT, EnableF32SrcMods>.ret;
   field Operand Src1Mod = getSrcMod<Src1VT, EnableF32SrcMods>.ret;
   field Operand Src2Mod = getSrcMod<Src2VT, EnableF32SrcMods>.ret;
   field Operand Src0ModDPP = getSrcModDPP<Src0VT>.ret;
   field Operand Src1ModDPP = getSrcModDPP<Src1VT>.ret;
+  field Operand Src2ModDPP = getSrcModDPP<Src2VT>.ret;
+  field Operand Src2ModVOP3DPP = getSrcModVOP3DPP<Src2VT, EnableF32SrcMods>.ret;
   field Operand Src0ModSDWA = getSrcModSDWA<Src0VT>.ret;
   field Operand Src1ModSDWA = getSrcModSDWA<Src1VT>.ret;
 
 
   field bit HasDst = !ne(DstVT.Value, untyped.Value);
   field bit HasDst32 = HasDst;
   field bit EmitDst = HasDst; // force dst encoding, see v_movreld_b32 special case
   field bit EmitDstSel = EmitDst;
field bit HasModifiers = !or(isModifierType<Src0VT>.ret,
                                HasOMod,
                                EnableF32SrcMods);
 
   field bit HasSrc0Mods = HasModifiers;
   field bit HasSrc1Mods = !if(HasModifiers, !or(HasSrc1FloatMods, HasSrc1IntMods), 0);
   field bit HasSrc2Mods = !if(HasModifiers, !or(HasSrc2FloatMods, HasSrc2IntMods), 0);
 
   field bit HasExt = getHasExt<NumSrcArgs, DstVT, Src0VT, Src1VT>.ret;
-  field bit HasExtDPP = getHasDPP<NumSrcArgs>.ret;
+  field bit HasExtVOP3DPP = getHasVOP3DPP<DstVT, Src0VT, Src1VT, Src2VT>.ret;
+  field bit HasExtDPP = !if(!or(getHasDPP<NumSrcArgs>.ret,
+                HasExtVOP3DPP), 1, 0);
   field bit HasExt32BitDPP = getHasExt32BitDPP<NumSrcArgs, DstVT, Src0VT, Src1VT>.ret;
   field bit HasExt64BitDPP = getHasExt64BitDPP<NumSrcArgs, DstVT, Src0VT, Src1VT>.ret;
   field bit HasExtSDWA = getHasSDWA<NumSrcArgs, DstVT, Src0VT, Src1VT>.ret;
   field bit HasExtSDWA9 = HasExtSDWA;
   field int NeedPatGen = PatGenMode.NoPattern;
 
   field bit IsMAI = 0;
   field bit IsVOP3P = 0;
   field bit IsDOT = 0;
   field bit IsSingle = 0;
 
   field Operand Src0PackedMod = !if(HasSrc0FloatMods, PackedF16InputMods, PackedI16InputMods);
   field Operand Src1PackedMod = !if(HasSrc1FloatMods, PackedF16InputMods, PackedI16InputMods);
   field Operand Src2PackedMod = !if(HasSrc2FloatMods, PackedF16InputMods, PackedI16InputMods);
 
   field dag Outs = !if(HasDst,(outs DstRC:$vdst),(outs));
 
   // VOP3b instructions are a special case with a second explicit
   // output. This is manually overridden for them.
   field dag Outs32 = Outs;
   field dag Outs64 = !if(HasDst,(outs DstRC64:$vdst),(outs));
   field dag OutsDPP = getOutsDPP<HasDst, DstVT, DstRCDPP>.ret;
   field dag OutsDPP8 = getOutsDPP<HasDst, DstVT, DstRCDPP>.ret;
+  field dag OutsVOP3DPP = OutsDPP;
+  field dag OutsVOP3DPP8 = OutsDPP8;
   field dag OutsSDWA = getOutsSDWA<HasDst, DstVT, DstRCSDWA>.ret;
 
   field dag Ins32 = getIns32<Src0RC32, Src1RC32, NumSrcArgs>.ret;
   field dag Ins64 = getIns64<Src0RC64, Src1RC64, Src2RC64, NumSrcArgs,
                              HasIntClamp, HasModifiers, HasSrc2Mods,
                              HasOMod, Src0Mod, Src1Mod, Src2Mod>.ret;
   field dag InsVOP3P = getInsVOP3P<Src0RC64, Src1RC64, Src2RC64,
                                    NumSrcArgs, HasClamp, HasOpSel,
                                    Src0PackedMod, Src1PackedMod, Src2PackedMod>.ret;
   field dag InsVOP3OpSel = getInsVOP3OpSel<Src0RC64, Src1RC64, Src2RC64,
                                 NumSrcArgs, HasClamp, HasOMod,
                                 getOpSelMod<Src0VT>.ret,
                                 getOpSelMod<Src1VT>.ret,
                                 getOpSelMod<Src2VT>.ret>.ret;
   field dag InsDPP = !if(HasExtDPP,
-                         getInsDPP<DstRCDPP, Src0DPP, Src1DPP, NumSrcArgs,
-                                   HasModifiers, Src0ModDPP, Src1ModDPP>.ret,
+                         getInsDPP<DstRCDPP, Src0DPP, Src1DPP, Src2DPP, NumSrcArgs,
+                                   HasModifiers, Src0ModDPP, Src1ModDPP, Src2ModDPP>.ret,
                          (ins));
-  field dag InsDPP16 = getInsDPP16<DstRCDPP, Src0DPP, Src1DPP, NumSrcArgs,
-                                   HasModifiers, Src0ModDPP, Src1ModDPP>.ret;
-  field dag InsDPP8 = getInsDPP8<DstRCDPP, Src0DPP, Src1DPP, NumSrcArgs, 0,
-                                 Src0ModDPP, Src1ModDPP>.ret;
+  field dag InsDPP16 = getInsDPP16<DstRCDPP, Src0DPP, Src1DPP, Src2DPP, NumSrcArgs,
+                                   HasModifiers, Src0ModDPP, Src1ModDPP, Src2ModDPP>.ret;
+  field dag InsDPP8 = getInsDPP8<DstRCDPP, Src0DPP, Src1DPP, Src2DPP,
+                                 NumSrcArgs, HasModifiers,
+                                 Src0ModDPP, Src1ModDPP, Src2ModDPP>.ret;
+  field dag InsVOP3Base = getInsVOP3Base<Src0VOP3DPP, Src1VOP3DPP,
+                  Src2VOP3DPP, NumSrcArgs, HasClamp, HasModifiers, HasSrc2Mods, HasOMod,
+                  Src0ModDPP, Src1ModDPP, Src2ModVOP3DPP, HasOpSel, IsVOP3P>.ret;
+  field dag InsVOP3DPP = getInsVOP3DPP<InsVOP3Base, DstRCDPP, NumSrcArgs>.ret;
+  field dag InsVOP3DPP16 = getInsVOP3DPP16<InsVOP3Base, DstRCDPP, NumSrcArgs>.ret;
+  field dag InsVOP3DPP8 = getInsVOP3DPP8<InsVOP3Base, DstRCDPP, NumSrcArgs>.ret;
   field dag InsSDWA = getInsSDWA<Src0SDWA, Src1SDWA, NumSrcArgs,
                                  HasSDWAOMod, Src0ModSDWA, Src1ModSDWA,
                                  DstVT>.ret;
 
-
   field string Asm32 = getAsm32<HasDst, NumSrcArgs, DstVT>.ret;
   field string Asm64 = getAsm64<HasDst, NumSrcArgs, HasIntClamp, HasModifiers, HasOMod, DstVT>.ret;
   field string AsmVOP3P = getAsmVOP3P<NumSrcArgs, HasModifiers, HasClamp, HasOpSel>.ret;
   field string AsmVOP3OpSel = getAsmVOP3OpSel<NumSrcArgs,
                                               HasClamp,
                                               HasSrc0FloatMods,
                                               HasSrc1FloatMods,
                                               HasSrc2FloatMods>.ret;
   field string AsmDPP = !if(HasExtDPP,
                             getAsmDPP<HasDst, NumSrcArgs, HasModifiers, DstVT>.ret, "");
   field string AsmDPP16 = getAsmDPP16<HasDst, NumSrcArgs, HasModifiers, DstVT>.ret;
   // DPP8 encoding has no fields for modifiers, and it is enforced by setting
   // the asm operand name via this HasModifiers flag
   field string AsmDPP8 = getAsmDPP8<HasDst, NumSrcArgs, 0 /*HasModifiers*/, DstVT>.ret;
+  field string AsmVOP3DPPBase = getAsmVOP3DPPBase<NumSrcArgs, HasDst, HasClamp,
+   HasOpSel, HasOMod, IsVOP3P, HasModifiers, HasSrc0FloatMods, HasSrc1FloatMods,
+   HasSrc2FloatMods, DstVT >.ret;
+  field string AsmVOP3DPP = getAsmVOP3DPP<AsmVOP3DPPBase>.ret;
+  field string AsmVOP3DPP16 = getAsmVOP3DPP16<AsmVOP3DPPBase>.ret;
+  field string AsmVOP3DPP8 = getAsmVOP3DPP8<AsmVOP3DPPBase>.ret;
   field string AsmSDWA = getAsmSDWA<HasDst, NumSrcArgs, DstVT>.ret;
   field string AsmSDWA9 = getAsmSDWA9<HasDst, HasSDWAOMod, NumSrcArgs, DstVT>.ret;
-
   field string TieRegDPP = "$old";
 }
 
   class VOP_NO_EXT <VOPProfile p> : VOPProfile <p.ArgVT> {
   let HasExt = 0;
   let HasExtDPP = 0;
+  let HasExtVOP3DPP = 0;
   let HasExt32BitDPP = 0;
   let HasExt64BitDPP = 0;
   let HasExtSDWA = 0;
   let HasExtSDWA9 = 0;
 }
 
 class VOP_PAT_GEN <VOPProfile p, int mode=PatGenMode.NoPattern> : VOPProfile <p.ArgVT> {
   let NeedPatGen = mode;

llvm/lib/Target/AMDGPU/SIRegisterInfo.td

 // This is for operands with the enum(9), VSrc encoding restriction,
 // but only allows VGPRs.
 def VRegSrc_32 : RegisterOperand<VGPR_32> {
   //let ParserMatchClass = RegImmMatcher<"VRegSrc32">;
   let DecoderMethod = "DecodeVS_32RegisterClass";
 }
 
 //===----------------------------------------------------------------------===//
+// VGPRSrc_*
+//===----------------------------------------------------------------------===//
+
+// An 8-bit RegisterOperand wrapper for a VGPR
+def VGPRSrc_32 : RegisterOperand<VGPR_32> {
+  let DecoderMethod = "DecodeVGPR_32RegisterClass";
+}
+
+//===----------------------------------------------------------------------===//
 //  ASrc_* Operands with an AccVGPR
 //===----------------------------------------------------------------------===//
 
 def ARegSrc_32 : RegisterOperand<AGPR_32> {
   let DecoderMethod = "DecodeAGPR_32RegisterClass";
   let EncoderMethod = "getAVOperandEncoding";
 }
 

llvm/lib/Target/AMDGPU/VOP2Instructions.td

 def VOP2b_I32_I1_I32_I32 : VOPProfile<[i32, i32, i32, untyped], 0, /*EnableClamp=*/1> {
   let Asm32 = "$vdst, vcc, $src0, $src1";
   let Asm64 = "$vdst, $sdst, $src0, $src1$clamp";
   let AsmSDWA = "$vdst, vcc, $src0_modifiers, $src1_modifiers$clamp $dst_sel $dst_unused $src0_sel $src1_sel";
   let AsmSDWA9 = "$vdst, vcc, $src0_modifiers, $src1_modifiers$clamp $dst_sel $dst_unused $src0_sel $src1_sel";
   let AsmDPP = "$vdst, vcc, $src0, $src1 $dpp_ctrl$row_mask$bank_mask$bound_ctrl";
   let AsmDPP8 = "$vdst, vcc, $src0, $src1 $dpp8$fi";
   let AsmDPP16 = AsmDPP#"$fi";
+  let InsDPP = (ins DstRCDPP:$old,
+                    Src0DPP:$src0,
+                    Src1DPP:$src1,
+                    dpp_ctrl:$dpp_ctrl, row_mask:$row_mask,
+                    bank_mask:$bank_mask, bound_ctrl:$bound_ctrl);
+  let InsDPP16 = !con(InsDPP, (ins FI:$fi));
+  let InsDPP8 = (ins DstRCDPP:$old,
+                    Src0DPP:$src0,
+                    Src1DPP:$src1,
+                    dpp8:$dpp8, FI:$fi);
   let Outs32 = (outs DstRC:$vdst);
   let Outs64 = (outs DstRC:$vdst, VOPDstS64orS32:$sdst);
 }
 
 // Write out to vcc or arbitrary SGPR and read in from vcc or
 // arbitrary SGPR.
 def VOP2b_I32_I1_I32_I32_I1 : VOPProfile<[i32, i32, i32, i1], 0, /*EnableClamp=*/1> {
+  let HasSrc2Mods = 0;
   let Asm32 = "$vdst, vcc, $src0, $src1, vcc";
   let Asm64 = "$vdst, $sdst, $src0, $src1, $src2$clamp";
   let AsmSDWA = "$vdst, vcc, $src0_modifiers, $src1_modifiers, vcc$clamp $dst_sel $dst_unused $src0_sel $src1_sel";
   let AsmSDWA9 = "$vdst, vcc, $src0_modifiers, $src1_modifiers, vcc$clamp $dst_sel $dst_unused $src0_sel $src1_sel";
   let AsmDPP = "$vdst, vcc, $src0, $src1, vcc $dpp_ctrl$row_mask$bank_mask$bound_ctrl";
   let AsmDPP8 = "$vdst, vcc, $src0, $src1, vcc $dpp8$fi";
   let AsmDPP16 = AsmDPP#"$fi";
   let Outs32 = (outs DstRC:$vdst);
let InsSDWA = (ins Src0ModSDWA:$src0_modifiers, Src0SDWA:$src0,
                      src0_sel:$src0_sel, src1_sel:$src1_sel);
 
   let InsDPP = (ins DstRCDPP:$old,
                     Src0DPP:$src0,
                     Src1DPP:$src1,
                     dpp_ctrl:$dpp_ctrl, row_mask:$row_mask,
                     bank_mask:$bank_mask, bound_ctrl:$bound_ctrl);
   let InsDPP16 = !con(InsDPP, (ins FI:$fi));
+  let InsDPP8 = (ins DstRCDPP:$old,
+                    Src0ModDPP:$src0_modifiers, Src0DPP:$src0,
+                    Src1ModDPP:$src1_modifiers, Src1DPP:$src1,
+                    dpp8:$dpp8, FI:$fi);
 
   let HasExt = 1;
   let HasExtDPP = 1;
   let HasExt32BitDPP = 1;
   let HasExtSDWA = 1;
   let HasExtSDWA9 = 1;
 }
 
 // Read in from vcc or arbitrary SGPR.
-def VOP2e_I32_I32_I32_I1 : VOPProfile<[i32, i32, i32, i1], /*EnableF32SrcMods=*/1> {
+class VOP2e_SGPR<list<ValueType> ArgVT> : VOPProfile<ArgVT, /*EnableF32SrcMods=*/1> {
   let Asm32 = "$vdst, $src0, $src1";
   let Asm64 = "$vdst, $src0_modifiers, $src1_modifiers, $src2";
   let AsmSDWA = "$vdst, $src0_modifiers, $src1_modifiers, vcc$clamp $dst_sel $dst_unused $src0_sel $src1_sel";
   let AsmSDWA9 = "$vdst, $src0_modifiers, $src1_modifiers, vcc$clamp $dst_sel $dst_unused $src0_sel $src1_sel";
   let AsmDPP = "$vdst, $src0, $src1, vcc $dpp_ctrl$row_mask$bank_mask$bound_ctrl";
   let AsmDPP8 = "$vdst, $src0, $src1, vcc $dpp8$fi";
   let AsmDPP16 = AsmDPP#"$fi";
 
let InsSDWA = (ins Src0ModSDWA:$src0_modifiers, Src0SDWA:$src0,
                      src0_sel:$src0_sel, src1_sel:$src1_sel);
 
   let InsDPP = (ins DstRCDPP:$old,
                     Src0ModDPP:$src0_modifiers, Src0DPP:$src0,
                     Src1ModDPP:$src1_modifiers, Src1DPP:$src1,
                     dpp_ctrl:$dpp_ctrl, row_mask:$row_mask,
                     bank_mask:$bank_mask, bound_ctrl:$bound_ctrl);
   let InsDPP16 = !con(InsDPP, (ins FI:$fi));
+  let InsDPP8 = (ins DstRCDPP:$old, Src0DPP:$src0, Src1DPP:$src1,
+                     dpp8:$dpp8, FI:$fi);
 
   let HasExt = 1;
   let HasExtDPP = 1;
   let HasExt32BitDPP = 1;
   let HasExtSDWA = 1;
   let HasExtSDWA9 = 1;
 }
 
-def VOP_READLANE : VOPProfile<[i32, i32, i32]> {
+def VOP2e_I32_I32_I32_I1 : VOP2e_SGPR<[i32, i32, i32, i1]>;
+
+def VOP_READLANE : VOPProfile<[i32, i32, i32, untyped]> {
   let Outs32 = (outs SReg_32:$vdst);
   let Outs64 = Outs32;
   let Ins32 = (ins VRegOrLds_32:$src0, SCSrc_b32:$src1);
   let Ins64 = Ins32;
   let Asm32 = " $vdst, $src0, $src1";
   let Asm64 = Asm32;
 
   let HasExt = 0;

llvm/lib/Target/AMDGPU/VOP3Instructions.td

 //===-- VOP3Instructions.td - Vector Instruction Definitions --------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // Special case for v_div_fmas_{f32|f64}, since it seems to be the
 // only VOP instruction that implicitly reads VCC.
 let Asm64 = " $vdst, $src0_modifiers, $src1_modifiers, $src2_modifiers$clamp$omod" in {
 def VOP_F32_F32_F32_F32_VCC : VOPProfile<[f32, f32, f32, f32]> {
   let Outs64 = (outs DstRC.RegClass:$vdst);
+  let HasExtVOP3DPP = 0;
+  let HasExtDPP = 0;
 }
 def VOP_F64_F64_F64_F64_VCC : VOPProfile<[f64, f64, f64, f64]> {
   let Outs64 = (outs DstRC.RegClass:$vdst);
 }
 }
 
 class VOP3b_Profile<ValueType vt> : VOPProfile<[vt, vt, vt, vt]> {
   let Outs64 = (outs DstRC:$vdst, VOPDstS64orS32:$sdst);
   let Asm64 = "$vdst, $sdst, $src0_modifiers, $src1_modifiers, $src2_modifiers$clamp$omod";
   let IsSingle = 1;
+  let HasExtVOP3DPP = 0;
+  let HasExtDPP = 0;
 }
 
 def VOP3b_F32_I1_F32_F32_F32 : VOP3b_Profile<f32>;
 def VOP3b_F64_I1_F64_F64_F64 : VOP3b_Profile<f64>;
 
 def VOP3b_I64_I1_I32_I32_I64 : VOPProfile<[i64, i32, i32, i64]> {
   let HasClamp = 1;
-  let IsSingle = 1;
 
+  let IsSingle = 1;
   let Outs64 = (outs DstRC:$vdst, VOPDstS64orS32:$sdst);
   let Asm64 = "$vdst, $sdst, $src0, $src1, $src2$clamp";
 }
 
 class V_MUL_PROF<VOPProfile P> : VOP3_Profile<P> {
+  let HasExtVOP3DPP = 0;
+  let HasExtDPP = 0;
 }
 
 def DIV_FIXUP_F32_PROF : VOP3_Profile<VOP_F32_F32_F32_F32> {
+  let HasExtVOP3DPP = 0;
+  let HasExtDPP = 0;
 }
 
 //===----------------------------------------------------------------------===//
 // VOP3 INTERP
 //===----------------------------------------------------------------------===//
 
 class VOP3Interp<string OpName, VOPProfile P, list<dag> pattern = []> :
                  VOP3_Pseudo<OpName, P, pattern> {
def VOP3_PERMLANE_Profile : VOP3_Profile<VOPProfile <[i32, i32, i32, i32]>, VOP3_OPSEL> {
   let Src0RC64 = VRegSrc_32;
   let Src1RC64 = SCSrc_b32;
   let Src2RC64 = SCSrc_b32;
   let InsVOP3OpSel = (ins IntOpSelMods:$src0_modifiers, VRegSrc_32:$src0,
                           IntOpSelMods:$src1_modifiers, SCSrc_b32:$src1,
                           IntOpSelMods:$src2_modifiers, SCSrc_b32:$src2,
                           VGPR_32:$vdst_in, op_sel0:$op_sel);
   let HasClamp = 0;
+  let HasExtVOP3DPP = 0;
+  let HasExtDPP = 0;
 }
 
 class PermlanePat<SDPatternOperator permlane,
   Instruction inst> : GCNPat<
   (permlane i32:$vdst_in, i32:$src0, i32:$src1, i32:$src2,
             timm:$fi, timm:$bc),
   (inst (as_i1timm $fi), VGPR_32:$src0, (as_i1timm $bc),
         SCSrc_b32:$src1, 0, SCSrc_b32:$src2, VGPR_32:$vdst_in)

 }
 
 let WaveSizePredicate = isWave32 in {
 def : DivFmasPat<f32, V_DIV_FMAS_F32_e64, VCC_LO>;
 def : DivFmasPat<f64, V_DIV_FMAS_F64_e64, VCC_LO>;
 }
 
 class VOP3_DOT_Profile<VOPProfile P, VOP3Features Features = VOP3_REGULAR> : VOP3_Profile<P, Features> {
+  // FIXME VOP3 DPP versions are unsupported
+  let HasExtVOP3DPP = 0;
   let HasClamp = 0;
   let HasOMod = 0;
   let InsVOP3OpSel = getInsVOP3OpSel<Src0RC64, Src1RC64, Src2RC64,
                                 NumSrcArgs, HasClamp, HasOMod,
                                 !if(isFloatType<Src0VT>.ret, FPVRegInputMods, IntOpSelMods),
                                 !if(isFloatType<Src1VT>.ret, FPVRegInputMods, IntOpSelMods),
                                 !if(isFloatType<Src2VT>.ret, FPVRegInputMods, IntOpSelMods)>.ret;
 }

 defm V_MAXMIN_F32          : VOP3_Realtriple_gfx11<0x25e>;
 defm V_MINMAX_F32          : VOP3_Realtriple_gfx11<0x25f>;
 defm V_MAXMIN_F16          : VOP3_Realtriple_gfx11<0x260>;
 defm V_MINMAX_F16          : VOP3_Realtriple_gfx11<0x261>;
 defm V_MAXMIN_U32          : VOP3_Realtriple_gfx11<0x262>;
 defm V_MINMAX_U32          : VOP3_Realtriple_gfx11<0x263>;
 defm V_MAXMIN_I32          : VOP3_Realtriple_gfx11<0x264>;
 defm V_MINMAX_I32          : VOP3_Realtriple_gfx11<0x265>;
-defm V_DOT2_F16_F16        : VOP3_Realtriple_gfx11<0x266>;
-defm V_DOT2_BF16_BF16      : VOP3_Realtriple_gfx11<0x267>;
+// FIXME VOP3 DPP Dot instructions are unsupported
+defm V_DOT2_F16_F16        : VOP3_Real_Base_gfx11<0x266>;
+defm V_DOT2_BF16_BF16      : VOP3_Real_Base_gfx11<0x267>;
 defm V_DIV_SCALE_F32       : VOP3be_Real_gfx11<0x2fc, "V_DIV_SCALE_F32", "v_div_scale_f32">;
 defm V_DIV_SCALE_F64       : VOP3be_Real_gfx11<0x2fd, "V_DIV_SCALE_F64", "v_div_scale_f64">;
 defm V_MAD_U64_U32_gfx11   : VOP3be_Real_gfx11<0x2fe, "V_MAD_U64_U32_gfx11", "v_mad_u64_u32">;
 defm V_MAD_I64_I32_gfx11   : VOP3be_Real_gfx11<0x2ff, "V_MAD_I64_I32_gfx11", "v_mad_i64_i32">;
 defm V_CVT_PK_I16_F32      : VOP3_Realtriple_gfx11<0x306>;
 defm V_CVT_PK_U16_F32      : VOP3_Realtriple_gfx11<0x307>;
 defm V_ADD_NC_I16          : VOP3_Realtriple_with_name_gfx11<0x30d, "V_ADD_I16", "v_add_nc_i16">;
 defm V_SUB_NC_I16          : VOP3_Realtriple_with_name_gfx11<0x30e, "V_SUB_I16", "v_sub_nc_i16">;

llvm/lib/Target/AMDGPU/VOPInstructions.td

class VOP_DPPe<VOPProfile P, bit IsDPP16=0> : Enc64 {
   let Inst{52}    = !if(P.HasSrc0Mods, src0_modifiers{0}, 0); // src0_neg
   let Inst{53}    = !if(P.HasSrc0Mods, src0_modifiers{1}, 0); // src0_abs
   let Inst{54}    = !if(P.HasSrc1Mods, src1_modifiers{0}, 0); // src1_neg
   let Inst{55}    = !if(P.HasSrc1Mods, src1_modifiers{1}, 0); // src1_abs
   let Inst{59-56} = bank_mask;
   let Inst{63-60} = row_mask;
 }
 
-class VOP_DPP_Pseudo <string OpName, VOPProfile P, list<dag> pattern=[]> :
-  InstSI <P.OutsDPP, P.InsDPP, OpName#P.AsmDPP, pattern>,
+class VOP3_DPPe_Fields_Base {
+  bits<9> dpp_ctrl;
+  bits<1> bound_ctrl;
+  bits<4> bank_mask;
+  bits<4> row_mask;
+  bit     fi;
+}
+class VOP3_DPPe_Fields : VOP3_DPPe_Fields_Base {
+  bits<8> src0;
+}
+
+// Common refers to common between DPP and DPP8
+class VOP3_DPPe_Common_Base<bits<10> op, VOPProfile P> : Enc96 {
+  bits<4> src0_modifiers;
+  bits<3> src1_modifiers;
+  bits<3> src2_modifiers;
+  bits<1> clamp;
+  bits<2> omod;
+
+  let Inst{8}     = !if(P.HasSrc0Mods, src0_modifiers{1}, 0);
+  let Inst{9}     = !if(P.HasSrc1Mods, src1_modifiers{1}, 0);
+  let Inst{10}    = !if(P.HasSrc2Mods, src2_modifiers{1}, 0);
+  // OPSEL must be set such that the low result only uses low inputs, and the high result only uses high inputs.
+  let Inst{11} = !if(P.HasOpSel,!if(P.HasSrc0Mods, src0_modifiers{2}, 0),?);
+  let Inst{12} = !if(P.HasOpSel,!if(P.HasSrc1Mods, src1_modifiers{2}, 0),?);
+  let Inst{13} = !if(P.HasOpSel,!if(P.HasSrc2Mods, src2_modifiers{2}, 0),?);
+  let Inst{14} = !if(P.HasOpSel,!if(P.HasSrc0Mods, src0_modifiers{3}, 0),?);
+  let Inst{15}    = !if(P.HasClamp, clamp, 0);
+  let Inst{25-16} = op;
+  let Inst{31-26} = 0x35;
+
+  let Inst{60-59} = !if(P.HasOMod, omod, 0);
+  let Inst{61}    = !if(P.HasSrc0Mods, src0_modifiers{0}, 0);
+  let Inst{62}    = !if(P.HasSrc1Mods, src1_modifiers{0}, 0);
+  let Inst{63}    = !if(P.HasSrc2Mods, src2_modifiers{0}, 0);
+}
+
+class VOP3_DPPe_Common<bits<10> op, VOPProfile P> : VOP3_DPPe_Common_Base<op, P> {
+  bits<8> vdst;
+  bits<9> src1;
+  bits<9> src2;
+
+  let Inst{7-0}   = !if(P.EmitDst, vdst{7-0}, 0);
+  let Inst{49-41} = !if(P.HasSrc1, src1, 0);
+  let Inst{58-50} = !if(P.HasSrc2, src2, 0);
+}
+
+class VOP_DPP_Pseudo <string OpName, VOPProfile P, list<dag> pattern=[],
+  dag Ins = P.InsDPP, string asmOps = P.AsmDPP> :
+  InstSI <P.OutsDPP, Ins, OpName#asmOps, pattern>,
   VOP <OpName>,
   SIMCInstr <OpName#"_dpp", SIEncodingFamily.NONE> {
 
   let isPseudo = 1;
   let isCodeGenOnly = 1;
 
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
   let UseNamedOperandTable = 1;
 
   let VALU = 1;
   let DPP = 1;
   let Size = 8;
 
   let ReadsModeReg = !or(isFloatType<P.DstVT>.ret, isFloatType<P.Src0VT>.ret);
 
   let mayRaiseFPException = ReadsModeReg;
   let Uses = !if(ReadsModeReg, [MODE, EXEC], [EXEC]);
   let isConvergent = 1;
 
   string Mnemonic = OpName;
-  string AsmOperands = P.AsmDPP;
+  string AsmOperands = asmOps;
 
   let AsmMatchConverter = !if(P.HasModifiers, "cvtDPP", "");
   let SubtargetPredicate = !if(P.HasExt64BitDPP, Has64BitDPP, HasDPP);
   let AssemblerPredicate = !if(P.HasExt64BitDPP, Has64BitDPP, HasDPP);
   let AsmVariantName = !if(P.HasExtDPP, AMDGPUAsmVariants.DPP,
                                         AMDGPUAsmVariants.Disable);
   let Constraints = !if(P.NumSrcArgs, P.TieRegDPP # " = $vdst", "");
   let DisableEncoding = !if(P.NumSrcArgs, P.TieRegDPP, "");
   let DecoderNamespace = "DPP";
 
   VOPProfile Pfl = P;
 }
 
+class VOP3_DPP_Pseudo <string OpName, VOPProfile P> :
+  VOP_DPP_Pseudo <OpName, P, [], P.InsVOP3DPP, P.AsmVOP3DPP> {
+  let PseudoInstr = OpName#"_e64"#"_dpp";
+  let OutOperandList = P.OutsVOP3DPP;
+  let Size = 12;
+  let VOP3 = 1;
+  let AsmMatchConverter = "cvtVOP3DPP";
+  let AsmVariantName = !if(P.HasExtVOP3DPP, AMDGPUAsmVariants.VOP3_DPP,
+                                            AMDGPUAsmVariants.Disable);
+}
+
 class VOP_DPP_Real <VOP_DPP_Pseudo ps, int EncodingFamily> :
   InstSI <ps.OutOperandList, ps.InOperandList, ps.Mnemonic # ps.AsmOperands, []>,
   SIMCInstr <ps.PseudoInstr, EncodingFamily> {
 
   let VALU = 1;
   let DPP = 1;
   let isPseudo = 0;
   let isCodeGenOnly = 0;
class VOP_DPP_Real <VOP_DPP_Pseudo ps, int EncodingFamily> :
   let DisableEncoding      = ps.DisableEncoding;
   let TSFlags              = ps.TSFlags;
   let SchedRW              = ps.SchedRW;
   let mayLoad              = ps.mayLoad;
   let mayStore             = ps.mayStore;
   let TRANS                = ps.TRANS;
 }
 
-class VOP_DPP <string OpName, VOPProfile P, bit IsDPP16,
-               dag InsDPP = !if(IsDPP16, P.InsDPP16, P.InsDPP),
-               string AsmDPP = !if(IsDPP16, P.AsmDPP16, P.AsmDPP)> :
-  InstSI <P.OutsDPP, InsDPP, OpName#AsmDPP, []>,
-  VOP_DPPe<P, IsDPP16> {
+class VOP_DPP_Base <string OpName, VOPProfile P,
+               dag InsDPP,
+               string AsmDPP > :
+  InstSI <P.OutsDPP, InsDPP, OpName#AsmDPP, []> {
 
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
   let UseNamedOperandTable = 1;
 
   let VALU = 1;
   let DPP = 1;
   let Size = 8;
 
   let AsmMatchConverter = !if(P.HasModifiers, "cvtDPP", "");
   let SubtargetPredicate = !if(P.HasExt64BitDPP, Has64BitDPP, HasDPP);
   let AssemblerPredicate = !if(P.HasExt64BitDPP, Has64BitDPP, HasDPP);
   let AsmVariantName = !if(P.HasExtDPP, AMDGPUAsmVariants.DPP,
                                         AMDGPUAsmVariants.Disable);
   let Constraints = !if(P.NumSrcArgs, P.TieRegDPP # " = $vdst", "");
   let DisableEncoding = !if(P.NumSrcArgs, P.TieRegDPP, "");
   let DecoderNamespace = "DPP";
 }
 
+class VOP_DPP <string OpName, VOPProfile P, bit IsDPP16,
+               dag InsDPP = !if(IsDPP16, P.InsDPP16, P.InsDPP),
+               string AsmDPP = !if(IsDPP16, P.AsmDPP16, P.AsmDPP)> :
+  VOP_DPP_Base<OpName, P, InsDPP, AsmDPP>, VOP_DPPe<P, IsDPP16>;
+
+class VOP3_DPP_Base <string OpName, VOPProfile P, bit IsDPP16,
+               dag InsDPP = !if(IsDPP16, P.InsVOP3DPP16, P.InsVOP3DPP),
+               string AsmDPP = !if(IsDPP16, P.AsmVOP3DPP16, P.AsmVOP3DPP)> :
+  VOP_DPP_Base<OpName, P, InsDPP, AsmDPP> {
+  let OutOperandList = P.OutsVOP3DPP;
+  let AsmMatchConverter = "cvtVOP3DPP";
+  let VOP3 = 1;
+  let AsmVariantName = !if(P.HasExtVOP3DPP, AMDGPUAsmVariants.VOP3_DPP,
+                                            AMDGPUAsmVariants.Disable);
+  let Size = 12;
+}
+
+class VOP3_DPP <bits<10> op, string OpName, VOPProfile P, bit IsDPP16,
+               dag InsDPP = !if(IsDPP16, P.InsVOP3DPP16, P.InsVOP3DPP),
+               string AsmDPP = !if(IsDPP16, P.AsmVOP3DPP16, P.AsmVOP3DPP)> :
+  VOP3_DPP_Base<OpName, P, IsDPP16, InsDPP, AsmDPP>, VOP3_DPPe_Common<op, P>,
+  VOP3_DPPe_Fields {
+
+  let Inst{40-32} = 0xfa;
+  let Inst{71-64} = !if(P.HasSrc0, src0{7-0}, 0);
+  let Inst{80-72} = dpp_ctrl;
+  let Inst{82}    = !if(IsDPP16, fi, ?);
+  let Inst{83}    = bound_ctrl;
+
+  // Inst{87-84} ignored by hw
+  let Inst{91-88} = bank_mask;
+  let Inst{95-92} = row_mask;
+}
+
 class VOP_DPP8e<VOPProfile P> : Enc64 {
   bits<8> src0;
   bits<24> dpp8;
   bits<9> fi;
 
   let Inst{39-32} = !if(P.HasSrc0, src0{7-0}, 0);
   let Inst{63-40} = dpp8{23-0};
 }
 
-class VOP_DPP8<string OpName, VOPProfile P> :
-  InstSI<P.OutsDPP8, P.InsDPP8, OpName#P.AsmDPP8, []>,
-  VOP_DPP8e<P> {
+class VOP3_DPP8e_Fields {
+  bits<8> src0;
+  bits<24> dpp8;
+  bits<9> fi;
+}
+
+class VOP_DPP8_Base<string OpName, VOPProfile P, dag InsDPP8 = P.InsDPP8, string AsmDPP8 = P.AsmDPP8> :
+  InstSI<P.OutsDPP8, InsDPP8, OpName#AsmDPP8, []> {
 
   let mayLoad = 0;
   let mayStore = 0;
   let hasSideEffects = 0;
   let UseNamedOperandTable = 1;
 
   let VALU = 1;
   let DPP = 1;
   let Size = 8;
 
   let AsmMatchConverter = "cvtDPP8";
   let SubtargetPredicate = HasDPP8;
   let AssemblerPredicate = HasDPP8;
-  let AsmVariantName = !if(P.HasExt, AMDGPUAsmVariants.DPP,
-                                     AMDGPUAsmVariants.Disable);
+  let AsmVariantName = AMDGPUAsmVariants.DPP;
   let Constraints = !if(P.NumSrcArgs, P.TieRegDPP # " = $vdst", "");
   let DisableEncoding = !if(P.NumSrcArgs, P.TieRegDPP, "");
 }
 
+class VOP_DPP8<string OpName, VOPProfile P> :
+  VOP_DPP8_Base<OpName, P>, VOP_DPP8e<P>;
+
+class VOP3_DPP8_Base<string OpName, VOPProfile P> :
+  VOP_DPP8_Base<OpName, P, P.InsVOP3DPP8, P.AsmVOP3DPP8> {
+  let OutOperandList = P.OutsVOP3DPP8;
+  let AsmMatchConverter = "cvtVOP3DPP8";
+  let AsmVariantName = !if(P.HasExtVOP3DPP, AMDGPUAsmVariants.VOP3_DPP,
+                                            AMDGPUAsmVariants.Disable);
+  let VOP3 = 1;
+  let Size = 12;
+}
+
+
+class VOP3_DPP8<bits<10> op, string OpName, VOPProfile P> :
+  VOP3_DPP8_Base<OpName, P>, VOP3_DPPe_Common<op, P>,
+  VOP3_DPP8e_Fields {
+
+  let Inst{40-32} = fi;
+  let Inst{71-64} = !if(P.HasSrc0, src0{7-0}, 0);
+  let Inst{95-72} = dpp8{23-0};
+}
+
 def DPP8Mode {
   int FI_0 = 0xE9;
   int FI_1 = 0xEA;
 }
 
 class getNumNodeArgs<SDPatternOperator Op> {
   SDNode N = !cast<SDNode>(Op);
   SDTypeProfile TP = N.TypeProfile;
!if(P.HasOpSel,
         "cvtVOP3OpSel",
         !if(!or(P.HasModifiers, P.HasOMod, P.HasIntClamp),
             "cvtVOP3",
             ""));
 }
 
 multiclass VOP3Inst<string OpName, VOPProfile P, SDPatternOperator node = null_frag> {
   def _e64 : VOP3InstBase<OpName, P, node>;
+  let SubtargetPredicate = isGFX11Plus in {
+    foreach _ = BoolToList<P.HasExtVOP3DPP>.ret in
+      def _e64_dpp : VOP3_DPP_Pseudo <OpName, P>;
+  } // end SubtargetPredicate = isGFX11Plus
+}
+
+//===----------------------------------------------------------------------===//
+// VOP3 DPP
+//===----------------------------------------------------------------------===//
+
+class Base_VOP3_DPP16<bits<10> op, VOP_DPP_Pseudo ps, string opName = ps.OpName>
+    : VOP3_DPP<op, opName, ps.Pfl, 1> {
+  let hasSideEffects = ps.hasSideEffects;
+  let Defs = ps.Defs;
+  let SchedRW = ps.SchedRW;
+  let Uses = ps.Uses;
+  let AssemblerPredicate = HasDPP16;
+  let SubtargetPredicate = HasDPP16;
+  let OtherPredicates = ps.OtherPredicates;
+}
+
+class VOP3_DPP16<bits<10> op, VOP_DPP_Pseudo ps, int subtarget,
+                 string opName = ps.OpName>
+    : Base_VOP3_DPP16<op, ps, opName>, SIMCInstr<ps.PseudoInstr, subtarget>;
+
+class Base_VOP3_DPP8<bits<10> op, VOP_Pseudo ps, string opName = ps.OpName>
+    : VOP3_DPP8<op, opName, ps.Pfl> {
+  let hasSideEffects = ps.hasSideEffects;
+  let Defs = ps.Defs;
+  let SchedRW = ps.SchedRW;
+  let Uses = ps.Uses;
+
+  let OtherPredicates = ps.OtherPredicates;
+}
+
+class Base_VOP3b_DPP16<bits<10> op, VOP_DPP_Pseudo ps,
+                       string opName = ps.OpName>
+    : Base_VOP3_DPP16<op, ps, opName> {
+  bits<7> sdst;
+  let Inst{14 - 8} = sdst;
+}
+
+class VOP3b_DPP8_Base<bits<10> op, VOP_Pseudo ps, string opName = ps.OpName>
+    : Base_VOP3_DPP8<op, ps, opName> {
+  bits<7> sdst;
+  let Inst{14 - 8} = sdst;
 }
 
 //===----------------------------------------------------------------------===//
 // VOP3 GFX11
 //===----------------------------------------------------------------------===//
 
 let AssemblerPredicate = isGFX11Only,
     DecoderNamespace = "GFX11" in {
let AssemblerPredicate = isGFX11Only,
   }
   // for READLANE/WRITELANE
   multiclass VOP3_Real_No_Suffix_gfx11<bits<10> op, string opName = NAME> {
     defvar ps = !cast<VOP_Pseudo>(opName);
       def _e64_gfx11 :
         VOP3_Real<ps, SIEncodingFamily.GFX11>,
         VOP3e_gfx11<op, ps.Pfl>;
   }
+  multiclass VOP3_Real_dpp_Base_gfx11<bits<10> op, string opName = NAME> {
+    def _e64_dpp_gfx11 : VOP3_DPP16<op, !cast<VOP_DPP_Pseudo>(opName#"_e64"#"_dpp"), SIEncodingFamily.GFX11> {
+      let DecoderNamespace = "DPPGFX11";
+      }
+  }
+  multiclass VOP3_Real_dpp_with_name_gfx11<bits<10> op, string opName,
+                                           string asmName> {
+    defvar ps = !cast<VOP3_Pseudo>(opName#"_e64");
+    let AsmString = asmName # ps.Pfl.AsmVOP3DPP16, DecoderNamespace = "DPPGFX11" in {
+      defm NAME : VOP3_Real_dpp_Base_gfx11<op, opName>;
+    }
+  }
+  multiclass VOP3_Real_dpp8_Base_gfx11<bits<10> op, string opName = NAME> {
+    defvar ps = !cast<VOP3_Pseudo>(opName#"_e64");
+    def _e64_dpp8_gfx11 : Base_VOP3_DPP8<op, ps> {
+      let DecoderNamespace = "DPP8GFX11";
+    }
+  }
+  multiclass VOP3_Real_dpp8_with_name_gfx11<bits<10> op, string opName,
+                                           string asmName> {
+    defvar ps = !cast<VOP3_Pseudo>(opName#"_e64");
+    let AsmString = asmName # ps.Pfl.AsmVOP3DPP8, DecoderNamespace = "DPP8GFX11"  in {
+      defm NAME : VOP3_Real_dpp8_Base_gfx11<op, opName>;
+    }
+  }
   multiclass VOP3be_Real_gfx11<bits<10> op, string opName, string asmName,
                                bit isSingle = 0> {
     defvar ps = !cast<VOP3_Pseudo>(opName#"_e64");
     let IsSingle = !or(isSingle, ps.Pfl.IsSingle) in
       def _e64_gfx11 :
         VOP3_Real<ps, SIEncodingFamily.GFX11, asmName>,
         VOP3be_gfx11<op, ps.Pfl> ;
   }
+  multiclass VOP3be_Real_dpp_gfx11<bits<10> op, string opName, string asmName> {
+    defvar ps = !cast<VOP3_Pseudo>(opName #"_e64");
+    defvar dpp_ps = !cast<VOP_DPP_Pseudo>(opName #"_e64" #"_dpp");
+    def _e64_dpp_gfx11 : Base_VOP3b_DPP16<op, dpp_ps, asmName>,
+                         SIMCInstr<dpp_ps.PseudoInstr, SIEncodingFamily.GFX11> {
+      let DecoderNamespace = "DPPGFX11";
+    }
+  }
+  multiclass VOP3be_Real_dpp8_gfx11<bits<10> op, string opName, string asmName> {
+    defvar ps = !cast<VOP3_Pseudo>(opName #"_e64");
+    def _e64_dpp8_gfx11 : VOP3b_DPP8_Base<op, ps, asmName> {
+      let DecoderNamespace = "DPP8GFX11";
+    }
+  }
 } // End AssemblerPredicate = isGFX11Only, DecoderNamespace = "GFX11"
 
 // VOP1 and VOP2 depend on these triple defs
 multiclass VOP3_Realtriple_gfx11<bits<10> op,
                                  bit isSingle = 0, string opName = NAME> :
-  VOP3_Real_Base_gfx11<op, opName, isSingle>;
+  VOP3_Real_Base_gfx11<op, opName, isSingle>,
+  VOP3_Real_dpp_Base_gfx11<op, opName>,
+  VOP3_Real_dpp8_Base_gfx11<op, opName>;
 
 multiclass VOP3Only_Realtriple_gfx11<bits<10> op> :
   VOP3_Realtriple_gfx11<op, 1>;
 
 multiclass VOP3_Realtriple_with_name_gfx11<bits<10> op, string opName,
                                            string asmName, bit isSingle = 0> :
-  VOP3_Real_with_name_gfx11<op, opName, asmName, isSingle>;
+  VOP3_Real_with_name_gfx11<op, opName, asmName, isSingle>,
+  VOP3_Real_dpp_with_name_gfx11<op, opName, asmName>,
+  VOP3_Real_dpp8_with_name_gfx11<op, opName, asmName>;
 
 multiclass VOP3Only_Realtriple_with_name_gfx11<bits<10> op, string opName,
                                            string asmName> :
   VOP3_Realtriple_with_name_gfx11<op, opName, asmName, 1>;
 
 multiclass VOP3be_Realtriple_gfx11<
     bits<10> op, bit isSingle = 0, string opName = NAME,
     string asmName = !cast<VOP_Pseudo>(opName#"_e64").Mnemonic> :
-  VOP3be_Real_gfx11<op, opName, asmName, isSingle>;
+  VOP3be_Real_gfx11<op, opName, asmName, isSingle>,
+  VOP3be_Real_dpp_gfx11<op, opName, asmName>,
+  VOP3be_Real_dpp8_gfx11<op, opName, asmName>;
 
 multiclass VOP3beOnly_Realtriple_gfx11<bits<10> op> :
   VOP3be_Realtriple_gfx11<op, 1>;
 
 include "VOPCInstructions.td"
 include "VOP1Instructions.td"
 include "VOP2Instructions.td"
 include "VOP3Instructions.td"

llvm/test/MC/AMDGPU/gfx11_asm_dpp.s

+// RUN: llvm-mc -arch=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -show-encoding %s | FileCheck --check-prefixes=GFX11 %s
+// RUN: llvm-mc -arch=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -show-encoding %s | FileCheck --check-prefixes=GFX11 %s
+
+; DPP8
+
+; VOP3
+v_bfe_u32_e64_dpp v0, v1, v2, v3 dpp8:[0,1,2,3,4,5,6,7]
+// GFX11: encoding: [0x00,0x00,0x10,0xd6,0xe9,0x04,0x0e,0x04,0x01,0x88,0xc6,0xfa]
+
+v_maxmin_f32_e64_dpp v0, v1, v2, v3 clamp dpp8:[0,1,2,3,4,5,6,7]
+// GFX11: encoding: [0x00,0x80,0x5e,0xd6,0xe9,0x04,0x0e,0x04,0x01,0x88,0xc6,0xfa]
+
+v_maxmin_f32_e64_dpp v0, v1, v2, v3 div:2 dpp8:[0,1,2,3,4,5,6,7] fi:1
+// GFX11: encoding: [0x00,0x00,0x5e,0xd6,0xea,0x04,0x0e,0x1c,0x01,0x88,0xc6,0xfa]
+
+v_minmax_f32_e64_dpp v0, -v1, -v2, -v3 dpp8:[0,1,2,3,4,5,6,7]
+// GFX11: encoding: [0x00,0x00,0x5f,0xd6,0xe9,0x04,0x0e,0xe4,0x01,0x88,0xc6,0xfa]
+
+v_minmax_f32_e64_dpp v0, abs(v1), v2, v3 dpp8:[0,1,2,3,4,5,6,7]
+// GFX11: encoding: [0x00,0x01,0x5f,0xd6,0xe9,0x04,0x0e,0x04,0x01,0x88,0xc6,0xfa]
+
+v_fma_f32_e64_dpp v80, v81, v82, v81 dpp8:[0,1,6,3,4,5,6,7]
+// GFX11: encoding: [0x50,0x00,0x13,0xd6,0xe9,0xa4,0x46,0x05,0x51,0x88,0xc7,0xfa]
+
+v_fma_f32_e64_dpp v80, v81, abs(v82), v81 dpp8:[0,1,6,3,4,5,6,7]
+// GFX11: encoding: [0x50,0x02,0x13,0xd6,0xe9,0xa4,0x46,0x05,0x51,0x88,0xc7,0xfa]
+
+v_max3_f32_e64_dpp v5, v1, v2, s3 dpp8:[7,6,5,4,3,2,1,0]
+// GFX11: encoding: [0x05,0x00,0x1c,0xd6,0xe9,0x04,0x0e,0x00,0x01,0x77,0x39,0x05]
+
+v_add3_u32_e64_dpp v5, v1, v2, -1 dpp8:[7,6,5,4,3,2,1,0]
+// GFX11: encoding: [0x05,0x00,0x55,0xd6,0xe9,0x04,0x06,0x03,0x01,0x77,0x39,0x05]
+
+v_add3_u32_e64_dpp v5, v1, v2, s4 dpp8:[7,6,5,4,3,2,1,0]
+// GFX11: encoding: [0x05,0x00,0x55,0xd6,0xe9,0x04,0x12,0x00,0x01,0x77,0x39,0x05]
+
+; DPP
+
+
+; VOP3
+v_bfe_u32_e64_dpp v0, v1, v2, v3 quad_perm:[3,2,1,0] row_mask:0x0 bank_mask:0x0
+// GFX11: encoding: [0x00,0x00,0x10,0xd6,0xfa,0x04,0x0e,0x04,0x01,0x1b,0x00,0x00]
+
+v_fma_f32_e64_dpp v93, abs(v94), v95, v94 quad_perm:[3,2,1,0] bank_mask:0xe
+// GFX11: encoding: [0x5d,0x01,0x13,0xd6,0xfa,0xbe,0x7a,0x05,0x5e,0x1b,0x00,0xfe]
+
+v_sub_nc_i32_e64_dpp v93, v94, v95 row_ror:7 bank_mask:0x1 bound_ctrl:0
+// GFX11: encoding: [0x5d,0x00,0x25,0xd7,0xfa,0xbe,0x02,0x00,0x5e,0x27,0x09,0xf1]
+
+v_lshl_or_b32_e64_dpp v255, v5, v0, vcc_hi row_xmask:0x6 row_mask:0x0 fi:1
+// GFX11: encoding: [0xff,0x00,0x56,0xd6,0xfa,0x00,0xae,0x01,0x05,0x66,0x05,0x0f]
+
+v_cubesc_f32_e64_dpp v5, v1, v2, 1 row_shr:4 row_mask:0xf bank_mask:0xf
+// GFX11: encoding: [0x05,0x00,0x0d,0xd6,0xfa,0x04,0x06,0x02,0x01,0x14,0x01,0xff]
+
+v_cubesc_f32_e64_dpp v5, v1, v2, s2 row_shr:4 row_mask:0xf bank_mask:0xf
+// GFX11: encoding: [0x05,0x00,0x0d,0xd6,0xfa,0x04,0x0a,0x00,0x01,0x14,0x01,0xff]

llvm/test/MC/AMDGPU/gfx11_err.s

 v_interp_p10_f32 v0, v1, v2, v3 wait_exp:8
 // GFX11: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
 
 v_interp_p2_f32 v0, -v1, v2, v3 wait_exp
 // GFX11: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
 
 global_atomic_cmpswap_x2 v[1:4], v3, v[5:8], off offset:2047 glc
 // GFX11: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+
+v_cubesc_f32_e64_dpp v5, v1, v2, 12345678 row_shr:4 row_mask:0xf bank_mask:0xf
+// GFX11: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+
+v_add3_u32_e64_dpp v5, v1, v2, 49812340 dpp8:[7,6,5,4,3,2,1,0]
+// GFX11: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+
+v_add3_u32_e64_dpp v5, v1, s1, v0 dpp8:[7,6,5,4,3,2,1,0]
+// GFX11: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction

llvm/test/MC/AMDGPU/gfx9-asm-err.s

 
 v_subrev_u16_e64 v5, v1, 0.5
 // GFX9ERR: error: literal operands are not supported
 
 v_subrev_u16_e64 v5, v1, -4.0
 // GFX9ERR: error: literal operands are not supported
 
 v_cvt_u32_f64 v5, v[0:1] quad_perm:[0,2,1,1] row_mask:0xf bank_mask:0xf
-// GFX9ERR: error: not a valid operand.
+// GFX9ERR: error: invalid operand for instruction
 
 global_load_lds_dword v[2:3], off
 // GFX9ERR: error: instruction not supported on this GPU
 
 global_load_dword v[2:3], off
 // GFX9ERR: error: invalid operands for instruction
 
 scratch_load_dword v2, off, offset:256
 // GFX9ERR: error: invalid operands for instruction

llvm/test/MC/AMDGPU/vop_dpp.s

 // NOSICI: error: not a valid operand.
 // VI9: v_mov_b32_dpp v0, v0 quad_perm:[1,3,0,1] row_mask:0xf bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x02,0x00,0x7e,0x00,0x4d,0x08,0xf1]
 v_mov_b32 v0, v0 quad_perm:[1,3,0,1] bank_mask:0x1 bound_ctrl:0
 
 //===----------------------------------------------------------------------===//
 // Check modifiers
 //===----------------------------------------------------------------------===//
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_add_f32_dpp v0, -v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x02,0x00,0x01,0x19,0xa1]
 v_add_f32 v0, -v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_add_f32_dpp v0, v0, |v0| row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x02,0x00,0x01,0x89,0xa1]
 v_add_f32 v0, v0, |v0| row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_add_f32_dpp v0, -v0, |v0| row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x02,0x00,0x01,0x99,0xa1]
 v_add_f32 v0, -v0, |v0| row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_add_f32_dpp v0, |v0|, -v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x02,0x00,0x01,0x69,0xa1]
 v_add_f32 v0, |v0|, -v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 //===----------------------------------------------------------------------===//
 // Check VOP1 opcodes
 //===----------------------------------------------------------------------===//
 
 // NOSICI: error: not a valid operand.
-// GCN: v_nop row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0 ; encoding: [0xfa,0x00,0x00,0x7e,0x00,0x01,0x09,0xa1]
+// GCN: v_nop row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x7e,0x00,0x01,0x09,0xa1]
 v_nop row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_u32_f32_dpp v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x0e,0x00,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_u32_f32 v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_fract_f32_dpp v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x36,0x00,0x7e,0x00,0x01,0x09,0xa1]
 v_fract_f32 v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_sin_f32_dpp v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x52,0x00,0x7e,0x00,0x01,0x09,0xa1]
 v_sin_f32 v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_mov_b32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x02,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_mov_b32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_f32_i32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x0a,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_f32_i32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_f32_u32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x0c,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_f32_u32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_i32_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x10,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_i32_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_f16_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x14,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_f16_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_f32_f16_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x16,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_f32_f16 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_rpi_i32_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x18,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_rpi_i32_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_flr_i32_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x1a,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_flr_i32_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_off_f32_i4_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x1c,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_off_f32_i4 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_f32_ubyte0_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x22,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_f32_ubyte0 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_f32_ubyte1_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x24,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_f32_ubyte1 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_f32_ubyte2_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x26,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_f32_ubyte2 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cvt_f32_ubyte3_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x28,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_f32_ubyte3 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_trunc_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x38,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_trunc_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_ceil_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x3a,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_ceil_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_rndne_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x3c,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_rndne_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_floor_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x3e,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_floor_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_exp_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x40,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_exp_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_log_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x42,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_log_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_rcp_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x44,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_rcp_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_rcp_iflag_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x46,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_rcp_iflag_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_rsq_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x48,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_rsq_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_sqrt_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x4e,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_sqrt_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_cos_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x54,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cos_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_not_b32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x56,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_not_b32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_bfrev_b32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x58,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_bfrev_b32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_ffbh_u32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x5a,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_ffbh_u32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_ffbl_b32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x5c,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_ffbl_b32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_ffbh_i32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x5e,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_ffbh_i32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_frexp_exp_i32_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x66,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_frexp_exp_i32_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_frexp_mant_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x68,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_frexp_mant_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // VI9: v_log_legacy_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x98,0x02,0x7e,0x00,0x01,0x09,0xa1]
 // NOSI: error: instruction not supported on this GPU
-// NOCI: error: not a valid operand.
+// NOCI: error: operands are not valid for this GPU or mode
 v_log_legacy_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // VI9: v_exp_legacy_f32_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x96,0x02,0x7e,0x00,0x01,0x09,0xa1]
 // NOSI: error: instruction not supported on this GPU
-// NOCI: error: not a valid operand.
+// NOCI: error: operands are not valid for this GPU or mode
 v_exp_legacy_f32 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: instruction not supported on this GPU
 // VI9: v_cvt_f16_u16_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x72,0x02,0x7e,0x00,0x01,0x09,0xa1]
 v_cvt_f16_u16 v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: instruction not supported on this GPU
 // VI9: v_cvt_f16_i16_dpp v1, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x74,0x02,0x7e,0x00,0x01,0x09,0xa1]

 // GFX9: v_screen_partition_4se_b32_dpp v5, v1 quad_perm:[0,1,2,3] row_mask:0x0 bank_mask:0x0 bound_ctrl:1 ; encoding: [0xfa,0x6e,0x0a,0x7e,0x01,0xe4,0x08,0x00]
 v_screen_partition_4se_b32_dpp v5, v1 quad_perm:[0,1,2,3] row_mask:0x0 bank_mask:0x0 bound_ctrl:0
 
 //===----------------------------------------------------------------------===//
 // Check VOP2 opcodes
 //===----------------------------------------------------------------------===//
 // ToDo: VOP2bInst instructions: v_add_u32, v_sub_u32 ... (vcc and ApplyMnemonic in AsmMatcherEmitter.cpp)
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_mac_f32_dpp v0, v0, v0  row_shl:1 row_mask:0xf bank_mask:0xf ; encoding: [0xfa,0x00,0x00,0x2c,0x00,0x01,0x01,0xff]
 v_mac_f32 v0, v0, v0 row_shl:1
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_mac_f32_dpp v0, v0, v0  row_shr:15 row_mask:0xf bank_mask:0xf ; encoding: [0xfa,0x00,0x00,0x2c,0x00,0x1f,0x01,0xff]
 v_mac_f32 v0, v0, v0 row_shr:0xf
 
-// NOSICI: error: not a valid operand.
-// VI9: v_mac_f32_dpp v0, v0, v0 quad_perm:[1,3,0,1] row_mask:0xa bank_mask:0xf bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x2c,0x00,0x4d,0x08,0xaf]
+// NOSICI: error: operands are not valid for this GPU or mode
+// VI9: v_mac_f32_dpp v0, v0, v0  quad_perm:[1,3,0,1] row_mask:0xa bank_mask:0xf bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x2c,0x00,0x4d,0x08,0xaf]
 v_mac_f32 v0, v0, v0 quad_perm:[1,3,0,1] row_mask:0xa bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_add_f32_dpp v0, v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x02,0x00,0x01,0x09,0xa1]
 v_add_f32 v0, v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_min_f32_dpp v0, v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x14,0x00,0x01,0x09,0xa1]
 v_min_f32 v0, v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_and_b32_dpp v0, v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x00,0x00,0x26,0x00,0x01,0x09,0xa1]
 v_and_b32 v0, v0, v0 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_mul_i32_i24_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x0c,0x02,0x01,0x09,0xa1]
 v_mul_i32_i24 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_sub_f32_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x04,0x02,0x01,0x09,0xa1]
 v_sub_f32 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_subrev_f32_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x06,0x02,0x01,0x09,0xa1]
 v_subrev_f32 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_mul_f32_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x0a,0x02,0x01,0x09,0xa1]
 v_mul_f32 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_mul_hi_i32_i24_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x0e,0x02,0x01,0x09,0xa1]
 v_mul_hi_i32_i24 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_mul_u32_u24_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x10,0x02,0x01,0x09,0xa1]
 v_mul_u32_u24 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_mul_hi_u32_u24_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x12,0x02,0x01,0x09,0xa1]
 v_mul_hi_u32_u24 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // VI9: v_max_f32_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x16,0x02,0x01,0x09,0xa1]
 v_max_f32 v1, v2 v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.
 // VI9: v_min_i32_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x18,0x02,0x01,0x09,0xa1]
 v_min_i32 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand.

 // VI9: v_min_i16_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x64,0x02,0x01,0x09,0xa1]
 v_min_i16 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: instruction not supported on this GPU
 // VI9: v_ldexp_f16_dpp v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x66,0x02,0x01,0x09,0xa1]
 v_ldexp_f16 v1, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: instruction not supported on this GPU
-// NOGFX9: error: not a valid operand.
+// NOGFX9: error: operands are not valid for this GPU or mode
 // VI: v_add_u32_dpp v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x32,0x02,0x01,0x09,0xa1]
 v_add_u32 v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: instruction not supported on this GPU
-// NOGFX9: error: not a valid operand.
+// NOGFX9: error: operands are not valid for this GPU or mode
 // VI: v_sub_u32_dpp v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x34,0x02,0x01,0x09,0xa1]
 v_sub_u32 v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: instruction not supported on this GPU
-// NOGFX9: error: not a valid operand.
+// NOGFX9: error: operands are not valid for this GPU or mode
 // VI: v_subrev_u32_dpp v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x36,0x02,0x01,0x09,0xa1]
 v_subrev_u32 v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // NOGFX9: error: instruction not supported on this GPU
 // VI: v_addc_u32_dpp v1, vcc, v2, v3, vcc row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x38,0x02,0x01,0x09,0xa1]
 v_addc_u32 v1, vcc, v2, v3, vcc row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // NOGFX9: error: instruction not supported on this GPU
 // VI: v_subb_u32_dpp v1, vcc, v2, v3, vcc row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x3a,0x02,0x01,0x09,0xa1]
 v_subb_u32 v1, vcc, v2, v3, vcc row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // NOGFX9: error: instruction not supported on this GPU
 // VI: v_subbrev_u32_dpp v1, vcc, v2, v3, vcc row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x3c,0x02,0x01,0x09,0xa1]
 v_subbrev_u32 v1, vcc, v2, v3, vcc row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // NOVI: error: instruction not supported on this GPU
 // GFX9: v_add_co_u32_dpp v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x32,0x02,0x01,0x09,0xa1]
 v_add_co_u32 v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // NOVI: error: instruction not supported on this GPU
 // GFX9: v_sub_co_u32_dpp v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x34,0x02,0x01,0x09,0xa1]
 v_sub_co_u32 v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand.
+// NOSICI: error: operands are not valid for this GPU or mode
 // NOVI: error: instruction not supported on this GPU
 // GFX9: v_subrev_co_u32_dpp v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x36,0x02,0x01,0x09,0xa1]
 v_subrev_co_u32 v1, vcc, v2, v3 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: instruction not supported on this GPU
 // NOVI: error: instruction not supported on this GPU
 // GFX9: v_addc_co_u32_dpp v1, vcc, v2, v3, vcc row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:1 ; encoding: [0xfa,0x06,0x02,0x38,0x02,0x01,0x09,0xa1]
 v_addc_co_u32 v1, vcc, v2, v3, vcc row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0

 // VI9:    v_cndmask_b32_dpp v5, v1, v2, vcc quad_perm:[0,1,2,3] row_mask:0x0 bank_mask:0x0 ; encoding: [0xfa,0x04,0x0a,0x00,0x01,0xe4,0x00,0x00]
 v_cndmask_b32_dpp v5, v1, v2, vcc quad_perm:[0,1,2,3] row_mask:0x0 bank_mask:0x0
 
 // NOSICI: error: not a valid operand.
 // VI9:    v_cndmask_b32_dpp v5, v1, v2, vcc row_shl:15 row_mask:0x0 bank_mask:0x0 ; encoding: [0xfa,0x04,0x0a,0x00,0x01,0x0f,0x01,0x00]
 v_cndmask_b32_dpp v5, v1, v2, vcc row_shl:15 row_mask:0x0 bank_mask:0x0
 
 //===----------------------------------------------------------------------===//
-// Check that immideates and scalar regs are not supported
+// Check that immediates and scalar regs are not supported
 //===----------------------------------------------------------------------===//
 
 // NOSICI: error: not a valid operand
 // NOVI: error: invalid operand for instruction
 // NOGFX9: error: invalid operand for instruction
 v_mov_b32 v0, 1 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand
 // NOVI: error: invalid operand for instruction
 // NOGFX9: error: invalid operand for instruction
 v_and_b32 v0, 42, v1 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand
+// NOSICI: error: invalid operand for instruction
 // NOVI: error: invalid operand for instruction
 // NOGFX9: error: invalid operand for instruction
 v_add_f32 v0, v1, 345 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand
 // NOVI: error: invalid operand for instruction
 // NOGFX9: error: invalid operand for instruction
 v_mov_b32 v0, s1 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 // NOSICI: error: not a valid operand
 // NOVI: error: invalid operand for instruction
 // NOGFX9: error: invalid operand for instruction
 v_and_b32 v0, s42, v1 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
-// NOSICI: error: not a valid operand
+// NOSICI: error: invalid operand for instruction
 // NOVI: error: invalid operand for instruction
 // NOGFX9: error: invalid operand for instruction
 v_add_f32 v0, v1, s45 row_shl:1 row_mask:0xa bank_mask:0x1 bound_ctrl:0
 
 //===----------------------------------------------------------------------===//
 // Validate register size checks (bug 37943)
 //===----------------------------------------------------------------------===//
 

llvm/test/MC/Disassembler/AMDGPU/gfx11_dasm_all.txt

 0x05,0x00,0x55,0xd6,0xff,0x05,0x0e,0x04
 
 # GFX11: v_add3_u32 v5, vcc_hi, v2, v3           ; encoding: [0x05,0x00,0x55,0xd6,0x6b,0x04,0x0e,0x04]
 0x05,0x00,0x55,0xd6,0x6b,0x04,0x0e,0x04
 
 # GFX11: v_add3_u32 v5, vcc_lo, v2, v3           ; encoding: [0x05,0x00,0x55,0xd6,0x6a,0x04,0x0e,0x04]
 0x05,0x00,0x55,0xd6,0x6a,0x04,0x0e,0x04
 
+# GFX11: v_add3_u32_e64_dpp v5, v1, v2, -1 dpp8:[7,6,5,4,3,2,1,0] ; encoding: [0x05,0x00,0x55,0xd6,0xe9,0x04,0x06,0x03,0x01,0x77,0x39,0x05]
+0x05,0x00,0x55,0xd6,0xe9,0x04,0x06,0x03,0x01,0x77,0x39,0x05
+
+# GFX11: v_add3_u32_e64_dpp v5, v1, v2, s4 dpp8:[7,6,5,4,3,2,1,0] ; encoding: [0x05,0x00,0x55,0xd6,0xe9,0x04,0x12,0x00,0x01,0x77,0x39,0x05]
+0x05,0x00,0x55,0xd6,0xe9,0x04,0x12,0x00,0x01,0x77,0x39,0x05
+
 # GFX11: v_add_f64 v[254:255], v[1:2], v[2:3]    ; encoding: [0xfe,0x00,0x27,0xd7,0x01,0x05,0x02,0x00]
 0xfe,0x00,0x27,0xd7,0x01,0x05,0x02,0x00
 
 # GFX11: v_add_f64 v[5:6], 0.5, v[2:3]           ; encoding: [0x05,0x00,0x27,0xd7,0xf0,0x04,0x02,0x00]
 0x05,0x00,0x27,0xd7,0xf0,0x04,0x02,0x00
 
 # GFX11: v_add_f64 v[5:6], 0, v[2:3]             ; encoding: [0x05,0x00,0x27,0xd7,0x80,0x04,0x02,0x00]
 0x05,0x00,0x27,0xd7,0x80,0x04,0x02,0x00

 0x05,0x00,0x10,0xd6,0xff,0x05,0x0e,0x04
 
 # GFX11: v_bfe_u32 v5, vcc_hi, v2, v3            ; encoding: [0x05,0x00,0x10,0xd6,0x6b,0x04,0x0e,0x04]
 0x05,0x00,0x10,0xd6,0x6b,0x04,0x0e,0x04
 
 # GFX11: v_bfe_u32 v5, vcc_lo, v2, v3            ; encoding: [0x05,0x00,0x10,0xd6,0x6a,0x04,0x0e,0x04]
 0x05,0x00,0x10,0xd6,0x6a,0x04,0x0e,0x04
 
+# GFX11: v_bfe_u32_e64_dpp v0, v1, v2, v3 dpp8:[0,1,2,3,4,5,6,7] ; encoding: [0x00,0x00,0x10,0xd6,0xe9,0x04,0x0e,0x04,0x01,0x88,0xc6,0xfa]
+0x00,0x00,0x10,0xd6,0xe9,0x04,0x0e,0x04,0x01,0x88,0xc6,0xfa
+
+# GFX11: v_bfe_u32_e64_dpp v0, v1, v2, v3 quad_perm:[3,2,1,0] row_mask:0x0 bank_mask:0x0 ; encoding: [0x00,0x00,0x10,0xd6,0xfa,0x04,0x0e,0x04,0x01,0x1b,0x00,0x00]
+0x00,0x00,0x10,0xd6,0xfa,0x04,0x0e,0x04,0x01,0x1b,0x00,0x00
+
 # GFX11: v_bfi_b32 v255, v1, v2, v3              ; encoding: [0xff,0x00,0x12,0xd6,0x01,0x05,0x0e,0x04]
 0xff,0x00,0x12,0xd6,0x01,0x05,0x0e,0x04
 
 # GFX11: v_bfi_b32 v5, 0.5, v2, v3               ; encoding: [0x05,0x00,0x12,0xd6,0xf0,0x04,0x0e,0x04]
 0x05,0x00,0x12,0xd6,0xf0,0x04,0x0e,0x04
 
 # GFX11: v_bfi_b32 v5, 0, v2, v3                 ; encoding: [0x05,0x00,0x12,0xd6,0x80,0x04,0x0e,0x04]
 0x05,0x00,0x12,0xd6,0x80,0x04,0x0e,0x04

 0x05,0x00,0x0d,0xd6,0xff,0x05,0x0e,0x04
 
 # GFX11: v_cubesc_f32 v5, vcc_hi, v2, v3         ; encoding: [0x05,0x00,0x0d,0xd6,0x6b,0x04,0x0e,0x04]
 0x05,0x00,0x0d,0xd6,0x6b,0x04,0x0e,0x04
 
 # GFX11: v_cubesc_f32 v5, vcc_lo, v2, v3         ; encoding: [0x05,0x00,0x0d,0xd6,0x6a,0x04,0x0e,0x04]
 0x05,0x00,0x0d,0xd6,0x6a,0x04,0x0e,0x04
 
+# GFX11: v_cubesc_f32_e64_dpp v5, v1, v2, 1 row_shr:4 row_mask:0xf bank_mask:0xf ; encoding: [0x05,0x00,0x0d,0xd6,0xfa,0x04,0x06,0x02,0x01,0x14,0x01,0xff]
+0x05,0x00,0x0d,0xd6,0xfa,0x04,0x06,0x02,0x01,0x14,0x01,0xff
+
+# GFX11: v_cubesc_f32_e64_dpp v5, v1, v2, s2 row_shr:4 row_mask:0xf bank_mask:0xf ; encoding: [0x05,0x00,0x0d,0xd6,0xfa,0x04,0x0a,0x00,0x01,0x14,0x01,0xff]
+0x05,0x00,0x0d,0xd6,0xfa,0x04,0x0a,0x00,0x01,0x14,0x01,0xff
+
 # GFX11: v_cubetc_f32 v255, v1, v2, v3           ; encoding: [0xff,0x00,0x0e,0xd6,0x01,0x05,0x0e,0x04]
 0xff,0x00,0x0e,0xd6,0x01,0x05,0x0e,0x04
 
 # GFX11: v_cubetc_f32 v5, 0.5, v2, v3            ; encoding: [0x05,0x00,0x0e,0xd6,0xf0,0x04,0x0e,0x04]
 0x05,0x00,0x0e,0xd6,0xf0,0x04,0x0e,0x04
 
 # GFX11: v_cubetc_f32 v5, 0, v2, v3              ; encoding: [0x05,0x00,0x0e,0xd6,0x80,0x04,0x0e,0x04]
 0x05,0x00,0x0e,0xd6,0x80,0x04,0x0e,0x04

 0x05,0x00,0x13,0xd6,0x6a,0x04,0x0e,0x04
 
 # GFX11: v_fma_f32 v5, v1, v2, null              ; encoding: [0x05,0x00,0x13,0xd6,0x01,0x05,0xf2,0x01]
 0x05,0x00,0x13,0xd6,0x01,0x05,0xf2,0x01
 
 # GFX11: v_fma_f32 v5, v1, null, v3              ; encoding: [0x05,0x00,0x13,0xd6,0x01,0xf9,0x0c,0x04]
 0x05,0x00,0x13,0xd6,0x01,0xf9,0x0c,0x04
 
+# GFX11: v_fma_f32_e64_dpp v80, v81, v82, v81 dpp8:[0,1,6,3,4,5,6,7] ; encoding: [0x50,0x00,0x13,0xd6,0xe9,0xa4,0x46,0x05,0x51,0x88,0xc7,0xfa]
+0x50,0x00,0x13,0xd6,0xe9,0xa4,0x46,0x05,0x51,0x88,0xc7,0xfa
+
+# GFX11: v_fma_f32_e64_dpp v80, v81, |v82|, v81 dpp8:[0,1,6,3,4,5,6,7] ; encoding: [0x50,0x02,0x13,0xd6,0xe9,0xa4,0x46,0x05,0x51,0x88,0xc7,0xfa]
+0x50,0x02,0x13,0xd6,0xe9,0xa4,0x46,0x05,0x51,0x88,0xc7,0xfa
+
+# GFX11: v_fma_f32_e64_dpp v93, |v94|, v95, v94 quad_perm:[3,2,1,0] row_mask:0xf bank_mask:0xe ; encoding: [0x5d,0x01,0x13,0xd6,0xfa,0xbe,0x7a,0x05,0x5e,0x1b,0x00,0xfe]
+0x5d,0x01,0x13,0xd6,0xfa,0xbe,0x7a,0x05,0x5e,0x1b,0x00,0xfe
+
 # GFX11: v_fma_dx9_zero_f32 v255, v1, v2, v3     ; encoding: [0xff,0x00,0x09,0xd6,0x01,0x05,0x0e,0x04]
 0xff,0x00,0x09,0xd6,0x01,0x05,0x0e,0x04
 
 # GFX11: v_fma_dx9_zero_f32 v5, 0.5, v2, v3      ; encoding: [0x05,0x00,0x09,0xd6,0xf0,0x04,0x0e,0x04]
 0x05,0x00,0x09,0xd6,0xf0,0x04,0x0e,0x04
 
 # GFX11: v_fma_dx9_zero_f32 v5, 0, v2, v3        ; encoding: [0x05,0x00,0x09,0xd6,0x80,0x04,0x0e,0x04]
 0x05,0x00,0x09,0xd6,0x80,0x04,0x0e,0x04

 0x05,0x00,0x56,0xd6,0xff,0x05,0x0e,0x04
 
 # GFX11: v_lshl_or_b32 v5, vcc_hi, v2, v3        ; encoding: [0x05,0x00,0x56,0xd6,0x6b,0x04,0x0e,0x04]
 0x05,0x00,0x56,0xd6,0x6b,0x04,0x0e,0x04
 
 # GFX11: v_lshl_or_b32 v5, vcc_lo, v2, v3        ; encoding: [0x05,0x00,0x56,0xd6,0x6a,0x04,0x0e,0x04]
 0x05,0x00,0x56,0xd6,0x6a,0x04,0x0e,0x04
 
+# GFX11: v_lshl_or_b32_e64_dpp v255, v5, v0, vcc_hi row_xmask:6 row_mask:0x0 bank_mask:0xf fi:1 ; encoding: [0xff,0x00,0x56,0xd6,0xfa,0x00,0xae,0x01,0x05,0x66,0x05,0x0f]
+0xff,0x00,0x56,0xd6,0xfa,0x00,0xae,0x01,0x05,0x66,0x05,0x0f
+
 # GFX11: v_lshlrev_b64 v[254:255], v1, v[2:3]    ; encoding: [0xfe,0x00,0x3c,0xd7,0x01,0x05,0x02,0x00]
 0xfe,0x00,0x3c,0xd7,0x01,0x05,0x02,0x00
 
 # GFX11: v_lshlrev_b64 v[5:6], 0.5, v[2:3]       ; encoding: [0x05,0x00,0x3c,0xd7,0xf0,0x04,0x02,0x00]
 0x05,0x00,0x3c,0xd7,0xf0,0x04,0x02,0x00
 
 # GFX11: v_lshlrev_b64 v[5:6], 0, v[2:3]         ; encoding: [0x05,0x00,0x3c,0xd7,0x80,0x04,0x02,0x00]
 0x05,0x00,0x3c,0xd7,0x80,0x04,0x02,0x00

 0x05,0x00,0x1c,0xd6,0xff,0x05,0x0e,0x04
 
 # GFX11: v_max3_f32 v5, vcc_hi, v2, v3           ; encoding: [0x05,0x00,0x1c,0xd6,0x6b,0x04,0x0e,0x04]
 0x05,0x00,0x1c,0xd6,0x6b,0x04,0x0e,0x04
 
 # GFX11: v_max3_f32 v5, vcc_lo, v2, v3           ; encoding: [0x05,0x00,0x1c,0xd6,0x6a,0x04,0x0e,0x04]
 0x05,0x00,0x1c,0xd6,0x6a,0x04,0x0e,0x04
 
+# GFX11: v_max3_f32_e64_dpp v5, v1, v2, s3 dpp8:[7,6,5,4,3,2,1,0] ; encoding: [0x05,0x00,0x1c,0xd6,0xe9,0x04,0x0e,0x00,0x01,0x77,0x39,0x05]
+0x05,0x00,0x1c,0xd6,0xe9,0x04,0x0e,0x00,0x01,0x77,0x39,0x05
+
 # GFX11: v_max3_i16 v255, v1, v2, v3             ; encoding: [0xff,0x00,0x4d,0xd6,0x01,0x05,0x0e,0x04]
 0xff,0x00,0x4d,0xd6,0x01,0x05,0x0e,0x04
 
 # GFX11: v_max3_i16 v5, 0, v2, v3                ; encoding: [0x05,0x00,0x4d,0xd6,0x80,0x04,0x0e,0x04]
 0x05,0x00,0x4d,0xd6,0x80,0x04,0x0e,0x04
 
 # GFX11: v_max3_i16 v5, 0x3800, v2, v3           ; encoding: [0x05,0x00,0x4d,0xd6,0xff,0x04,0x0e,0x04,0x00,0x38,0x00,0x00]
 0x05,0x00,0x4d,0xd6,0xff,0x04,0x0e,0x04,0x00,0x38,0x00,0x00

 0x05,0x00,0x2a,0xd7,0x01,0xd5,0x00,0x00
 
 # GFX11: v_max_f64 v[5:6], v[254:255], v[2:3]    ; encoding: [0x05,0x00,0x2a,0xd7,0xfe,0x05,0x02,0x00]
 0x05,0x00,0x2a,0xd7,0xfe,0x05,0x02,0x00
 
 # GFX11: v_max_f64 v[5:6], vcc, v[2:3]           ; encoding: [0x05,0x00,0x2a,0xd7,0x6a,0x04,0x02,0x00]
 0x05,0x00,0x2a,0xd7,0x6a,0x04,0x02,0x00
 
+# GFX11: v_maxmin_f32_e64_dpp v0, v1, v2, v3 clamp dpp8:[0,1,2,3,4,5,6,7] ; encoding: [0x00,0x80,0x5e,0xd6,0xe9,0x04,0x0e,0x04,0x01,0x88,0xc6,0xfa]
+0x00,0x80,0x5e,0xd6,0xe9,0x04,0x0e,0x04,0x01,0x88,0xc6,0xfa
+
+# GFX11: v_maxmin_f32_e64_dpp v0, v1, v2, v3 div:2 dpp8:[0,1,2,3,4,5,6,7] fi:1 ; encoding: [0x00,0x00,0x5e,0xd6,0xea,0x04,0x0e,0x1c,0x01,0x88,0xc6,0xfa]
+0x00,0x00,0x5e,0xd6,0xea,0x04,0x0e,0x1c,0x01,0x88,0xc6,0xfa
+
 # GFX11: v_med3_f16 v255, v1, v2, v3             ; encoding: [0xff,0x00,0x4f,0xd6,0x01,0x05,0x0e,0x04]
 0xff,0x00,0x4f,0xd6,0x01,0x05,0x0e,0x04
 
 # GFX11: v_med3_f16 v5, 0.5, v2, v3              ; encoding: [0x05,0x00,0x4f,0xd6,0xf0,0x04,0x0e,0x04]
 0x05,0x00,0x4f,0xd6,0xf0,0x04,0x0e,0x04
 
 # GFX11: v_med3_f16 v5, 0, v2, v3                ; encoding: [0x05,0x00,0x4f,0xd6,0x80,0x04,0x0e,0x04]
 0x05,0x00,0x4f,0xd6,0x80,0x04,0x0e,0x04

 0x05,0x00,0x1b,0xd6,0xff,0x05,0x0e,0x04
 
 # GFX11: v_min3_u32 v5, vcc_hi, v2, v3           ; encoding: [0x05,0x00,0x1b,0xd6,0x6b,0x04,0x0e,0x04]
 0x05,0x00,0x1b,0xd6,0x6b,0x04,0x0e,0x04
 
 # GFX11: v_min3_u32 v5, vcc_lo, v2, v3           ; encoding: [0x05,0x00,0x1b,0xd6,0x6a,0x04,0x0e,0x04]
 0x05,0x00,0x1b,0xd6,0x6a,0x04,0x0e,0x04
 
+# GFX11: v_minmax_f32_e64_dpp v0, -v1, -v2, -v3 dpp8:[0,1,2,3,4,5,6,7] ; encoding: [0x00,0x00,0x5f,0xd6,0xe9,0x04,0x0e,0xe4,0x01,0x88,0xc6,0xfa]
+0x00,0x00,0x5f,0xd6,0xe9,0x04,0x0e,0xe4,0x01,0x88,0xc6,0xfa
+
+# GFX11: v_minmax_f32_e64_dpp v0, |v1|, v2, v3 dpp8:[0,1,2,3,4,5,6,7] ; encoding: [0x00,0x01,0x5f,0xd6,0xe9,0x04,0x0e,0x04,0x01,0x88,0xc6,0xfa]
+0x00,0x01,0x5f,0xd6,0xe9,0x04,0x0e,0x04,0x01,0x88,0xc6,0xfa
+
 # GFX11: v_min_f64 v[254:255], v[1:2], v[2:3]    ; encoding: [0xfe,0x00,0x29,0xd7,0x01,0x05,0x02,0x00]
 0xfe,0x00,0x29,0xd7,0x01,0x05,0x02,0x00
 
 # GFX11: v_min_f64 v[5:6], 0.5, v[2:3]           ; encoding: [0x05,0x00,0x29,0xd7,0xf0,0x04,0x02,0x00]
 0x05,0x00,0x29,0xd7,0xf0,0x04,0x02,0x00
 
 # GFX11: v_min_f64 v[5:6], 0, v[2:3]             ; encoding: [0x05,0x00,0x29,0xd7,0x80,0x04,0x02,0x00]
 0x05,0x00,0x29,0xd7,0x80,0x04,0x02,0x00

 0x01,0x69,0x0a,0x7e
 
 # GFX11: v_sqrt_f64_e32 v[5:6], v[254:255]       ; encoding: [0xfe,0x69,0x0a,0x7e]
 0xfe,0x69,0x0a,0x7e
 
 # GFX11: v_sqrt_f64_e32 v[5:6], vcc              ; encoding: [0x6a,0x68,0x0a,0x7e]
 0x6a,0x68,0x0a,0x7e
 
+# GFX11: v_sub_nc_i32_e64_dpp v93, v94, v95 row_ror:7 row_mask:0xf bank_mask:0x1 bound_ctrl:1 ; encoding: [0x5d,0x00,0x25,0xd7,0xfa,0xbe,0x02,0x00,0x5e,0x27,0x09,0xf1]
+0x5d,0x00,0x25,0xd7,0xfa,0xbe,0x02,0x00,0x5e,0x27,0x09,0xf1
+
 # GFX11: v_trig_preop_f64 v[254:255], v[1:2], v2 ; encoding: [0xfe,0x00,0x2f,0xd7,0x01,0x05,0x02,0x00]
 0xfe,0x00,0x2f,0xd7,0x01,0x05,0x02,0x00
 
 # GFX11: v_trig_preop_f64 v[5:6], 0.5, v2        ; encoding: [0x05,0x00,0x2f,0xd7,0xf0,0x04,0x02,0x00]
 0x05,0x00,0x2f,0xd7,0xf0,0x04,0x02,0x00
 
 # GFX11: v_trig_preop_f64 v[5:6], 0, v2          ; encoding: [0x05,0x00,0x2f,0xd7,0x80,0x04,0x02,0x00]
 0x05,0x00,0x2f,0xd7,0x80,0x04,0x02,0x00