[AMDGPU][MC] Added support of SCC, VCCZ and EXECZ operands

[dp]

Summary

See bug 39292: https://bugs.llvm.org/show_bug.cgi?id=39292

[rampitec]

I think you need to handle these registers in the SIInstrInfo::usesConstantBus() and AMDGPUAsmParser::usesConstantBus()/AMDGPU::isSGPR().

[dp]

I may be missing something, but these registers are already handled by AMDGPU::isSGPR() because they are defined in SReg_32RegClass.
The same is true for AMDGPUAsmParser::usesConstantBus.

I can add a check to SIInstrInfo::usesConstantBus() but is not it used by codegen only?
As codegen cannot utilize these registers right now, I do not see any use case and will be unable to add any relevant tests.

[rampitec]

In D62660#1523651, @dp wrote:

I may be missing something, but these registers are already handled by AMDGPU::isSGPR() because they are defined in SReg_32RegClass.
The same is true for AMDGPUAsmParser::usesConstantBus.

I can add a check to SIInstrInfo::usesConstantBus() but is not it used by codegen only?
As codegen cannot utilize these registers right now, I do not see any use case and will be unable to add any relevant tests.

Right, isSGPR() uses SReg_32, so this is fine, and AsmParser is also fine. But SIInstrInfo::usesConstantBus() checks for SGPR_32 and SGPR_64, thus missing it. I am not sure if it is possible to write a test, but I would guess an inline asm could be written to violate constant bus limit. I would at least list them there to have no issues if codegen will start using these.

[dp]

Corrected SIInstrInfo::usesConstantBus

[arsenm]

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

3890–3895

This looks like a separate change

lib/Target/AMDGPU/SIInstrInfo.cpp

2834–2849

Can we avoid this switch by replacing the AMDGPU::SGPR_32RegClass.contains with SReg_32?

[dp]

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

3890–3895

No, this was previously checked by this code:

if (isInlineValue(RegNo))
  return !isCI() && !isSI() && !isVI();

As isInlineValue has changed I had to correct the check as well

lib/Target/AMDGPU/SIInstrInfo.cpp

2834–2849

I'll correct this, thanks!

[rampitec]

lib/Target/AMDGPU/SIInstrInfo.cpp

2834–2849

It should not count implicit use of many registers, we need to be careful here as SReg_* is too broad.

[dp]

Corrected SIInstrInfo::usesConstantBus

[rampitec]

lib/Target/AMDGPU/SIInstrInfo.cpp

2836

Implicit exec is also free.

[dp]

lib/Target/AMDGPU/SIInstrInfo.cpp

2836

This statement asserts that only M0 and VCC use constant bus when implicit so all other registers are free (including exec). What am I missing here?

[rampitec]

LGTM

lib/Target/AMDGPU/SIInstrInfo.cpp

2836

Right, OK.

lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

for (const MachineInstr &MI : MBB) {
         case AMDGPU::TBA:
         case AMDGPU::TBA_LO:
         case AMDGPU::TBA_HI:
         case AMDGPU::TMA:
         case AMDGPU::TMA_LO:
         case AMDGPU::TMA_HI:
           llvm_unreachable("trap handler registers should not be used");
 
+        case AMDGPU::SRC_VCCZ:
+          llvm_unreachable("src_vccz register should not be used");
+
+        case AMDGPU::SRC_EXECZ:
+          llvm_unreachable("src_execz register should not be used");
+
+        case AMDGPU::SRC_SCC:
+          llvm_unreachable("src_scc register should not be used");
+
         default:
           break;
         }
 
         if (AMDGPU::SReg_32RegClass.contains(Reg)) {
           assert(!AMDGPU::TTMP_32RegClass.contains(Reg) &&
                  "trap handler registers should not be used");
           IsSGPR = true;

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

 static bool isInlineValue(unsigned Reg) {
   switch (Reg) {
   case AMDGPU::SRC_SHARED_BASE:
   case AMDGPU::SRC_SHARED_LIMIT:
   case AMDGPU::SRC_PRIVATE_BASE:
   case AMDGPU::SRC_PRIVATE_LIMIT:
   case AMDGPU::SRC_POPS_EXITING_WAVE_ID:
     return true;
+  case AMDGPU::SRC_VCCZ:
+  case AMDGPU::SRC_EXECZ:
+  case AMDGPU::SRC_SCC:
+    return true;
   default:
     return false;
   }
 }
 
 bool AMDGPUOperand::isInlineValue() const {
   return isRegKind() && ::isInlineValue(getReg());
 }
return StringSwitch<unsigned>(RegName)
     .Case("src_private_base", AMDGPU::SRC_PRIVATE_BASE)
     .Case("private_limit", AMDGPU::SRC_PRIVATE_LIMIT)
     .Case("src_private_limit", AMDGPU::SRC_PRIVATE_LIMIT)
     .Case("pops_exiting_wave_id", AMDGPU::SRC_POPS_EXITING_WAVE_ID)
     .Case("src_pops_exiting_wave_id", AMDGPU::SRC_POPS_EXITING_WAVE_ID)
     .Case("lds_direct", AMDGPU::LDS_DIRECT)
     .Case("src_lds_direct", AMDGPU::LDS_DIRECT)
     .Case("m0", AMDGPU::M0)
-    .Case("scc", AMDGPU::SCC)
+    .Case("vccz", AMDGPU::SRC_VCCZ)
+    .Case("src_vccz", AMDGPU::SRC_VCCZ)
+    .Case("execz", AMDGPU::SRC_EXECZ)
+    .Case("src_execz", AMDGPU::SRC_EXECZ)
+    .Case("scc", AMDGPU::SRC_SCC)
+    .Case("src_scc", AMDGPU::SRC_SCC)
     .Case("tba", AMDGPU::TBA)
     .Case("tma", AMDGPU::TMA)
     .Case("flat_scratch_lo", AMDGPU::FLAT_SCR_LO)
     .Case("flat_scratch_hi", AMDGPU::FLAT_SCR_HI)
     .Case("xnack_mask_lo", AMDGPU::XNACK_MASK_LO)
     .Case("xnack_mask_hi", AMDGPU::XNACK_MASK_HI)
     .Case("vcc_lo", AMDGPU::VCC_LO)
     .Case("vcc_hi", AMDGPU::VCC_HI)
bool AMDGPUAsmParser::subtargetHasRegister(const MCRegisterInfo &MRI,
   // GFX10 has 2 more SGPRs 104 and 105.
   for (MCRegAliasIterator R(AMDGPU::SGPR104_SGPR105, &MRI, true);
        R.isValid(); ++R) {
     if (*R == RegNo)
       return hasSGPR104_SGPR105();
   }
 
   switch (RegNo) {
+  case AMDGPU::SRC_SHARED_BASE:
+  case AMDGPU::SRC_SHARED_LIMIT:
+  case AMDGPU::SRC_PRIVATE_BASE:
+  case AMDGPU::SRC_PRIVATE_LIMIT:
+  case AMDGPU::SRC_POPS_EXITING_WAVE_ID:
+    return !isCI() && !isSI() && !isVI();
   case AMDGPU::TBA:
   case AMDGPU::TBA_LO:
   case AMDGPU::TBA_HI:
   case AMDGPU::TMA:
   case AMDGPU::TMA_LO:
   case AMDGPU::TMA_HI:
     return !isGFX9() && !isGFX10();
   case AMDGPU::XNACK_MASK:
   case AMDGPU::XNACK_MASK_LO:
   case AMDGPU::XNACK_MASK_HI:
     return !isCI() && !isSI() && !isGFX10() && hasXNACK();
   case AMDGPU::SGPR_NULL:
     return isGFX10();
   default:
     break;
   }
 
-  if (isInlineValue(RegNo))
-    return !isCI() && !isSI() && !isVI();
-
   if (isCI())
     return true;
 
   if (isSI() || isGFX10()) {
     // No flat_scr on SI.
     // On GFX10 flat scratch is not a valid register operand and can only be
     // accessed with s_setreg/s_getreg.
     switch (RegNo) {

lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

MCOperand AMDGPUDisassembler::decodeSpecialReg32(unsigned Val) const {
   case 125: return createRegOperand(SGPR_NULL);
   case 126: return createRegOperand(EXEC_LO);
   case 127: return createRegOperand(EXEC_HI);
   case 235: return createRegOperand(SRC_SHARED_BASE);
   case 236: return createRegOperand(SRC_SHARED_LIMIT);
   case 237: return createRegOperand(SRC_PRIVATE_BASE);
   case 238: return createRegOperand(SRC_PRIVATE_LIMIT);
   case 239: return createRegOperand(SRC_POPS_EXITING_WAVE_ID);
-    // ToDo: no support for vccz register
-  case 251: break;
-    // ToDo: no support for execz register
-  case 252: break;
-  case 253: return createRegOperand(SCC);
+  case 251: return createRegOperand(SRC_VCCZ);
+  case 252: return createRegOperand(SRC_EXECZ);
+  case 253: return createRegOperand(SRC_SCC);
   case 254: return createRegOperand(LDS_DIRECT);
   default: break;
   }
   return errOperand(Val, "unknown operand encoding " + Twine(Val));
 }
 
 MCOperand AMDGPUDisassembler::decodeSpecialReg64(unsigned Val) const {
   using namespace AMDGPU;
 
   switch (Val) {
   case 102: return createRegOperand(FLAT_SCR);
   case 104: return createRegOperand(XNACK_MASK);
   case 106: return createRegOperand(VCC);
   case 108: return createRegOperand(TBA);
   case 110: return createRegOperand(TMA);
   case 126: return createRegOperand(EXEC);
   case 235: return createRegOperand(SRC_SHARED_BASE);
   case 236: return createRegOperand(SRC_SHARED_LIMIT);
   case 237: return createRegOperand(SRC_PRIVATE_BASE);
   case 238: return createRegOperand(SRC_PRIVATE_LIMIT);
   case 239: return createRegOperand(SRC_POPS_EXITING_WAVE_ID);
+  case 251: return createRegOperand(SRC_VCCZ);
+  case 252: return createRegOperand(SRC_EXECZ);
+  case 253: return createRegOperand(SRC_SCC);
   default: break;
   }
   return errOperand(Val, "unknown operand encoding " + Twine(Val));
 }
 
 MCOperand AMDGPUDisassembler::decodeSDWASrc(const OpWidthTy Width,
                                             const unsigned Val) const {
   using namespace AMDGPU::SDWA;

lib/Target/AMDGPU/MCTargetDesc/AMDGPUInstPrinter.cpp

 }
 
 void AMDGPUInstPrinter::printRegOperand(unsigned RegNo, raw_ostream &O,
                                         const MCRegisterInfo &MRI) {
   switch (RegNo) {
   case AMDGPU::VCC:
     O << "vcc";
     return;
-  case AMDGPU::SCC:
-    O << "scc";
+  case AMDGPU::SRC_VCCZ:
+    O << "src_vccz";
+    return;
+  case AMDGPU::SRC_EXECZ:
+    O << "src_execz";
+    return;
+  case AMDGPU::SRC_SCC:
+    O << "src_scc";
     return;
   case AMDGPU::EXEC:
     O << "exec";
     return;
   case AMDGPU::M0:
     O << "m0";
     return;
   case AMDGPU::SGPR_NULL:
void AMDGPUInstPrinter::printRegOperand(unsigned RegNo, raw_ostream &O,
   case AMDGPU::XNACK_MASK_HI:
     O << "xnack_mask_hi";
     return;
   case AMDGPU::FP_REG:
   case AMDGPU::SP_REG:
   case AMDGPU::SCRATCH_WAVE_OFFSET_REG:
   case AMDGPU::PRIVATE_RSRC_REG:
     llvm_unreachable("pseudo-register should not ever be emitted");
+  case AMDGPU::SCC:
+    llvm_unreachable("pseudo scc should not ever be emitted");
   default:
     break;
   }
 
   // The low 8 bits of the encoding value is the register index, for both VGPRs
   // and SGPRs.
   unsigned RegIdx = MRI.getEncodingValue(RegNo) & ((1 << 8) - 1);
 

lib/Target/AMDGPU/SIInstrInfo.cpp

bool SIInstrInfo::usesConstantBus(const MachineRegisterInfo &MRI,
   // FLAT_SCR is just an SGPR pair.
   if (!MO.isImplicit() && (MO.getReg() == AMDGPU::FLAT_SCR))
     return true;
 
   // EXEC register uses the constant bus.
   if (!MO.isImplicit() && MO.getReg() == AMDGPU::EXEC)
     return true;
 
   // SGPRs use the constant bus
-  return (MO.getReg() == AMDGPU::VCC || MO.getReg() == AMDGPU::M0 ||
-          (!MO.isImplicit() &&
-           (AMDGPU::SGPR_32RegClass.contains(MO.getReg()) ||
-            AMDGPU::SGPR_64RegClass.contains(MO.getReg()))));
+  switch (MO.getReg()) {
+    case AMDGPU::M0:
+    case AMDGPU::VCC:
+    case AMDGPU::SRC_SCC:
+    case AMDGPU::SRC_VCCZ:
+    case AMDGPU::SRC_EXECZ:
+    case AMDGPU::SRC_SHARED_BASE:
+    case AMDGPU::SRC_SHARED_LIMIT:
+    case AMDGPU::SRC_PRIVATE_BASE:
+    case AMDGPU::SRC_PRIVATE_LIMIT:
+    case AMDGPU::SRC_POPS_EXITING_WAVE_ID:
+      return true;
+    default:
+      break;
+  }
+
+  return !MO.isImplicit() &&
+          AMDGPU::SGPR_64RegClass.contains(MO.getReg()));
 }
 
 static unsigned findImplicitSGPRRead(const MachineInstr &MI) {
   for (const MachineOperand &MO : MI.implicit_operands()) {
     // We only care about reads.
     if (MO.isDef())
       continue;
 

lib/Target/AMDGPU/SIRegisterInfo.cpp

BitVector SIRegisterInfo::getReservedRegs(const MachineFunction &MF) const {
   // EXEC_LO and EXEC_HI could be allocated and used as regular register, but
   // this seems likely to result in bugs, so I'm marking them as reserved.
   reserveRegisterTuples(Reserved, AMDGPU::EXEC);
   reserveRegisterTuples(Reserved, AMDGPU::FLAT_SCR);
 
   // M0 has to be reserved so that llvm accepts it as a live-in into a block.
   reserveRegisterTuples(Reserved, AMDGPU::M0);
 
+  // Reserve src_vccz, src_execz, src_scc.
+  reserveRegisterTuples(Reserved, AMDGPU::SRC_VCCZ);
+  reserveRegisterTuples(Reserved, AMDGPU::SRC_EXECZ);
+  reserveRegisterTuples(Reserved, AMDGPU::SRC_SCC);
+
   // Reserve the memory aperture registers.
   reserveRegisterTuples(Reserved, AMDGPU::SRC_SHARED_BASE);
   reserveRegisterTuples(Reserved, AMDGPU::SRC_SHARED_LIMIT);
   reserveRegisterTuples(Reserved, AMDGPU::SRC_PRIVATE_BASE);
   reserveRegisterTuples(Reserved, AMDGPU::SRC_PRIVATE_LIMIT);
 
   // Reserve src_pops_exiting_wave_id - support is not implemented in Codegen.
   reserveRegisterTuples(Reserved, AMDGPU::SRC_POPS_EXITING_WAVE_ID);

lib/Target/AMDGPU/SIRegisterInfo.td

 
 def EXEC : RegisterWithSubRegs<"EXEC", [EXEC_LO, EXEC_HI]>,
            DwarfRegAlias<EXEC_LO> {
   let Namespace = "AMDGPU";
   let SubRegIndices = [sub0, sub1];
   let HWEncoding = 126;
 }
 
-def SCC : SIReg<"scc", 253>;
+// 32-bit real registers, for MC only.
+// May be used with both 32-bit and 64-bit operands.
+def SRC_VCCZ : SIReg<"src_vccz", 251>;
+def SRC_EXECZ : SIReg<"src_execz", 252>;
+def SRC_SCC : SIReg<"src_scc", 253>;
+
+// 1-bit pseudo register, for codegen only.
+// Should never be emitted.
+def SCC : SIReg<"">;
+
 def M0 : SIReg <"m0", 124>;
 def SGPR_NULL : SIReg<"null", 125>;
 
 def SRC_SHARED_BASE : SIReg<"src_shared_base", 235>;
 def SRC_SHARED_LIMIT : SIReg<"src_shared_limit", 236>;
 def SRC_PRIVATE_BASE : SIReg<"src_private_base", 237>;
 def SRC_PRIVATE_LIMIT : SIReg<"src_private_limit", 238>;
 def SRC_POPS_EXITING_WAVE_ID : SIReg<"src_pops_exiting_wave_id", 239>;
def LDS_DIRECT_CLASS : RegisterClass<"AMDGPU", [i32, f32, i16, f16, v2i16, v2f16], 32,
   let CopyCost = -1;
 }
 
 // Subset of SReg_32 without M0 for SMRD instructions and alike.
 // See comments in SIInstructions.td for more info.
 def SReg_32_XM0_XEXEC : RegisterClass<"AMDGPU", [i32, f32, i16, f16, v2i16, v2f16], 32,
   (add SGPR_32, VCC_LO, VCC_HI, FLAT_SCR_LO, FLAT_SCR_HI, XNACK_MASK_LO, XNACK_MASK_HI,
    SGPR_NULL, TTMP_32, TMA_LO, TMA_HI, TBA_LO, TBA_HI, SRC_SHARED_BASE, SRC_SHARED_LIMIT,
-   SRC_PRIVATE_BASE, SRC_PRIVATE_LIMIT, SRC_POPS_EXITING_WAVE_ID)> {
+   SRC_PRIVATE_BASE, SRC_PRIVATE_LIMIT, SRC_POPS_EXITING_WAVE_ID,
+   SRC_VCCZ, SRC_EXECZ, SRC_SCC)> {
   let AllocationPriority = 8;
 }
 
 def SReg_32_XEXEC_HI : RegisterClass<"AMDGPU", [i32, f32, i16, f16, v2i16, v2f16], 32,
   (add SReg_32_XM0_XEXEC, EXEC_LO, M0_CLASS)> {
   let AllocationPriority = 8;
 }
 

test/MC/AMDGPU/literals.s

 // RUN: not llvm-mc -arch=amdgcn -show-encoding %s | FileCheck %s --check-prefix=GCN --check-prefix=SI --check-prefix=SICI
 // RUN: not llvm-mc -arch=amdgcn -mcpu=tahiti -show-encoding %s | FileCheck %s --check-prefix=GCN --check-prefix=SI --check-prefix=SICI
-// RUN: not llvm-mc -arch=amdgcn -mcpu=bonaire -show-encoding %s | FileCheck %s --check-prefix=GCN --check-prefix=SICI --check-prefix=CIVI
+// RUN: not llvm-mc -arch=amdgcn -mcpu=bonaire -show-encoding %s | FileCheck %s --check-prefix=GCN --check-prefix=SICI --check-prefix=CIVI --check-prefix=CI
 // RUN: not llvm-mc -arch=amdgcn -mcpu=tonga -show-encoding %s | FileCheck %s --check-prefix=GCN --check-prefix=CIVI --check-prefix=GFX89
 // RUN: not llvm-mc -arch=amdgcn -mcpu=gfx900 -show-encoding %s | FileCheck %s --check-prefix=GCN --check-prefix=CIVI --check-prefix=GFX89 --check-prefix=GFX9
 
 // RUN: not llvm-mc -arch=amdgcn -show-encoding %s 2>&1 | FileCheck %s --check-prefix=NOGCN --check-prefix=NOSI --check-prefix=NOSICI --check-prefix=NOSICIVI
 // RUN: not llvm-mc -arch=amdgcn -mcpu=tahiti -show-encoding %s 2>&1 | FileCheck %s --check-prefix=NOGCN --check-prefix=NOSI --check-prefix=NOSICI --check-prefix=NOSICIVI
 // RUN: not llvm-mc -arch=amdgcn -mcpu=bonaire -show-encoding %s 2>&1 | FileCheck %s --check-prefix=NOGCN --check-prefix=NOSICI --check-prefix=NOCIVI --check-prefix=NOSICIVI
 // RUN: not llvm-mc -arch=amdgcn -mcpu=tonga -show-encoding %s 2>&1 | FileCheck %s --check-prefix=NOGCN --check-prefix=NOSICIVI --check-prefix=NOVI --check-prefix=NOGFX89
 // RUN: not llvm-mc -arch=amdgcn -mcpu=gfx900 -show-encoding %s 2>&1 | FileCheck %s --check-prefix=NOGCN --check-prefix=NOGFX89 --check-prefix=NOGFX9

 
 // NOGCN: error: invalid operand for instruction
 v_trunc_f64 v[0:1], 0x100000001
 
 // NOGCN: error: invalid operand for instruction
 v_trunc_f64 v[0:1], 0x1fffffff000
 
 //---------------------------------------------------------------------------//
-// named inline values like shared_base
+// named inline values: scc, vccz, execz
+//---------------------------------------------------------------------------//
+
+// SICI: buffer_atomic_add v0, off, s[0:3], src_scc offset:4095 ; encoding: [0xff,0x0f,0xc8,0xe0,0x00,0x00,0x00,0xfd]
+// GFX89: buffer_atomic_add v0, off, s[0:3], src_scc offset:4095 ; encoding: [0xff,0x0f,0x08,0xe1,0x00,0x00,0x00,0xfd]
+buffer_atomic_add v0, off, s[0:3], scc offset:4095
+
+// SICI: s_add_i32 s0, src_vccz, s0      ; encoding: [0xfb,0x00,0x00,0x81]
+// GFX89: s_add_i32 s0, src_vccz, s0      ; encoding: [0xfb,0x00,0x00,0x81]
+s_add_i32 s0, vccz, s0
+
+// SICI: s_add_i32 s0, src_execz, s0      ; encoding: [0xfc,0x00,0x00,0x81]
+// GFX89: s_add_i32 s0, src_execz, s0      ; encoding: [0xfc,0x00,0x00,0x81]
+s_add_i32 s0, execz, s0
+
+// SICI: s_add_i32 s0, src_scc, s0       ; encoding: [0xfd,0x00,0x00,0x81]
+// GFX89: s_add_i32 s0, src_scc, s0       ; encoding: [0xfd,0x00,0x00,0x81]
+s_add_i32 s0, scc, s0
+
+// SICI: s_and_b64 s[0:1], s[0:1], src_vccz ; encoding: [0x00,0xfb,0x80,0x87]
+// GFX89: s_and_b64 s[0:1], s[0:1], src_vccz ; encoding: [0x00,0xfb,0x80,0x86]
+s_and_b64 s[0:1], s[0:1], src_vccz
+
+// SICI: s_and_b64 s[0:1], s[0:1], src_execz ; encoding: [0x00,0xfc,0x80,0x87]
+// GFX89: s_and_b64 s[0:1], s[0:1], src_execz ; encoding: [0x00,0xfc,0x80,0x86]
+s_and_b64 s[0:1], s[0:1], src_execz
+
+// SICI: s_and_b64 s[0:1], s[0:1], src_scc ; encoding: [0x00,0xfd,0x80,0x87]
+// GFX89: s_and_b64 s[0:1], s[0:1], src_scc ; encoding: [0x00,0xfd,0x80,0x86]
+s_and_b64 s[0:1], s[0:1], src_scc
+
+// NOSICI: error: instruction not supported on this GPU
+// GFX89: v_add_u16_e32 v0, src_vccz, v0  ; encoding: [0xfb,0x00,0x00,0x4c]
+v_add_u16 v0, vccz, v0
+
+// NOSICI: error: not a valid operand
+// NOVI: error: invalid operand for instruction
+// GFX9: v_add_u16_sdwa v0, src_scc, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD ; encoding: [0xf9,0x00,0x00,0x4c,0xfd,0x06,0x86,0x06]
+v_add_u16_sdwa v0, scc, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+
+// NOSICI: error: not a valid operand
+// NOVI: error: invalid operand for instruction
+// GFX9: v_add_u16_sdwa v0, v0, src_scc dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD ; encoding: [0xf9,0xfa,0x01,0x4c,0x00,0x06,0x06,0x86]
+v_add_u16_sdwa v0, v0, scc dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
+
+// NOSICIVI: error: instruction not supported on this GPU
+// GFX9: v_add_u32_e32 v0, src_execz, v0 ; encoding: [0xfc,0x00,0x00,0x68]
+v_add_u32 v0, execz, v0
+
+// NOSICIVI: error: instruction not supported on this GPU
+// GFX9: v_add_u32_e64 v0, src_scc, v0   ; encoding: [0x00,0x00,0x34,0xd1,0xfd,0x00,0x02,0x00]
+v_add_u32_e64 v0, scc, v0
+
+// SICI: v_cmp_eq_i64_e32 vcc, src_scc, v[0:1] ; encoding: [0xfd,0x00,0x44,0x7d]
+// GFX89: v_cmp_eq_i64_e32 vcc, src_scc, v[0:1] ; encoding: [0xfd,0x00,0xc4,0x7d]
+v_cmp_eq_i64 vcc, scc, v[0:1]
+
+// NOSICI: error: instruction not supported on this GPU
+// GFX89: v_max_f16_e32 v0, src_execz, v0 ; encoding: [0xfc,0x00,0x00,0x5a]
+v_max_f16 v0, execz, v0
+
+// SICI: v_max_f32_e32 v0, src_vccz, v0  ; encoding: [0xfb,0x00,0x00,0x20]
+// GFX89: v_max_f32_e32 v0, src_vccz, v0  ; encoding: [0xfb,0x00,0x00,0x16]
+v_max_f32 v0, vccz, v0
+
+// SICI: v_max_f64 v[0:1], src_scc, v[0:1] ; encoding: [0x00,0x00,0xce,0xd2,0xfd,0x00,0x02,0x00]
+// GFX89: v_max_f64 v[0:1], src_scc, v[0:1] ; encoding: [0x00,0x00,0x83,0xd2,0xfd,0x00,0x02,0x00]
+v_max_f64 v[0:1], scc, v[0:1]
+
+// NOSICIVI: error: instruction not supported on this GPU
+// GFX9: v_pk_add_f16 v0, src_execz, v0  ; encoding: [0x00,0x00,0x8f,0xd3,0xfc,0x00,0x02,0x18]
+v_pk_add_f16 v0, execz, v0
+
+// NOSICI: error: not a valid operand
+// GFX89: v_ceil_f16_e64 v0, -src_vccz    ; encoding: [0x00,0x00,0x85,0xd1,0xfb,0x00,0x00,0x20]
+v_ceil_f16 v0, neg(vccz)
+
+// NOSICI: error: not a valid operand
+// GFX89: v_ceil_f16_e64 v0, |src_scc|    ; encoding: [0x00,0x01,0x85,0xd1,0xfd,0x00,0x00,0x00]
+v_ceil_f16 v0, abs(scc)
+
+// NOSI: error: not a valid operand
+// CI: v_ceil_f64_e64 v[5:6], |src_execz| ; encoding: [0x05,0x01,0x30,0xd3,0xfc,0x00,0x00,0x00]
+// GFX89: v_ceil_f64_e64 v[5:6], |src_execz| ; encoding: [0x05,0x01,0x58,0xd1,0xfc,0x00,0x00,0x00]
+v_ceil_f64 v[5:6], |execz|
+
+// NOSI: error: not a valid operand
+// CI: v_ceil_f64_e64 v[5:6], -vcc     ; encoding: [0x05,0x00,0x30,0xd3,0x6a,0x00,0x00,0x20]
+// GFX89: v_ceil_f64_e64 v[5:6], -vcc     ; encoding: [0x05,0x00,0x58,0xd1,0x6a,0x00,0x00,0x20]
+v_ceil_f64 v[5:6], -vcc
+
+// SICI: v_ceil_f32_e64 v0, -src_vccz    ; encoding: [0x00,0x00,0x44,0xd3,0xfb,0x00,0x00,0x20]
+// GFX89: v_ceil_f32_e64 v0, -src_vccz    ; encoding: [0x00,0x00,0x5d,0xd1,0xfb,0x00,0x00,0x20]
+v_ceil_f32 v0, -vccz
+
+// SICI: v_ceil_f32_e64 v0, |src_execz|  ; encoding: [0x00,0x01,0x44,0xd3,0xfc,0x00,0x00,0x00]
+// GFX89: v_ceil_f32_e64 v0, |src_execz|  ; encoding: [0x00,0x01,0x5d,0xd1,0xfc,0x00,0x00,0x00]
+v_ceil_f32 v0, |execz|
+
+// NOSICI: error: not a valid operand
+// NOVI: error: invalid operand for instruction
+// GFX9: v_ceil_f16_sdwa v5, |src_vccz| dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x8a,0x0a,0x7e,0xfb,0x16,0xa6,0x00]
+v_ceil_f16_sdwa v5, |vccz| dst_sel:DWORD dst_unused:UNUSED_PRESERVE
+
+// NOSICI: error: not a valid operand
+// NOVI: error: invalid operand for instruction
+// GFX9: v_ceil_f16_sdwa v5, -src_scc dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x8a,0x0a,0x7e,0xfd,0x16,0x96,0x00]
+v_ceil_f16_sdwa v5, -scc dst_sel:DWORD dst_unused:UNUSED_PRESERVE
+
+// NOSICIVI: error: invalid operand for instruction
+// GFX9: v_ceil_f32_sdwa v5, src_vccz dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x3a,0x0a,0x7e,0xfb,0x16,0x86,0x00]
+v_ceil_f32_sdwa v5, vccz dst_sel:DWORD src0_sel:DWORD
+
+// NOSICIVI: error: invalid operand for instruction
+// GFX9: v_ceil_f32_sdwa v5, |src_execz| dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x3a,0x0a,0x7e,0xfc,0x16,0xa6,0x00]
+v_ceil_f32_sdwa v5, |execz| dst_sel:DWORD src0_sel:DWORD
+
+//---------------------------------------------------------------------------//
+// named inline values: shared_base, shared_limit, private_base, etc
 //---------------------------------------------------------------------------//
 
 // NOSICIVI: error: failed parsing operand.
 // GFX9: buffer_atomic_add v0, off, s[0:3], src_shared_base offset:4095 ; encoding: [0xff,0x0f,0x08,0xe1,0x00,0x00,0x00,0xeb]
 buffer_atomic_add v0, off, s[0:3], src_shared_base offset:4095
 
 // NOSICIVI: error: failed parsing operand.
 // GFX9: s_add_i32 s0, src_shared_base, s0 ; encoding: [0xeb,0x00,0x00,0x81]

 //---------------------------------------------------------------------------//
 // named inline values compete with other scalars for constant bus access
 //---------------------------------------------------------------------------//
 
 // NOSICIVI: error: failed parsing operand.
 // NOGFX9: error: invalid operand (violates constant bus restrictions)
 v_add_u32 v0, private_base, s0
 
+// NOSICIVI: error: instruction not supported on this GPU
+// NOGFX9: error: invalid operand (violates constant bus restrictions)
+v_add_u32 v0, scc, s0
+
 // v_div_fmas implicitly reads VCC
 // NOSICIVI: error: failed parsing operand.
 // NOGFX9: error: invalid operand (violates constant bus restrictions)
 v_div_fmas_f32 v0, shared_base, v0, v1
 
 // v_div_fmas implicitly reads VCC
 // NOSICIVI: error: failed parsing operand.
 // NOGFX9: error: invalid operand (violates constant bus restrictions)
 v_div_fmas_f32 v0, v0, shared_limit, v1
 
 // v_div_fmas implicitly reads VCC
 // NOSICIVI: error: failed parsing operand.
 // NOGFX9: error: invalid operand (violates constant bus restrictions)
 v_div_fmas_f32 v0, v0, v1, private_limit
 
+// v_div_fmas implicitly reads VCC
+// NOGCN: error: invalid operand (violates constant bus restrictions)
+v_div_fmas_f32 v0, execz, v0, v1
+
+// v_div_fmas implicitly reads VCC
+// NOGCN: error: invalid operand (violates constant bus restrictions)
+v_div_fmas_f32 v0, v0, scc, v1
+
+// v_div_fmas implicitly reads VCC
+// NOGCN: error: invalid operand (violates constant bus restrictions)
+v_div_fmas_f32 v0, v0, v1, vccz
+
 // v_addc_co_u32 implicitly reads VCC (VOP2)
 // NOSICIVI: error: failed parsing operand.
 // NOGFX9: error: invalid operand (violates constant bus restrictions)
 v_addc_co_u32 v0, vcc, shared_base, v0, vcc
 
 // NOSICIVI: error: failed parsing operand.
 // NOGFX9: error: invalid operand (violates constant bus restrictions)
 v_madak_f32 v0, shared_base, v0, 0x11213141
 
+// NOGCN: error: invalid operand (violates constant bus restrictions)
+v_madak_f32 v0, scc, v0, 0x11213141
+
 // NOSICIVI: error: failed parsing operand.
 // NOGFX9: error: invalid operand (violates constant bus restrictions)
 v_cmp_eq_f32 s[0:1], private_base, private_limit
 
 // NOSICIVI: error: failed parsing operand.
 // NOGFX9: error: invalid operand (violates constant bus restrictions)
 v_cmp_eq_f32 s[0:1], private_base, s0
 
+// NOGCN: error: invalid operand (violates constant bus restrictions)
+v_cmp_eq_f32 s[0:1], execz, s0
+
 // NOSICIVI: error: failed parsing operand.
 // NOGFX9: error: invalid operand (violates constant bus restrictions)
 v_pk_add_f16 v255, private_base, private_limit
+
+// NOSICIVI: error: instruction not supported on this GPU
+// NOGFX9: error: invalid operand (violates constant bus restrictions)
+v_pk_add_f16 v255, vccz, execz

test/MC/Disassembler/AMDGPU/literal_gfx9.txt

 # GFX9: v_ceil_f16_sdwa v5, -src_shared_base dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x8a,0x0a,0x7e,0xeb,0x16,0x96,0x00]
 0xf9,0x8a,0x0a,0x7e,0xeb,0x16,0x96,0x00
 
 # GFX9: v_ceil_f32_sdwa v5, src_shared_base dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x3a,0x0a,0x7e,0xeb,0x16,0x86,0x00]
 0xf9,0x3a,0x0a,0x7e,0xeb,0x16,0x86,0x00
 
 # GFX9: v_ceil_f32_sdwa v5, |src_shared_base| dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x3a,0x0a,0x7e,0xeb,0x16,0xa6,0x00]
 0xf9,0x3a,0x0a,0x7e,0xeb,0x16,0xa6,0x00
+
+# GFX9: buffer_atomic_add v0, off, s[0:3], src_scc offset:4095 ; encoding: [0xff,0x0f,0x08,0xe1,0x00,0x00,0x00,0xfd]
+0xff,0x0f,0x08,0xe1,0x00,0x00,0x00,0xfd
+
+# GFX9: s_add_i32 s0, src_vccz, s0      ; encoding: [0xfb,0x00,0x00,0x81]
+0xfb,0x00,0x00,0x81
+
+# GFX9: s_add_i32 s0, src_execz, s0      ; encoding: [0xfc,0x00,0x00,0x81]
+0xfc,0x00,0x00,0x81
+
+# GFX9: s_add_i32 s0, src_scc, s0       ; encoding: [0xfd,0x00,0x00,0x81]
+0xfd,0x00,0x00,0x81
+
+# GFX9: s_and_b64 s[0:1], s[0:1], src_vccz ; encoding: [0x00,0xfb,0x80,0x86]
+0x00,0xfb,0x80,0x86
+
+# GFX9: s_and_b64 s[0:1], s[0:1], src_execz ; encoding: [0x00,0xfc,0x80,0x86]
+0x00,0xfc,0x80,0x86
+
+# GFX9: s_and_b64 s[0:1], s[0:1], src_scc ; encoding: [0x00,0xfd,0x80,0x86]
+0x00,0xfd,0x80,0x86
+
+# GFX9: v_add_u16_e32 v0, src_vccz, v0  ; encoding: [0xfb,0x00,0x00,0x4c]
+0xfb,0x00,0x00,0x4c
+
+# GFX9: v_add_u16_sdwa v0, src_scc, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD ; encoding: [0xf9,0x00,0x00,0x4c,0xfd,0x06,0x86,0x06]
+0xf9,0x00,0x00,0x4c,0xfd,0x06,0x86,0x06
+
+# GFX9: v_add_u16_sdwa v0, v0, src_scc dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD ; encoding: [0xf9,0xfa,0x01,0x4c,0x00,0x06,0x06,0x86]
+0xf9,0xfa,0x01,0x4c,0x00,0x06,0x06,0x86
+
+# GFX9: v_add_u32_e32 v0, src_execz, v0 ; encoding: [0xfc,0x00,0x00,0x68]
+0xfc,0x00,0x00,0x68
+
+# GFX9: v_add_u32_e64 v0, src_scc, v0   ; encoding: [0x00,0x00,0x34,0xd1,0xfd,0x00,0x02,0x00]
+0x00,0x00,0x34,0xd1,0xfd,0x00,0x02,0x00
+
+# GFX9: v_cmp_eq_i64_e32 vcc, src_scc, v[0:1] ; encoding: [0xfd,0x00,0xc4,0x7d]
+0xfd,0x00,0xc4,0x7d
+
+# GFX9: v_max_f16_e32 v0, src_execz, v0 ; encoding: [0xfc,0x00,0x00,0x5a]
+0xfc,0x00,0x00,0x5a
+
+# GFX9: v_max_f32_e32 v0, src_vccz, v0  ; encoding: [0xfb,0x00,0x00,0x16]
+0xfb,0x00,0x00,0x16
+
+# GFX9: v_max_f64 v[0:1], src_scc, v[0:1] ; encoding: [0x00,0x00,0x83,0xd2,0xfd,0x00,0x02,0x00]
+0x00,0x00,0x83,0xd2,0xfd,0x00,0x02,0x00
+
+# GFX9: v_pk_add_f16 v0, src_execz, v0  ; encoding: [0x00,0x00,0x8f,0xd3,0xfc,0x00,0x02,0x18]
+0x00,0x00,0x8f,0xd3,0xfc,0x00,0x02,0x18
+
+# GFX9: v_ceil_f16_e64 v0, -src_vccz    ; encoding: [0x00,0x00,0x85,0xd1,0xfb,0x00,0x00,0x20]
+0x00,0x00,0x85,0xd1,0xfb,0x00,0x00,0x20
+
+# GFX9: v_ceil_f16_e64 v0, |src_scc|    ; encoding: [0x00,0x01,0x85,0xd1,0xfd,0x00,0x00,0x00]
+0x00,0x01,0x85,0xd1,0xfd,0x00,0x00,0x00
+
+# GFX9: v_ceil_f64_e64 v[5:6], |src_execz| ; encoding: [0x05,0x01,0x58,0xd1,0xfc,0x00,0x00,0x00]
+0x05,0x01,0x58,0xd1,0xfc,0x00,0x00,0x00
+
+# GFX9: v_ceil_f64_e64 v[5:6], -vcc     ; encoding: [0x05,0x00,0x58,0xd1,0x6a,0x00,0x00,0x20]
+0x05,0x00,0x58,0xd1,0x6a,0x00,0x00,0x20
+
+# GFX9: v_ceil_f32_e64 v0, -src_vccz    ; encoding: [0x00,0x00,0x5d,0xd1,0xfb,0x00,0x00,0x20]
+0x00,0x00,0x5d,0xd1,0xfb,0x00,0x00,0x20
+
+# GFX9: v_ceil_f32_e64 v0, |src_execz|  ; encoding: [0x00,0x01,0x5d,0xd1,0xfc,0x00,0x00,0x00]
+0x00,0x01,0x5d,0xd1,0xfc,0x00,0x00,0x00
+
+# GFX9: v_ceil_f16_sdwa v5, |src_vccz| dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x8a,0x0a,0x7e,0xfb,0x16,0xa6,0x00]
+0xf9,0x8a,0x0a,0x7e,0xfb,0x16,0xa6,0x00
+
+# GFX9: v_ceil_f16_sdwa v5, -src_scc dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x8a,0x0a,0x7e,0xfd,0x16,0x96,0x00]
+0xf9,0x8a,0x0a,0x7e,0xfd,0x16,0x96,0x00
+
+# GFX9: v_ceil_f32_sdwa v5, src_vccz dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x3a,0x0a,0x7e,0xfb,0x16,0x86,0x00]
+0xf9,0x3a,0x0a,0x7e,0xfb,0x16,0x86,0x00
+
+# GFX9: v_ceil_f32_sdwa v5, |src_execz| dst_sel:DWORD dst_unused:UNUSED_PRESERVE src0_sel:DWORD ; encoding: [0xf9,0x3a,0x0a,0x7e,0xfc,0x16,0xa6,0x00]
+0xf9,0x3a,0x0a,0x7e,0xfc,0x16,0xa6,0x00