[AMDGPU][llvm-mc] Add support for sendmsg(...) syntax.

[artem.tamazov]

Summary

Added support for sendmsg(MSG[, OP[, STREAM_ID]]) syntax
in s_sendmsg and s_sendmsghalt instructions.
The syntax matches the SP3 assembler/disassembler rules.
That is why implicit inputs (like M0 and EXEC) are not printed
to disassembly output anymore.

sendmsg(...) allows only _supported_ message types and attributes,
even if literals are used instead of symbolic names.
However, raw literal (without "sendmsg") still can be used,
and that allows for any 16-bit value.

Tests updated/added.

[arsenm]

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

1837–1839 ↗

(On Diff #55249)

cases at same indentation as switch

1883 ↗

(On Diff #55249)

There should be an enum for the symbolic name/values

1884 ↗

(On Diff #55249)

demorgans law this

1893 ↗

(On Diff #55249)

Capitalize and period

1901 ↗

(On Diff #55249)

Why is this loop needed?

lib/Target/AMDGPU/InstPrinter/AMDGPUInstPrinter.cpp

778 ↗

(On Diff #55249)

llvm_unreachable

786 ↗

(On Diff #55249)

llvm_unreachable

790 ↗

(On Diff #55249)

Single quotes

800 ↗

(On Diff #55249)

Space before (

805 ↗

(On Diff #55249)

llvm_unreachable

807 ↗

(On Diff #55249)

Single quotes

810 ↗

(On Diff #55249)

Another while(0)

[SamWot]

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

1838 ↗

(On Diff #55249)

This should be MatchOperand_NoMatch. Otherwise this will prevent other operand types to be parsed.

1845–1846 ↗

(On Diff #55249)

What are those error messages?
Is it "unrecognized instruction mnemonic"?

lib/Target/AMDGPU/SIInstructions.td

25–27 ↗

(On Diff #55249)

This and other Operands and AsmOperandClasses should be moved to SIInstrInfo.td. It might be done in different change.

[artem.tamazov]

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

1884 ↗

(On Diff #55249)

I expect extension of this expression to something like

if (! ((1 <= Operation.Id && Operation.Id <= 4)
    ||(IsSomeNewAmdHw() && 5 <= Operation.Id && Operation.Id <= 6))) {...

If you not insist, I would better keep current code to minimize future changes.

1901 ↗

(On Diff #55249)

Actually this is not a loop. "break" within do block; jumps just after while. That avoids gotos.

lib/Target/AMDGPU/InstPrinter/AMDGPUInstPrinter.cpp

810 ↗

(On Diff #55249)

Another idiom to avoid goto.

[artem.tamazov]

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

1838 ↗

(On Diff #55249)

Only immediates are supported in this context.

1845–1846 ↗

(On Diff #55249)

No. An immediate must fit into 16 bits.

[artem.tamazov]

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

1838 ↗

(On Diff #55249)

Pardon, immediates and sendmsg(...) handmade "type" are supported, but not other types. So we don't need to parse those.

[SamWot]

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

1838 ↗

(On Diff #55249)

LLVM parser tries to parse various operand types until it meets Success or ParseFail. If it meets NoMatch then it tryies next operand type. So if some instruction would have SendMsgImm and some other operands allowed in same place (for example if SendMsgImm is optional) then returning ParseFail will prevent trying to parse other operand.
For now this might work but I suggest to replace it with NoMatch here.

[artem.tamazov]

lib/Target/AMDGPU/SIInstructions.td

25–27 ↗

(On Diff #55249)

Will do separately.

[artem.tamazov]

Review-related fixes.

[artem.tamazov]

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

1883 ↗

(On Diff #55249)

Matthew, please let me know if you want to rework hwreg(...) support in the same manner.

[artem.tamazov]

ParseFail -> NoMatch. Ready to next review round.

[SamWot]

LGTM

[artem.tamazov]

(I am going to submit this soon if no objections.)

llvm/trunk/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MC/MCSymbolELF.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/ELF.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/raw_ostream.h"
 
+// FIXME ODR: Move this to some common place for AsmParser and InstPrinter
+namespace llvm {
+namespace AMDGPU {
+namespace SendMsg {
+
+// This must be in sync with llvm::AMDGPU::SendMsg::Id enum members.
+static
+const char* const IdSymbolic[] = {
+  nullptr,
+  "MSG_INTERRUPT",
+  "MSG_GS",
+  "MSG_GS_DONE",
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  "MSG_SYSMSG"
+};
+
+// These two must be in sync with llvm::AMDGPU::SendMsg::Op enum members.
+static
+const char* const OpSysSymbolic[] = {
+  nullptr,
+  "SYSMSG_OP_ECC_ERR_INTERRUPT",
+  "SYSMSG_OP_REG_RD",
+  "SYSMSG_OP_HOST_TRAP_ACK",
+  "SYSMSG_OP_TTRACE_PC"
+};
+
+static
+const char* const OpGsSymbolic[] = {
+  "GS_OP_NOP",
+  "GS_OP_CUT",
+  "GS_OP_EMIT",
+  "GS_OP_EMIT_CUT"
+};
+
+} // namespace SendMsg
+} // namespace AMDGPU
+} // namespace llvm
+
 using namespace llvm;
 
 namespace {
 
 struct OptionalOperand;
 
 enum RegisterKind { IS_UNKNOWN, IS_VGPR, IS_SGPR, IS_TTMP, IS_SPECIAL };
 
enum ImmTy {
     ImmTySdwaSel,
     ImmTySdwaDstUnused,
     ImmTyDMask,
     ImmTyUNorm,
     ImmTyDA,
     ImmTyR128,
     ImmTyLWE,
     ImmTyHwreg,
+    ImmTySendMsg,
   };
 
   struct TokOp {
     const char *Data;
     unsigned Length;
   };
 
   struct ImmOp {
void printImmTy(raw_ostream& OS, ImmTy Type) const {
     case ImmTySdwaSel: OS << "SdwaSel"; break;
     case ImmTySdwaDstUnused: OS << "SdwaDstUnused"; break;
     case ImmTyDMask: OS << "DMask"; break;
     case ImmTyUNorm: OS << "UNorm"; break;
     case ImmTyDA: OS << "DA"; break;
     case ImmTyR128: OS << "R128"; break;
     case ImmTyLWE: OS << "LWE"; break;
     case ImmTyHwreg: OS << "Hwreg"; break;
+    case ImmTySendMsg: OS << "SendMsg"; break;
     }
   }
 
   void print(raw_ostream &OS) const override {
     switch (Kind) {
     case Register:
       OS << "<register " << getReg() << " mods: " << Reg.Modifiers << '>';
       break;
static AMDGPUOperand::Ptr CreateExpr(const class MCExpr *Expr, SMLoc S) {
     Op->Expr = Expr;
     Op->StartLoc = S;
     Op->EndLoc = S;
     return Op;
   }
 
   bool isSWaitCnt() const;
   bool isHwreg() const;
+  bool isSendMsg() const;
   bool isMubufOffset() const;
   bool isSMRDOffset() const;
   bool isSMRDLiteralOffset() const;
   bool isDPPCtrl() const;
 };
 
 class AMDGPUAsmParser : public MCTargetAsmParser {
   const MCInstrInfo &MII;
public:
 
   void cvtDSOffset01(MCInst &Inst, const OperandVector &Operands);
   void cvtDS(MCInst &Inst, const OperandVector &Operands);
 
   bool parseCnt(int64_t &IntVal);
   OperandMatchResultTy parseSWaitCntOps(OperandVector &Operands);
   bool parseHwregOperand(int64_t &HwRegCode, int64_t &Offset, int64_t &Width, bool &IsIdentifier);
   OperandMatchResultTy parseHwreg(OperandVector &Operands);
+private:
+  struct OperandInfoTy {
+    int64_t Id;
+    bool IsSymbolic;
+    OperandInfoTy(int64_t Id_) : Id(Id_), IsSymbolic(false) { }
+  };
+  bool parseSendMsg(OperandInfoTy &Msg, OperandInfoTy &Operation, int64_t &StreamId);
+public:
+  OperandMatchResultTy parseSendMsgOp(OperandVector &Operands);
   OperandMatchResultTy parseSOppBrTarget(OperandVector &Operands);
   AMDGPUOperand::Ptr defaultHwreg() const;
 
 
   void cvtMubuf(MCInst &Inst, const OperandVector &Operands);
   AMDGPUOperand::Ptr defaultMubufOffset() const;
   AMDGPUOperand::Ptr defaultGLC() const;
   AMDGPUOperand::Ptr defaultSLC() const;

 bool AMDGPUOperand::isHwreg() const {
   return isImmTy(ImmTyHwreg);
 }
 
 AMDGPUOperand::Ptr AMDGPUAsmParser::defaultHwreg() const {
   return AMDGPUOperand::CreateImm(0, SMLoc(), AMDGPUOperand::ImmTyHwreg);
 }
 
+bool AMDGPUAsmParser::parseSendMsg(OperandInfoTy &Msg, OperandInfoTy &Operation, int64_t &StreamId) {
+  using namespace llvm::AMDGPU::SendMsg;
+
+  if (Parser.getTok().getString() != "sendmsg")
+    return true;
+  Parser.Lex();
+
+  if (getLexer().isNot(AsmToken::LParen))
+    return true;
+  Parser.Lex();
+
+  if (getLexer().is(AsmToken::Identifier)) {
+    Msg.IsSymbolic = true;
+    Msg.Id = ID_UNKNOWN_;
+    const std::string tok = Parser.getTok().getString();
+    for (int i = ID_GAPS_FIRST_; i < ID_GAPS_LAST_; ++i) {
+      switch(i) {
+        default: continue; // Omit gaps.
+        case ID_INTERRUPT: case ID_GS: case ID_GS_DONE:  case ID_SYSMSG: break;
+      }
+      if (tok == IdSymbolic[i]) {
+        Msg.Id = i;
+        break;
+      }
+    }
+    Parser.Lex();
+  } else {
+    Msg.IsSymbolic = false;
+    if (getLexer().isNot(AsmToken::Integer))
+      return true;
+    if (getParser().parseAbsoluteExpression(Msg.Id))
+      return true;
+    if (getLexer().is(AsmToken::Integer))
+      if (getParser().parseAbsoluteExpression(Msg.Id))
+        Msg.Id = ID_UNKNOWN_;
+  }
+  if (Msg.Id == ID_UNKNOWN_) // Don't know how to parse the rest.
+    return false;
+
+  if (!(Msg.Id == ID_GS || Msg.Id == ID_GS_DONE || Msg.Id == ID_SYSMSG)) {
+    if (getLexer().isNot(AsmToken::RParen))
+      return true;
+    Parser.Lex();
+    return false;
+  }
+
+  if (getLexer().isNot(AsmToken::Comma))
+    return true;
+  Parser.Lex();
+
+  assert(Msg.Id == ID_GS || Msg.Id == ID_GS_DONE || Msg.Id == ID_SYSMSG);
+  Operation.Id = ID_UNKNOWN_;
+  if (getLexer().is(AsmToken::Identifier)) {
+    Operation.IsSymbolic = true;
+    const char* const *S = (Msg.Id == ID_SYSMSG) ? OpSysSymbolic : OpGsSymbolic;
+    const int F = (Msg.Id == ID_SYSMSG) ? OP_SYS_FIRST_ : OP_GS_FIRST_;
+    const int L = (Msg.Id == ID_SYSMSG) ? OP_SYS_LAST_ : OP_GS_LAST_;
+    const std::string Tok = Parser.getTok().getString();
+    for (int i = F; i < L; ++i) {
+      if (Tok == S[i]) {
+        Operation.Id = i;
+        break;
+      }
+    }
+    Parser.Lex();
+  } else {
+    Operation.IsSymbolic = false;
+    if (getLexer().isNot(AsmToken::Integer))
+      return true;
+    if (getParser().parseAbsoluteExpression(Operation.Id))
+      return true;
+  }
+
+  if ((Msg.Id == ID_GS || Msg.Id == ID_GS_DONE) && Operation.Id != OP_GS_NOP) {
+    // Stream id is optional.
+    if (getLexer().is(AsmToken::RParen)) {
+      Parser.Lex();
+      return false;
+    }
+
+    if (getLexer().isNot(AsmToken::Comma))
+      return true;
+    Parser.Lex();
+
+    if (getLexer().isNot(AsmToken::Integer))
+      return true;
+    if (getParser().parseAbsoluteExpression(StreamId))
+      return true;
+  }
+
+  if (getLexer().isNot(AsmToken::RParen))
+    return true;
+  Parser.Lex();
+  return false;
+}
+
+AMDGPUAsmParser::OperandMatchResultTy
+AMDGPUAsmParser::parseSendMsgOp(OperandVector &Operands) {
+  using namespace llvm::AMDGPU::SendMsg;
+
+  int64_t Imm16Val = 0;
+  SMLoc S = Parser.getTok().getLoc();
+
+  switch(getLexer().getKind()) {
+  default:
+    return MatchOperand_NoMatch;
+  case AsmToken::Integer:
+    // The operand can be an integer value.
+    if (getParser().parseAbsoluteExpression(Imm16Val))
+      return MatchOperand_NoMatch;
+    if (!isInt<16>(Imm16Val) && !isUInt<16>(Imm16Val)) {
+      Error(S, "invalid immediate: only 16-bit values are legal");
+      // Do not return error code, but create an imm operand anyway and proceed
+      // to the next operand, if any. That avoids unneccessary error messages.
+    }
+    break;
+  case AsmToken::Identifier: {
+      OperandInfoTy Msg(ID_UNKNOWN_);
+      OperandInfoTy Operation(OP_UNKNOWN_);
+      int64_t StreamId = STREAM_ID_DEFAULT;
+      if (parseSendMsg(Msg, Operation, StreamId))
+        return MatchOperand_NoMatch;
+      do {
+        // Validate and encode message ID.
+        if (! ((ID_INTERRUPT <= Msg.Id && Msg.Id <= ID_GS_DONE)
+                || Msg.Id == ID_SYSMSG)) {
+          if (Msg.IsSymbolic)
+            Error(S, "invalid/unsupported symbolic name of message");
+          else
+            Error(S, "invalid/unsupported code of message");
+          break;
+        }
+        Imm16Val = Msg.Id;
+        // Validate and encode operation ID.
+        if (Msg.Id == ID_GS || Msg.Id == ID_GS_DONE) {
+          if (! (OP_GS_FIRST_ <= Operation.Id && Operation.Id < OP_GS_LAST_)) {
+            if (Operation.IsSymbolic)
+              Error(S, "invalid symbolic name of GS_OP");
+            else
+              Error(S, "invalid code of GS_OP: only 2-bit values are legal");
+            break;
+          }
+          if (Operation.Id == OP_GS_NOP
+              && Msg.Id != ID_GS_DONE) {
+            Error(S, "invalid GS_OP: NOP is for GS_DONE only");
+            break;
+          }
+          Imm16Val |= (Operation.Id << OP_SHIFT_);
+        }
+        if (Msg.Id == ID_SYSMSG) {
+          if (! (OP_SYS_FIRST_ <= Operation.Id && Operation.Id < OP_SYS_LAST_)) {
+            if (Operation.IsSymbolic)
+              Error(S, "invalid/unsupported symbolic name of SYSMSG_OP");
+            else
+              Error(S, "invalid/unsupported code of SYSMSG_OP");
+            break;
+          }
+          Imm16Val |= (Operation.Id << OP_SHIFT_);
+        }
+        // Validate and encode stream ID.
+        if ((Msg.Id == ID_GS || Msg.Id == ID_GS_DONE) && Operation.Id != OP_GS_NOP) {
+          if (! (STREAM_ID_FIRST_ <= StreamId && StreamId < STREAM_ID_LAST_)) {
+            Error(S, "invalid stream id: only 2-bit values are legal");
+            break;
+          }
+          Imm16Val |= (StreamId << STREAM_ID_SHIFT_);
+        }
+      } while (0);
+    }
+    break;
+  }
+  Operands.push_back(AMDGPUOperand::CreateImm(Imm16Val, S, AMDGPUOperand::ImmTySendMsg));
+  return MatchOperand_Success;
+}
+
+bool AMDGPUOperand::isSendMsg() const {
+  return isImmTy(ImmTySendMsg);
+}
+
 //===----------------------------------------------------------------------===//
 // sopp branch targets
 //===----------------------------------------------------------------------===//
 
 AMDGPUAsmParser::OperandMatchResultTy
 AMDGPUAsmParser::parseSOppBrTarget(OperandVector &Operands) {
   SMLoc S = Parser.getTok().getLoc();
 

llvm/trunk/lib/Target/AMDGPU/InstPrinter/AMDGPUInstPrinter.cpp

 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/raw_ostream.h"
 
 #include <string>
 
+// FIXME ODR: Move this to some common place for AsmParser and InstPrinter
+namespace llvm {
+namespace AMDGPU {
+namespace SendMsg {
+
+// This must be in sync with llvm::AMDGPU::SendMsg::Id enum members.
+static
+const char* const IdSymbolic[] = {
+  nullptr,
+  "MSG_INTERRUPT",
+  "MSG_GS",
+  "MSG_GS_DONE",
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  nullptr,
+  "MSG_SYSMSG"
+};
+
+// These two must be in sync with llvm::AMDGPU::SendMsg::Op enum members.
+static
+const char* const OpSysSymbolic[] = {
+  nullptr,
+  "SYSMSG_OP_ECC_ERR_INTERRUPT",
+  "SYSMSG_OP_REG_RD",
+  "SYSMSG_OP_HOST_TRAP_ACK",
+  "SYSMSG_OP_TTRACE_PC"
+};
+
+static
+const char* const OpGsSymbolic[] = {
+  "GS_OP_NOP",
+  "GS_OP_CUT",
+  "GS_OP_EMIT",
+  "GS_OP_EMIT_CUT"
+};
+
+} // namespace SendMsg
+} // namespace AMDGPU
+} // namespace llvm
+
 using namespace llvm;
 
 void AMDGPUInstPrinter::printInst(const MCInst *MI, raw_ostream &OS,
                                   StringRef Annot, const MCSubtargetInfo &STI) {
   OS.flush();
   printInstruction(MI, OS);
 
   printAnnotation(OS, Annot);
if (KCacheMode > 0) {
     int KCacheAddr = MI->getOperand(OpNo + 2).getImm();
     int LineSize = (KCacheMode == 1) ? 16 : 32;
     O << KCacheAddr * 16 << '-' << KCacheAddr * 16 + LineSize;
   }
 }
 
 void AMDGPUInstPrinter::printSendMsg(const MCInst *MI, unsigned OpNo,
                                      raw_ostream &O) {
-  unsigned SImm16 = MI->getOperand(OpNo).getImm();
-  unsigned Msg = SImm16 & 0xF;
-  if (Msg == 2 || Msg == 3) {
-    unsigned Op = (SImm16 >> 4) & 0xF;
-    if (Msg == 3)
-      O << "Gs_done(";
-    else
-      O << "Gs(";
-    if (Op == 0) {
-      O << "nop";
-    } else {
-      unsigned Stream = (SImm16 >> 8) & 0x3;
-      if (Op == 1)
-        O << "cut";
-      else if (Op == 2)
-        O << "emit";
-      else if (Op == 3)
-        O << "emit-cut";
-      O << " stream " << Stream;
-    }
-    O << "), [m0] ";
-  } else if (Msg == 1)
-    O << "interrupt ";
-  else if (Msg == 15)
-    O << "system ";
-  else
-    O << "unknown(" << Msg << ") ";
+  using namespace llvm::AMDGPU::SendMsg;
+
+  const unsigned SImm16 = MI->getOperand(OpNo).getImm();
+  const unsigned Id = SImm16 & ID_MASK_;
+  do {
+    if (Id == ID_INTERRUPT) {
+      if ((SImm16 & ~ID_MASK_) != 0) // Unused/unknown bits must be 0.
+        break;
+      O << "sendmsg(" << IdSymbolic[Id] << ')';
+      return;
+    }
+    if (Id == ID_GS || Id == ID_GS_DONE) {
+      if ((SImm16 & ~(ID_MASK_|OP_GS_MASK_|STREAM_ID_MASK_)) != 0) // Unused/unknown bits must be 0.
+        break;
+      const unsigned OpGs = (SImm16 & OP_GS_MASK_) >> OP_SHIFT_;
+      const unsigned StreamId = (SImm16 & STREAM_ID_MASK_) >> STREAM_ID_SHIFT_;
+      if (OpGs == OP_GS_NOP && Id != ID_GS_DONE) // NOP to be used for GS_DONE only.
+        break;
+      if (OpGs == OP_GS_NOP && StreamId != 0) // NOP does not use/define stream id bits.
+        break;
+      O << "sendmsg(" << IdSymbolic[Id] << ", " << OpGsSymbolic[OpGs];
+      if (OpGs != OP_GS_NOP) {  O << ", " << StreamId; }
+      O << ')';
+      return;
+    }
+    if (Id == ID_SYSMSG) {
+      if ((SImm16 & ~(ID_MASK_|OP_SYS_MASK_)) != 0) // Unused/unknown bits must be 0.
+        break;
+      const unsigned OpSys = (SImm16 & OP_SYS_MASK_) >> OP_SHIFT_;
+      if (! (OP_SYS_FIRST_ <= OpSys && OpSys < OP_SYS_LAST_)) // Unused/unknown.
+        break;
+      O << "sendmsg(" << IdSymbolic[Id] << ", " << OpSysSymbolic[OpSys] << ')';
+      return;
+    }
+  } while (0);
+  O << SImm16; // Unknown simm16 code.
 }
 
 void AMDGPUInstPrinter::printWaitFlag(const MCInst *MI, unsigned OpNo,
                                       raw_ostream &O) {
   unsigned SImm16 = MI->getOperand(OpNo).getImm();
   unsigned Vmcnt = SImm16 & 0xF;
   unsigned Expcnt = (SImm16 >> 4) & 0x7;
   unsigned Lgkmcnt = (SImm16 >> 8) & 0xF;

llvm/trunk/lib/Target/AMDGPU/SIDefines.h

namespace SIOutMods {
   enum {
     NONE = 0,
     MUL2 = 1,
     MUL4 = 2,
     DIV2 = 3
   };
 }
 
+namespace llvm {
+namespace AMDGPU {
+namespace SendMsg { // Encoding of SIMM16 used in s_sendmsg* insns.
+
+enum Id { // Message ID, width(3) [3:0].
+  ID_UNKNOWN_ = -1,
+  ID_INTERRUPT = 1,
+  ID_GS,
+  ID_GS_DONE,
+  ID_SYSMSG = 15,
+  ID_GAPS_LAST_, // Indicate that sequence has gaps.
+  ID_GAPS_FIRST_ = ID_INTERRUPT,
+  ID_MASK_ = 0xf
+};
+
+enum Op { // Both GS and SYS operation IDs.
+  OP_UNKNOWN_ = -1,
+  // width(2) [5:4]
+  OP_GS_NOP = 0,
+  OP_GS_CUT,
+  OP_GS_EMIT,
+  OP_GS_EMIT_CUT,
+  OP_GS_LAST_,
+  OP_GS_FIRST_ = OP_GS_NOP,
+  OP_GS_MASK_ = (0x3 << 4),
+  // width(3) [6:4]
+  OP_SYS_ECC_ERR_INTERRUPT = 1,
+  OP_SYS_REG_RD,
+  OP_SYS_HOST_TRAP_ACK,
+  OP_SYS_TTRACE_PC,
+  OP_SYS_LAST_,
+  OP_SYS_FIRST_ = OP_SYS_ECC_ERR_INTERRUPT,
+  OP_SYS_MASK_ = (0x7 << 4),
+  OP_SHIFT_ = 4
+};
+
+enum StreamId { // Stream ID, (2) [9:8].
+  STREAM_ID_DEFAULT = 0,
+  STREAM_ID_LAST_ = 4,
+  STREAM_ID_FIRST_ = STREAM_ID_DEFAULT,
+  STREAM_ID_MASK_ = (0x3 << 8),
+  STREAM_ID_SHIFT_ = 8
+};
+
+} // namespace SendMsg
+} // namespace AMDGPU
+} // namespace llvm
+
 #define R_00B028_SPI_SHADER_PGM_RSRC1_PS                                0x00B028
 #define R_00B02C_SPI_SHADER_PGM_RSRC2_PS                                0x00B02C
 #define   S_00B02C_EXTRA_LDS_SIZE(x)                                  (((x) & 0xFF) << 8)
 #define R_00B128_SPI_SHADER_PGM_RSRC1_VS                                0x00B128
 #define R_00B228_SPI_SHADER_PGM_RSRC1_GS                                0x00B228
 #define R_00B848_COMPUTE_PGM_RSRC1                                      0x00B848
 #define   S_00B028_VGPRS(x)                                           (((x) & 0x3F) << 0)
 #define   S_00B028_SGPRS(x)                                           (((x) & 0x0F) << 6)

llvm/trunk/lib/Target/AMDGPU/SIInstrInfo.td

class NamedOperandU16<string Name, AsmOperandClass MatchClass> : Operand<i16> {
   let ParserMatchClass = MatchClass;
 }
 
 class NamedOperandU32<string Name, AsmOperandClass MatchClass> : Operand<i32> {
   let PrintMethod = "print"#Name;
   let ParserMatchClass = MatchClass;
 }
 
+def SendMsgMatchClass : AsmOperandClass {
+  let Name = "SendMsg";
+  let PredicateMethod = "isSendMsg";
+  let ParserMethod = "parseSendMsgOp";
+  let RenderMethod = "addImmOperands";
+}
+
 let OperandType = "OPERAND_IMMEDIATE" in {
 
 def offen : NamedOperandBit<"Offen", NamedMatchClass<"Offen">>;
 def idxen : NamedOperandBit<"Idxen", NamedMatchClass<"Idxen">>;
 def addr64 : NamedOperandBit<"Addr64", NamedMatchClass<"Addr64">>;
 
 def offset : NamedOperandU16<"Offset", NamedMatchClass<"Offset">>;
 def offset0 : NamedOperandU8<"Offset0", NamedMatchClass<"Offset0">>;

llvm/trunk/lib/Target/AMDGPU/SIInstructions.td

 def INTERP : InterpSlots;
 
 def InterpSlot : Operand<i32> {
   let PrintMethod = "printInterpSlot";
 }
 
 def SendMsgImm : Operand<i32> {
   let PrintMethod = "printSendMsg";
+  let ParserMatchClass = SendMsgMatchClass;
 }
 
 def isGCN : Predicate<"Subtarget->getGeneration() "
                       ">= AMDGPUSubtarget::SOUTHERN_ISLANDS">,
             AssemblerPredicate<"FeatureGCN">;
 def isSI : Predicate<"Subtarget->getGeneration() "
                       "== AMDGPUSubtarget::SOUTHERN_ISLANDS">,
            AssemblerPredicate<"FeatureSouthernIslands">;

 
 let Uses = [EXEC, M0] in {
   // FIXME: Should this be mayLoad+mayStore?
   def S_SENDMSG : SOPP <0x00000010, (ins SendMsgImm:$simm16), "s_sendmsg $simm16",
       [(AMDGPUsendmsg (i32 imm:$simm16))]
   >;
 } // End Uses = [EXEC, M0]
 
-def S_SENDMSGHALT : SOPP <0x00000011, (ins i16imm:$simm16), "s_sendmsghalt $simm16">;
+def S_SENDMSGHALT : SOPP <0x00000011, (ins SendMsgImm:$simm16), "s_sendmsghalt $simm16">;
 def S_TRAP : SOPP <0x00000012, (ins i16imm:$simm16), "s_trap $simm16">;
 def S_ICACHE_INV : SOPP <0x00000013, (ins), "s_icache_inv"> {
   let simm16 = 0;
 }
 def S_INCPERFLEVEL : SOPP <0x00000014, (ins i16imm:$simm16), "s_incperflevel $simm16">;
 def S_DECPERFLEVEL : SOPP <0x00000015, (ins i16imm:$simm16), "s_decperflevel $simm16">;
 def S_TTRACEDATA : SOPP <0x00000016, (ins), "s_ttracedata"> {
   let simm16 = 0;

llvm/trunk/test/CodeGen/AMDGPU/llvm.SI.sendmsg-m0.ll

 ;RUN: llc < %s -march=amdgcn -mcpu=verde -verify-machineinstrs | FileCheck --check-prefix=SI --check-prefix=BOTH %s
 ;RUN: llc < %s -march=amdgcn -mcpu=tonga -verify-machineinstrs | FileCheck --check-prefix=VI --check-prefix=BOTH %s
 
 ; BOTH-LABEL: {{^}}main:
 ; BOTH: s_mov_b32 m0, s0
 ; VI-NEXT: s_nop 0
-; BOTH-NEXT: s_sendmsg Gs_done(nop)
+; BOTH-NEXT: sendmsg(MSG_GS_DONE, GS_OP_NOP)
 ; BOTH-NEXT: s_endpgm
 
 define amdgpu_gs void @main(i32 inreg %a) #0 {
 main_body:
   call void @llvm.SI.sendmsg(i32 3, i32 %a)
   ret void
 }
 
 ; Function Attrs: nounwind
 declare void @llvm.SI.sendmsg(i32, i32) #1
 
 attributes #0 = { "unsafe-fp-math"="true" }
 attributes #1 = { nounwind }

llvm/trunk/test/CodeGen/AMDGPU/llvm.SI.sendmsg.ll

 ;RUN: llc < %s -march=amdgcn -mcpu=verde -verify-machineinstrs | FileCheck %s
 ;RUN: llc < %s -march=amdgcn -mcpu=tonga -verify-machineinstrs | FileCheck %s
 
 ; CHECK-LABEL: {{^}}main:
 ; CHECK: s_mov_b32 m0, 0
 ; CHECK-NOT: s_mov_b32 m0
-; CHECK: s_sendmsg Gs(emit stream 0)
-; CHECK: s_sendmsg Gs(cut stream 1)
-; CHECK: s_sendmsg Gs(emit-cut stream 2)
-; CHECK: s_sendmsg Gs_done(nop)
+; CHECK: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT, 0)
+; CHECK: s_sendmsg sendmsg(MSG_GS, GS_OP_CUT, 1)
+; CHECK: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT_CUT, 2)
+; CHECK: s_sendmsg sendmsg(MSG_GS_DONE, GS_OP_NOP)
 
 define void @main() {
 main_body:
   call void @llvm.SI.sendmsg(i32 34, i32 0);
   call void @llvm.SI.sendmsg(i32 274, i32 0);
   call void @llvm.SI.sendmsg(i32 562, i32 0);
   call void @llvm.SI.sendmsg(i32 3, i32 0);
   ret void
 }
 
 ; Function Attrs: nounwind
 declare void @llvm.SI.sendmsg(i32, i32) #0
 
 attributes #0 = { nounwind }

llvm/trunk/test/MC/AMDGPU/sopp-err.s

+// RUN: not llvm-mc -arch=amdgcn -show-encoding %s 2>&1 | FileCheck --check-prefix=GCN --check-prefix=SICI %s
+// RUN: not llvm-mc -arch=amdgcn -mcpu=SI -show-encoding %s 2>&1 | FileCheck --check-prefix=GCN --check-prefix=SICI %s
+// RUN: not llvm-mc -arch=amdgcn -mcpu=fiji -show-encoding %s 2>&1 | FileCheck --check-prefix=GCN --check-prefix=VI %s
+
+s_sendmsg sendmsg(11)
+// GCN: error: invalid/unsupported code of message
+
+s_sendmsg sendmsg(MSG_INTERRUPTX)
+// GCN: error: invalid/unsupported symbolic name of message
+
+s_sendmsg sendmsg(MSG_INTERRUPT, 0)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(MSG_GS)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(MSG_GS, GS_OP_NOP)
+// GCN: error: invalid GS_OP: NOP is for GS_DONE only
+
+s_sendmsg sendmsg(MSG_GS, GS_OP_CUT, 0, 0)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(MSG_GSX, GS_OP_CUT, 0)
+// GCN: error: invalid/unsupported symbolic name of message
+
+s_sendmsg sendmsg(MSG_GS, GS_OP_CUTX, 0)
+// GCN: error: invalid symbolic name of GS_OP
+
+s_sendmsg sendmsg(MSG_GS, GS_OP_CUT, 4)
+// GCN: error: invalid stream id: only 2-bit values are legal
+
+s_sendmsg sendmsg(2)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(2, 0)
+// GCN: error: invalid GS_OP: NOP is for GS_DONE only
+
+s_sendmsg sendmsg(2, 3, 0, 0)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(2, 4, 1)
+// GCN: error: invalid code of GS_OP: only 2-bit values are legal
+
+s_sendmsg sendmsg(2, 2, 4)
+// GCN: error: invalid stream id: only 2-bit values are legal
+
+s_sendmsg sendmsg(2, 2, 0, 0)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(MSG_GS_DONE, GS_OP_NOP, 0)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(15)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(15, 1, 0)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(15, 0)
+// GCN: error: invalid/unsupported code of SYSMSG_OP
+
+s_sendmsg sendmsg(15, 5)
+// GCN: error: invalid/unsupported code of SYSMSG_OP
+
+s_sendmsg sendmsg(MSG_SYSMSG)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(MSG_SYSMSG, SYSMSG_OP_ECC_ERR_INTERRUPT, 0)
+// GCN: error: not a valid operand
+
+s_sendmsg sendmsg(MSG_SYSMSG, 0)
+// GCN: error: invalid/unsupported code of SYSMSG_OP
+
+s_sendmsg sendmsg(MSG_SYSMSG, 5)
+// GCN: error: invalid/unsupported code of SYSMSG_OP

llvm/trunk/test/MC/AMDGPU/sopp.s

 
 s_sleep 10
 // GCN: s_sleep 10 ; encoding: [0x0a,0x00,0x8e,0xbf]
 
 s_setprio 1
 // GCN: s_setprio 1 ; encoding: [0x01,0x00,0x8f,0xbf]
 
 s_sendmsg 2
-// GCN: s_sendmsg Gs(nop), [m0] ; encoding: [0x02,0x00,0x90,0xbf]
+// GCN: s_sendmsg 2 ; encoding: [0x02,0x00,0x90,0xbf]
+
+s_sendmsg 0x1
+// GCN: s_sendmsg sendmsg(MSG_INTERRUPT) ; encoding: [0x01,0x00,0x90,0xbf]
+
+s_sendmsg sendmsg(1)
+// GCN: s_sendmsg sendmsg(MSG_INTERRUPT) ; encoding: [0x01,0x00,0x90,0xbf]
+
+s_sendmsg sendmsg(MSG_INTERRUPT)
+// GCN: s_sendmsg sendmsg(MSG_INTERRUPT) ; encoding: [0x01,0x00,0x90,0xbf]
+
+s_sendmsg 0x12
+// GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_CUT, 0) ; encoding: [0x12,0x00,0x90,0xbf]
+
+s_sendmsg sendmsg(2, 1)
+// GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_CUT, 0) ; encoding: [0x12,0x00,0x90,0xbf]
+
+s_sendmsg sendmsg(MSG_GS, GS_OP_CUT, 0)
+// GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_CUT, 0) ; encoding: [0x12,0x00,0x90,0xbf]
+
+s_sendmsg 0x122
+// GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT, 1) ; encoding: [0x22,0x01,0x90,0xbf]
+
+s_sendmsg sendmsg(2, 2, 1)
+// GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT, 1) ; encoding: [0x22,0x01,0x90,0xbf]
+
+s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT, 1)
+// GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT, 1) ; encoding: [0x22,0x01,0x90,0xbf]
+
+s_sendmsg 0x232
+// GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT_CUT, 2) ; encoding: [0x32,0x02,0x90,0xbf]
+
+s_sendmsg sendmsg(2, 3, 2)
+// GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT_CUT, 2) ; encoding: [0x32,0x02,0x90,0xbf]
+
+s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT_CUT, 2)
+// GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT_CUT, 2) ; encoding: [0x32,0x02,0x90,0xbf]
+
+s_sendmsg 0x3
+// GCN: s_sendmsg sendmsg(MSG_GS_DONE, GS_OP_NOP) ; encoding: [0x03,0x00,0x90,0xbf]
+
+s_sendmsg sendmsg(3, 0)
+// GCN: s_sendmsg sendmsg(MSG_GS_DONE, GS_OP_NOP) ; encoding: [0x03,0x00,0x90,0xbf]
+
+s_sendmsg sendmsg(MSG_GS_DONE, GS_OP_NOP)
+// GCN: s_sendmsg sendmsg(MSG_GS_DONE, GS_OP_NOP) ; encoding: [0x03,0x00,0x90,0xbf]
+
+s_sendmsg 0x4
+// GCN: s_sendmsg 4 ; encoding: [0x04,0x00,0x90,0xbf]
+
+s_sendmsg 11
+// GCN: s_sendmsg 11 ; encoding: [0x0b,0x00,0x90,0xbf]
+
+s_sendmsg 0x1f
+// GCN: s_sendmsg sendmsg(MSG_SYSMSG, SYSMSG_OP_ECC_ERR_INTERRUPT) ; encoding: [0x1f,0x00,0x90,0xbf]
+
+s_sendmsg sendmsg(15, 1)
+// GCN: s_sendmsg sendmsg(MSG_SYSMSG, SYSMSG_OP_ECC_ERR_INTERRUPT) ; encoding: [0x1f,0x00,0x90,0xbf]
+
+s_sendmsg sendmsg(MSG_SYSMSG, SYSMSG_OP_ECC_ERR_INTERRUPT)
+// GCN: s_sendmsg sendmsg(MSG_SYSMSG, SYSMSG_OP_ECC_ERR_INTERRUPT) ; encoding: [0x1f,0x00,0x90,0xbf]
+
+s_sendmsg 0x6f
+// GCN: s_sendmsg 111 ; encoding: [0x6f,0x00,0x90,0xbf]
 
 s_sendmsghalt 3
-// GCN: s_sendmsghalt 3 ; encoding: [0x03,0x00,0x91,0xbf]
+// GCN: s_sendmsghalt sendmsg(MSG_GS_DONE, GS_OP_NOP) ; encoding: [0x03,0x00,0x91,0xbf]
+
+s_sendmsghalt sendmsg(MSG_GS, GS_OP_EMIT, 1)
+// GCN: s_sendmsghalt sendmsg(MSG_GS, GS_OP_EMIT, 1) ; encoding: [0x22,0x01,0x91,0xbf]
 
 s_trap 4
 // GCN: s_trap 4 ; encoding: [0x04,0x00,0x92,0xbf]
 
 s_icache_inv
 // GCN: s_icache_inv ; encoding: [0x00,0x00,0x93,0xbf]
 
 s_incperflevel 5
 // GCN: s_incperflevel 5 ; encoding: [0x05,0x00,0x94,0xbf]
 
 s_decperflevel 6
 // GCN: s_decperflevel 6 ; encoding: [0x06,0x00,0x95,0xbf]
 
 s_ttracedata
 // GCN: s_ttracedata ; encoding: [0x00,0x00,0x96,0xbf]

llvm/trunk/test/MC/Disassembler/AMDGPU/sopp_vi.txt

 0x09 0x00 0x8d 0xbf
 
 # GCN: s_sleep 10 ; encoding: [0x0a,0x00,0x8e,0xbf]
 0x0a 0x00 0x8e 0xbf
 
 # GCN: s_setprio 1 ; encoding: [0x01,0x00,0x8f,0xbf]
 0x01 0x00 0x8f 0xbf
 
-# GCN: s_sendmsg Gs(nop), [m0] ; encoding: [0x02,0x00,0x90,0xbf]
+# GCN: s_sendmsg 2 ; encoding: [0x02,0x00,0x90,0xbf]
 0x02 0x00 0x90 0xbf
 
-# GCN: s_sendmsghalt 3 ; encoding: [0x03,0x00,0x91,0xbf]
+# GCN: s_sendmsg sendmsg(MSG_INTERRUPT) ; encoding: [0x01,0x00,0x90,0xbf]
+0x01 0x00 0x90 0xbf
+
+# GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_CUT, 0) ; encoding: [0x12,0x00,0x90,0xbf]
+0x12 0x00 0x90 0xbf
+
+# GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT, 1) ; encoding: [0x22,0x01,0x90,0xbf]
+0x22 0x01 0x90 0xbf
+
+# GCN: s_sendmsg sendmsg(MSG_GS, GS_OP_EMIT_CUT, 2) ; encoding: [0x32,0x02,0x90,0xbf]
+0x32 0x02 0x90 0xbf
+
+# GCN: s_sendmsg sendmsg(MSG_GS_DONE, GS_OP_NOP) ; encoding: [0x03,0x00,0x90,0xbf]
+0x03 0x00 0x90 0xbf
+
+# GCN: s_sendmsg 11 ; encoding: [0x0b,0x00,0x90,0xbf]
+0x0b 0x00 0x90 0xbf
+
+# GCN: s_sendmsg sendmsg(MSG_SYSMSG, SYSMSG_OP_ECC_ERR_INTERRUPT) ; encoding: [0x1f,0x00,0x90,0xbf]
+0x1f 0x00 0x90 0xbf
+
+# GCN: s_sendmsg 111 ; encoding: [0x6f,0x00,0x90,0xbf]
+0x6f 0x00 0x90 0xbf
+
+# GCN: s_sendmsghalt sendmsg(MSG_GS_DONE, GS_OP_NOP) ; encoding: [0x03,0x00,0x91,0xbf]
 0x03 0x00 0x91 0xbf
 
+# GCN: s_sendmsghalt sendmsg(MSG_GS, GS_OP_EMIT, 1) ; encoding: [0x22,0x01,0x91,0xbf]
+0x22 0x01 0x91 0xbf
+
+# GCN: s_sendmsghalt 111 ; encoding: [0x6f,0x00,0x91,0xbf]
+0x6f 0x00 0x91 0xbf
+
 # GCN: s_trap 4 ; encoding: [0x04,0x00,0x92,0xbf]
 0x04 0x00 0x92 0xbf
 
 # GCN: s_icache_inv ; encoding: [0x00,0x00,0x93,0xbf]
 0x00 0x00 0x93 0xbf
 
 # GCN: s_incperflevel 5 ; encoding: [0x05,0x00,0x94,0xbf]
 0x05 0x00 0x94 0xbf
 
 # GCN: s_decperflevel 6 ; encoding: [0x06,0x00,0x95,0xbf]
 0x06 0x00 0x95 0xbf
 
 # GCN: s_ttracedata ; encoding: [0x00,0x00,0x96,0xbf]
 0x00 0x00 0x96 0xbf