[TableGen] AsmMatcher: support for default values for optional operands

[SamWot]

Summary

This change allows to specify "DefaultMethod" for optional operand (IsOptional = 1) in AsmOperandClass that return default value for operand. This is used in convertToMCInst to set default values in MCInst.
Previously if you wanted to set default value for operand you had to create custom converter method. With this change it is possible to use standard converters even when optional operands presented.

[SamWot]

Ping

[ab]

I haven't looked closely yet, but have you considered doing this entirely in tablegen, either via default operand lists (like ARM's OptionalDefOperand) or even some default value string?

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

363–365

These look independent; feel free to commit right away (or submit another patch).

[SamWot]

In D18242#384742, @ab wrote:

I haven't looked closely yet, but have you considered doing this entirely in tablegen, either via default operand lists (like ARM's OptionalDefOperand) or even some default value string?

I actualy thought about this but decided that DefaultMethod is better (at least for now) because usage of OptionalDefOperand or default string value would require big changes in MCTargetAsmParser’s interface and all back-ends. MCTargetAsmParser need some way to create arguments (MCParsedAsmOperand) with default values and insert them in instructions. Currently MCTargetAsmParser has no interface to create operands and it’s hard to create it because every target’s MCParsedAsmOperand implementation differs a lot. With DefaultMethod responsibility to create default arguments lies on back-end developers and no interface changes in MCTargetAsmParser required.

Another solution can be that MCTargetAsmParser would directly create MCOperands instead of MCParsedAsmOperand. But in this case it is there is no way to use RenderMethod for this operand.

[SamWot]

Ping

[SamWot]

Ping

[ab]

I'm no fan of this, but I think you're right, there's not much better we can do right now.

What do you think of only generating the extra code only if there's at least one optional operand class? That would avoid needing a mask on other targets.

include/llvm/Target/Target.td

623

return default -> returns the default

623–624

"This method is only used if.."

625

IsOptional == 1, no?

628–630

Why pass anything? You don't use the Opcode, and the Operands are only used for the location:

• what happens when the first and only operand is optional?
• loc of the preceding operand is probably more accurate; but I think just using SMLoc() makes the most sense
• does anything look at locations at this point anyway? If there's an error it would have been caught earlier, no?

utils/TableGen/AsmMatcherEmitter.cpp

206

which method?

[SamWot]

Fixes and new code to check if target has optional operands.

include/llvm/Target/Target.td

628–630

I added Opcode and Operands to arguments list because this arguments seems to be useful in defaultMethod. It is not hard to imagine situation when default value for operand differs depending on instruction opcode or other operands presented in instruction. But if you think that they are unnecessary I can remove them.

There is always first operand in OperandVector - instruction mnemonic.

[ab]

include/llvm/Target/Target.td

628–630

I added Opcode and Operands to arguments list because this arguments seems to be useful in defaultMethod. It is not hard to imagine situation when default value for operand differs depending on instruction opcode or other operands presented in instruction. But if you think that they are unnecessary I can remove them.

Eh, if you have a patch in mind that would use them, OK, but otherwise, what do you think of leaving them out, and adding them when they turn out to be necessary?

There is always first operand in OperandVector - instruction mnemonic.

You're right, forgot about that!

[SamWot]

Removed unused arguments in defaultMethod.
Rebased on top of master.

[ab]

LGTM; thanks Sam!

include/llvm/Target/Target.td

class AsmOperandClass {
   /// Set to 1 if this operand is optional and not always required. Typically,
   /// the AsmParser will emit an error when it finishes parsing an
   /// instruction if it hasn't matched all the operands yet.  However, this
   /// error will be suppressed if all of the remaining unmatched operands are
   /// marked as IsOptional.
   ///
   /// Optional arguments must be at the end of the operand list.
   bit IsOptional = 0;
+
+  /// The name of the method on the target specific asm parser that returns the
+  /// default operand for this optional operand. This method is only used if
+  /// IsOptional == 1. If not set, this will default to "defaultFooOperands",
+  /// where Foo is the AsmOperandClass name. The method signature should be:
+  ///   std::unique_ptr<MCParsedAsmOperand> defaultFooOperands(
+  ///           unsigned Opcode,
+  ///           const OperandVector &Operands) const;
+  string DefaultMethod = ?;
 }
 
 def ImmAsmOperand : AsmOperandClass {
   let Name = "Imm";
 }
 
 /// Operand Types - These provide the built-in operand types that may be used
 /// by a target.  Targets can optionally provide their own operand types as

lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

 #include "AMDKernelCodeT.h"
 #include "MCTargetDesc/AMDGPUMCTargetDesc.h"
 #include "MCTargetDesc/AMDGPUTargetStreamer.h"
 #include "SIDefines.h"
 #include "Utils/AMDGPUBaseInfo.h"
 #include "Utils/AMDKernelCodeTUtils.h"
 #include "llvm/ADT/APFloat.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/SmallBitVector.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCParser/MCAsmLexer.h"
class AMDGPUOperand : public MCParsedAsmOperand {
 
   SMLoc StartLoc, EndLoc;
 
 public:
   AMDGPUOperand(enum KindTy K) : MCParsedAsmOperand(), Kind(K) {}
 
   MCContext *Ctx;
 
+  typedef std::unique_ptr<AMDGPUOperand> Ptr;
+
   enum ImmTy {
     ImmTyNone,
     ImmTyDSOffset0,
     ImmTyDSOffset1,
     ImmTyGDS,
     ImmTyOffset,
     ImmTyGLC,
     ImmTySLC,
case Token:
       OS << '\'' << getToken() << '\'';
       break;
     case Expression:
       OS << "<expr " << *Expr << '>';
       break;
     }
   }
 
-  static std::unique_ptr<AMDGPUOperand> CreateImm(int64_t Val, SMLoc Loc,
-                                                  enum ImmTy Type = ImmTyNone,
-                                                  bool IsFPImm = false) {
+  static AMDGPUOperand::Ptr CreateImm(int64_t Val, SMLoc Loc,
     auto Op = llvm::make_unique<AMDGPUOperand>(Immediate);
     Op->Imm.Val = Val;
     Op->Imm.IsFPImm = IsFPImm;
     Op->Imm.Type = Type;
     Op->Imm.Modifiers = 0;
     Op->StartLoc = Loc;
     Op->EndLoc = Loc;
     return Op;
   }
 
-  static std::unique_ptr<AMDGPUOperand> CreateToken(StringRef Str, SMLoc Loc,
-                                           bool HasExplicitEncodingSize = true) {
+  static AMDGPUOperand::Ptr CreateToken(StringRef Str, SMLoc Loc,
     auto Res = llvm::make_unique<AMDGPUOperand>(Token);
     Res->Tok.Data = Str.data();
     Res->Tok.Length = Str.size();
     Res->StartLoc = Loc;
     Res->EndLoc = Loc;
     return Res;
   }
 
-  static std::unique_ptr<AMDGPUOperand> CreateReg(unsigned RegNo, SMLoc S,
-                                                  SMLoc E,
-                                                  const MCRegisterInfo *TRI,
-                                                  const MCSubtargetInfo *STI,
-                                                  bool ForceVOP3) {
+  static AMDGPUOperand::Ptr CreateReg(unsigned RegNo, SMLoc S,
     auto Op = llvm::make_unique<AMDGPUOperand>(Register);
     Op->Reg.RegNo = RegNo;
     Op->Reg.TRI = TRI;
     Op->Reg.STI = STI;
     Op->Reg.Modifiers = 0;
     Op->Reg.IsForcedVOP3 = ForceVOP3;
     Op->StartLoc = S;
     Op->EndLoc = E;
     return Op;
   }
 
-  static std::unique_ptr<AMDGPUOperand> CreateExpr(const class MCExpr *Expr, SMLoc S) {
+  static AMDGPUOperand::Ptr CreateExpr(const class MCExpr *Expr, SMLoc S) {
     auto Op = llvm::make_unique<AMDGPUOperand>(Expression);
     Op->Expr = Expr;
     Op->StartLoc = S;
     Op->EndLoc = S;
     return Op;
   }
 
   bool isDSOffset() const;
public:
   bool parseCnt(int64_t &IntVal);
   OperandMatchResultTy parseSWaitCntOps(OperandVector &Operands);
   bool parseHwreg(int64_t &HwRegCode, int64_t &Offset, int64_t &Width);
   OperandMatchResultTy parseHwregOp(OperandVector &Operands);
   OperandMatchResultTy parseSOppBrTarget(OperandVector &Operands);
 
   OperandMatchResultTy parseFlatOptionalOps(OperandVector &Operands);
   OperandMatchResultTy parseFlatAtomicOptionalOps(OperandVector &Operands);
-  void cvtFlat(MCInst &Inst, const OperandVector &Operands);
-  void cvtFlatAtomic(MCInst &Inst, const OperandVector &Operands);
 
   void cvtMubuf(MCInst &Inst, const OperandVector &Operands);
   OperandMatchResultTy parseOffset(OperandVector &Operands);
   OperandMatchResultTy parseMubufOptionalOps(OperandVector &Operands);
   OperandMatchResultTy parseGLC(OperandVector &Operands);
   OperandMatchResultTy parseSLC(OperandVector &Operands);
   OperandMatchResultTy parseTFE(OperandVector &Operands);
+  AMDGPUOperand::Ptr defaultMubufOffset(unsigned Opcode,
+                                        const OperandVector &Operands) const;
+  AMDGPUOperand::Ptr defaultGLC(unsigned Opcode,
+                                const OperandVector &Operands) const;
+  AMDGPUOperand::Ptr defaultSLC(unsigned Opcode,
+                                const OperandVector &Operands) const;
+  AMDGPUOperand::Ptr defaultTFE(unsigned Opcode,
+                                const OperandVector &Operands) const;
+
 
   OperandMatchResultTy parseDMask(OperandVector &Operands);
   OperandMatchResultTy parseUNorm(OperandVector &Operands);
   OperandMatchResultTy parseDA(OperandVector &Operands);
   OperandMatchResultTy parseR128(OperandVector &Operands);
   OperandMatchResultTy parseLWE(OperandVector &Operands);
+  AMDGPUOperand::Ptr defaultDMask(unsigned Opcode,
+                                  const OperandVector &Operands) const;
+  AMDGPUOperand::Ptr defaultUNorm(unsigned Opcode,
+                                  const OperandVector &Operands) const;
+  AMDGPUOperand::Ptr defaultDA(unsigned Opcode,
+                               const OperandVector &Operands) const;
+  AMDGPUOperand::Ptr defaultR128(unsigned Opcode,
+                                 const OperandVector &Operands) const;
+  AMDGPUOperand::Ptr defaultLWE(unsigned Opcode,
+                                const OperandVector &Operands) const;
 
   void cvtId(MCInst &Inst, const OperandVector &Operands);
   void cvtVOP3_2_mod(MCInst &Inst, const OperandVector &Operands);
-  void cvtVOP3_2_nomod(MCInst &Inst, const OperandVector &Operands);
-  void cvtVOP3_only(MCInst &Inst, const OperandVector &Operands);
   void cvtVOP3(MCInst &Inst, const OperandVector &Operands);
 
   void cvtMIMG(MCInst &Inst, const OperandVector &Operands);
   void cvtMIMGAtomic(MCInst &Inst, const OperandVector &Operands);
   OperandMatchResultTy parseVOP3OptionalOps(OperandVector &Operands);
 
   OperandMatchResultTy parseDPPCtrlOps(OperandVector &Operands);
   OperandMatchResultTy parseDPPOptionalOps(OperandVector &Operands);
-  void cvtDPP_mod(MCInst &Inst, const OperandVector &Operands);
-  void cvtDPP_nomod(MCInst &Inst, const OperandVector &Operands);
-  void cvtDPP(MCInst &Inst, const OperandVector &Operands, bool HasMods);
+  AMDGPUOperand::Ptr defaultRowMask(unsigned Opcode,
+                                    const OperandVector &Operands) const;
+  AMDGPUOperand::Ptr defaultBankMask(unsigned Opcode,
+                                     const OperandVector &Operands) const;
+  AMDGPUOperand::Ptr defaultBoundCtrl(unsigned Opcode,
+                                      const OperandVector &Operands) const;
+  void cvtDPP(MCInst &Inst, const OperandVector &Operands);
 };
 
 struct OptionalOperand {
   const char *Name;
   AMDGPUOperand::ImmTy Type;
   bool IsBit;
   int64_t Default;
   bool (*ConvertResult)(int64_t&);
AMDGPUAsmParser::parseFlatOptionalOps(OperandVector &Operands) {
   return parseOptionalOps(FlatOptionalOps, Operands);
 }
 
 AMDGPUAsmParser::OperandMatchResultTy
 AMDGPUAsmParser::parseFlatAtomicOptionalOps(OperandVector &Operands) {
   return parseOptionalOps(FlatAtomicOptionalOps, Operands);
 }
 
-void AMDGPUAsmParser::cvtFlat(MCInst &Inst,
-                               const OperandVector &Operands) {
-  OptionalImmIndexMap OptionalIdx;
-
-  for (unsigned i = 1, e = Operands.size(); i != e; ++i) {
-    AMDGPUOperand &Op = ((AMDGPUOperand &)*Operands[i]);
-
-    // Add the register arguments
-    if (Op.isReg()) {
-      Op.addRegOperands(Inst, 1);
-      continue;
-    }
-
-    OptionalIdx[Op.getImmTy()] = i;
-  }
-  addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyGLC);
-  addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTySLC);
-  addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyTFE);
-}
-
-
-void AMDGPUAsmParser::cvtFlatAtomic(MCInst &Inst,
-                               const OperandVector &Operands) {
-  OptionalImmIndexMap OptionalIdx;
-
-  for (unsigned i = 1, e = Operands.size(); i != e; ++i) {
-    AMDGPUOperand &Op = ((AMDGPUOperand &)*Operands[i]);
-
-    // Add the register arguments
-    if (Op.isReg()) {
-      Op.addRegOperands(Inst, 1);
-      continue;
-    }
-
-    // Handle 'glc' token for flat atomics.
-    if (Op.isToken()) {
-      continue;
-    }
-
-    // Handle optional arguments
-    OptionalIdx[Op.getImmTy()] = i;
-  }
-  addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTySLC);
-  addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyTFE);
-}
-
 //===----------------------------------------------------------------------===//
 // mubuf
 //===----------------------------------------------------------------------===//
 
 static const OptionalOperand MubufOptionalOps [] = {
   {"offset", AMDGPUOperand::ImmTyOffset, false, 0, nullptr},
   {"glc",    AMDGPUOperand::ImmTyGLC, true, 0, nullptr},
   {"slc",    AMDGPUOperand::ImmTySLC, true, 0, nullptr},

 AMDGPUAsmParser::parseTFE(OperandVector &Operands) {
   return parseNamedBit("tfe", Operands);
 }
 
 bool AMDGPUOperand::isMubufOffset() const {
   return isImmTy(ImmTyOffset) && isUInt<12>(getImm());
 }
 
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultMubufOffset(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyOffset);
+}
+
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultGLC(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyGLC);
+}
+
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultSLC(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTySLC);
+}
+
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultTFE(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyTFE);
+}
+
 void AMDGPUAsmParser::cvtMubuf(MCInst &Inst,
                                const OperandVector &Operands) {
   OptionalImmIndexMap OptionalIdx;
 
   for (unsigned i = 1, e = Operands.size(); i != e; ++i) {
     AMDGPUOperand &Op = ((AMDGPUOperand &)*Operands[i]);
 
     // Add the register arguments
AMDGPUAsmParser::parseR128(OperandVector &Operands) {
   return parseNamedBit("r128", Operands, AMDGPUOperand::ImmTyR128);
 }
 
 AMDGPUAsmParser::OperandMatchResultTy
 AMDGPUAsmParser::parseLWE(OperandVector &Operands) {
   return parseNamedBit("lwe", Operands, AMDGPUOperand::ImmTyLWE);
 }
 
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultDMask(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyDMask);
+}
+
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultUNorm(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyUNorm);
+}
+
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultDA(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyDA);
+}
+
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultR128(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyR128);
+}
+
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultLWE(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyLWE);
+}
+
 //===----------------------------------------------------------------------===//
 // smrd
 //===----------------------------------------------------------------------===//
 
 bool AMDGPUOperand::isSMRDOffset() const {
 
   // FIXME: Support 20-bit offsets on VI.  We need to to pass subtarget
   // information here.
void AMDGPUAsmParser::cvtVOP3_2_mod(MCInst &Inst, const OperandVector &Operands) {
   uint64_t TSFlags = MII.get(Inst.getOpcode()).TSFlags;
   if (TSFlags & SIInstrFlags::VOP3) {
     cvtVOP3(Inst, Operands);
   } else {
     cvtId(Inst, Operands);
   }
 }
 
-void AMDGPUAsmParser::cvtVOP3_2_nomod(MCInst &Inst, const OperandVector &Operands) {
-  if (operandsHaveModifiers(Operands)) {
-    cvtVOP3(Inst, Operands);
-  } else {
-    cvtId(Inst, Operands);
-  }
-}
-
-void AMDGPUAsmParser::cvtVOP3_only(MCInst &Inst, const OperandVector &Operands) {
-  cvtVOP3(Inst, Operands);
-}
-
 void AMDGPUAsmParser::cvtVOP3(MCInst &Inst, const OperandVector &Operands) {
   OptionalImmIndexMap OptionalIdx;
   unsigned I = 1;
   const MCInstrDesc &Desc = MII.get(Inst.getOpcode());
   for (unsigned J = 0; J < Desc.getNumDefs(); ++J) {
     ((AMDGPUOperand &)*Operands[I++]).addRegOperands(Inst, 1);
   }
 
if (Op.isImmTy(AMDGPUOperand::ImmTyDppBoundCtrl)) {
       Operands.push_back(
         AMDGPUOperand::CreateImm(1, S, AMDGPUOperand::ImmTyDppBoundCtrl));
         return MatchOperand_Success;
     }
   }
   return Res;
 }
 
-void AMDGPUAsmParser::cvtDPP_mod(MCInst &Inst, const OperandVector &Operands) {
-  cvtDPP(Inst, Operands, true);
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultRowMask(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0xf, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyDppRowMask);
 }
 
-void AMDGPUAsmParser::cvtDPP_nomod(MCInst &Inst, const OperandVector &Operands) {
-  cvtDPP(Inst, Operands, false);
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultBankMask(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0xf, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyDppBankMask);
 }
 
-void AMDGPUAsmParser::cvtDPP(MCInst &Inst, const OperandVector &Operands,
-                             bool HasMods) {
+AMDGPUOperand::Ptr AMDGPUAsmParser::defaultBoundCtrl(unsigned Opcode,
+                                      const OperandVector &Operands) const {
+  return AMDGPUOperand::CreateImm(0, Operands[0]->getStartLoc(),
+                                  AMDGPUOperand::ImmTyDppBoundCtrl);
+}
+
+void AMDGPUAsmParser::cvtDPP(MCInst &Inst, const OperandVector &Operands) {
   OptionalImmIndexMap OptionalIdx;
 
   unsigned I = 1;
   const MCInstrDesc &Desc = MII.get(Inst.getOpcode());
   for (unsigned J = 0; J < Desc.getNumDefs(); ++J) {
     ((AMDGPUOperand &)*Operands[I++]).addRegOperands(Inst, 1);
   }
 
   for (unsigned E = Operands.size(); I != E; ++I) {
     AMDGPUOperand &Op = ((AMDGPUOperand &)*Operands[I]);
     // Add the register arguments
-    if (!HasMods && Op.isReg()) {
-      Op.addRegOperands(Inst, 1);
-    } else if (HasMods && Op.isRegOrImmWithInputMods()) {
+    if (Op.isRegOrImmWithInputMods()) {
+      // We convert only instructions with modifiers
       Op.addRegOrImmWithInputModsOperands(Inst, 2);
     } else if (Op.isDPPCtrl()) {
       Op.addImmOperands(Inst, 1);
     } else if (Op.isImm()) {
       // Handle optional arguments
       OptionalIdx[Op.getImmTy()] = I;
     } else {
       llvm_unreachable("Invalid operand type");

lib/Target/AMDGPU/SIInstrFormats.td

class VOP3Common <dag outs, dag ins, string asm, list<dag> pattern, bit HasMods = 0, bit VOP3Only = 0> :
   // zero (or less than zero).
   let AddedComplexity = -1000;
 
   let VOP3 = 1;
   let VALU = 1;
 
   let AsmMatchConverter =
     !if(!eq(VOP3Only,1),
-        "cvtVOP3_only",
-        !if(!eq(HasMods,1), "cvtVOP3_2_mod", "cvtVOP3_2_nomod"));
+        "cvtVOP3",
+        !if(!eq(HasMods,1), "cvtVOP3_2_mod", ""));
+
   let isCodeGenOnly = 0;
 
   int Size = 8;
 
   // Because SGPRs may be allowed if there are multiple operands, we
   // need a post-isel hook to insert copies in order to avoid
   // violating constant bus requirements.
   let hasPostISelHook = 1;
class FLAT <bits<7> op, dag outs, dag ins, string asm, list<dag> pattern> :
   // and are not considered done until both have been decremented.
   let VM_CNT = 1;
   let LGKM_CNT = 1;
 
   let Uses = [EXEC, FLAT_SCR]; // M0
 
   let UseNamedOperandTable = 1;
   let hasSideEffects = 0;
-  let AsmMatchConverter = "cvtFlat";
   let SchedRW = [WriteVMEM];
 }
 
 class MIMG <dag outs, dag ins, string asm, list<dag> pattern> :
     InstSI <outs, ins, asm, pattern> {
 
   let VM_CNT = 1;
   let EXP_CNT = 1;
   let MIMG = 1;
   let Uses = [EXEC];
 
   let UseNamedOperandTable = 1;
   let hasSideEffects = 0; // XXX ????
 }

lib/Target/AMDGPU/SIInstrInfo.td

 include "SIInstrFormats.td"
 include "VIInstrFormats.td"
 
 def MubufOffsetMatchClass : AsmOperandClass {
   let Name = "MubufOffset";
   let ParserMethod = "parseMubufOptionalOps";
   let RenderMethod = "addImmOperands";
   let IsOptional = 1;
+  let DefaultMethod = "defaultMubufOffset";
 }
 
 class DSOffsetBaseMatchClass <string parser> : AsmOperandClass {
   let Name = "DSOffset"#parser;
   let ParserMethod = parser;
   let RenderMethod = "addImmOperands";
   let PredicateMethod = "isDSOffset";
   let IsOptional = 1;

 def GDS01MatchClass : GDSBaseMatchClass <"parseDSOff01OptionalOps">;
 
 class GLCBaseMatchClass <string parser> : AsmOperandClass {
   let Name = "GLC"#parser;
   let PredicateMethod = "isGLC";
   let ParserMethod = parser;
   let RenderMethod = "addImmOperands";
   let IsOptional = 1;
+  let DefaultMethod = "defaultGLC";
 }
 
 def GLCMubufMatchClass : GLCBaseMatchClass <"parseMubufOptionalOps">;
 def GLCFlatMatchClass : GLCBaseMatchClass <"parseFlatOptionalOps">;
 
 class SLCBaseMatchClass <string parser> : AsmOperandClass {
   let Name = "SLC"#parser;
   let PredicateMethod = "isSLC";
   let ParserMethod = parser;
   let RenderMethod = "addImmOperands";
   let IsOptional = 1;
+  let DefaultMethod = "defaultSLC";
 }
 
 def SLCMubufMatchClass : SLCBaseMatchClass <"parseMubufOptionalOps">;
 def SLCFlatMatchClass : SLCBaseMatchClass <"parseFlatOptionalOps">;
 def SLCFlatAtomicMatchClass : SLCBaseMatchClass <"parseFlatAtomicOptionalOps">;
 
 class TFEBaseMatchClass <string parser> : AsmOperandClass {
   let Name = "TFE"#parser;
   let PredicateMethod = "isTFE";
   let ParserMethod = parser;
   let RenderMethod = "addImmOperands";
   let IsOptional = 1;
+  let DefaultMethod = "defaultTFE";
 }
 
 def TFEMubufMatchClass : TFEBaseMatchClass <"parseMubufOptionalOps">;
 def TFEFlatMatchClass : TFEBaseMatchClass <"parseFlatOptionalOps">;
 def TFEFlatAtomicMatchClass : TFEBaseMatchClass <"parseFlatAtomicOptionalOps">;
 
 def OModMatchClass : AsmOperandClass {
   let Name = "OMod";

 }
 
 class DPPOptionalMatchClass <string OpName>: AsmOperandClass {
   let Name = "DPPOptional"#OpName;
   let PredicateMethod = "is"#OpName;
   let ParserMethod = "parseDPPOptionalOps";
   let RenderMethod = "addImmOperands";
   let IsOptional = 1;
+  let DefaultMethod = "default"#OpName;
 }
 
 class OptionalImmAsmOperand <string OpName> : AsmOperandClass {
   let Name = "Imm"#OpName;
   let PredicateMethod = "isImm";
   let RenderMethod = "addImmOperands";
   let IsOptional = 1;
 }
 
 def DMaskMatchClass : AsmOperandClass {
   let Name = "DMask";
   let PredicateMethod = "isDMask";
   let ParserMethod = "parseDMask";
   let RenderMethod = "addImmOperands";
   let IsOptional = 1;
+  let DefaultMethod = "defaultDMask";
 }
 
 class NamedBitMatchClass<string BitName> : AsmOperandClass {
   let Name = "Imm"#BitName;
   let PredicateMethod = "is"#BitName;
   let ParserMethod = "parse"#BitName;
   let RenderMethod = "addImmOperands";
   let IsOptional = 1;
+  let DefaultMethod = "default"#BitName;
 }
 
 class NamedBitOperand<string BitName> : Operand<i1> {
   let PrintMethod = "print"#BitName;
 }
 
 def HwregMatchClass : AsmOperandClass {
   let Name = "Hwreg";
multiclass FLAT_Store_Helper <flat op, string asm_name,
   }
 }
 
 multiclass FLAT_ATOMIC <flat op, string asm_name, RegisterClass vdst_rc,
     RegisterClass data_rc = vdst_rc,
     dag outs_noret = (outs),
     string asm_noret = asm_name#" $addr, $data"#"$slc"#"$tfe"> {
 
-  let mayLoad = 1, mayStore = 1, glc = 0, vdst = 0, AsmMatchConverter = "cvtFlatAtomic" in {
+  let mayLoad = 1, mayStore = 1, glc = 0, vdst = 0 in {
     def "" : FLAT_Pseudo <NAME, outs_noret,
                           (ins VReg_64:$addr, data_rc:$data,
                                slc_flat_atomic:$slc, tfe_flat_atomic:$tfe), []>,
              AtomicNoRet <NAME, 0>;
 
     def _ci : FLAT_Real_ci <op.CI, NAME, outs_noret,
                             (ins VReg_64:$addr, data_rc:$data,
                                  slc_flat_atomic:$slc, tfe_flat_atomic:$tfe),
                             asm_noret>;
 
     def _vi : FLAT_Real_vi <op.VI, NAME, outs_noret,
                             (ins VReg_64:$addr, data_rc:$data,
                                  slc_flat_atomic:$slc, tfe_flat_atomic:$tfe),
                             asm_noret>;
   }
 
-  let glc = 1, hasPostISelHook = 1, AsmMatchConverter = "cvtFlatAtomic" in {
+  let glc = 1, hasPostISelHook = 1 in {
     defm _RTN : FLAT_AtomicRet_m <op, (outs vdst_rc:$vdst),
                         (ins VReg_64:$addr, data_rc:$data, slc_flat_atomic:$slc,
                              tfe_flat_atomic:$tfe),
                         asm_name#" $vdst, $addr, $data glc"#"$slc"#"$tfe", []>;
   }
 }
 
 class MIMG_Mask <string op, int channels> {

lib/Target/AMDGPU/VIInstrFormats.td

class VOP3be_vi <bits<10> op> : Enc64 {
   let Inst{63} = src2_modifiers{0};
 }
 
 class VOP_DPP <dag outs, dag ins, string asm, list<dag> pattern, bit HasMods = 0> :
     VOPAnyCommon <outs, ins, asm, pattern> {
   let DPP = 1;
   let Size = 8;
 
-  let AsmMatchConverter = !if(!eq(HasMods,1), "cvtDPP_mod", "cvtDPP_nomod");
+  let AsmMatchConverter = !if(!eq(HasMods,1), "cvtDPP", "");
 }
 
 class VOP_DPPe : Enc64 {
   bits<2> src0_modifiers;
   bits<8> src0;
   bits<2> src1_modifiers;
   bits<9> dpp_ctrl;
   bits<1> bound_ctrl;

utils/TableGen/AsmMatcherEmitter.cpp

struct ClassInfo {
   RegisterSet Registers;
 
   /// For custom match classes: the diagnostic kind for when the predicate fails.
   std::string DiagnosticType;
 
   /// Is this operand optional and not always required.
   bool IsOptional;
 
+  /// DefaultMethod - The name of the method that returns the default operand
+  /// for optional operand
+  std::string DefaultMethod;
+
 public:
   /// isRegisterClass() - Check if this is a register class.
   bool isRegisterClass() const {
     return Kind >= RegisterClass0 && Kind < UserClass0;
   }
 
   /// isUserClass() - Check if this is a user defined class.
   bool isUserClass() const {
const SubtargetFeatureInfo *getSubtargetFeature(Record *Def) const {
     assert(Def->isSubClassOf("Predicate") && "Invalid predicate type!");
     const auto &I = SubtargetFeatures.find(Def);
     return I == SubtargetFeatures.end() ? nullptr : &I->second;
   }
 
   RecordKeeper &getRecords() const {
     return Records;
   }
+
+  bool hasOptionalOperands() const {
+    return std::find_if(Classes.begin(), Classes.end(),
+                        [](const ClassInfo& Class){ return Class.IsOptional; })
+              != Classes.end();
+  }
 };
 
 } // end anonymous namespace
 
 void MatchableInfo::dump() const {
   errs() << TheDef->getName() << " -- " << "flattened:\"" << AsmString <<"\"\n";
 
   for (unsigned i = 0, e = AsmOperands.size(); i != e; ++i) {
if (!Entry) {
     Entry->ClassName = "Token";
     Entry->Name = "MCK_" + getEnumNameForToken(Token);
     Entry->ValueName = Token;
     Entry->PredicateMethod = "<invalid>";
     Entry->RenderMethod = "<invalid>";
     Entry->ParserMethod = "";
     Entry->DiagnosticType = "";
     Entry->IsOptional = false;
+    Entry->DefaultMethod = "<invalid>";
   }
 
   return Entry;
 }
 
 ClassInfo *
 AsmMatcherInfo::getOperandClass(const CGIOperandList::OperandInfo &OI,
                                 int SubOpIdx) {
for (const RegisterSet &RS : RegisterSets) {
     CI->Name = "MCK_Reg" + utostr(Index);
     CI->ValueName = "";
     CI->PredicateMethod = ""; // unused
     CI->RenderMethod = "addRegOperands";
     CI->Registers = RS;
     // FIXME: diagnostic type.
     CI->DiagnosticType = "";
     CI->IsOptional = false;
+    CI->DefaultMethod = ""; // unused
     RegisterSetClasses.insert(std::make_pair(RS, CI));
     ++Index;
   }
 
   // Find the superclasses; we could compute only the subgroup lattice edges,
   // but there isn't really a point.
   for (const RegisterSet &RS : RegisterSets) {
     ClassInfo *CI = RegisterSetClasses[RS];
for (Record *Rec : AsmOperands) {
     Init *DiagnosticType = Rec->getValueInit("DiagnosticType");
     if (StringInit *SI = dyn_cast<StringInit>(DiagnosticType))
       CI->DiagnosticType = SI->getValue();
 
     Init *IsOptional = Rec->getValueInit("IsOptional");
     if (BitInit *BI = dyn_cast<BitInit>(IsOptional))
       CI->IsOptional = BI->getValue();
 
+    // Get or construct the default method name.
+    Init *DMName = Rec->getValueInit("DefaultMethod");
+    if (StringInit *SI = dyn_cast<StringInit>(DMName)) {
+      CI->DefaultMethod = SI->getValue();
+    } else {
+      assert(isa<UnsetInit>(DMName) && "Unexpected DefaultMethod field!");
+      CI->DefaultMethod = "default" + CI->ClassName + "Operands";
+    }
+
     ++Index;
   }
 }
 
 AsmMatcherInfo::AsmMatcherInfo(Record *asmParser,
                                CodeGenTarget &target,
                                RecordKeeper &records)
   : Records(records), AsmParser(asmParser), Target(target) {
static unsigned getConverterOperandID(const std::string &Name,
 
   assert(ID < Table.size());
 
   return ID;
 }
 
 static void emitConvertFuncs(CodeGenTarget &Target, StringRef ClassName,
                              std::vector<std::unique_ptr<MatchableInfo>> &Infos,
-                             bool HasMnemonicFirst, raw_ostream &OS) {
+                             bool HasMnemonicFirst, bool HasOptionalOperands,
+                             raw_ostream &OS) {
   SmallSetVector<std::string, 16> OperandConversionKinds;
   SmallSetVector<std::string, 16> InstructionConversionKinds;
   std::vector<std::vector<uint8_t> > ConversionTable;
   size_t MaxRowLength = 2; // minimum is custom converter plus terminator.
 
   // TargetOperandClass - This is the target's operand class, like X86Operand.
   std::string TargetOperandClass = Target.getName() + "Operand";
 
   // Write the convert function to a separate stream, so we can drop it after
   // the enum. We'll build up the conversion handlers for the individual
   // operand types opportunistically as we encounter them.
   std::string ConvertFnBody;
   raw_string_ostream CvtOS(ConvertFnBody);
   // Start the unified conversion function.
+  if (HasOptionalOperands) {
     CvtOS << "void " << Target.getName() << ClassName << "::\n"
           << "convertToMCInst(unsigned Kind, MCInst &Inst, "
           << "unsigned Opcode,\n"
-        << "                const OperandVector"
-        << " &Operands) {\n"
-        << "  assert(Kind < CVT_NUM_SIGNATURES && \"Invalid signature!\");\n"
-        << "  const uint8_t *Converter = ConversionTable[Kind];\n"
-        << "  Inst.setOpcode(Opcode);\n"
-        << "  for (const uint8_t *p = Converter; *p; p+= 2) {\n"
-        << "    switch (*p) {\n"
-        << "    default: llvm_unreachable(\"invalid conversion entry!\");\n"
-        << "    case CVT_Reg:\n"
-        << "      static_cast<" << TargetOperandClass
-        << "&>(*Operands[*(p + 1)]).addRegOperands(Inst, 1);\n"
-        << "      break;\n"
-        << "    case CVT_Tied:\n"
-        << "      Inst.addOperand(Inst.getOperand(*(p + 1)));\n"
-        << "      break;\n";
+          << "                const OperandVector &Operands,\n"
+          << "                const SmallBitVector &OptionalOperandsMask) {\n";
+  } else {
+    CvtOS << "void " << Target.getName() << ClassName << "::\n"
+          << "convertToMCInst(unsigned Kind, MCInst &Inst, "
+          << "unsigned Opcode,\n"
+          << "                const OperandVector &Operands) {\n";
+  }
+  CvtOS << "  assert(Kind < CVT_NUM_SIGNATURES && \"Invalid signature!\");\n";
+  CvtOS << "  const uint8_t *Converter = ConversionTable[Kind];\n";
+  if (HasOptionalOperands) {
+    CvtOS << "  unsigned NumDefaults = 0;\n";
+  }
+  CvtOS << "  unsigned OpIdx;\n";
+  CvtOS << "  Inst.setOpcode(Opcode);\n";
+  CvtOS << "  for (const uint8_t *p = Converter; *p; p+= 2) {\n";
+  if (HasOptionalOperands) {
+    CvtOS << "    OpIdx = *(p + 1) - NumDefaults;\n";
+  } else {
+    CvtOS << "    OpIdx = *(p + 1);\n";
+  }
+  CvtOS << "    switch (*p) {\n";
+  CvtOS << "    default: llvm_unreachable(\"invalid conversion entry!\");\n";
+  CvtOS << "    case CVT_Reg:\n";
+  CvtOS << "      static_cast<" << TargetOperandClass
+        << "&>(*Operands[OpIdx]).addRegOperands(Inst, 1);\n";
+  CvtOS << "      break;\n";
+  CvtOS << "    case CVT_Tied:\n";
+  CvtOS << "      Inst.addOperand(Inst.getOperand(OpIdx));\n";
+  CvtOS << "      break;\n";
 
   std::string OperandFnBody;
   raw_string_ostream OpOS(OperandFnBody);
   // Start the operand number lookup function.
   OpOS << "void " << Target.getName() << ClassName << "::\n"
        << "convertToMapAndConstraints(unsigned Kind,\n";
   OpOS.indent(27);
   OpOS << "const OperandVector &Operands) {\n"
for (unsigned i = 0, e = II->ResOperands.size(); i != e; ++i) {
         Signature += Class;
         Signature += utostr(OpInfo.MINumOperands);
         Signature += "_" + itostr(OpInfo.AsmOperandNum);
 
         // Add the conversion kind, if necessary, and get the associated ID
         // the index of its entry in the vector).
         std::string Name = "CVT_" + (Op.Class->isRegisterClass() ? "Reg" :
                                      Op.Class->RenderMethod);
+        if (Op.Class->IsOptional) {
+          // For optional operands we must also care about DefaultMethod
+          assert(HasOptionalOperands);
+          Name += "_" + Op.Class->DefaultMethod;
+        }
         Name = getEnumNameForToken(Name);
 
         bool IsNewConverter = false;
         unsigned ID = getConverterOperandID(Name, OperandConversionKinds,
                                             IsNewConverter);
 
         // Add the operand entry to the instruction kind conversion row.
         ConversionRow.push_back(ID);
         ConversionRow.push_back(OpInfo.AsmOperandNum + HasMnemonicFirst);
 
         if (!IsNewConverter)
           break;
 
         // This is a new operand kind. Add a handler for it to the
         // converter driver.
-        CvtOS << "    case " << Name << ":\n"
-              << "      static_cast<" << TargetOperandClass
-              << "&>(*Operands[*(p + 1)])." << Op.Class->RenderMethod
+        CvtOS << "    case " << Name << ":\n";
+        if (Op.Class->IsOptional) {
+          // If optional operand is not present in actual instruction then we
+          // should call its DefaultMethod before RenderMethod
+          assert(HasOptionalOperands);
+          CvtOS << "      if (OptionalOperandsMask[*(p + 1) - 1]) {\n"
+                << "        " << Op.Class->DefaultMethod << "(Opcode, Operands)"
+                << "->" << Op.Class->RenderMethod << "(Inst, "
+                << OpInfo.MINumOperands << ");\n"
+                << "        ++NumDefaults;\n"
+                << "      } else {\n"
+                << "        static_cast<" << TargetOperandClass
+                << "&>(*Operands[OpIdx])." << Op.Class->RenderMethod
                 << "(Inst, " << OpInfo.MINumOperands << ");\n"
-              << "      break;\n";
+                << "      }\n";
+        } else {
+          CvtOS << "      static_cast<" << TargetOperandClass
+                << "&>(*Operands[OpIdx])." << Op.Class->RenderMethod
+                << "(Inst, " << OpInfo.MINumOperands << ");\n";
+        }
+        CvtOS << "      break;\n";
 
         // Add a handler for the operand number lookup.
         OpOS << "    case " << Name << ":\n"
              << "      Operands[*(p + 1)]->setMCOperandNum(NumMCOperands);\n";
 
         if (Op.Class->isRegisterClass())
           OpOS << "      Operands[*(p + 1)]->setConstraint(\"r\");\n";
         else
if (NumAmbiguous)
       errs() << "warning: " << NumAmbiguous
              << " ambiguous matchables!\n";
   });
 
   // Compute the information on the custom operand parsing.
   Info.buildOperandMatchInfo();
 
   bool HasMnemonicFirst = AsmParser->getValueAsBit("HasMnemonicFirst");
+  bool HasOptionalOperands = Info.hasOptionalOperands();
 
   // Write the output.
 
   // Information for the class declaration.
   OS << "\n#ifdef GET_ASSEMBLER_HEADER\n";
   OS << "#undef GET_ASSEMBLER_HEADER\n";
   OS << "  // This should be included into the middle of the declaration of\n";
   OS << "  // your subclasses implementation of MCTargetAsmParser.\n";
   OS << "  uint64_t ComputeAvailableFeatures(const FeatureBitset& FB) const;\n";
+  if (HasOptionalOperands) {
+    OS << "  void convertToMCInst(unsigned Kind, MCInst &Inst, "
+       << "unsigned Opcode,\n"
+       << "                       const OperandVector &Operands,\n"
+       << "                       const SmallBitVector &OptionalOperandsMask);\n";
+  } else {
     OS << "  void convertToMCInst(unsigned Kind, MCInst &Inst, "
        << "unsigned Opcode,\n"
-     << "                       const OperandVector "
-     << "&Operands);\n";
+       << "                       const OperandVector &Operands);\n";
+  }
   OS << "  void convertToMapAndConstraints(unsigned Kind,\n                ";
   OS << "           const OperandVector &Operands) override;\n";
   if (HasMnemonicFirst)
     OS << "  bool mnemonicIsValid(StringRef Mnemonic, unsigned VariantID);\n";
   OS << "  unsigned MatchInstructionImpl(const OperandVector &Operands,\n"
      << "                                MCInst &Inst,\n"
      << "                                uint64_t &ErrorInfo,"
      << " bool matchingInlineAsm,\n"
#endif // NDEBUG
   OS << "#undef GET_MATCHER_IMPLEMENTATION\n\n";
 
   // Generate the function that remaps for mnemonic aliases.
   bool HasMnemonicAliases = emitMnemonicAliases(OS, Info, Target);
 
   // Generate the convertToMCInst function to convert operands into an MCInst.
   // Also, generate the convertToMapAndConstraints function for MS-style inline
   // assembly.  The latter doesn't actually generate a MCInst.
-  emitConvertFuncs(Target, ClassName, Info.Matchables, HasMnemonicFirst, OS);
+  emitConvertFuncs(Target, ClassName, Info.Matchables, HasMnemonicFirst,
+                   HasOptionalOperands, OS);
 
   // Emit the enumeration for classes which participate in matching.
   emitMatchClassEnumeration(Target, Info.Classes, OS);
 
   // Emit the routine to match token strings to their match class.
   emitMatchTokenString(Target, Info.Classes, OS);
 
   // Emit the subclass predicate routine.
#endif // NDEBUG
   }
 
   // Emit code to compute the class list for this operand vector.
   OS << "  // Some state to try to produce better error messages.\n";
   OS << "  bool HadMatchOtherThanFeatures = false;\n";
   OS << "  bool HadMatchOtherThanPredicate = false;\n";
   OS << "  unsigned RetCode = Match_InvalidOperand;\n";
   OS << "  uint64_t MissingFeatures = ~0ULL;\n";
+  if (HasOptionalOperands) {
+    OS << "  SmallBitVector OptionalOperandsMask(" << MaxNumOperands << ");\n";
+  }
   OS << "  // Set ErrorInfo to the operand that mismatches if it is\n";
   OS << "  // wrong for all instances of the instruction.\n";
   OS << "  ErrorInfo = ~0ULL;\n";
 
   // Emit code to search the table.
   OS << "  // Find the appropriate table for this asm variant.\n";
   OS << "  const MatchEntry *Start, *End;\n";
   OS << "  switch (VariantID) {\n";
#endif // NDEBUG
 
   if (HasMnemonicFirst) {
     OS << "    // equal_range guarantees that instruction mnemonic matches.\n";
     OS << "    assert(Mnemonic == it->getMnemonic());\n";
   }
 
   // Emit check that the subclasses match.
   OS << "    bool OperandsValid = true;\n";
+  if (HasOptionalOperands) {
+    OS << "    OptionalOperandsMask.reset(0, " << MaxNumOperands << ");\n";
+  }
   OS << "    for (unsigned FormalIdx = " << (HasMnemonicFirst ? "0" : "SIndex")
      << ", ActualIdx = " << (HasMnemonicFirst ? "1" : "SIndex")
      << "; FormalIdx != " << MaxNumOperands << "; ++FormalIdx) {\n";
   OS << "      auto Formal = "
      << "static_cast<MatchClassKind>(it->Classes[FormalIdx]);\n";
   OS << "      if (ActualIdx >= Operands.size()) {\n";
   OS << "        OperandsValid = (Formal == " <<"InvalidMatchClass) || "
                                  "isSubclass(Formal, OptionalMatchClass);\n";
   OS << "        if (!OperandsValid) ErrorInfo = ActualIdx;\n";
+  if (HasOptionalOperands) {
+    OS << "        OptionalOperandsMask.set(FormalIdx, " << MaxNumOperands
+       << ");\n";
+  }
   OS << "        break;\n";
   OS << "      }\n";
   OS << "      MCParsedAsmOperand &Actual = *Operands[ActualIdx];\n";
   OS << "      unsigned Diag = validateOperandClass(Actual, Formal);\n";
   OS << "      if (Diag == Match_Success) {\n";
   OS << "        ++ActualIdx;\n";
   OS << "        continue;\n";
   OS << "      }\n";
   OS << "      // If the generic handler indicates an invalid operand\n";
   OS << "      // failure, check for a special case.\n";
   OS << "      if (Diag == Match_InvalidOperand) {\n";
   OS << "        Diag = validateTargetOperandClass(Actual, Formal);\n";
   OS << "        if (Diag == Match_Success) {\n";
   OS << "          ++ActualIdx;\n";
   OS << "          continue;\n";
   OS << "        }\n";
   OS << "      }\n";
   OS << "      // If current formal operand wasn't matched and it is optional\n"
      << "      // then try to match next formal operand\n";
   OS << "      if (Diag == Match_InvalidOperand "
-     << "&& isSubclass(Formal, OptionalMatchClass))\n";
+     << "&& isSubclass(Formal, OptionalMatchClass)) {\n";
+  if (HasOptionalOperands) {
+    OS << "        OptionalOperandsMask.set(FormalIdx);\n";
+  }
   OS << "        continue;\n";
+  OS << "      }\n";
   OS << "      // If this operand is broken for all of the instances of this\n";
   OS << "      // mnemonic, keep track of it so we can report loc info.\n";
   OS << "      // If we already had a match that only failed due to a\n";
   OS << "      // target predicate, that diagnostic is preferred.\n";
   OS << "      if (!HadMatchOtherThanPredicate &&\n";
   OS << "          (it == MnemonicRange.first || ErrorInfo <= ActualIdx)) {\n";
   OS << "        ErrorInfo = ActualIdx;\n";
   OS << "        // InvalidOperand is the default. Prefer specificity.\n";
#endif // NDEBUG
   OS << "    Inst.clear();\n\n";
   OS << "    if (matchingInlineAsm) {\n";
   OS << "      Inst.setOpcode(it->Opcode);\n";
   OS << "      convertToMapAndConstraints(it->ConvertFn, Operands);\n";
   OS << "      return Match_Success;\n";
   OS << "    }\n\n";
   OS << "    // We have selected a definite instruction, convert the parsed\n"
      << "    // operands into the appropriate MCInst.\n";
+  if (HasOptionalOperands) {
+    OS << "    convertToMCInst(it->ConvertFn, Inst, it->Opcode, Operands,\n"
+       << "                    OptionalOperandsMask);\n";
+  } else {
     OS << "    convertToMCInst(it->ConvertFn, Inst, it->Opcode, Operands);\n";
+  }
   OS << "\n";
 
   // Verify the instruction with the target-specific match predicate function.
   OS << "    // We have a potential match. Check the target predicate to\n"
      << "    // handle any context sensitive constraints.\n"
      << "    unsigned MatchResult;\n"
      << "    if ((MatchResult = checkTargetMatchPredicate(Inst)) !="
      << " Match_Success) {\n"