[TableGen] Support custom decoders for variable length instructions

[myhsu]

Summary

Just like the encoder directive for variable-length instructions, this patch adds a new decoder directive to allow custom decoder function on an operand.

Right now, due to the design of DecoderEmitter each operand can only have a single custom decoder in a given instruction.

[0x59616e]

llvm/utils/TableGen/DecoderEmitter.cpp

1896

nit: Use !empty for better readability.

llvm/utils/TableGen/VarLenCodeEmitterGen.cpp

165–170

nit: getCustomCoders will return empty StringRef automatically if DI->getNumArgs() == 2. So this if statement is unnecessary.

195–198

ditto

215–227

Use early exit to reduce indentation for better readability.

221–222

nit: Avoid declaring multiple variables in the same statement;

[0x59616e]

llvm/include/llvm/Target/Target.td

806–807

Under what circumstances will we need different decoders for the same operand ?

[0x59616e]

llvm/utils/TableGen/VarLenCodeEmitterGen.cpp

212

getCustomCoders seems unnecessary to be a member function of VarLenInst ?

[myhsu]

Addressed feedbacks

llvm/include/llvm/Target/Target.td

806–807

I figure it's probably rare but for instance, the binary encoding of an operand was separated in two different places so the decoder has to puzzle them back.

llvm/utils/TableGen/VarLenCodeEmitterGen.cpp

165–170

good catch, thanks!

[barannikov88]

I'm not very familiar with varlen decoder, hence only general notes.

llvm/include/llvm/Target/Target.td

806–807

Is it diagnosed somewhere?

llvm/utils/TableGen/VarLenCodeEmitterGen.cpp

88

• The guideline is to limit anonymous namespaces to class declarations and use static everywhere else.
• The function does not protect against users, it needs proper error checking.

[RKSimon]

Should we add a description to llvm/docs/TableGen/ProgRef.rst?

[Paul-C-Anagnostopoulos]

The TableGen Programmer's Reference does not include descriptions of backend facilities.

[RKSimon]

In D142079#4071770, @Paul-C-Anagnostopoulos wrote:

The TableGen Programmer's Reference does not include descriptions of backend facilities.

So maybe BackGuide.rst ?

[Paul-C-Anagnostopoulos]

Yes, that makes sense.

[myhsu]

In D142079#4071791, @RKSimon wrote:

In D142079#4071770, @Paul-C-Anagnostopoulos wrote:

The TableGen Programmer's Reference does not include descriptions of backend facilities.

So maybe BackGuide.rst ?

I don't think it will be a good place since it's for TableGen backend developers while this patch only adds new TG directives for disassembler developers. Right now, IMO, the most related document is actually Writing an LLVM Backend, which is the only place mentioning TG syntax for writing instruction encodings (for fixed-length instructions, of course). Instead of cluttering with Writing an LLVM Backend I feel like a better way will be creating a separate page for variable-length instruction encoding / decoding.

[myhsu]

• Pull getCustomCoders out of anonymous namespace
• Report a fatal error if a encoder or decoder directive is not followed by a function name.

llvm/include/llvm/Target/Target.td

806–807

I double check with the code and found that it requires some works to report such diagnostics (e.g. print a warning if two different decoders are used on the same operand), which personally I think it's not worth it.

[0x59616e]

LGTM. Thanks for this amazing work ;)

llvm/include/llvm/Target/Target.td

 /// Which represents a 4-bit encoding for an instruction operand named `$src`.
 def operand;
 /// Similar to `operand`, we can reference only part of the operand's encoding:
 ///     (slice "$src", 6, 8)
 ///     (slice "$src", 8, 6)
 /// Both DAG represent bit 6 to 8 (total of 3 bits) in the encoding of operand
 /// `$src`.
 def slice;
-/// You can use `encoder` to specify a custom encoder function for a specific
-/// `operand` or `encoder` directive. For example:
+/// You can use `encoder` or `decoder` to specify a custom encoder or decoder
+/// function for a specific `operand` or `slice` directive. For example:
 ///     (operand "$src", 4, (encoder "encodeMyImm"))
 ///     (slice "$src", 8, 6, (encoder "encodeMyReg"))
+///     (operand "$src", 4, (encoder "encodeMyImm"), (decoder "decodeMyImm"))
+/// The ordering of `encoder` and `decoder` in the same `operand` or `slice`
+/// doesn't matter.
+/// Note that currently we cannot assign different decoders in the same
+/// (instruction) operand.
 def encoder;
+def decoder;
 
 /// PointerLikeRegClass - Values that are designed to have pointer width are
 /// derived from this.  TableGen treats the register class as having a symbolic
 /// type that it doesn't know, and resolves the actual regclass to use by using
 /// the TargetRegisterInfo::getPointerRegClass() hook at codegen time.
 class PointerLikeRegClass<int Kind> {
   int RegClassKind = Kind;
 }

llvm/test/TableGen/VarLenDecoder.td

class MyVarInst<MyMemOperand memory_op> : Instruction {
 
   let OutOperandList = (outs GR64:$dst);
   let InOperandList  = (ins memory_op:$src);
 }
 
 def FOO16 : MyVarInst<MemOp16> {
   let Inst = (ascend
       (descend (operand "$dst", 3), 0b01000, (operand "$src.reg", 3)),
-      (slice "$src.offset", 15, 0)
+      (slice "$src.offset", 15, 0, (decoder "myCustomDecoder"))
   );
 }
 def FOO32 : MyVarInst<MemOp32> {
   let Inst = (ascend
-      (descend (operand "$dst", 3), 0b01001, (operand "$src.reg", 3)),
+      (descend (operand "$dst", 3), 0b01001,
+      (operand "$src.reg", 3, (decoder "myCustomDecoder"))),
       (slice "$src.offset", 31, 16),
       (slice "$src.offset", 15, 0)
   );
 }
 
 // CHECK:      MCD::OPC_ExtractField, 3, 5,  // Inst{7-3} ...
 // CHECK-NEXT: MCD::OPC_FilterValue, 8, 4, 0, 0, // Skip to: 12
 // CHECK-NEXT: MCD::OPC_Decode, [[#OPCODE:]], 1, 0, // Opcode: FOO16
 // CHECK-NEXT: MCD::OPC_FilterValue, 9, 4, 0, 0, // Skip to: 21
 // CHECK-NEXT: MCD::OPC_Decode, [[#OPCODE+1]], 1, 1, // Opcode: FOO32
 // CHECK-NEXT: MCD::OPC_Fail,
 
 // Instruction length table
 // CHECK: 27,
 // CHECK-NEXT: 43,
 // CHECK-NEXT: };
 
 // CHECK:      case 0:
 // CHECK-NEXT: tmp = fieldFromInstruction(insn, 8, 3);
 // CHECK-NEXT: if (!Check(S, DecodeRegClassRegisterClass(MI, tmp, Address, Decoder))) { return MCDisassembler::Fail; }
 // CHECK-NEXT: tmp = fieldFromInstruction(insn, 0, 3);
 // CHECK-NEXT: if (!Check(S, DecodeRegClassRegisterClass(MI, tmp, Address, Decoder))) { return MCDisassembler::Fail; }
 // CHECK-NEXT: tmp = fieldFromInstruction(insn, 11, 16);
-// CHECK-NEXT: MI.addOperand(MCOperand::createImm(tmp));
+// CHECK-NEXT: if (!Check(S, myCustomDecoder(MI, tmp, Address, Decoder))) { return MCDisassembler::Fail; }
 // CHECK-NEXT: return S;
 // CHECK-NEXT: case 1:
 // CHECK-NEXT: tmp = fieldFromInstruction(insn, 8, 3);
 // CHECK-NEXT: if (!Check(S, DecodeRegClassRegisterClass(MI, tmp, Address, Decoder))) { return MCDisassembler::Fail; }
 // CHECK-NEXT: tmp = fieldFromInstruction(insn, 0, 3);
-// CHECK-NEXT: if (!Check(S, DecodeRegClassRegisterClass(MI, tmp, Address, Decoder))) { return MCDisassembler::Fail; }
+// CHECK-NEXT: if (!Check(S, myCustomDecoder(MI, tmp, Address, Decoder))) { return MCDisassembler::Fail; }
 // CHECK-NEXT: tmp = 0x0;
 // CHECK-NEXT: insertBits(tmp, fieldFromInstruction(insn, 11, 16), 16, 16);
 // CHECK-NEXT: insertBits(tmp, fieldFromInstruction(insn, 27, 16), 0, 16);
 // CHECK-NEXT: MI.addOperand(MCOperand::createImm(tmp));
 // CHECK-NEXT: return S;
 
 // CHECK-LABEL: case MCD::OPC_ExtractField: {
 // CHECK: makeUp(insn, Start + Len);
 
 // CHECK-LABEL: case MCD::OPC_CheckField: {
 // CHECK: makeUp(insn, Start + Len);
 
 // CHECK-LABEL: case MCD::OPC_Decode: {
 // CHECK: Len = InstrLenTable[Opc];
 // CHECK-NEXT: makeUp(insn, Len);

llvm/utils/TableGen/DecoderEmitter.cpp

for (auto &EncodingSegment : VLI) {
     }
 
     if (!OpName.empty()) {
       auto OpSubOpPair =
           const_cast<CodeGenInstruction &>(CGI).Operands.ParseOperandName(
               OpName);
       unsigned OpIdx = CGI.Operands.getFlattenedOperandNumber(OpSubOpPair);
       Operands[OpIdx].addField(CurrBitPos, EncodingSegment.BitWidth, Offset);
+      if (!EncodingSegment.CustomDecoder.empty())
+        Operands[OpIdx].Decoder = EncodingSegment.CustomDecoder.str();
 
       int TiedReg = TiedTo[OpSubOpPair.first];
       if (TiedReg != -1) {
         unsigned OpIdx = CGI.Operands.getFlattenedOperandNumber(
             std::make_pair(TiedReg, OpSubOpPair.second));
         Operands[OpIdx].addField(CurrBitPos, EncodingSegment.BitWidth, Offset);
       }
     }

llvm/utils/TableGen/VarLenCodeEmitterGen.h

 #include "llvm/TableGen/Record.h"
 
 namespace llvm {
 
 struct EncodingSegment {
   unsigned BitWidth;
   const Init *Value;
   StringRef CustomEncoder = "";
+  StringRef CustomDecoder = "";
 };
 
 class VarLenInst {
   const RecordVal *TheDef;
   size_t NumBits;
 
   // Set if any of the segment is not fixed value.
   bool HasDynamicSegment;
 
   SmallVector<EncodingSegment, 4> Segments;
 
   void buildRec(const DagInit *DI);
 
-  StringRef getCustomEncoderName(const Init *EI) const {
-    if (const auto *DI = dyn_cast<DagInit>(EI)) {
-      if (DI->getNumArgs() && isa<StringInit>(DI->getArg(0)))
-        return cast<StringInit>(DI->getArg(0))->getValue();
-    }
-    return "";
-  }
-
 public:
   VarLenInst() : TheDef(nullptr), NumBits(0U), HasDynamicSegment(false) {}
 
   explicit VarLenInst(const DagInit *DI, const RecordVal *TheDef);
 
   /// Number of bits
   size_t size() const { return NumBits; }
 

llvm/utils/TableGen/VarLenCodeEmitterGen.cpp

class VarLenCodeEmitterGen {
   std::string getInstructionCaseForEncoding(Record *R, Record *EncodingDef,
                                             CodeGenTarget &Target);
 
 public:
   explicit VarLenCodeEmitterGen(RecordKeeper &R) : Records(R) {}
 
   void run(raw_ostream &OS);
 };
-
 } // end anonymous namespace
 
+// Get the name of custom encoder or decoder, if there is any.
+// Returns `{encoder name, decoder name}`.
+static std::pair<StringRef, StringRef> getCustomCoders(ArrayRef<Init *> Args) {
+  std::pair<StringRef, StringRef> Result;
+  for (const auto *Arg : Args) {
+    const auto *DI = dyn_cast<DagInit>(Arg);
+    if (!DI)
+      continue;
+    const Init *Op = DI->getOperator();
+    if (!isa<DefInit>(Op))
+      continue;
+    // syntax: `(<encoder | decoder> "function name")`
+    StringRef OpName = cast<DefInit>(Op)->getDef()->getName();
+    if (OpName != "encoder" && OpName != "decoder")
+      continue;
+    if (!DI->getNumArgs() || !isa<StringInit>(DI->getArg(0)))
+      PrintFatalError("expected '" + OpName +
+                      "' directive to be followed by a custom function name.");
+    StringRef FuncName = cast<StringInit>(DI->getArg(0))->getValue();
+    if (OpName == "encoder")
+      Result.first = FuncName;
+    else
+      Result.second = FuncName;
+  }
+  return Result;
+}
+
 VarLenInst::VarLenInst(const DagInit *DI, const RecordVal *TheDef)
     : TheDef(TheDef), NumBits(0U) {
   buildRec(DI);
   for (const auto &S : Segments)
     NumBits += S.BitWidth;
 }
 
 void VarLenInst::buildRec(const DagInit *DI) {
for (; i != e; i += s) {
       } else if (const auto *SubDI = dyn_cast<DagInit>(Arg)) {
         buildRec(SubDI);
       } else {
         PrintFatalError(TheDef->getLoc(), "Unrecognized type of argument in `" +
                                               Op + "`: " + Arg->getAsString());
       }
     }
   } else if (Op == "operand") {
-    // (operand <operand name>, <# of bits>, [(encoder <custom encoder>)])
+    // (operand <operand name>, <# of bits>,
+    //          [(encoder <custom encoder>)][, (decoder <custom decoder>)])
     if (DI->getNumArgs() < 2)
       PrintFatalError(TheDef->getLoc(),
                       "Expecting at least 2 arguments for `operand`");
     HasDynamicSegment = true;
     const Init *OperandName = DI->getArg(0), *NumBits = DI->getArg(1);
     if (!isa<StringInit>(OperandName) || !isa<IntInit>(NumBits))
       PrintFatalError(TheDef->getLoc(), "Invalid argument types for `operand`");
 
     auto NumBitsVal = cast<IntInit>(NumBits)->getValue();
     if (NumBitsVal <= 0)
       PrintFatalError(TheDef->getLoc(), "Invalid number of bits for `operand`");
 
-    StringRef CustomEncoder;
-    if (DI->getNumArgs() >= 3)
-      CustomEncoder = getCustomEncoderName(DI->getArg(2));
-    Segments.push_back(
-        {static_cast<unsigned>(NumBitsVal), OperandName, CustomEncoder});
+    auto [CustomEncoder, CustomDecoder] =
+        getCustomCoders(DI->getArgs().slice(2));
+    Segments.push_back({static_cast<unsigned>(NumBitsVal), OperandName,
+                        CustomEncoder, CustomDecoder});
   } else if (Op == "slice") {
     // (slice <operand name>, <high / low bit>, <low / high bit>,
-    //        [(encoder <custom encoder>)])
+    //        [(encoder <custom encoder>)][, (decoder <custom decoder>)])
     if (DI->getNumArgs() < 3)
       PrintFatalError(TheDef->getLoc(),
                       "Expecting at least 3 arguments for `slice`");
     HasDynamicSegment = true;
     Init *OperandName = DI->getArg(0), *HiBit = DI->getArg(1),
          *LoBit = DI->getArg(2);
     if (!isa<StringInit>(OperandName) || !isa<IntInit>(HiBit) ||
         !isa<IntInit>(LoBit))
       PrintFatalError(TheDef->getLoc(), "Invalid argument types for `slice`");
 
     auto HiBitVal = cast<IntInit>(HiBit)->getValue(),
          LoBitVal = cast<IntInit>(LoBit)->getValue();
     if (HiBitVal < 0 || LoBitVal < 0)
       PrintFatalError(TheDef->getLoc(), "Invalid bit range for `slice`");
     bool NeedSwap = false;
     unsigned NumBits = 0U;
     if (HiBitVal < LoBitVal) {
       NeedSwap = true;
       NumBits = static_cast<unsigned>(LoBitVal - HiBitVal + 1);
     } else {
       NumBits = static_cast<unsigned>(HiBitVal - LoBitVal + 1);
     }
 
-    StringRef CustomEncoder;
-    if (DI->getNumArgs() >= 4)
-      CustomEncoder = getCustomEncoderName(DI->getArg(3));
+    auto [CustomEncoder, CustomDecoder] =
+        getCustomCoders(DI->getArgs().slice(3));
 
     if (NeedSwap) {
       // Normalization: Hi bit should always be the second argument.
       Init *const NewArgs[] = {OperandName, LoBit, HiBit};
       Segments.push_back({NumBits,
                           DagInit::get(DI->getOperator(), nullptr, NewArgs, {}),
-                          CustomEncoder});
+                          CustomEncoder, CustomDecoder});
     } else {
-      Segments.push_back({NumBits, DI, CustomEncoder});
+      Segments.push_back({NumBits, DI, CustomEncoder, CustomDecoder});
     }
   }
 }
 
 void VarLenCodeEmitterGen::run(raw_ostream &OS) {
   CodeGenTarget Target(Records);
   auto Insts = Records.getAllDerivedDefinitions("Instruction");
 
   auto NumberedInstructions = Target.getInstructionsByEnumValue();
   const CodeGenHwModes &HWM = Target.getHwModes();
 
   // The set of HwModes used by instruction encodings.
   std::set<unsigned> HwModes;
   for (const CodeGenInstruction *CGI : NumberedInstructions) {
     Record *R = CGI->TheDef;
 
     // Create the corresponding VarLenInst instance.
     if (R->getValueAsString("Namespace") == "TargetOpcode" ||
         R->getValueAsBit("isPseudo"))
       continue;
 
     if (const RecordVal *RV = R->getValue("EncodingInfos")) {
       if (auto *DI = dyn_cast_or_null<DefInit>(RV->getValue())) {
         EncodingInfoByHwMode EBM(DI->getDef(), HWM);
         for (auto &KV : EBM) {
           HwModes.insert(KV.first);
           Record *EncodingDef = KV.second;
           RecordVal *RV = EncodingDef->getValue("Inst");
           DagInit *DI = cast<DagInit>(RV->getValue());