[Docs] Add tablegen backend for target opcode documentatio

[olista01]

Summary

This is a tablegen backend to generate documentation for the opcodes that exist for each target. For each opcode, it lists the assembly string, the names and types of all operands, and the flags and predicates that apply to the opcode.

As far as I can tell, the current table-generated documentation is built for llvm.org by some method external to the CMake scripts. What need doing to get this documentation visible on llvm.org?

[olista01]

Ping

[olista01]

Ping.

[kristof.beyls]

I wanted to look at the documentation produced for a specific instruction today, as part of investigation PR35157.
I wanted to understand what the operands of VST1d64TPseudoWB_fixed are in the Arm backend.
After downloading this patch and building/running it; I saw it produced the following documentation for that instruction:

VST1d64TPseudoWB\_fixed
=======================

Flags: ``mayStore``, ``isPredicable``, ``hasExtraSrcRegAllocReq``, ``isCodeGenOnly``

* DEF ``GPR:$wb``

* USE ``addrmode6:$addr`` (**tied to wb**)

* USE ``QQPR:$src``

* USE ``pred:$p``

Predicates: ``HasNEON``

However, this doesn't seem to print all of the operands. It seemingly only prints the operands that are represented somehow in the assembly representation of the instruction?
When looking in ARMGenInstrInfo.inc, for the same instruction, I find (after formatting and adding a few comments to improve readability):

  { 2209,
    /*NumOperands*/     6,
    /*NumDefs*/ 1,
    /*Size*/    4,
    /*SchedClass*/      646,
    /*Flags*/   0|(1ULL<<MCID::MayStore)|(1ULL<<MCID::Predicable)|(1ULL<<MCID::ExtraSrcRegAllocReq),
    /*TSFlags*/ 0x10006ULL,
    /*ImplicitUses*/ nullptr,
    /*ImplicitDefs*/ nullptr,
    /*MCOperandInfo* */OperandInfo275,
    /*DeprecatedFeature*/ -1 ,
    /*?*/ nullptr },  // Inst #2209 = VST1d64TPseudoWB_fixed
....
static const MCOperandInfo OperandInfo275[] = {
  { ARM::GPRRegClassID, 0, MCOI::OPERAND_REGISTER, 0 },
  { ARM::GPRRegClassID, 0, MCOI::OPERAND_MEMORY, ((0 << 16) | (1 << MCOI::TIED_TO)) },
  { -1, 0, MCOI::OPERAND_MEMORY, 0 },
  { ARM::QQPRRegClassID, 0, MCOI::OPERAND_REGISTER, 0 },
  { -1, 0|(1<<MCOI::Predicate), MCOI::OPERAND_UNKNOWN, 0 },
  { -1, 0|(1<<MCOI::Predicate), MCOI::OPERAND_UNKNOWN, 0 },
};

This shows the instruction has 6 operands, 2 more than the 4 operands printed in the documentation.
Could the output in the documentation be adapted so that it prints all operands?

[olista01]

Instead of only printing the operands that appear in the assembly string (some of which correspond to more then one MachineOperand), print a flattened list of operands. Kristof's example instruction now generates this documentation:

VST1d64TPseudoWB\_fixed
=======================

Flags: ``mayStore``, ``isPredicable``, ``hasExtraSrcRegAllocReq``, ``isCodeGenOnly``

* DEF ``GPR:$wb``

* USE ``addrmode6/GPR:$addr.addr``

* USE ``addrmode6/i32imm:$addr.align``

* USE ``QQPR:$src``

* USE ``pred/i32imm:$p.anon0``

* USE ``pred/i32imm:$p.anon1``

Constraints: ``$addr.addr = $wb``

Predicates: ``HasNEON``

[kristof.beyls]

LGTM, modulo the one comment about the file header.

utils/TableGen/InstrDocsEmitter.cpp

1–7 ↗

(On Diff #122233)

This new file needs a header.

52–54 ↗

(On Diff #122233)

It's probably debatable whether it's useful or not to also print documentation for the target-independent instructions. But let's leave this as is for now.

llvm/trunk/utils/TableGen/CMakeLists.txt

add_tablegen(llvm-tblgen LLVM
   DAGISelMatcher.cpp
   DFAPacketizerEmitter.cpp
   DisassemblerEmitter.cpp
   FastISelEmitter.cpp
   FixedLenDecoderEmitter.cpp
   GlobalISelEmitter.cpp
   InfoByHwMode.cpp
   InstrInfoEmitter.cpp
+  InstrDocsEmitter.cpp
   IntrinsicEmitter.cpp
   OptParserEmitter.cpp
   PseudoLoweringEmitter.cpp
   RegisterBankEmitter.cpp
   RegisterInfoEmitter.cpp
   SearchableTableEmitter.cpp
   SubtargetEmitter.cpp
   SubtargetFeatureInfo.cpp

llvm/trunk/utils/TableGen/InstrDocsEmitter.cpp

+//===- InstrDocsEmitter.cpp - Opcode Documentation Generator --------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// InstrDocsEmitter generates restructured text documentation for the opcodes
+// that can be used by MachineInstr. For each opcode, the documentation lists:
+// * Opcode name
+// * Assembly string
+// * Flags (e.g. mayLoad, isBranch, ...)
+// * Operands, including type and name
+// * Operand constraints
+// * Implicit register uses & defs
+// * Predicates
+//
+//===----------------------------------------------------------------------===//
+
+#include "CodeGenDAGPatterns.h"
+#include "CodeGenInstruction.h"
+#include "CodeGenTarget.h"
+#include "TableGenBackends.h"
+#include "llvm/TableGen/Record.h"
+#include <string>
+#include <vector>
+
+using namespace llvm;
+
+namespace llvm {
+
+void writeTitle(StringRef Str, raw_ostream &OS, char Kind = '-') {
+  OS << std::string(Str.size(), Kind) << "\n" << Str << "\n"
+     << std::string(Str.size(), Kind) << "\n";
+}
+
+void writeHeader(StringRef Str, raw_ostream &OS, char Kind = '-') {
+  OS << Str << "\n" << std::string(Str.size(), Kind) << "\n";
+}
+
+std::string escapeForRST(StringRef Str) {
+  std::string Result;
+  Result.reserve(Str.size() + 4);
+  for (char C : Str) {
+    switch (C) {
+    // We want special characters to be shown as their C escape codes.
+    case '\n': Result += "\\n"; break;
+    case '\t': Result += "\\t"; break;
+    // Underscore at the end of a line has a special meaning in rst.
+    case '_': Result += "\\_"; break;
+    default: Result += C;
+    }
+  }
+  return Result;
+}
+
+void EmitInstrDocs(RecordKeeper &RK, raw_ostream &OS) {
+  CodeGenDAGPatterns CDP(RK);
+  CodeGenTarget &Target = CDP.getTargetInfo();
+  unsigned VariantCount = Target.getAsmParserVariantCount();
+
+  // Page title.
+  std::string Title = Target.getName();
+  Title += " Instructions";
+  writeTitle(Title, OS);
+  OS << "\n";
+
+  for (const CodeGenInstruction *II : Target.getInstructionsByEnumValue()) {
+    Record *Inst = II->TheDef;
+
+    // Don't print the target-independent instructions.
+    if (II->Namespace == "TargetOpcode")
+      continue;
+
+    // Heading (instruction name).
+    writeHeader(escapeForRST(Inst->getName()), OS, '=');
+    OS << "\n";
+
+    // Assembly string(s).
+    if (!II->AsmString.empty()) {
+      for (unsigned VarNum = 0; VarNum < VariantCount; ++VarNum) {
+        Record *AsmVariant = Target.getAsmParserVariant(VarNum);
+        OS << "Assembly string";
+        if (VariantCount != 1)
+          OS << " (" << AsmVariant->getValueAsString("Name") << ")";
+        std::string AsmString =
+            CodeGenInstruction::FlattenAsmStringVariants(II->AsmString, VarNum);
+        // We trim spaces at each end of the asm string because rst needs the
+        // formatting backticks to be next to a non-whitespace character.
+        OS << ": ``" << escapeForRST(StringRef(AsmString).trim(" "))
+           << "``\n\n";
+      }
+    }
+
+    // Boolean flags.
+    std::vector<const char *> FlagStrings;
+#define xstr(s) str(s)
+#define str(s) #s
+#define FLAG(f) if (II->f) { FlagStrings.push_back(str(f)); }
+    FLAG(isReturn)
+    FLAG(isBranch)
+    FLAG(isIndirectBranch)
+    FLAG(isCompare)
+    FLAG(isMoveImm)
+    FLAG(isBitcast)
+    FLAG(isSelect)
+    FLAG(isBarrier)
+    FLAG(isCall)
+    FLAG(isAdd)
+    FLAG(canFoldAsLoad)
+    FLAG(mayLoad)
+    //FLAG(mayLoad_Unset) // Deliberately omitted.
+    FLAG(mayStore)
+    //FLAG(mayStore_Unset) // Deliberately omitted.
+    FLAG(isPredicable)
+    FLAG(isConvertibleToThreeAddress)
+    FLAG(isCommutable)
+    FLAG(isTerminator)
+    FLAG(isReMaterializable)
+    FLAG(hasDelaySlot)
+    FLAG(usesCustomInserter)
+    FLAG(hasPostISelHook)
+    FLAG(hasCtrlDep)
+    FLAG(isNotDuplicable)
+    FLAG(hasSideEffects)
+    //FLAG(hasSideEffects_Unset) // Deliberately omitted.
+    FLAG(isAsCheapAsAMove)
+    FLAG(hasExtraSrcRegAllocReq)
+    FLAG(hasExtraDefRegAllocReq)
+    FLAG(isCodeGenOnly)
+    FLAG(isPseudo)
+    FLAG(isRegSequence)
+    FLAG(isExtractSubreg)
+    FLAG(isInsertSubreg)
+    FLAG(isConvergent)
+    FLAG(hasNoSchedulingInfo)
+    if (!FlagStrings.empty()) {
+      OS << "Flags: ";
+      bool IsFirst = true;
+      for (auto FlagString : FlagStrings) {
+        if (!IsFirst)
+          OS << ", ";
+        OS << "``" << FlagString << "``";
+        IsFirst = false;
+      }
+      OS << "\n\n";
+    }
+
+    // Operands.
+    for (unsigned i = 0; i < II->Operands.size(); ++i) {
+      bool IsDef = i < II->Operands.NumDefs;
+      auto Op = II->Operands[i];
+
+      if (Op.MINumOperands > 1) {
+        // This operand corresponds to multiple operands on the
+        // MachineInstruction, so print all of them, showing the types and
+        // names of both the compound operand and the basic operands it
+        // contains.
+        for (unsigned SubOpIdx = 0; SubOpIdx < Op.MINumOperands; ++SubOpIdx) {
+          Record *SubRec =
+              cast<DefInit>(Op.MIOperandInfo->getArg(SubOpIdx))->getDef();
+          StringRef SubOpName = Op.MIOperandInfo->getArgNameStr(SubOpIdx);
+          StringRef SubOpTypeName = SubRec->getName();
+
+          OS << "* " << (IsDef ? "DEF" : "USE") << " ``" << Op.Rec->getName()
+             << "/" << SubOpTypeName << ":$" << Op.Name << ".";
+          // Not all sub-operands are named, make up a name for these.
+          if (SubOpName.empty())
+            OS << "anon" << SubOpIdx;
+          else
+            OS << SubOpName;
+          OS << "``\n\n";
+        }
+      } else {
+        // The operand corresponds to only one MachineInstruction operand.
+        OS << "* " << (IsDef ? "DEF" : "USE") << " ``" << Op.Rec->getName()
+           << ":$" << Op.Name << "``\n\n";
+      }
+    }
+
+    // Constraints.
+    StringRef Constraints = Inst->getValueAsString("Constraints");
+    if (!Constraints.empty()) {
+      OS << "Constraints: ``" << Constraints << "``\n\n";
+    }
+
+    // Implicit definitions.
+    if (!II->ImplicitDefs.empty()) {
+      OS << "Implicit defs: ";
+      bool IsFirst = true;
+      for (Record *Def : II->ImplicitDefs) {
+        if (!IsFirst)
+          OS << ", ";
+        OS << "``" << Def->getName() << "``";
+        IsFirst = false;
+      }
+      OS << "\n\n";
+    }
+
+    // Implicit uses.
+    if (!II->ImplicitUses.empty()) {
+      OS << "Implicit uses: ";
+      bool IsFirst = true;
+      for (Record *Use : II->ImplicitUses) {
+        if (!IsFirst)
+          OS << ", ";
+        OS << "``" << Use->getName() << "``";
+        IsFirst = false;
+      }
+      OS << "\n\n";
+    }
+
+    // Predicates.
+    std::vector<Record *> Predicates =
+        II->TheDef->getValueAsListOfDefs("Predicates");
+    if (!Predicates.empty()) {
+      OS << "Predicates: ";
+      bool IsFirst = true;
+      for (Record *P : Predicates) {
+        if (!IsFirst)
+          OS << ", ";
+        OS << "``" << P->getName() << "``";
+        IsFirst = false;
+      }
+      OS << "\n\n";
+    }
+  }
+}
+
+} // end llvm namespace

llvm/trunk/utils/TableGen/TableGen.cpp

 
 using namespace llvm;
 
 enum ActionType {
   PrintRecords,
   GenEmitter,
   GenRegisterInfo,
   GenInstrInfo,
+  GenInstrDocs,
   GenAsmWriter,
   GenAsmMatcher,
   GenDisassembler,
   GenPseudoLowering,
   GenCallingConv,
   GenDAGISel,
   GenDFAPacketizer,
   GenFastISel,
Action(cl::desc("Action to perform:"),
          cl::values(clEnumValN(PrintRecords, "print-records",
                                "Print all records to stdout (default)"),
                     clEnumValN(GenEmitter, "gen-emitter",
                                "Generate machine code emitter"),
                     clEnumValN(GenRegisterInfo, "gen-register-info",
                                "Generate registers and register classes info"),
                     clEnumValN(GenInstrInfo, "gen-instr-info",
                                "Generate instruction descriptions"),
+                    clEnumValN(GenInstrDocs, "gen-instr-docs",
+                               "Generate instruction documentation"),
                     clEnumValN(GenCallingConv, "gen-callingconv",
                                "Generate calling convention descriptions"),
                     clEnumValN(GenAsmWriter, "gen-asm-writer",
                                "Generate assembly writer"),
                     clEnumValN(GenDisassembler, "gen-disassembler",
                                "Generate disassembler"),
                     clEnumValN(GenPseudoLowering, "gen-pseudo-lowering",
                                "Generate pseudo instruction lowering"),
case GenEmitter:
     EmitCodeEmitter(Records, OS);
     break;
   case GenRegisterInfo:
     EmitRegisterInfo(Records, OS);
     break;
   case GenInstrInfo:
     EmitInstrInfo(Records, OS);
     break;
+  case GenInstrDocs:
+    EmitInstrDocs(Records, OS);
+    break;
   case GenCallingConv:
     EmitCallingConv(Records, OS);
     break;
   case GenAsmWriter:
     EmitAsmWriter(Records, OS);
     break;
   case GenAsmMatcher:
     EmitAsmMatcher(Records, OS);

llvm/trunk/utils/TableGen/TableGenBackends.h

 void EmitAsmWriter(RecordKeeper &RK, raw_ostream &OS);
 void EmitCallingConv(RecordKeeper &RK, raw_ostream &OS);
 void EmitCodeEmitter(RecordKeeper &RK, raw_ostream &OS);
 void EmitDAGISel(RecordKeeper &RK, raw_ostream &OS);
 void EmitDFAPacketizer(RecordKeeper &RK, raw_ostream &OS);
 void EmitDisassembler(RecordKeeper &RK, raw_ostream &OS);
 void EmitFastISel(RecordKeeper &RK, raw_ostream &OS);
 void EmitInstrInfo(RecordKeeper &RK, raw_ostream &OS);
+void EmitInstrDocs(RecordKeeper &RK, raw_ostream &OS);
 void EmitPseudoLowering(RecordKeeper &RK, raw_ostream &OS);
 void EmitRegisterInfo(RecordKeeper &RK, raw_ostream &OS);
 void EmitSubtarget(RecordKeeper &RK, raw_ostream &OS);
 void EmitMapTable(RecordKeeper &RK, raw_ostream &OS);
 void EmitOptParser(RecordKeeper &RK, raw_ostream &OS);
 void EmitCTags(RecordKeeper &RK, raw_ostream &OS);
 void EmitAttributes(RecordKeeper &RK, raw_ostream &OS);
 void EmitSearchableTables(RecordKeeper &RK, raw_ostream &OS);
 void EmitGlobalISel(RecordKeeper &RK, raw_ostream &OS);
 void EmitX86EVEX2VEXTables(RecordKeeper &RK, raw_ostream &OS);
 void EmitX86FoldTables(RecordKeeper &RK, raw_ostream &OS);
 void EmitRegisterBank(RecordKeeper &RK, raw_ostream &OS);
 
 } // End llvm namespace
 
 #endif