[MLIR] ODS TypeDef documentation

[jdd]

Summary

Adds documentation for the new ODS TypeDef support.

[jdd]

I'm working on this in this branch: https://github.com/teqdruid/llvm-project/tree/ods-types-doc. It may be faster to edit directly. I can either push this branch upstream or give reviews/editors access to my fork.

[mehdi_amini]

mlir/docs/OpDefinitions.md

1415

As a title, it may be more clear to expand this.

The name "TypeDef" is a bit confusing to me by the way, as it first got me to think about C++ typedefs.

1433

1454

(offtopic with the doc)
Have you thought / planned for declarative assembly?

1462–1463

How does it work when there is no printer/parser for a Type?

1538

Nit: can you wrap this line?

[jdd]

mlir/docs/OpDefinitions.md

1415

Yeah, well 'Type' was already taken in ODS.

1454

I had a way to do this in my original patch, but it wasn't not consistent with the Op declarative format so we decided to remove it.

[jdd]

• Updating doc as per Mehdi's comments

[jdd]

Also, this documentation is very slightly wrong without https://reviews.llvm.org/D89438

[rriddle]

LGTM for the current documentation after resolving comments. The questions related to where the methods are generated and merging TypeDef with something else aren't blocking on this revision.

mlir/docs/OpDefinitions.md

1415

The use of Data is weird to me given that not all types represent "Data". I would just remove it.

1415

Can you merge the tablegen TypeDef class with DialectType?

1433

The indent here is off.

1437

Can you just make this one line?

1449

LLVM/MLIR don't have code on the same line as the if, can you add a newline to each of these.

1468

is in -> as in

1479

nit: Set ... to generate or Setting ... will generate.

1482

nit: Use a -> Use the

1524

nit: StorageAllocator allocator -> StorageAllocator &allocator

1532–1533

1547

Why are they placed in the dialect's namespace and not local/static to where the .inc file is included?

1580

nit: Drop the extra newlines here.

[jdd]

• Changes based on River's feedback.

mlir/docs/OpDefinitions.md

1415

RE merging: I'm not sure what the implications of that would be. Was also considering having TypeDef extend Type and make it have a predicate which only matches itself. Either way, I'm considering both future improvements.

1547

They're declared in the header file so you don't have to #include the cpp in the same place as you put your Dialect::parseType definition.

[rriddle]

mlir/docs/OpDefinitions.md

1547

For the general infra we should try as hard as possible not to expose internal implementation details to public users. In most cases(all the ones that I can think of both upstream and downstream), the user wouldn't want to expose these methods and they would also be included in the same file as the parseType/printType implementations. If a user wants to do that differently, they can always manually expose it themselves but I wouldn't want that to be the default.

[jdd]

mlir/docs/OpDefinitions.md

1547

In understand but couldn't think of an alternative since I didn't want to require users to include the implementation in the same file as Dialect::parseType. Though maybe that's appropriate. Isolate all the Type code to one cpp file.

mlir/docs/OpDefinitions.md

 # Operation Definition Specification (ODS)
 
 In addition to specializing the `mlir::Op` C++ template, MLIR also supports
-defining operations in a table-driven manner. This is achieved via
-[TableGen][TableGen], which is both a generic language and its tooling to
+defining operations and data types in a table-driven manner. This is achieved
+via [TableGen][TableGen], which is both a generic language and its tooling to
 maintain records of domain-specific information. Facts regarding an operation
-are specified concisely into a TableGen record, which will be expanded into an
-equivalent `mlir::Op` C++ template specialization at compiler build time.
+are specified concisely into a TableGen record, which will be expanded into
+an equivalent `mlir::Op` C++ template specialization at compiler build time.
 
 This manual explains in detail all the available mechanisms for defining
 operations in such a table-driven manner. It aims to be a specification instead
 of a tutorial. Please refer to [Quickstart tutorial to adding MLIR graph
 rewrite](Tutorials/QuickstartRewrites.md) for the latter.
 
 In addition to detailing each mechanism, this manual also tries to capture
 best practices. They are rendered as quoted bullet points.
llvm::Optional<MyBitEnum> symbolizeMyBitEnum(uint32_t value) {
   // Special case for all bits unset.
   if (value == 0) return MyBitEnum::None;
 
   if (value & ~(1u | 2u | 4u)) return llvm::None;
   return static_cast<MyBitEnum>(value);
 }
 ```
 
+## Type Definitions
+
+MLIR defines the TypeDef class hierarchy to enable generation of data types
+from their specifications. A type is defined by specializing the TypeDef
+class with concrete contents for all the fields it requires. For example, an
+integer type could be defined as:
+
+```tablegen
+// All of the types will extend this class.
+class Test_Type<string name> : TypeDef<Test_Dialect, name> { }
+
+// An alternate int type.
+def IntegerType : Test_Type<"TestInteger"> {
+  let mnemonic = "int";
+
+  let summary = "An integer type with special semantics";
+
+  let description = [{
+    An alternate integer type. This type differentiates itself from the
+    standard integer type by not having a SignednessSemantics parameter, just
+    a width.
+  }];
+
+  let parameters = (ins "unsigned":$width);
+
+  // We define the printer inline.
+  let printer = [{
+    $_printer << "int<" << getImpl()->width << ">";
+  }];
+
+  // The parser is defined here also.
+  let parser = [{
+    if (parser.parseLess())
+      return Type();
+    int width;
+    if ($_parser.parseInteger(width))
+      return Type();
+    if ($_parser.parseGreater())
+      return Type();
+    return get(ctxt, width);
+  }];
+```
+
+### Type name
+
+The name of the C++ class which gets generated defaults to
+`<classParamName>Type` (e.g. `TestIntegerType` in the above example). This
+can be overridden via the the `cppClassName` field. The field `mnemonic` is
+to specify the asm name for parsing. It is optional and not specifying it
+will imply that no parser or printer methods are attached to this class.
+
+### Type documentation
+
+The `summary` and `description` fields exist and are to be used the same way
+as in Operations. Namely, the summary should be a one-liner and `description`
+should be a longer explanation.
+
+### Type parameters
+
+The `parameters` field is a list of the types parameters. If no parameters
+are specified (the default), this type is considered a singleton type.
+Parameters are in the `"c++Type":$paramName` format.
+To use C++ types as parameters which need allocation in the storage
+constructor, there are two options:
+
+- Set `hasCustomStorageConstructor` to generate the TypeStorage class with
+a constructor which is just declared -- no definition -- so you can write it
+yourself.
+- Use the `TypeParameter` tablegen class instead of the "c++Type" string.
+
+### TypeParameter tablegen class
+
+This is used to further specify attributes about each of the types
+parameters. It includes documentation (`description` and `syntax`), the C++
+type to use, and a custom allocator to use in the storage constructor method.
+
+```tablegen
+// DO NOT DO THIS!
+let parameters = (ins
+  "ArrayRef<int>":$dims);
+```
+
+The default storage constructor blindly copies fields by value. It does not
+know anything about the types. In this case, the ArrayRef<int> requires
+allocation with `dims = allocator.copyInto(dims)`.
+
+You can specify the necessary constuctor by specializing the `TypeParameter`
+tblgen class:
+
+```tablegen
+class ArrayRefIntParam :
+    TypeParameter<"::llvm::ArrayRef<int>", "Array of ints"> {
+  let allocator = [{$_dst = $_allocator.copyInto($_self);}];
+}
+
+...
+
+let parameters = (ins
+  ArrayRefIntParam:$dims);
+```
+
+The `allocator` code block has the following substitutions:
+- `$_allocator` is the TypeStorageAllocator in which to allocate objects.
+- `$_dst` is the variable in which to place the allocated data.
+
+MLIR includes several specialized classes for common situations:
+- `StringRefParameter<descriptionOfParam>` for StringRefs.
+- `ArrayRefParameter<arrayOf, descriptionOfParam>` for ArrayRefs of value
+types
+- `SelfAllocationParameter<descriptionOfParam>` for C++ classes which contain
+a method called `allocateInto(StorageAllocator &allocator)` to allocate
+itself into `allocator`.
+- `ArrayRefOfSelfAllocationParameter<arrayOf, descriptionOfParam>` for arrays
+of objects which self-allocate as per the last specialization.
+
+If we were to use one of these included specializations:
+
+```tablegen
+let parameters = (ins
+  ArrayRefParameter<"int", "The dimensions">:$dims
+);
+```
+
+### Parsing and printing
+
+If a mnemonic is specified, the `printer` and `parser` code fields are active.
+The rules for both are:
+- If null, generate just the declaration.
+- If non-null and non-empty, use the code in the definition. The `$_printer`
+or `$_parser` substitutions are valid and should be used.
+- It is an error to have an empty code block.
+
+For each dialect, two "dispatch" functions will be created: one for parsing
+and one for printing. You should add calls to these in your
+`Dialect::printType` and `Dialect::parseType` methods. They are created in
+the dialect's namespace and their function signatures are:
+```c++
+Type generatedTypeParser(MLIRContext* ctxt, DialectAsmParser& parser,
+                         StringRef mnemonic);
+LogicalResult generatedTypePrinter(Type type, DialectAsmPrinter& printer);
+```
+
+The mnemonic, parser, and printer fields are optional. If they're not
+defined, the generated code will not include any parsing or printing code and
+omit the type from the dispatch functions above. In this case, the dialect
+author is responsible for parsing/printing the types in `Dialect::printType`
+and `Dialect::parseType`.
+
+### Other fields
+
+- If the `genStorageClass` field is set to 1 (the default) a storage class is
+generated with member variables corresponding to each of the specified
+`parameters`.
+- If the `genAccessors` field is 1 (the default) accessor methods will be
+generated on the Type class (e.g. `int getWidth() const` in the example
+above).
+- If the `genVerifyInvariantsDecl` field is set, a declaration for a method
+`static LogicalResult verifyConstructionInvariants(Location, parameters...)`
+is added to the class as well as a `getChecked(Location, parameters...)`
+method which gets the result of `verifyConstructionInvariants` before calling
+`get`.
+- The `storageClass` field can be used to set the name of the storage class.
+- The `storageNamespace` field is used to set the namespace where the storage
+class should sit. Defaults to "detail".
+- The `extraClassDeclaration` field is used to include extra code in the
+class declaration.
+
 ## Debugging Tips
 
 ### Run `mlir-tblgen` to see the generated content
 
 TableGen syntax sometimes can be obscure; reading the generated content can be
 a very helpful way to understand and debug issues. To build `mlir-tblgen`, run
 `cmake --build . --target mlir-tblgen` in your build directory and find the
 `mlir-tblgen` binary in the `bin/` subdirectory. All the supported generators