[MLIR] [LangRef] Clarify `dialect-item` parsing

[bollu]

Summary

Draw attention to the fact that the grammar is written in such a way
that it ensures balanced parentheses. Also add a note on how external
consumers of MLIR could easily parse such a grammar.

We take the time to restructure the examples, and add clarification
about illegal attribute declarations.

[bollu]

Rebase patch to latest MLIR

[Groverkss]

mlir/docs/LangRef.md

677

Is this supposed to be pretty-dialect-item-contents?

681

704

718–727

I think you can remove this.

[ayrivera]

Hi @bollu ,

Unfortunately, I don't have experience with MLIR, so I can't comment about what these changes are related to. As for the markdown file, it looks good to me after the fixes pointed by @Groverkss.

mlir/docs/LangRef.md

 
 // Using the type alias.
 "foo"(%x) : !avx_m128 -> ()
 ```
 
 ### Dialect Types
 
 Similarly to operations, dialects may define custom extensions to the type
-system.
+system. Note that an opaque dialect item must be of the form
+`my-dialect<"string-payload">`. A pretty dialect type must be
+of the form `my-dialect.type-name<structured-payload>`. The `structured-payload`
+is, roughly speaking, made of sequences of `pretty-dialect-iterm-contents`,
+where a `pretty-dialect-item-contents` is a tree of **balanced parentheses**
+made up of `()`, `{}`, `[]`, and `<>`.
+
+Thus, a lax parsing strategy of for `pretty-dialect-item-body` is to parse
+a `<`, followed by consuming balanced parens of `<>`, `[]`, `{}`, and ending with a `>`.
 
 ```
 dialect-namespace ::= bare-id
 
 opaque-dialect-item ::= dialect-namespace '<' string-literal '>'
 
 pretty-dialect-item ::= dialect-namespace '.' pretty-dialect-item-lead-ident
                                               pretty-dialect-item-body?
 
 pretty-dialect-item-lead-ident ::= '[A-Za-z][A-Za-z0-9._]*'
 pretty-dialect-item-body ::= '<' pretty-dialect-item-contents+ '>'
 pretty-dialect-item-contents ::= pretty-dialect-item-body
                               | '(' pretty-dialect-item-contents+ ')'
                               | '[' pretty-dialect-item-contents+ ']'
                               | '{' pretty-dialect-item-contents+ '}'
                               | '[^[<({>\])}\0]+'
 
 dialect-type ::= '!' opaque-dialect-item
 dialect-type ::= '!' pretty-dialect-item
 ```
 
-Dialect types can be specified in a verbose form, e.g. like this:
-
-```mlir
-// LLVM type that wraps around llvm IR types.
-!llvm<"i32*">
-
-// Tensor flow string type.
-!tf.string
+Some examples of legal and illegal `pretty-dialect-items` is given below:
 
-// Complex type
-!foo<"something<abcd>">
-
-// Even more complex type
-!foo<"something<a%%123^^^>>>">
+```
+my-dialect.r<foo[]> -- Legal
+my-dialect.r<bar[baz<>]> -- Legal
+my-dialect.r<1 < 2> -- Illegal: imbalanced `<`
+my-dialect.r<illegal[> -- Illegal: imbalanced `[`
+my-dialect.r<illegal]> -- Illegal: imbalanced `]`
 ```
 
-Dialect types that are simple enough can use the pretty format, which is a
-lighter weight syntax that is equivalent to the above forms:
+Some examples of dialect types specified in verbose and pretty form
+are given below:
 
 ```mlir
-// Tensor flow string type.
-!tf.string
+// RUN: mlir-opt --allow-unregistered-dialect 
+
+// Pretty and generic: simple type
+"foo.op" () : () -> !foo.i32 // pretty
+"foo.op" () : () -> !foo<"i32"> // generic
+
+// Pretty and generic: type with argument
+"foo.op" () : () -> !foo.bar<i32> // pretty
+"foo.op" () : () -> !foo<"bar<i32>"> // generic
 
-// Complex type
-!foo.something<abcd>
+// Only generic: complex type
+!foo<"something<a%%123^^^>>>"> // generic
+// NO pretty form, because imbalanced.
 ```
 
 Sufficiently complex dialect types are required to use the verbose form for
 generality. For example, the more complex type shown above wouldn't be valid in
 the lighter syntax: `!foo.something<a%%123^^^>>>` because it contains characters
 that are not allowed in the lighter syntax, as well as unbalanced `<>`
 characters.