[mlir][Linalg] Provide a Tablegen backend to specify named Linalg ops declaratively

[nicolasvasilache]

Summary

This revision demonstrates some basic support for declaratively defining Linalg "named" ops.
Such named ops form the backbone of operations that are ubiquitous in the ML application domain.

This revision closely related to the definition of a "Tensor Computation Primitives Dialect"
and demonstrates that certain ops can be expressed as declarative configurations of the
linalg.generic op.

For now, this only supports linalg.matvec (fixed-rank) and linalg.fill (rank-polymorphic)
and replaces manual C++ implementations with a Tablegen'd equivalent.

These 2 ops are already enough to exhibit interesting behaviors, tradeoffs and open questions:

1. reference maps/iterators as static vs instance methods and requirements for rank-polymorphism
2. behavior in the presence / absence of maps/iterators
3. handling of other attributes than the ones from linalg.generic
4. ability to sugar the parsing/pretty-printing
5. (future) declarative specification of properties that will automate matchers and emitters to

loop, scalar and vector form

6. (future) specification of mapping to library call / HW ISA

[nicolasvasilache]

roundtrip.mlir and tile.mlir have lerger NFC changes, they will be rebased, please ignore those for now.

[merge_guards_bot]

Unit tests: unknown.

clang-tidy: unknown.

clang-format: unknown.

Build artifacts: diff.json, CMakeCache.txt, console-log.txt

Pre-merge checks is in beta. Report issue. Please join beta or enable it for your project.

[jpienaar]

The previous rev was reverted as during discussion this feature was seen as unclear if we wanted to support, was changing the functionality & contract of the tool. Opening a new rev while rolling in the previous change does not resolve that discussion.

[nicolasvasilache]

In D73826#1853198, @jpienaar wrote:

The previous rev was reverted as during discussion this feature was seen as unclear if we wanted to support, was changing the functionality & contract of the tool. Opening a new rev while rolling in the previous change does not resolve that discussion.

No it does not and does not intend to resolve the discussion but to reopen it with a concrete use case.

This revision shows one possible implementation of "named" Linalg ops specified in a declarative fashion.
As often in that space, one has to try to build it to understand it, the 6 points in my commit message describe some of the tradeoffs involved.

A clear open problem is how to improve the core infra support to make it better, more natural and future proof.
The fact that this revision builds on previously reverted code is circumstantial, this was the simplest way to show people where Linalg intends to go.

I'll mark as WIP and RFC to make it clearer that this is not intended to land as is.

[mehdi_amini]

In D73826#1853198, @jpienaar wrote:

The previous rev was reverted as during discussion this feature was seen as unclear if we wanted to support, was changing the functionality & contract of the tool. Opening a new rev while rolling in the previous change does not resolve that discussion.

At least now there is a motivation / example of uses to support the discussion!

I'm still not convinced that hooking into Tablegen C++ is something we want Dialect author to do, so I'd like to look closer into what you're trying to do here and see how we could achieve this without injecting code into TableGen emitters.

[nicolasvasilache]

In D73826#1853293, @mehdi_amini wrote:

In D73826#1853198, @jpienaar wrote:

The previous rev was reverted as during discussion this feature was seen as unclear if we wanted to support, was changing the functionality & contract of the tool. Opening a new rev while rolling in the previous change does not resolve that discussion.

At least now there is a motivation / example of uses to support the discussion!

I'm still not convinced that hooking into Tablegen C++ is something we want Dialect author to do, so I'd like to look closer into what you're trying to do here and see how we could achieve this without injecting code into TableGen emitters.

I am not either and I am looking forward to getting better suggestions and moving the needle on this longstanding issue in Linalg.

[jpienaar]

In D73826#1853301, @nicolasvasilache wrote:

In D73826#1853293, @mehdi_amini wrote:

In D73826#1853198, @jpienaar wrote:

The previous rev was reverted as during discussion this feature was seen as unclear if we wanted to support, was changing the functionality & contract of the tool. Opening a new rev while rolling in the previous change does not resolve that discussion.

At least now there is a motivation / example of uses to support the discussion!

I'm still not convinced that hooking into Tablegen C++ is something we want Dialect author to do, so I'd like to look closer into what you're trying to do here and see how we could achieve this without injecting code into TableGen emitters.

I am not either and I am looking forward to getting better suggestions and moving the needle on this longstanding issue in Linalg.

The above functionality could already be handled with OpInterfaces + the way export generation is handled for TFLite dialect [which doesn't use OpInterfaces as that predated it, but could], one need not change OpDefinitionGen to get this behavior.

Another benefit to coupling with OpInterfaces is that it also serves to pushed one to think more about the core properties being taken advantage of.

[nicolasvasilache]

@jpienaar again, no need to pattern-match and overfit on the current implementation details as they are subject to change.

Your comment on how similar things are already achieved elsewhere without OpGen extensions is useful.
To be even more useful, could you point to a specific commit or place in the code that one can look out to understand how this is done in practice?

Thanks!

[nicolasvasilache]

@jpienaar @mehdi_amini !pingplease.

The above functionality could already be handled with OpInterfaces + the way export generation is handled for TFLite dialect [which doesn't use OpInterfaces as that predated it, but could], one need not change OpDefinitionGen to get this behavior.

This is not actionable for me, I do not see how to declaratively express AffineMaps in a declarative fashion in Tablegen and generating the proper OpInterface methods without the OpDefinitionGen hook.
Concretely, the following is the full declarative definition of the MatvecOp attributes proposed by this revision and replaces a bunch of C++ code scattered in various places:

def MatvecOp : LinalgNamedStructured_Op<"matvec", [NInputs<2>, NOutputs<1>]> {
  let arguments = (ins AnyStridedMemRefOfRank<2>,
                       AnyStridedMemRefOfRank<1>,
                       AnyStridedMemRefOfRank<1>);
  let hasLibraryImpl = 1;
  let iterators = ["i", "r_j"];
  let iterators_types = ["parallel", "reduction"];
  //   A(i, r_j) * B(r_j) -> C(i)
  let input_indexing_maps = [AffineExpressions<["i", "r_j"]>,
                             AffineExpressions<["r_j"]>];
  let output_indexing_maps = [AffineExpressions<["i"]>];
  let hasFolder = 1;
}

please provide something concrete and actionable or reconsider a temporary solution to unblock progress.
I am happy to use any mechanism deemed appropriate that works, I do not see one with my current understanding of the stack.

Lastly, this revision raises 6 points that are important to solve in the grander vision of Linalg and Tensor Compute Primitives, we should move the needle so we can start answering those key questions.

[antiagainst]

The main value I see with the OpDefinitionGen hook is that it allows one to automatically insert traits and methods into the generated C++ op classes. This is preferable when a dialect's ops have a large variation of potential traits and methods, and that information is already encoded with other fields in the op definitions.

Use Linalg ops as an example, let's say we want to introduce a separate OpInterface for the newly introduced iterator_types, and let's call it HasIteratorTypesInterface. We still want to use the declarative way (let iterators_types = ["parallel", "reduction"];) for specifying them as shown by Nicolas. Then the ODS hook gives us a way to automatically insert the HasIteratorTypesInterface::Trait into the C++ op class and synthesize the implementation for any op containing let iterators_types = [...]; without requiring the op to explicitly list the trait in the ODS. (And IMO it makes sense to omit given the trait is just derived information from let iterators_types = [...];). Additionally it allows some Linalg ops to implement HasIteratorTypesInterface while some not, and we can use hasTrait<HasIteratorTypesInterface::Trait> to quickly check whether an op has iterator types defined. Similarly, one can have a separate hasIndexingMapsInterface for other newly introduced fields.

But I'm not sure that's the direction Nicolas is intended to evolve the Linalg dialect. Instead of having smaller and more atomic op interfaces and let each op to opt in suitable ones automatically, maybe the approach of having one gigantic op interface like today and requiring pretty much every op to implement it even if some methods does not make sense (so we have llvm_unreachable in the implementation) for some op, will still do for Linalg ops and carry us a long way. Then the above does not really matter.

Concretely for this patch to make progress, I think one way to avoid using ODS hook, as pointed out by Jacques, is to have a completely new TableGen backend that parses the whole LinalgStructuredOps.td, filters the ones that have LinalgNamedStructured_Op, generate the op interface implementation C++ code into a separate file, and then include that separately into the main Linalg impl file. Without using ODS hooks, one need to handle the class method signature by oneself though; but it shouldn't be too burdensome. I personally view ODS hooks as a simpler and more integrated way for even this case, but as I said in the above, it's not the main value I think ODS hooks bring. :)

[nicolasvasilache]

Rebase on top of NFC changes and spin out a separate tablegen backend.

[nicolasvasilache]

Continue the impl.

[nicolasvasilache]

@mehdi_amini @jpienaar @rriddle @ftynse @antiagainst I have rebased and updated the revision description as well as the impl.
I would be interested in your opinion on where this is going and if it matches the expected behavior you may have.
If you see some red flags please let me know, I hope to have this working and pushable by tomorrow.

Thanks!

[flaub]

Just throwing an idea out there, but what if instead of 'named' ops (which seems to be the source of the need to extend TableGen), we could use something like the 'contraction' in PlaidML's tile dialect. It's not as flexible as arbitrary generic ops, but it seems to cover most of the ML workloads we've thrown at it. Here's what the ContractionOp looks like: https://github.com/plaidml/plaidml/blob/plaidml-v1/pmlc/dialect/tile/ir/ops.td#L232 (you can ignore the other ops in there, they are meant for the construction phase, for turning values into the type system via attributes/AffineMaps, etc). I'm thinking we really only need 2 kinds of ops: Contraction op and a generic Elementwise op. Bonus: both of these forms are auto-differentiable. We could add the other properties of structured ops.

I realize this probably isn't the right forum for this discussion, we could start a thread on discourse if that's more appropriate.

[mehdi_amini]

In D73826#1865623, @antiagainst wrote:

The main value I see with the OpDefinitionGen hook is that it allows one to automatically insert traits and methods into the generated C++ op classes. This is preferable when a dialect's ops have a large variation of potential traits and methods, and that information is already encoded with other fields in the op definitions

There is no disagreement that this may be useful for some cases: but we should strive to improve ODS itself to provide this facilities. I don't think we want dialect author to write Tablegen backends: this defeats the whole point of ODS in my opinion!

[mehdi_amini]

@flaub Thanks that's very interesting! It reminds me: when are you up for a presentation on the Tile layer at an open design meeting? ;)

[nicolasvasilache]

@flaub so actually, one of the (multiple) reasons I am pushing for this is to have the ability to add a generic contraction op and a generic pointwise op as special cases :)
We already have vector.contract added by andydavis@ with similar semantics and it would make the rewrite to vector form trivial.

So linalg.contract and linalg.pointwise are simple extensions of this and have the nice properties you mention.
I am unclear atm if we want autodiff to happen here (in Linalg) or at a higher level, we should discuss this in a live meeting since you have the most experience.

Still, linalg also wants to be future-proof and has additional expressiveness to represent e.g. Cholesky on triangular matrices, stencils, stacked RNNs and non-dense data types + gather/scatter in the future (but the road is still quite long until all that is available)

Lastly, this will also allow generating matchers and easily map the generic form to a library (in simple contract and pointwise cases but I conjecture also on much more complex cases, TBD :) ).

[antiagainst]

In D73826#1873559, @mehdi_amini wrote:

In D73826#1865623, @antiagainst wrote:

The main value I see with the OpDefinitionGen hook is that it allows one to automatically insert traits and methods into the generated C++ op classes. This is preferable when a dialect's ops have a large variation of potential traits and methods, and that information is already encoded with other fields in the op definitions

There is no disagreement that this may be useful for some cases: but we should strive to improve ODS itself to provide this facilities. I don't think we want dialect author to write Tablegen backends: this defeats the whole point of ODS in my opinion!

Well actually the new impl indeed changes to write a new TableGen backends. :) But I get your idea here. Yup I agree that we should strive to improve ODS itself and avoid dialect author to write TableGen expansion logic as much as possible. So features that apply to any dialect should be factored out and landed into main OpDefinitionGen. But there are cases it's inevitable like the Linalg case showed here: we have custom syntax defined for op definitions only reasonable to Linalg dialect. One can either completely roll its own TableGen backend or hook into OpDefinitionGen to avoid some boilerplate. I would argue the later saves developers' efforts and it's enhancing the functionality of ODS for dialect-specific use cases. :)

[nicolasvasilache]

Finish the revision and update CMakeLists.txt.

[antiagainst]

LGTM, but may want another pair of eyes to check the Linalg specific logic. :)

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredInterface.td

1 ↗

(On Diff #244322)

Super nit: Do we need to provide the "Linalg StructuredIfce" part with strange abbr.? Doesn't seem to provide anything to me. :)

mlir/tools/mlir-tblgen/LinalgNamedOpsGen.cpp

31

I'd prefer to have explicit using like the following that doing this.

93

Nit: just use char to be clear here?

[nicolasvasilache]

Update CMake.

[nicolasvasilache]

Address review comments.

[jpienaar]

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredOps.td

56–57

Should this comment be outside of the "section heading" block?

290–296

Why not have a trait that declares these and then have you backend generate the definitions? (else it seems that to know you have this function you need to hardcode the names vs casting to the op interface)

327

TableGen (and therefore ODS) does not have namespaces, please follow what we are doign everywhere else and prefacing the dialect as namespace. Else one can't write patterns using these ops without concern of collisions.

(true above and below too)

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredOpsInterface.td

45

Why unsigned but comment says true?

55

space after value? (same below too)

167

Is the only advantage here that you don't need to specify type? If so, why not use infer type op interface?

180

So this is different from clone in that you can specify location too?

[jpienaar]

In D73826#1874647, @antiagainst wrote:

In D73826#1873559, @mehdi_amini wrote:

In D73826#1865623, @antiagainst wrote:

The main value I see with the OpDefinitionGen hook is that it allows one to automatically insert traits and methods into the generated C++ op classes. This is preferable when a dialect's ops have a large variation of potential traits and methods, and that information is already encoded with other fields in the op definitions

There is no disagreement that this may be useful for some cases: but we should strive to improve ODS itself to provide this facilities. I don't think we want dialect author to write Tablegen backends: this defeats the whole point of ODS in my opinion!

Well actually the new impl indeed changes to write a new TableGen backends. :) But I get your idea here. Yup I agree that we should strive to improve ODS itself and avoid dialect author to write TableGen expansion logic as much as possible. So features that apply to any dialect should be factored out and landed into main OpDefinitionGen. But there are cases it's inevitable like the Linalg case showed here: we have custom syntax defined for op definitions only reasonable to Linalg dialect. One can either completely roll its own TableGen backend or hook into OpDefinitionGen to avoid some boilerplate. I would argue the later saves developers' efforts and it's enhancing the functionality of ODS for dialect-specific use cases. :)

I think a problem is that we have ODS now becoming a custom DSL for every dialect as every dialect can perform arbitrary interpretation and so there is no need to move towards common/reusable abstractions. ODS is restrictive for a reason :)

The extension of mlir-tblgen from being ops defined in dialect independent manner & translate utilities for dialects, to custom DSL for dialect was snuck in and so while we have that landed, I think of that more as a bug that should be fixed instead rather than something we should point to as existance.

In D73826#1863982, @nicolasvasilache wrote:

@jpienaar @mehdi_amini !pingplease.

The above functionality could already be handled with OpInterfaces + the way export generation is handled for TFLite dialect [which doesn't use OpInterfaces as that predated it, but could], one need not change OpDefinitionGen to get this behavior.

This is not actionable for me, I do not see how to declaratively express AffineMaps in a declarative fashion in Tablegen and generating the proper OpInterface methods without the OpDefinitionGen hook.

This change still does not do that, you do not have a way of specifying AffineMaps, you allow some subclass of them in a new format way and in a way that is dialect specific. I think this makes using ODS as Linalg DSL easier, but I'm not sure even there the added complexity is work it (and you could do this just using basic string interpolation locally with the same amount of portability and analyzability as you are providing here, nothing else in the system is gaining much from this and whether you had it in TableGen itself or as generator post seem to provide equal benefit here and if general patterns arose from which dialect indepent support could be added we could refactor).

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredOps.td

338–340

Just to be clear, so all of this is so that you can write

let input_indexing_maps = [AffineExpressions<["i", "r_j"]>,
                             AffineExpressions<["r_j"]>];
let output_indexing_maps = [AffineExpressions<["i"]>];

along with the op and from this you are generating

return SmallVector<AffineMap, 4>{AffineMap::get(2, 0, {i, j}),
                                   AffineMap::get(2, 0, {j}),
                                   AffineMap::get(2, 0, {i})};

?

[mehdi_amini]

FYI:

mlir/lib/Dialect/Linalg/IR/LinalgOps.cpp:795:22: warning: unused function 'verify' [-Wunused-function]
static LogicalResult verify(FillOp op) {
                     ^
1 warning generated.

In D73826#1876891, @jpienaar wrote:

There is no disagreement that this may be useful for some cases: but we should strive to improve ODS itself to provide this facilities. I don't think we want dialect author to write Tablegen backends: this defeats the whole point of ODS in my opinion!

Well actually the new impl indeed changes to write a new TableGen backends. :) But I get your idea here. Yup I agree that we should strive to improve ODS itself and avoid dialect author to write TableGen expansion logic as much as possible. So features that apply to any dialect should be factored out and landed into main OpDefinitionGen. But there are cases it's inevitable like the Linalg case showed here: we have custom syntax defined for op definitions only reasonable to Linalg dialect. One can either completely roll its own TableGen backend or hook into OpDefinitionGen to avoid some boilerplate. I would argue the later saves developers' efforts and it's enhancing the functionality of ODS for dialect-specific use cases. :)

I think a problem is that we have ODS now becoming a custom DSL for every dialect as every dialect can perform arbitrary interpretation and so there is no need to move towards common/reusable abstractions. ODS is restrictive for a reason :)

That's my initial position as well: the bar should be rather high to get into Tablegen backend, I'd rather not do this for every single "sugar" that can be added: at some point we'll have custom embedded parsers in Tablegen and create brand new languages...
That means there is a tradeoff and we'd like to make sure we consider and weigh a bit more the alternatives, including having something a bit more ugly in ODS if it avoid custom backends. Hence the current question in the diff to try to do this.

The extension of mlir-tblgen from being ops defined in dialect independent manner & translate utilities for dialects, to custom DSL for dialect was snuck in and so while we have that landed, I think of that more as a bug that should be fixed instead rather than something we should point to as existance.

+1

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredOps.td

263–264

the comment seems identical to line 55, but it is formatted like if it intends to open a new section

266

I don't parse this sentence

338–340

I'm interested in the answer here, I'd like to make sure we understand the problem this is solving :)

[nicolasvasilache]

@jpienaar and @mehdi_amini thanks for your fast turnaround on this!

Ultimately, my objective still is and has always been to write many DSL ops as a linalg.generic with some variant of Einsum/TC notation: e.g. A(i, r_j) * B(r_j) -> C(i) and have ops be automatically generated from this almost mathematical form.
We have had quite some experience in that area and I claim it is a fundamentally "good thing to have".
Once that exists, it can be pattern-matched to library calls automatically + resist transformations such as tiling and fusion since we have all the IR in place to do that.

This revision is a baby step that wants to act as a forcing function on a topic that seems to be, well.. , interesting: it is good to hear the objections clearly as to where Tablegen-based infrastructure is allowed to evolve.

Is there a world in which we can have such an Einsum-based parser on textual format to specify Linalg ops?
Does this have to be a language outside of MLIR? (I would certainly hope not).
I think https://reviews.llvm.org/D73405 sets a nice precedent and I would very much want to see something like this for specifying Linalg ops declaratively.

If this Einsum-based parser + Linalg Tablegen backend will never be acceptable future, I would love to understand why not and what alternatives people would recommend.

Thanks for your insights!

[mehdi_amini]

Is there a world in which we can have such an Einsum-based parser on textual format to specify Linalg ops?
Does this have to be a language outside of MLIR? (I would certainly hope not).

I don't understand how these two questions can go together as they seem entirely disconnected to me, so I'm probably not understanding the questions in the first place :)

I think https://reviews.llvm.org/D73405 sets a nice precedent and I would very much want to see something like this for specifying Linalg ops declaratively.

The main reason I have been reluctant to move on here is exactly because I would very much want to see something like this for specifying Linalg ops declaratively!
Where "something like this" is a feature that is generic enough to generalize and be reused across dialects, like D73405 :)

[nicolasvasilache]

In D73826#1877588, @mehdi_amini wrote:

Is there a world in which we can have such an Einsum-based parser on textual format to specify Linalg ops?
Does this have to be a language outside of MLIR? (I would certainly hope not).

I don't understand how these two questions can go together as they seem entirely disconnected to me, so I'm probably not understanding the questions in the first place :)

I think https://reviews.llvm.org/D73405 sets a nice precedent and I would very much want to see something like this for specifying Linalg ops declaratively.

The main reason I have been reluctant to move on here is exactly because I would very much want to see something like this for specifying Linalg ops declaratively!
Where "something like this" is a feature that is generic enough to generalize and be reused across dialects, like D73405 :)

Ok so if we agree on the principle I think this is a great start then.
My main concern is a cost benefit tradeoff: I am super interested in the result of this but do not have the background or resources to do it in a way that is portable and reusable across dialects. It still took me an inordinate amount of time to get to what you see here: this is not my area of expertise.
However I did start talking about this to various folks more than 3 months ago but everyone is very busy and it has reached a point where it is blocking.
So I went ahead to move the needle.

Is it acceptable to move this in a new binary say ‘linalg-named-opgen-dontlookhowugly-pleasehelpimprove” so I am not blocked. When the right people have cycles and they see the end result they can start lending a hand.

[mehdi_amini]

In D73826#1877588, @mehdi_amini wrote:

Is there a world in which we can have such an Einsum-based parser on textual format to specify Linalg ops?
Does this have to be a language outside of MLIR? (I would certainly hope not).

I don't understand how these two questions can go together as they seem entirely disconnected to me, so I'm probably not understanding the questions in the first place :)

I think https://reviews.llvm.org/D73405 sets a nice precedent and I would very much want to see something like this for specifying Linalg ops declaratively.

The main reason I have been reluctant to move on here is exactly because I would very much want to see something like this for specifying Linalg ops declaratively!
Where "something like this" is a feature that is generic enough to generalize and be reused across dialects, like D73405 :)

In D73826#1878406, @nicolasvasilache wrote:

In D73826#1877588, @mehdi_amini wrote:

Is there a world in which we can have such an Einsum-based parser on textual format to specify Linalg ops?
Does this have to be a language outside of MLIR? (I would certainly hope not).

I don't understand how these two questions can go together as they seem entirely disconnected to me, so I'm probably not understanding the questions in the first place :)

I think https://reviews.llvm.org/D73405 sets a nice precedent and I would very much want to see something like this for specifying Linalg ops declaratively.

The main reason I have been reluctant to move on here is exactly because I would very much want to see something like this for specifying Linalg ops declaratively!
Where "something like this" is a feature that is generic enough to generalize and be reused across dialects, like D73405 :)

Ok so if we agree on the principle I think this is a great start then.
My main concern is a cost benefit tradeoff: I am super interested in the result of this but do not have the background or resources to do it in a way that is portable and reusable across dialects. It still took me an inordinate amount of time to get to what you see here: this is not my area of expertise.
However I did start talking about this to various folks more than 3 months ago but everyone is very busy and it has reached a point where it is blocking.

Fair enough.
Even though:

• I don't remember you talking to me (or Jacques) about this 3 months ago (the first time this was mentioned was when Lei pushed the Tablegen hook and you mentioned you would use it later, but there was no detail at that time, and it was last month I believe).
• It isn't clear what is really blocked here. We tried to ask before:

Just to be clear, so all of this is so that you can write

let input_indexing_maps = [AffineExpressions<["i", "r_j"]>,
                             AffineExpressions<["r_j"]>];
let output_indexing_maps = [AffineExpressions<["i"]>];

along with the op and from this you are generating

return SmallVector<AffineMap, 4>{AffineMap::get(2, 0, {i, j}),
                                   AffineMap::get(2, 0, {j}),
                                   AffineMap::get(2, 0, {i})};

But we didn't really get an answer to the questions. I also mentioned "That means there is a tradeoff and we'd like to make sure we consider and weigh a bit more the alternatives, including having something a bit more ugly in ODS if it avoid custom backends. Hence the current question in the diff to try to do this."

Is it acceptable to move this in a new binary say ‘linalg-named-opgen-dontlookhowugly-pleasehelpimprove” so I am not blocked. When the right people have cycles and they see the end result they can start lending a hand.

It may be, but I'd be more comfortable supporting this if you could first help us figuring out the answers to the questions asked a few days ago. I'm still not sure I got the complete picture about the urgency (what is blocked exactly? why is it important right now?) and the alternatives.

[nicolasvasilache]

we didn't really get an answer to the questions. I also mentioned

I believe I did answer very clearly : the objective is to evolve into an einsum-like specification. I’ll paste it here again: a string such as “C(i, j) += A(i, k) * B(k, j)” is enough to specify the linalg.generic configuration for a named matmul op and this generalizes to many ops.

Where "something like this" is a feature that is generic enough to generalize and be reused across dialects, like D73405 :)

And yet we did not wait for having a nice declarative generic assembly form to be able to have Tablegen backed ops, did we?
As everything that has ever been contributed to MLIR, things are done on a per need basis. How is this different?

Now if you happen to have cycles to help with this, I’ll be thrilled! So could you please describe uses of einsum-like notation (or generalizations) for other dialects? Still, does coming up with such examples and a common solution need to slow down the progress of Linalg?

I don't remember you talking to me (or Jacques) about this 3 months ago

We originally talked about this with Jacques, Lei and River independently for some time, most recently back around Nov, there is an open internal bug that I can reference when I come back from vacation. At the time I don’t remember you being involved or having contributed to Tablegen aspects so it makes sense you may miss context. Note that at the time custom Tablegen backends were explicitly suggested as a solution (this is based on my recollection though so may be a bit fuzzy after all this time :) I’d have to look at the bug for details). Now that I need it for growing the set of Linalg ops, I am trying to male progress. This also happens to be an explicit KR, feel free to look it up internally or I can point you to it when I am back in office.

Bottom line, while I appreciate the objective of being future proof and general, I do not see a justification for blocking on this in this instance, place and time. I think this goes contrary to how MLIR contributors have been making progress: on a per need basis.

If the worry is that mlir-tblgen should not be augmented with DSL specific behavior I agree and I am happy to spin off a Linalg-specific binary that will be NFC for everything else Tablegen.

More generally I think we are touching a broader point: we want to allow and even encourage progress in specialized areas/dialects. When it is time for refactoring and generalization, we will do it, as usual, on a per-need basis. A very good example of this is the OpsInterface, which refactored and reused the Model/Concept you yourself proposed to solve the “inheritance” issue Linalg had in very beginning. This type of iteration takes months and benefits from existing concrete use cases.

Please let me know something actionnable that we can apply to unblock the declarative specification of named Linalg ops.

Thanks @joker-eph!

[mehdi_amini]

In D73826#1878508, @nicolasvasilache wrote:

we didn't really get an answer to the questions. I also mentioned

I believe I did answer very clearly : the objective is to evolve into an einsum-like specification. I’ll paste it here again: a string such as “C(i, j) += A(i, k) * B(k, j)” is enough to specify the linalg.generic configuration for a named matmul op and this generalizes to many ops.

I don't see how this patch is getting in this direction though.

If the goal is to write a TensorComprehension specifications for the ops, it seems like you may end-up generating ODS directly from such specifications. In any case, we could discuss this direction, but this is another thing that you didn't bring before.

Where "something like this" is a feature that is generic enough to generalize and be reused across dialects, like D73405 :)

And yet we did not wait for having a nice declarative generic assembly form to be able to have Tablegen backed ops, did we?
As everything that has ever been contributed to MLIR, things are done on a per need basis. How is this different?

When we didn't have this feature, we were writing inline C++ in ODS, or adding extra method that were calling C++ implementation.

Now if you happen to have cycles to help with this, I’ll be thrilled! So could you please describe uses of einsum-like notation (or generalizations) for other dialects?

Still, does coming up with such examples and a common solution need to slow down the progress of Linalg?

You haven't clarified how this is "slowing down the progress of Linalg" so far. But at the same time "Linalg" is a very fuzzy things for most people...

More generally I think we are touching a broader point: we want to allow and even encourage progress in specialized areas/dialects. When it is time for refactoring and generalization, we will do it, as usual, on a per-need basis. A very good example of this is the OpsInterface, which refactored and reused the Model/Concept you yourself proposed to solve the “inheritance” issue Linalg had in very beginning. This type of iteration takes months and benefits from existing concrete use cases.

Please let me know something actionnable that we can apply to unblock the declarative specification of named Linalg ops.

The immediate actionable direction I can provide is the same as before: please start answering questions, and trying to build more consensus behind what you're trying to achieve. I don't think being more and more pushy on this is gonna suddenly enlighten anyone and convince them, at least not me.
So I'll re-ask:

1. It isn't clear what is really blocked here, where is the urgency, especially with respect to current alternatives that I'd like to clarify. You just claimed progress is being blocked without specifics.
2. Jacques' question is a simple yes or no : "Just to be clear, so all of this is so that you can write"

let input_indexing_maps = [AffineExpressions<["i", "r_j"]>,
                             AffineExpressions<["r_j"]>];
let output_indexing_maps = [AffineExpressions<["i"]>];

along with the op and from this you are generating

return SmallVector<AffineMap, 4>{AffineMap::get(2, 0, {i, j}),
                                   AffineMap::get(2, 0, {j}),
                                   AffineMap::get(2, 0, {i})};

[nicolasvasilache]

Slowly getting back to this after vacation, I'll rebase on top of an NFC change so people see less noise.

[nicolasvasilache]

This is now absolete as of: https://reviews.llvm.org/D79135

mlir/include/mlir/Dialect/Linalg/IR/CMakeLists.txt

 add_mlir_dialect(LinalgOps LinalgDoc)
 set(LLVM_TARGET_DEFINITIONS LinalgStructuredOps.td)
 mlir_tablegen(LinalgStructuredOps.h.inc -gen-op-decls)
 mlir_tablegen(LinalgStructuredOps.cpp.inc -gen-op-defs)
+add_public_tablegen_target(MLIRLinalgStructuredOpsIncGen)
+
+set(LLVM_TARGET_DEFINITIONS LinalgStructuredOpsInterface.td)
 mlir_tablegen(LinalgStructuredOpsInterfaces.h.inc -gen-op-interface-decls)
 mlir_tablegen(LinalgStructuredOpsInterfaces.cpp.inc -gen-op-interface-defs)
-add_public_tablegen_target(MLIRLinalgStructuredOpsIncGen)
+add_public_tablegen_target(MLIRLinalgStructuredOpsInterfaceIncGen)
 
+set(LLVM_TARGET_DEFINITIONS LinalgStructuredOps.td)
+mlir_tablegen(LinalgNamedStructuredOpsInterface.cpp.inc -gen-linalg-named-structured-ops-defs)
+add_public_tablegen_target(MLIRLinalgNamedStructuredOpsInterfaceIncGen)

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredOps.td

 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LINALG_STRUCTURED_OPS
 #define LINALG_STRUCTURED_OPS
 
 include "mlir/Dialect/AffineOps/AffineOpsBase.td"
 include "mlir/Dialect/Linalg/IR/LinalgBase.td"
+include "mlir/Dialect/Linalg/IR/LinalgStructuredOpsInterface.td"
 
 // The Linalg `NInputs` trait provides the API for ops that are known
 // to have a specified number of inputs, all passed as operands.
 // See Linalg/LinalgTraits.h for implementation details an usage.
 class NInputs<int args_in> :
   NativeOpTrait<"linalg::NInputs<" # !cast<string>(args_in) # ">::Impl"> {}
 
 // The Linalg `NOutputs` trait provides the API for ops that are known
 // to have a specified number of outputs, all passed as operands.
 // See Linalg/LinalgTraits.h for implementation details an usage.
 class NOutputs<int args_out> :
   NativeOpTrait<"linalg::NOutputs<" # !cast<string>(args_out) # ">::Impl"> {}
 
 def StructuredOpTraits : NativeOpTrait<"linalg::StructuredOpTraits">;
 
-// The linalg 'LinalgStructuredInterface' provides access to the 'LinalgOp'
-// interface.
-def LinalgStructuredInterface : OpInterface<"LinalgOp"> {
-  let methods = [
-    //========================================================================//
-    // Loop types handling.
-    //========================================================================//
-    InterfaceMethod<
-      "Return the number of parallel loops within the current operation.",
-      "unsigned", "getNumParallelLoops"
-    >,
-    InterfaceMethod<
-      "Return the number of reduction loops within the current operation.",
-      "unsigned", "getNumReductionLoops"
-    >,
-    InterfaceMethod<
-      "Return the number of window loops within the current operation.",
-      "unsigned", "getNumWindowLoops"
-    >,
-    InterfaceMethod<
-      "Return the number of loops within the current operation.",
-      "unsigned", "getNumLoops">,
-
-    InterfaceMethod<
-      [{Returns true if the current operation has only one loop and it's a
-        reduction loop}],
-      "unsigned", "hasSingleReductionLoop">,
-
-    //========================================================================//
-    // Input arguments handling.
-    //========================================================================//
-    InterfaceMethod<
-      "Return the number of inputs from the current operation.",
-      "unsigned", "getNumInputs"
-    >,
-    InterfaceMethod<"Return the input view at the given index.",
-      "Value ", "getInput", (ins "unsigned":$i)
-    >,
-    InterfaceMethod<[{
-        Return the index of the given input value `v`, or `None` if the value is
-        not an input.
-      }],
-      "llvm::Optional<unsigned>", "getIndexOfInput", (ins "Value ":$v)
-    >,
-    InterfaceMethod<
-      "Return the input operands from the current operation.",
-      "Operation::operand_range", "getInputs"
-    >,
-    InterfaceMethod<[{
-        Return the type of the input shape at the given index.
-      }], "ShapedType", "getInputShapedType", (ins "unsigned":$i)>,
-    InterfaceMethod<[{
-        Return the subset of input operands that are of ranked tensor type.
-      }], "SmallVector<RankedTensorType, 4>", "getInputTensorTypes">,
-
-    //========================================================================//
-    // Output arguments handling.
-    //========================================================================//
-    InterfaceMethod<
-      "Return the number of outputs from the current operation.",
-      "unsigned", "getNumOutputs"
-    >,
-    InterfaceMethod<"Return the output buffer at the given index.",
-      "Value ", "getOutputBuffer", (ins "unsigned":$i)
-    >,
-    InterfaceMethod<[{
-        Return the index of the given buffer value, or `None` if the value is
-        not part of the output buffers.
-      }],
-      "llvm::Optional<unsigned>", "getIndexOfOutputBuffer", (ins "Value ":$view)
-    >,
-    InterfaceMethod<[{
-        Return the type of the output buffer at the given index.
-      }], "MemRefType", "getOutputBufferType", (ins "unsigned":$i)>,
-    InterfaceMethod<[{
-        Return the results that are of ranked tensor type.
-      }], "SmallVector<RankedTensorType, 4>", "getOutputTensorTypes">,
-    InterfaceMethod<
-      "Return the output buffers (operands) from the current operation.",
-      "Operation::operand_range", "getOutputBuffers"
-    >,
-
-    //========================================================================//
-    // Input and Output arguments handling.
-    //========================================================================//
-    InterfaceMethod<
-      "Return the number of inputs and outputs, irrespective of their buffer "
-      "or tensor type.",
-      "unsigned", "getNumInputsAndOutputs"
-    >,
-    InterfaceMethod<
-      "Return the number of inputs, irrespective of their buffer or tensor "
-      "type, and output buffers",
-      "unsigned", "getNumInputsAndOutputBuffers"
-    >,
-    InterfaceMethod<
-      "Return the range over inputs (irrespective of type) and output buffers.",
-      "Operation::operand_range", "getInputsAndOutputBuffers"
-    >,
-
-    //========================================================================//
-    // Other interface methods.
-    //========================================================================//
-    InterfaceMethod<
-      "Return the reference iterators for this named op (if any are specied). "
-      "These reference iterators are used to specify the default behavior of "
-      "the op. Typically this would be a static method but in order to allow "
-      "rank-polymorphic ops, this needs to be per object instance. Named ops "
-      "must define referenceIterators, even if empty for the 0-D case. "
-      "Generic ops on the other hand have a None `referenceIterators`",
-      "llvm::Optional<SmallVector<StringRef, 8>>", "referenceIterators"
-    >,
-    InterfaceMethod<
-      "Return the reference indexing maps for this named op (if any are "
-      "specified). Typically this would be a static method but in order to "
-      "allow rank-polymorphic ops, this needs to be per object instance. Named "
-      "ops must define referenceIterators, even if empty for the 0-D case. "
-      "Generic ops on the other hand have a None `referenceIndexingMaps`",
-      "llvm::Optional<SmallVector<AffineMap, 8>>", "referenceIndexingMaps"
-    >,
-    InterfaceMethod<
-      "Return the iterator types attribute within the current operation.",
-      "ArrayAttr", "iterator_types"
-    >,
-    InterfaceMethod<
-      "Return the indexing maps attribute within the current operation.",
-      "ArrayAttr", "indexing_maps"
-    >,
-    InterfaceMethod<"Return the input or output indexing map at index `i`.",
-      "AffineMap", "getIndexingMap", (ins "unsigned":$i)
-    >,
-    InterfaceMethod<"Return the input indexing map at index `i`.",
-      "AffineMap", "getInputIndexingMap", (ins "unsigned":$i)
-    >,
-    InterfaceMethod<"Return the output indexing map at index `i`.",
-      "AffineMap", "getOutputIndexingMap", (ins "unsigned":$i)
-    >,
-    InterfaceMethod<[{
-        Return whether the op has only MemRef input and outputs.
-      }], "bool", "hasBufferSemantics">,
-    InterfaceMethod<[{
-        Return whether the op has only RankedTensor input and outputs.
-      }], "bool", "hasTensorSemantics">,
-
-    //========================================================================//
-    // Other static interface methods.
-    //========================================================================//
-    StaticInterfaceMethod<[{
-        Create an operation of the current type with the given location,
-        operands, and attributes.
-      }],
-      "Operation *", "create",
-      (ins "OpBuilder &":$builder, "Location":$loc,
-           "ValueRange":$operands,
-           "ArrayRef<NamedAttribute>":$attributes), [{
-        return builder.create<ConcreteOp>(loc, ArrayRef<Type>{}, operands,
-                                          attributes);
-      }]
-    >,
-    InterfaceMethod<[{
-        Clone the current operation with the given location and operands. This
-        is used to abstract away the optional underlying region creation.
-      }],
-      "Operation *", "clone",
-      (ins "OpBuilder &":$b, "Location":$loc, "ValueRange":$operands), [{
-        BlockAndValueMapping map;
-        unsigned numRegions = op.getOperation()->getNumRegions();
-        Operation *res = create(b, loc, operands, op.getAttrs());
-        assert(res->getNumRegions() == numRegions && "inconsistent # regions");
-        for (unsigned ridx = 0; ridx < numRegions; ++ridx)
-          op.getOperation()->getRegion(ridx).cloneInto(
-            &res->getRegion(ridx), map);
-        return res;
-      }]
-    >
-  ];
-}
-
 // Base Tablegen class for Linalg ops.
 // Linalg ops that correspond to library calls operate on linalg::View as their
 // first operands. These may be optionally followed by non-view operands
 // depending on the specific Linalg op.
 class LinalgStructuredBase_Op<string mnemonic, list<OpTrait> props>
   : Op<Linalg_Dialect, mnemonic,
        !listconcat(props, [StructuredOpTraits, LinalgStructuredInterface])> {
 }
 
 class LinalgStructured_Op<string mnemonic, list<OpTrait> props>
   : LinalgStructuredBase_Op<mnemonic, props> {
   code libraryCallName = [{
     std::string getLibraryCallName() {
       return generateLibraryCallName(getOperation());
     }
   }];
   let assemblyFormat = "`(` operands `)` attr-dict `:` type(operands)";
 }
 
 ////////////////////////////////////////////////////////////////////////////////
-// Named Linalg ops, implemented as special configurations of a generic op.
+// Named Linalg ops, implemented as special configurations of generic ops.
+// At the moment these are not declarative and require a bunch of C++ code.
+// These should be migrated to the declarative configurations as they evolve.
 ////////////////////////////////////////////////////////////////////////////////
 def CopyOp : LinalgStructured_Op<"copy", [NInputs<1>, NOutputs<1>]> {
   let description = [{
     Copies the data in the input view into the output view.
 
     Usage:
       ```mlir
       linalg.copy(%arg0, %arg1) : memref<?xf32, stride_specification>,
llvm::Optional<SmallVector<AffineMap, 8>> referenceIndexingMaps() {
       llvm_unreachable("NYI referenceIndexingMaps for CopyOp");
     }
   }];
   let verifier = [{ return ::verify(*this); }];
 
   let hasFolder = 1;
 }
 
-def FillOp : LinalgStructured_Op<"fill", [NInputs<0>, NOutputs<1>]> {
-  let arguments = (ins AnyStridedMemRef:$output,
-                   AnyTypeOf<[AnyFloat, AnyInteger, AnyVector]>:$value);
-  let extraClassDeclaration = libraryCallName # [{
-    // Defined in C++ for now.
-    // TODO(ntv): auto-generate.
-    ArrayAttr indexing_maps();
-
-    // Rank-polymorphic.
-    //   filling_value -> O(ivs) with parallel iterators.
-    llvm::Optional<SmallVector<StringRef, 8>> referenceIterators() {
-      unsigned nPar = output().getType().cast<ShapedType>().getRank();
-      return SmallVector<StringRef, 8>(nPar, getParallelIteratorTypeName());
-    }
-
-    llvm::Optional<SmallVector<AffineMap, 8>> referenceIndexingMaps() {
-      llvm_unreachable("NYI referenceIndexingMaps for CopyOp");
-    }
-  }];
-  let verifier = [{ return ::verify(*this); }];
-  
-  let hasFolder = 1;
-}
-
 def DotOp : LinalgStructured_Op<"dot", [NInputs<2>, NOutputs<1>]> {
   let arguments = (ins AnyStridedMemRefOfRank<1>,
                        AnyStridedMemRefOfRank<1>,
                        AnyStridedMemRefOfRank<0>);
   let extraClassDeclaration = libraryCallName # [{
     // Defined in C++ for now.
     // TODO(ntv): auto-generate.
     ArrayAttr indexing_maps();
 
     llvm::Optional<SmallVector<StringRef, 8>> referenceIterators() {
       return SmallVector<StringRef, 8>{getReductionIteratorTypeName()};
     }
 
     llvm::Optional<SmallVector<AffineMap, 8>> referenceIndexingMaps() {
       llvm_unreachable("NYI referenceIndexingMaps for DotOp");
     }
   }];
 
   let hasFolder = 1;
 }
 
-def MatvecOp : LinalgStructured_Op<"matvec", [NInputs<2>, NOutputs<1>]> {
-  let arguments = (ins AnyStridedMemRefOfRank<2>,
-                       AnyStridedMemRefOfRank<1>,
-                       AnyStridedMemRefOfRank<1>);
-  let extraClassDeclaration = libraryCallName # [{
-    // Defined in C++ for now.
-    // TODO(ntv): auto-generate.
-    ArrayAttr indexing_maps();
-
-    llvm::Optional<SmallVector<StringRef, 8>> referenceIterators() {
-      return SmallVector<StringRef, 8>{
-        getParallelIteratorTypeName(),
-        getReductionIteratorTypeName()};
-    }
-
-    llvm::Optional<SmallVector<AffineMap, 8>> referenceIndexingMaps() {
-      llvm_unreachable("NYI referenceIndexingMaps for MatvecOp");
-    }
-  }];
-
-  let hasFolder = 1;
-}
-
 def MatmulOp : LinalgStructured_Op<"matmul", [NInputs<2>, NOutputs<1>]> {
   let arguments = (ins AnyStridedMemRefOfRank<2>,
                        AnyStridedMemRefOfRank<2>,
                        AnyStridedMemRefOfRank<2>);
   let extraClassDeclaration = libraryCallName # [{
     // Defined in C++ for now.
     // TODO(ntv): auto-generate.
     ArrayAttr indexing_maps();
def ConvOp : LinalgStructured_Op<"conv", [NInputs<2>, NOutputs<1>]> {
   let arguments = (ins AnyStridedMemRef:$filter, AnyStridedMemRef:$input,
                    AnyStridedMemRef:$output,
                    OptionalAttr<I64ArrayAttr>:$strides,
                    OptionalAttr<I64ArrayAttr>:$dilations);
   let extraClassDeclaration = libraryCallName # [{
     // TODO(ntv) extend to support more than 1 dimensions and potentially
     // grouping too.
     unsigned getNumBatchDimensions() { return 1; }
+
     unsigned getNumInputFeatureDimensions() { return 1; }
+
     unsigned getNumOutputFeatureDimensions() { return 1; }
 
     // Defined in C++ for now.
     // TODO(ntv): auto-generate.
     ArrayAttr indexing_maps();
 
     llvm::Optional<SmallVector<StringRef, 8>> referenceIterators() {
       // Outer parallel loops are always the number of output dimensions; i.e.
let extraClassDeclaration = libraryCallName # [{
     }
   }];
 
   let verifier = [{ return ::verify(*this); }];
 
   let hasFolder = 1;
 }
 
+////////////////////////////////////////////////////////////////////////////////
+// Named Linalg ops, implemented as special configurations of generic ops.
+// At the moment these are not fully declarative special C++ code in the form of
+// specifying the reference iterators and indexing maps.
+// This is necessary to obtain rank-polymorphic behavior and until a better
+// solution is implemented.
+////////////////////////////////////////////////////////////////////////////////
+
+def NamedStructuredOpTraits : NativeOpTrait<"linalg::NamedStructuredOpTraits">;
+def NamedStructuredOpLibraryCallTraits :
+  NativeOpTrait<"linalg::NamedStructuredOpLibraryCallTraits">;
+
+def FillOp : LinalgStructured_Op<"fill", [
+    NInputs<0>,
+    NOutputs<1>,
+    NamedStructuredOpTraits]> {
+  let arguments = (ins AnyStridedMemRef:$output,
+                   AnyTypeOf<[AnyFloat, AnyInteger, AnyVector]>:$value);
+  let extraClassDeclaration = libraryCallName # [{
+    // Rank-polymorphic.
+    //   filling_value -> O(ivs) with parallel iterators.
+    llvm::Optional<SmallVector<StringRef, 8>> referenceIterators() {
+      unsigned nPar = output().getType().cast<ShapedType>().getRank();
+      return SmallVector<StringRef, 8>(nPar, getParallelIteratorTypeName());
+    }
+
+    llvm::Optional<SmallVector<AffineMap, 8>> referenceIndexingMaps() {
+      auto ctx = getContext();
+      unsigned rank = output().getType().cast<ShapedType>().getRank();
+      if (rank == 0)
+        return SmallVector<AffineMap, 8>{
+          AffineMap::get(0, 0, {getAffineConstantExpr(0, ctx)})};
+      return SmallVector<AffineMap, 8>{
+        AffineMap::getMultiDimIdentityMap(rank, ctx)};
+    }
+  }];
+
+  let hasFolder = 1;
+}
+
+////////////////////////////////////////////////////////////////////////////////
+// Named Linalg ops, implemented as a declarative configurations of generic ops.
+// For now these ops have fixed rank which allows us to immediately specify
+// their core properties declaratively.
+// The big missing piece atm is the region which is currently still C++.
+// In the future we will also want to automatically generate the matchers and
+// rewriters to allow rewriting a linalg.generic into a named op.
+////////////////////////////////////////////////////////////////////////////////
+
+class AffineExpressions<list<string> map> {
+  list<string> indexing_maps = map;
+}
+
+class LinalgNamedStructured_Op<string mnemonic, list<OpTrait> props>
+  : Op<Linalg_Dialect, mnemonic,
+       !listconcat(props, [StructuredOpTraits, LinalgStructuredInterface])> {
+  list<string> iterators = ?;
+  list<string> iterators_types = ?;
+  list<AffineExpressions> input_indexing_maps = ?;
+  list<AffineExpressions> output_indexing_maps = ?;
+  bit hasLibraryImpl = 0;
+  let assemblyFormat = "`(` operands `)` attr-dict `:` type(operands)";
+}
+
+def MatvecOp : LinalgNamedStructured_Op<"matvec", [
+     NInputs<2>,
+     NOutputs<1>,
+     NamedStructuredOpTraits,
+     NamedStructuredOpLibraryCallTraits]> {
+  let arguments = (ins AnyStridedMemRefOfRank<2>,
+                       AnyStridedMemRefOfRank<1>,
+                       AnyStridedMemRefOfRank<1>);
+  let iterators = ["i", "r_j"];
+  let iterators_types = ["parallel", "reduction"];
+  //   A(i, r_j) * B(r_j) -> C(i)
+  let input_indexing_maps = [AffineExpressions<["i", "r_j"]>,
+                             AffineExpressions<["r_j"]>];
+  let output_indexing_maps = [AffineExpressions<["i"]>];
+
+  let hasLibraryImpl = 1;
+  let hasFolder = 1;
+}
+
+////////////////////////////////////////////////////////////////////////////////
+// Generic Linalg ops.
+////////////////////////////////////////////////////////////////////////////////
 def LinalgOperand: Type<
   Or<[AnyRankedTensor.predicate, AnyStridedMemRef.predicate]>>;
 
 class LinalgOperandOfRank<int rank>: Type<
   And<[
     LinalgOperand.predicate,
     CPred<"$_self.cast<ShapedType>().getRank() == " # rank>]
   >>;
 
-////////////////////////////////////////////////////////////////////////////////
-// Generic Linalg ops.
-////////////////////////////////////////////////////////////////////////////////
 class GenericOpBase<string mnemonic> : LinalgStructuredBase_Op<mnemonic, []> {
   let arguments = (ins Variadic<LinalgOperand>:$views,
                    I64Attr:$args_in,
                    I64Attr:$args_out,
                    AffineMapArrayAttr:$indexing_maps,
                    ArrayAttr:$iterator_types,
                    OptionalAttr<StrAttr>:$doc,
                    OptionalAttr<FlatSymbolRefAttr>:$fun,

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredOpsInterface.td

+//===- LinalgStructuredInterface.td- Linalg StructuredIfce -*- tablegen -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This is the definition file for the structured interface for Linalg ops.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LINALG_STRUCTURED_INTERFACE
+#define LINALG_STRUCTURED_INTERFACE
+
+include "mlir/Dialect/Linalg/IR/LinalgBase.td"
+
+// The linalg 'LinalgStructuredInterface' provides access to the 'LinalgOp'
+// interface.
+def LinalgStructuredInterface : OpInterface<"LinalgOp"> {
+  let methods = [
+    //========================================================================//
+    // Loop types handling.
+    //========================================================================//
+    InterfaceMethod<
+      "Return the number of parallel loops within the current operation.",
+      "unsigned", "getNumParallelLoops"
+    >,
+    InterfaceMethod<
+      "Return the number of reduction loops within the current operation.",
+      "unsigned", "getNumReductionLoops"
+    >,
+    InterfaceMethod<
+      "Return the number of window loops within the current operation.",
+      "unsigned", "getNumWindowLoops"
+    >,
+    InterfaceMethod<
+      "Return the number of loops within the current operation.",
+      "unsigned", "getNumLoops">,
+
+    InterfaceMethod<
+      [{Returns true if the current operation has only one loop and it's a
+        reduction loop}],
+      "unsigned", "hasSingleReductionLoop">,
+
+    //========================================================================//
+    // Input arguments handling.
+    //========================================================================//
+    InterfaceMethod<
+      "Return the number of inputs from the current operation.",
+      "unsigned", "getNumInputs"
+    >,
+    InterfaceMethod<"Return the input view at the given index.",
+      "Value ", "getInput", (ins "unsigned":$i)
+    >,
+    InterfaceMethod<[{
+        Return the index of the given input value `v`, or `None` if the value is
+        not an input.
+      }],
+      "llvm::Optional<unsigned>", "getIndexOfInput", (ins "Value ":$v)
+    >,
+    InterfaceMethod<
+      "Return the input operands from the current operation.",
+      "Operation::operand_range", "getInputs"
+    >,
+    InterfaceMethod<[{
+        Return the type of the input shape at the given index.
+      }], "ShapedType", "getInputShapedType", (ins "unsigned":$i)>,
+    InterfaceMethod<[{
+        Return the subset of input operands that are of ranked tensor type.
+      }], "SmallVector<RankedTensorType, 4>", "getInputTensorTypes">,
+
+    //========================================================================//
+    // Output arguments handling.
+    //========================================================================//
+    InterfaceMethod<
+      "Return the number of outputs from the current operation.",
+      "unsigned", "getNumOutputs"
+    >,
+    InterfaceMethod<"Return the output buffer at the given index.",
+      "Value ", "getOutputBuffer", (ins "unsigned":$i)
+    >,
+    InterfaceMethod<[{
+        Return the index of the given buffer value, or `None` if the value is
+        not part of the output buffers.
+      }],
+      "llvm::Optional<unsigned>", "getIndexOfOutputBuffer", (ins "Value ":$view)
+    >,
+    InterfaceMethod<[{
+        Return the type of the output buffer at the given index.
+      }], "MemRefType", "getOutputBufferType", (ins "unsigned":$i)>,
+    InterfaceMethod<[{
+        Return the results that are of ranked tensor type.
+      }], "SmallVector<RankedTensorType, 4>", "getOutputTensorTypes">,
+    InterfaceMethod<
+      "Return the output buffers (operands) from the current operation.",
+      "Operation::operand_range", "getOutputBuffers"
+    >,
+
+    //========================================================================//
+    // Input and Output arguments handling.
+    //========================================================================//
+    InterfaceMethod<
+      "Return the number of inputs and outputs, irrespective of their buffer "
+      "or tensor type.",
+      "unsigned", "getNumInputsAndOutputs"
+    >,
+    InterfaceMethod<
+      "Return the number of inputs, irrespective of their buffer or tensor "
+      "type, and output buffers",
+      "unsigned", "getNumInputsAndOutputBuffers"
+    >,
+    InterfaceMethod<
+      "Return the range over inputs (irrespective of type) and output buffers.",
+      "Operation::operand_range", "getInputsAndOutputBuffers"
+    >,
+
+    //========================================================================//
+    // Other interface methods.
+    //========================================================================//
+    InterfaceMethod<
+      "Return the reference iterators for this named op (if any are specied). "
+      "These reference iterators are used to specify the default behavior of "
+      "the op. Typically this would be a static method but in order to allow "
+      "rank-polymorphic ops, this needs to be per object instance. Named ops "
+      "must define referenceIterators, even if empty for the 0-D case. "
+      "Generic ops on the other hand have a None `referenceIterators`",
+      "llvm::Optional<SmallVector<StringRef, 8>>", "referenceIterators"
+    >,
+    InterfaceMethod<
+      "Return the reference indexing maps for this named op (if any are "
+      "specified). Typically this would be a static method but in order to "
+      "allow rank-polymorphic ops, this needs to be per object instance. Named "
+      "ops must define referenceIterators, even if empty for the 0-D case. "
+      "Generic ops on the other hand have a None `referenceIndexingMaps`",
+      "llvm::Optional<SmallVector<AffineMap, 8>>", "referenceIndexingMaps"
+    >,
+    InterfaceMethod<
+      "Return the iterator types attribute within the current operation.",
+      "ArrayAttr", "iterator_types"
+    >,
+    InterfaceMethod<
+      "Return the indexing maps attribute within the current operation.",
+      "ArrayAttr", "indexing_maps"
+    >,
+    InterfaceMethod<"Return the input or output indexing map at index `i`.",
+      "AffineMap", "getIndexingMap", (ins "unsigned":$i)
+    >,
+    InterfaceMethod<"Return the input indexing map at index `i`.",
+      "AffineMap", "getInputIndexingMap", (ins "unsigned":$i)
+    >,
+    InterfaceMethod<"Return the output indexing map at index `i`.",
+      "AffineMap", "getOutputIndexingMap", (ins "unsigned":$i)
+    >,
+    InterfaceMethod<[{
+        Return whether the op has only MemRef input and outputs.
+      }], "bool", "hasBufferSemantics">,
+    InterfaceMethod<[{
+        Return whether the op has only RankedTensor input and outputs.
+      }], "bool", "hasTensorSemantics">,
+
+    //========================================================================//
+    // Other static interface methods.
+    //========================================================================//
+    StaticInterfaceMethod<[{
+        Create an operation of the current type with the given location,
+        operands, and attributes.
+      }],
+      "Operation *", "create",
+      (ins "OpBuilder &":$builder, "Location":$loc,
+           "ValueRange":$operands,
+           "ArrayRef<NamedAttribute>":$attributes), [{
+        return builder.create<ConcreteOp>(loc, ArrayRef<Type>{}, operands,
+                                          attributes);
+      }]
+    >,
+    InterfaceMethod<[{
+        Clone the current operation with the given location and operands. This
+        is used to abstract away the optional underlying region creation.
+      }],
+      "Operation *", "clone",
+      (ins "OpBuilder &":$b, "Location":$loc, "ValueRange":$operands), [{
+        BlockAndValueMapping map;
+        unsigned numRegions = op.getOperation()->getNumRegions();
+        Operation *res = create(b, loc, operands, op.getAttrs());
+        assert(res->getNumRegions() == numRegions && "inconsistent # regions");
+        for (unsigned ridx = 0; ridx < numRegions; ++ridx)
+          op.getOperation()->getRegion(ridx).cloneInto(
+            &res->getRegion(ridx), map);
+        return res;
+      }]
+    >
+  ];
+}
+
+#endif // LINALG_STRUCTURED_INTERFACE

mlir/include/mlir/Dialect/Linalg/IR/LinalgTraits.h

ArrayAttr iterator_types() {
 
     // If there is no reference, this must be a generic op.
     // TODO(ntv): Traits are used to define ops. Split into cpp to avoid
     // cyclic dependency.
     auto name = this->getOperation()->getName().getStringRef();
     if (!maybeReferenceIteratorTypes && name != "generic" &&
         name != "indexed_generic") {
       this->getOperation()->dump();
-      llvm_unreachable("Op missing ");
+      llvm_unreachable("Op missing referenceIterators");
     }
 
-    // If we have a reference, build the reference attribute.
+    // If we have a reference, build the reference attribute and set it in the
+    // op before returning.
     auto *ctx = this->getOperation()->getContext();
     auto attrRange = llvm::map_range(*maybeReferenceIteratorTypes,
                                      [ctx](StringRef str) -> Attribute {
                                        return StringAttr::get(str, ctx);
                                      });
     auto attr = ArrayAttr::get(llvm::to_vector<4>(attrRange), ctx);
     // TODO(ntv): Need to memoize this. Can't just store as an attribute atm as
     // it will impact parser, printer and tests.
public:
   static LogicalResult verifyTrait(Operation *op) {
     auto nOperands = cast<ConcreteType>(op).getNumInputsAndOutputBuffers();
     if (failed(OpTrait::impl::verifyAtLeastNOperands(op, nOperands)))
       return failure();
     return success();
   }
 };
 
+/// This class provides the API for named Linalg StructuredOps.
+template <typename ConcreteType>
+class NamedStructuredOpTraits
+    : public OpTrait::TraitBase<ConcreteType, NamedStructuredOpTraits> {
+public:
+  llvm::Optional<SmallVector<StringRef, 8>> referenceIterators();
+  llvm::Optional<SmallVector<AffineMap, 8>> referenceIndexingMaps();
+};
+
+/// This class provides the API mapping named Linalg StructuredOps to a library
+/// call.
+template <typename ConcreteType>
+class NamedStructuredOpLibraryCallTraits
+    : public OpTrait::TraitBase<ConcreteType,
+                                NamedStructuredOpLibraryCallTraits> {
+public:
+  std::string getLibraryCallName();
+};
+
 } // namespace linalg
 } // namespace OpTrait
 } // namespace mlir
 
 #endif // MLIR_DIALECT_LINALG_LINALGTRAITS_H_

mlir/lib/Dialect/Linalg/IR/CMakeLists.txt

add_llvm_library(MLIRLinalgOps
   intrinsics_gen
   )
 
 add_dependencies(MLIRLinalgOps
 
   ${LIBS}
   MLIRLinalgOpsIncGen
   MLIRLinalgStructuredOpsIncGen
+  MLIRLinalgStructuredOpsInterfaceIncGen
+  MLIRLinalgNamedStructuredOpsInterfaceIncGen
   )
 target_link_libraries(MLIRLinalgOps ${LIBS})

mlir/lib/Dialect/Linalg/IR/LinalgOps.cpp

static AffineMap extractOrIdentityMap(Optional<AffineMap> maybeMap,
   if (maybeMap)
     return maybeMap.getValue();
   if (rank == 0)
     return AffineMap();
   return AffineMap::getMultiDimIdentityMap(rank, context);
 }
 
 namespace mlir {
+namespace OpTrait {
+namespace linalg {
+
+// Linalg "named" traits depend on
+#include "mlir/Dialect/Linalg/IR/LinalgNamedStructuredOps.cpp.inc"
+
+} // namespace linalg
+} // namespace OpTrait
+} // namespace mlir
+
+namespace mlir {
 namespace linalg {
 
 #include "mlir/Dialect/Linalg/IR/LinalgStructuredOpsInterfaces.cpp.inc"
 
 #define GET_OP_CLASSES
 #include "mlir/Dialect/Linalg/IR/LinalgOps.cpp.inc"
 
 #define GET_OP_CLASSES
if (auto copyOp = dyn_cast<CopyOp>(op)) {
     auto maybeInputMap = copyOp.inputPermutation();
     auto maybeOutputMap = copyOp.outputPermutation();
     unsigned inputRank = copyOp.getInputShapedType(0).getRank();
     unsigned outputRank = copyOp.getOutputShapedType(0).getRank();
     return SmallVector<AffineMap, 4>{
         extractOrIdentityMap(maybeInputMap, inputRank, context),
         extractOrIdentityMap(maybeOutputMap, outputRank, context)};
   }
-  if (auto fillOp = dyn_cast<FillOp>(op)) {
-    // filling_value -> O(ivs)
-    unsigned rank = fillOp.getNumParallelLoops();
-    return SmallVector<AffineMap, 4>{
-        extractOrIdentityMap(llvm::None, rank, context)};
-  }
   auto i = getAffineDimExpr(0, context);
   auto j = getAffineDimExpr(1, context);
   auto k = getAffineDimExpr(2, context);
   if (isa<DotOp>(op))
     // A(r_i) * B(r_i) -> C()
     return SmallVector<AffineMap, 4>{AffineMap::get(1, 0, {i}),
                                      AffineMap::get(1, 0, {i}), AffineMap()};
-  if (isa<MatvecOp>(op))
-    //   A(i, r_j) * B(r_j) -> C(i)
-    return SmallVector<AffineMap, 4>{AffineMap::get(2, 0, {i, j}),
-                                     AffineMap::get(2, 0, {j}),
-                                     AffineMap::get(2, 0, {i})};
   if (isa<MatmulOp>(op))
     //   A(i, r_k) * B(r_k, j) -> C(i, j)
     return SmallVector<AffineMap, 4>{AffineMap::get(3, 0, {i, k}),
                                      AffineMap::get(3, 0, {k, j}),
                                      AffineMap::get(3, 0, {i, j})};
   if (auto convOp = dyn_cast<ConvOp>(op)) {
     //   F(z0, ..., zN-1, q, k) * I(b, x0 + z0, ..., xN-1 + zN-1, q) ->
     //     O(b, x0, ..., xN-1, k)
ArrayAttr mlir::linalg::ConvOp::indexing_maps() {
   return getIndexingMaps(getOperation());
 }
 ArrayAttr mlir::linalg::CopyOp::indexing_maps() {
   return getIndexingMaps(getOperation());
 }
 ArrayAttr mlir::linalg::DotOp::indexing_maps() {
   return getIndexingMaps(getOperation());
 }
-ArrayAttr mlir::linalg::FillOp::indexing_maps() {
-  return getIndexingMaps(getOperation());
-}
 ArrayAttr mlir::linalg::MatmulOp::indexing_maps() {
   return getIndexingMaps(getOperation());
 }
-ArrayAttr mlir::linalg::MatvecOp::indexing_maps() {
-  return getIndexingMaps(getOperation());
-}
 
 // TODO(ntv, rriddle): Consider making all this boilerplate easy to autogenerate
 // with Tablegen. This seems a desirable property in the context of OpInterfaces
 // where a Linalg "named" op **isa** LinalgOp.
 LogicalResult ConvOp::fold(ArrayRef<Attribute>,
                            SmallVectorImpl<OpFoldResult> &) {
   return foldMemRefCast(*this);
 }

mlir/tools/mlir-tblgen/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   MLIRTableGen
   Support
   )
 
 add_tablegen(mlir-tblgen MLIR
   EnumsGen.cpp
+  LinalgNamedOpsGen.cpp
   LLVMIRConversionGen.cpp
   LLVMIRIntrinsicGen.cpp
   mlir-tblgen.cpp
   OpDefinitionsGen.cpp
   OpDocGen.cpp
   OpFormatGen.cpp
   OpInterfacesGen.cpp
   ReferenceImplGen.cpp
   RewriterGen.cpp
   SPIRVUtilsGen.cpp
   StructsGen.cpp
   )
 set_target_properties(mlir-tblgen PROPERTIES FOLDER "Tablegenning")

mlir/tools/mlir-tblgen/LinalgNamedOpsGen.cpp

+//===- LinalgNamedOpsGen.cpp - MLIR Linalg op generator -------------------===//
+//
+// Part of the MLIR Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// LinalgNamedOpsGen uses the description of structured operations in the Linalg
+// dialect to generate op definitions, parsers, pretty-printers and matchers
+// (e.g. linalg.matmul).
+//
+//===----------------------------------------------------------------------===//
+
+#include "mlir/Support/STLExtras.h"
+#include "mlir/TableGen/GenInfo.h"
+#include "mlir/TableGen/OpClass.h"
+#include "mlir/TableGen/Operator.h"
+
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/TableGen/Error.h"
+#include "llvm/TableGen/Record.h"
+#include "llvm/TableGen/TableGenBackend.h"
+
+using namespace llvm;
+using namespace mlir;
+using namespace mlir::tblgen;
+
+using mlir::tblgen::Operator;
+
+// Fills a function with signature "void iterator_types()".
+void iteratorTypesBody(const Record &r, llvm::raw_ostream &os) {
+  auto iteratorTypes = r.getValueAsListOfStrings("iterators_types");
+  os << "  return SmallVector<StringRef, 8>{";
+  interleaveComma(iteratorTypes, os,
+                  [&os](StringRef str) { os << "\"" << str << "\""; });
+  os << "};";
+}
+
+// Fils a function with signature "ArrayAttr iterator_maps()".
+void indexingMapsBody(const Record &r, llvm::raw_ostream &os) {
+  auto iterators = r.getValueAsListOfStrings("iterators");
+  unsigned numDims = iterators.size();
+  /* return AffineMap::get({...}, ctx); */
+  auto ins = r.getValueAsListOfDefs("input_indexing_maps");
+  auto outs = r.getValueAsListOfDefs("output_indexing_maps");
+  SmallVector<const Record *, 4> maps(ins.begin(), ins.end());
+  maps.append(outs.begin(), outs.end());
+
+  /* AffineExpr ...; */
+  os << "  MLIRContext *ctx = getOperation()->getContext();\n";
+  interleaveComma(iterators, os << "  AffineExpr ");
+  os << ";\n";
+  /* bindDims(ctx, ...); */
+  interleaveComma(iterators, os << "  bindDims(ctx, ");
+  os << ");\n";
+  os << "  return SmallVector<AffineMap, 8>{";
+  interleaveComma(maps, os, [&os, numDims](const Record *def) {
+    os << "\n    AffineMap::get(" << numDims << ", 0, {";
+    interleaveComma(def->getValueAsListOfStrings("indexing_maps"), os,
+                    [&os, numDims](const StringRef &str) {
+                      os << "\n      simplifyAffineExpr(" << str << ", "
+                         << numDims << ", 0)";
+                    });
+    os << "})";
+  });
+  os << "};";
+}
+
+template <typename Lambda>
+void emitMethodWithBody(StringRef traitName, StringRef opName,
+                        StringRef resultType, StringRef funName,
+                        StringRef operands, const Record &r,
+                        llvm::raw_ostream &os, Lambda fun) {
+  os << "template<>\n";
+  os << resultType << " " << traitName << "<" << opName << ">::" << funName
+     << "(" << operands << ") {\n";
+  fun(r, os);
+  os << "\n}\n\n";
+}
+
+bool emitLinalgNamedOpsInterfaceFunctions(const llvm::RecordKeeper &records,
+                                          llvm::raw_ostream &os) {
+  llvm::emitSourceFileHeader("Operations for Linalg Named Structured Ops", os);
+  os << "#include \"mlir/Dialect/Linalg/IR/LinalgTraits.h\"\n\n";
+
+  auto defs = records.getAllDerivedDefinitions("LinalgNamedStructured_Op");
+  for (const llvm::Record *r : defs) {
+    const auto *traitName = "NamedStructuredOpTraits";
+    auto opName = r->getName();
+    const auto *resultType = "llvm::Optional<SmallVector<StringRef, 8>>";
+    const auto *funName = "referenceIterators";
+    emitMethodWithBody(traitName, opName, resultType, funName, /*operands=*/"",
+                       *r, os, iteratorTypesBody);
+
+    resultType = "llvm::Optional<SmallVector<AffineMap, 8>>";
+    funName = "referenceIndexingMaps";
+    emitMethodWithBody(traitName, opName, resultType, funName, /*operands=*/"",
+                       *r, os, indexingMapsBody);
+
+    if (r->getValueAsBit("hasLibraryImpl")) {
+      traitName = "NamedStructuredOpLibraryCallTraits";
+      resultType = "std::string";
+      funName = "getLibraryCallName";
+      emitMethodWithBody(
+          traitName, opName, resultType, funName, /*operands=*/"", *r, os,
+          [](const Record &r, llvm::raw_ostream &os) {
+            os << "  return generateLibraryCallName(getOperation());";
+          });
+    }
+  }
+
+  return false;
+}
+
+static mlir::GenRegistration
+    genLinalgNamedOps("gen-linalg-named-structured-ops-defs",
+                      "Generate Linalg named StructuredOps",
+                      emitLinalgNamedOpsInterfaceFunctions);