This is an archive of the discontinued LLVM Phabricator instance.

[mlir][linalg] Add support for arbitrary element types for named operations
AbandonedPublic

Authored by andidr on Jan 24 2022, 2:11 AM.

Details

Summary

Linalg named operations are currently limited to tensors and memrefs
composed of floating point, integer or index elements and using any
other element type triggers an assertion.

This change adds support for arbitrary element types for named
operations through a new type interface
LinalgArithmeticOperatorTypeInterface and two operation interfaces
LinalgAddOpInterface and LinalgMulOpInterface.

The type interface is used to instantiate operations that implement
arithmetic operations for a type (e.g., instantiating arith::AddFOp
for an addition on floats) transparently. The two operation interfaces
allow for the identification of instances of operations implementing
additions and multiplications, required by ContractionOpInterface.

Implementations of LinalgArithmeticOperatorTypeInterface are
provided for the builtin float and integer types and implementations
of LinalgAddOpInterface and LinalgAddOpInterface are provided for
arith::AddFOp, arith::AddIOp, arith::MulFOp, and
arith::MulIOp, respectively.

Diff Detail

Event Timeline

andidr created this revision.Jan 24 2022, 2:11 AM
andidr requested review of this revision.Jan 24 2022, 2:11 AM
andidr edited the summary of this revision. (Show Details)Jan 24 2022, 2:42 AM
andidr updated this revision to Diff 402482.Jan 24 2022, 5:13 AM

Fixed formatting issue in mlir/lib/Dialect/Linalg/IR/LinalgTypeInterfaces.cpp.

ormris removed a subscriber: ormris.Jan 24 2022, 11:05 AM

Thanks for moving this forward!

I added a few detail comments but I would like to ask a high-level question first. Wouldn't it make sense to have a single main interface method that takes the enum value, op builder, and a value range and then do the dispatch/error handling etc internally in the interface? Ideally, when adding a new operation we want to touch as little places as possible. At the moment, we need to adapt the enum, the applyfn_, the interface, and at least one interface implementation. Having an interface that takes an enum instead of having one method per operation time should reduce that by one and also simplifies a follow up refactoring to remove applyfn_

iface.create(builder, ArithFn::Add, ValueRange{lhs, rhs})
iface.create(builder, ArithFn::Exp, ValueRange{x})
mlir/include/mlir/Dialect/Linalg/IR/LinalgInterfaces.td
1068

I would probably go with LinalgArithmeticOpTypeInterface or even LinalgArithOpTypeInterface? If you prefer the long name I would probably use Operation rather than Operator since add / mul are really operations in the MLIR sense. I associate operators to matmul, convolution, etc...

1105

nit: Shouldn't this be "::mlir::OpBuilder &"?

mlir/include/mlir/Dialect/Linalg/IR/LinalgTypeInterfaces.h
2

nit: type

38

nit: comments should always end with a '.'

below as well.

mlir/lib/Dialect/Linalg/IR/LinalgDefaultTypeInterfaceImplementations.cpp
22

nit: . missing at the end of the comment.

28

I think 't', 'lhs', and 'rhs' can be passed in by value?

73

nit: . point at the end of the comment missing.

mlir/lib/Dialect/Linalg/IR/LinalgOps.cpp
245

I think - possibly in a follow up revision - we may want to drop the arithfn_add and co and call the interface directly from the yaml generation tool (https://github.com/llvm/llvm-project/blob/bb1fe369774adb86a6bdeec3f56684b6a01c7ff3/mlir/tools/mlir-linalg-ods-gen/mlir-linalg-ods-yaml-gen.cpp#L960).

336

nit: . missing at the end of the comment.

348

nit: . missing at the end of the comment.

353

if possible I would use the strings we use in the yaml files (https://github.com/llvm/llvm-project/blob/bb1fe369774adb86a6bdeec3f56684b6a01c7ff3/mlir/python/mlir/dialects/linalg/opdsl/lang/comprehension.py#L377). These are also the post fixes used for arithfn_add, arithfn_mul etc.

gysit added inline comments.Jan 24 2022, 12:30 PM
mlir/include/mlir/Dialect/Linalg/IR/LinalgInterfaces.td
1068

Another alternative is also ArithFnTypeInterface to name stuff similar to the Python side.

andidr marked 8 inline comments as done.Jan 25 2022, 3:10 AM

Thanks for the feedback!

I added a few detail comments but I would like to ask a high-level question first. Wouldn't it make sense to have a single main interface method that takes the enum value, op builder, and a value range and then do the dispatch/error handling etc internally in the interface? Ideally, when adding a new operation we want to touch as little places as possible. At the moment, we need to adapt the enum, the applyfn_, the interface, and at least one interface implementation. Having an interface that takes an enum instead of having one method per operation time should reduce that by one and also simplifies a follow up refactoring to remove applyfn_

iface.create(builder, ArithFn::Add, ValueRange{lhs, rhs})
iface.create(builder, ArithFn::Exp, ValueRange{x})

I agree that having separate interface methods results in a rather large interface and that directly using the enum is more compact. However, when extending the set of arithmetic operators, handling the enum in a generic interface method results either in a compilation error for all implementations due to the new value not being handled in the switch statement or requires a boilerplate default: branch that is the same across all implementations. The former breaks modularity, as it requires all implementations to be adapted upon a change in Linalg itself, while the latter requires re-implementation of the default behavior in many places.

I also found that using separate interface methods is more true to the spirit of interfaces: what the interface is used for is directly specified in the interface without an additional level of abstraction and default behavior is specified by the default implementation.

All in all, I am not strongly against a generic interface method, but found the current implementation more elegant despite the fact that it is more verbose.

mlir/lib/Dialect/Linalg/IR/LinalgOps.cpp
245

Yes, indeed. I was thinking about doing this right away in this revision, but I came to the conclusion that it's more transparent to have that change in a separate commit.

andidr updated this revision to Diff 402822.Jan 25 2022, 3:11 AM
andidr marked an inline comment as done.

Addressed comments by @gysit

andidr added inline comments.Jan 25 2022, 3:17 AM
mlir/include/mlir/Dialect/Linalg/IR/LinalgInterfaces.td
1068

Agreed that the name is probably too long. I refrained from abbreviating Operator with Op, since Op is usually used to refer to an operation. Also, I'd like to keep the term operator, since the interface is really a mapping from an operator to an operation.

gysit added a comment.Jan 25 2022, 5:00 AM

However, when extending the set of arithmetic operators, handling the enum in a generic interface method results either in a compilation error for all implementations due to the new value not being handled in the switch statement or requires a boilerplate default: branch that is the same across all implementations.

I was thinking of having a default: branch in all implementations that calls a handleNotImplemented method that is part of 'LinalgArithmeticOperatorTypeInterface' (similarly there could be a handleNotSupported. It may be possible to combine the two approaches and have an additional method on the interface that implements the switch case and calls the createAdd, createSub, etc. methods.

Agreed that the name is probably too long. I refrained from abbreviating Operator with Op, since Op is usually used to refer to an operation. Also, I'd like to keep the term operator, since the interface is really a mapping from an operator to an operation.

Right Op is a bit overused. That is actually the reason why we went for function respectively its abbreviation Fn in OpDSL.

Also since this is quite a deep change, let me tag a more experienced reviewer as well @nicolasvasilache

Can you re-upload the patch with full-context please? (arc manages this all automatically).

More immediate question to the patch: it isn't clear to me how much it this fundamentally coupled to the dialect. It seems from the interface description that it is intended to help named op decomposition.
In general interfaces correspond to something that is defined/required by transformations, and implemented by the dialect that owns the operations. The positioning of this interface in the Linalg/IR folder does not match this view directly, this isn't defining the "IR".

mlir/include/mlir/Dialect/Linalg/IR/LinalgOpInterfaces.h
52

Why does it need to be publicly exposed?

andidr updated this revision to Diff 403177.Jan 26 2022, 2:05 AM
andidr marked an inline comment as done.

Updated according to @mehdi_amini's comments.

Thanks @mehdi_amini for the feedback!

Can you re-upload the patch with full-context please? (arc manages this all automatically).

Done.

More immediate question to the patch: it isn't clear to me how much it this fundamentally coupled to the dialect. It seems from the interface description that it is intended to help named op decomposition.
In general interfaces correspond to something that is defined/required by transformations, and implemented by the dialect that owns the operations. The positioning of this interface in the Linalg/IR folder does not match this view directly, this isn't defining the "IR".

In the current implementation, one could argue that the interfaces do
indeed define the IR, since they define, for which types named
operations can be instantiated. They exist for the rather pratical
goal of enabling linalg to choose the correct operation in order to
apply an arithmetic operator on operands of a given type.

From a more abstract perspective, the purpose of the type interface is
to attach the semantics of arithmetic operators to the operations of a
type. It is a fundamental question to me whether such semantics should
be lifted out of the dialect and exist as a concept within MLIR itself
(e.g., in the form of separate type interfaces for the different
arithmetic operators, which can be implemented idividually for a given
type). I assume that this is only justified if there is a need for
such a concept across multiple dialects or within the basic
infrastructure itself. Until now, this doesn't seem to be the case,
but I'd be interested to hear your thoughts on this.

The current implementation with
LinalgArithmeticOperatorTypeInterface is an attempt to establish a
mapping from arithmetic operators to operations without invading the
global space of concepts in MLIR. Your response suggests that this may
not be a proper use of type interfaces. Let me know if you think that
there are better alternatives.

mlir/include/mlir/Dialect/Linalg/IR/LinalgOpInterfaces.h
52

It doesn't. Thanks for spotting this! Updated the revision, such that the definition does not become visible outside of the linalg code itself.

mravishankar resigned from this revision.Feb 8 2022, 8:23 PM