Page MenuHomePhabricator

[mlir][TOSA]Add optional attributes to TOSA custom op

Authored by eric-k256 on Oct 31 2022, 4:05 PM.



The implementation_attrs attr allows passing of backend specific
attributes to TOSA custom ops.

Also adds a config option to avoid namespace collisions on the

Diff Detail

Event Timeline

eric-k256 created this revision.Oct 31 2022, 4:05 PM
eric-k256 requested review of this revision.Oct 31 2022, 4:05 PM
rsuderman accepted this revision.Nov 1 2022, 3:09 PM
This revision is now accepted and ready to land.Nov 1 2022, 3:09 PM
This revision was automatically updated to reflect the committed changes.
silvas added a subscriber: silvas.Nov 3 2022, 4:45 AM
silvas added inline comments.

Can we make this more flexible than just a string? I really don't want to have to stringify a bunch of metadata. A typical example from Torch-MLIR is

def Torch_AtenPadOp : Torch_Op<"aten.pad", [
  ]> {
  let summary = "Generated op for `aten::pad : (Tensor, int[], str, float?) -> (Tensor)`";
  let arguments = (ins

So we would need a list of integers, a string mode, and an optional float. My ideal representation for this would be something like an ArrayAttr of (dense array attr $pad, StringAttr $mode, FloatAttr | "none") -- I'm not sure about the representation for "none".

The alternative would be to JSONify everything :/

eric-k256 added inline comments.Nov 3 2022, 2:43 PM

I thought a bit about options for the attributes when making the change. I can certainly see that string may not be the best way. My other thought was to do a DenseI8ArrayAttr, which makes things slightly easier than strings. It still forces the frontend to encode the attributes in a way easily digested by the backend. That encoding can define the representation of none. From there, going to your idea of an ArrayAttr isn't too big a change. Are there places where an attribute doesn't pack neatly into an existing attribute type? Your example of none comes to mind. Could we use OpaqueAttr here?

Whatever the solution is, the contents of the attributes will still be opaque to TOSA itself. In your example, the backend consuming a tosa.cusom of 'aten.pad' needs to know that attribute 0 is the pad list, 1 is mode, etc. Whether that's done through an ArrayAttr or DenseI8ArrayAttr, that doesn't change.

silvas added inline comments.Nov 4 2022, 3:09 AM

Does using MLIR native attributes affect flatbuffer serialization of the TOSA MLIR? I think that using JSON might actually be better in that case, even if it is more awkward.

eric-k256 added inline comments.Nov 4 2022, 8:38 AM

It creates some issues, but they are solvable. We would need to put an encoding scheme in the serialization to encode any MLIR attribute. Anyone consuming the serialized form would need to be able to undo the encoding.

In the prototype serialization code I wrote, I'm already writing the string out as a byte array, as MLIR strings can have embedded null bytes, but flatbuffers doesn't. It's easier to write/read as bytes rather than spending the effort to stringify.