This is an archive of the discontinued LLVM Phabricator instance.

[mlir] Add a new builtin DenseResourceElementsAttr
ClosedPublic

Authored by rriddle on Jul 18 2022, 8:59 AM.

Details

Summary

This attributes is intended cover the current set of use cases that abuse
DenseElementsAttr, e.g. when the data is large. Using resources for large
data is one of the major reasons why they were added; e.g. they can be
deallocated mid-compilation, they support a wide variety of data origins
(e.g, heap allocated, mmap'd, etc.), they can support mutation, etc.

I considered at length not having a builtin variant of this, and instead
having multiple versions of this attribute for dialects that are interested,
but they all boiled down to the exact same attribute definition. Given the
generality of this attribute, it feels more aligned to keep it next to DenseArrayAttr
(given that DenseArrayAttr covers the "small" case, and DenseResourcesElementsAttr
covers the "large" case). The underlying infra used to build this attribute is
general, and having a builtin attribute doesn't preclude users from defining
their own when it makes sense (they can even share a blob manager with the
builtin dialect to avoid data duplication).

Diff Detail

Event Timeline

rriddle created this revision.Jul 18 2022, 8:59 AM
rriddle requested review of this revision.Jul 18 2022, 8:59 AM
rriddle added reviewers: mehdi_amini, jpienaar.

I haven't added tests yet, uploaded early to link from OpaqueElementsAttr removal (https://reviews.llvm.org/D129917).

I'm not convinced by this right now: it seems to me that this has a lot of the short-comings that DenseElementsAttr has, in particular the verification logic has to be externalized in the ops using it. This seems fine on the surface (invariants are enforced, it's not the "JSON of IR") until you start writing C++ using this API: everywhere a DenseElementAttr is passed the verification context is loose and makes the code harder to understand and maintain.
This is why DenseArrayAttr was added with explicit definitions of all the variants: to provide a "closed" API (when you get a DenseIt64ArrayAttr you're manipulating an ArrayRef<int64_t> and it is reflected in the API usage without any cast at the use site).

What I'd like to see here is that we define op so that the C++ class returned in the API is "type safe": it has a C++ API / accessors that matches the constraints of the verifier.

I'm not convinced by this right now: it seems to me that this has a lot of the short-comings that DenseElementsAttr has, in particular the verification logic has to be externalized in the ops using it. This seems fine on the surface (invariants are enforced, it's not the "JSON of IR") until you start writing C++ using this API: everywhere a DenseElementAttr is passed the verification context is loose and makes the code harder to understand and maintain.
This is why DenseArrayAttr was added with explicit definitions of all the variants: to provide a "closed" API (when you get a DenseIt64ArrayAttr you're manipulating an ArrayRef<int64_t> and it is reflected in the API usage without any cast at the use site).

What I'd like to see here is that we define op so that the C++ class returned in the API is "type safe": it has a C++ API / accessors that matches the constraints of the verifier.

The patch isn't done as-is, I uploaded early to give some context to the OpaqueElementsAttr patch. Realistically, the part that isn't done in this patch so far is the user facing API. The intention, from my perspective, is to have this be a sibling in some ways to DenseArrayAttr (with the inevitable goal of killing DenseElementsAttr in favor of a combination of these two). I'm fine with modeling the API in a similar way to DenseArrayAttr if you prefer.

One problem long-term though with killing DenseElementsAttr, is what to do about non-native types that we still want to model arrays for (e.g. i4, etc.)

(I missed the fact that this was sent in connection with the other review)

For i4: I'm not saying that we should provide an ArrayRef API everywhere: iterator and ranges are fine as needed.
The part I'm pointing at the discrepancy between the C++ API and the verifier/ODS invariants.
So if we have a DenseI4ResourceElementsAttr n array of i4, how would we implement it by hand in C++? What would the "perfect" API look like?
It may something like operator[] returns a int32 accessing a given element of the array and masking/shifting it. But a DenseF64ResourceElementsAttr would have the same API returning a double.

(I missed the fact that this was sent in connection with the other review)

For i4: I'm not saying that we should provide an ArrayRef API everywhere: iterator and ranges are fine as needed.
The part I'm pointing at the discrepancy between the C++ API and the verifier/ODS invariants.
So if we have a DenseI4ResourceElementsAttr n array of i4, how would we implement it by hand in C++? What would the "perfect" API look like?
It may something like operator[] returns a int32 accessing a given element of the array and masking/shifting it. But a DenseF64ResourceElementsAttr would have the same API returning a double.

Right, right. I agree there. I think that as part of evolving the ElementsAttr APIs, I think moving to having concrete sub-classes represent distinct use cases makes sense. It's much more maintainable than just assuming something you can iterate i32 because it has an i32 element type. The concentration of all APIs on DenseElementsAttr is a historical thing at this point. I think we could have things like e.g., APIntElementsAttr that explicitly know and provide accessors for APInt. From my perspective, I think a good path forward really would be:

DenseArrayAttr -> Used for "small" data that gets allocated in the context
DenseResourceElementsAttr -> Used for "large", generally external

On top of that though, we have the selective classes for known data types (i.e. how DenseArrayAttr has multiple classes that expose controlled APIs). We can also do this for ElementsAttr in general, i.e. have classes that inherit from ElementsAttr but have guarantees on iteration (e.g. the APIntElementsAttr example above). The main thing I think we should reach for here is having a cohesive API between the attributes.

rriddle updated this revision to Diff 446079.Jul 20 2022, 1:48 AM
rriddle edited the summary of this revision. (Show Details)
rriddle updated this revision to Diff 447534.Jul 25 2022, 6:26 PM

Looks good, primary question about builtin dialect here & if the changes to API addressed Mehdi's concerns (I think it does)

mlir/include/mlir/IR/BuiltinAttributes.td
441

ElementsAttr ?

442

Is there any instance where this isn't actually a DenseElementsAttr ? (ElementsAttr is more general and could be used in more spots, so this is more OOC and as densely here triggers that)

468

inserts where?

477

Is public: needed here?

mlir/lib/AsmParser/AttributeParser.cpp
945 ↗(On Diff #447534)

Is builtin used only as we need some spot to stick it?

mlir/lib/AsmParser/Parser.cpp
345 ↗(On Diff #447534)

dyn_cast_if_present ? Also when would dialect be null here? (the error below would segfault in that case I believe)

mlir/test/IR/invalid-file-metadata.mlir
65 ↗(On Diff #447534)

Use shape dialect? (I could see ml_program and TOSA getting support ~soon, while I don't imagine a shape that large that we'd want it as a resource, arith would be another option)

Looks good overall, but I'd like to see positive tests (not only "invalid") and more coverage for the added APIs.

mlir/include/mlir/IR/BuiltinAttributes.td
447

I think these are copy/pasted and not updated.

rriddle updated this revision to Diff 448420.Jul 28 2022, 12:48 PM
rriddle marked 7 inline comments as done.
rriddle added inline comments.Jul 28 2022, 12:49 PM
mlir/include/mlir/IR/BuiltinAttributes.td
441

This is how we spell it in the rest of the docs, so I wanted to be consistent there.

442

I don't think there's a supported case right now where this isn't dense, otherwise it'd be difficult to reason about this generically (you can't reason about the data generically at all). If we wanted to expand beyond that, we'd need to understand the use case better IMO.

477

Yeah, given that other stuff may be emitted after this section (AFAIK we don't always emit the extraDecls section last).

mlir/lib/AsmParser/AttributeParser.cpp
945 ↗(On Diff #447534)

To some extent, the builtin dialect manages all of the resources referenced by DenseResourceElementsAttr. This is statically guaranteed by the DenseResourceElementsHandle, which is a handle only for the builtin dialect.

mehdi_amini added inline comments.Jul 28 2022, 12:58 PM
mlir/test/IR/dense-resource-elements-attr.mlir
1 ↗(On Diff #448420)

Why this -allow-unregistered-dialect ?

mehdi_amini accepted this revision.Jul 28 2022, 1:20 PM
This revision is now accepted and ready to land.Jul 28 2022, 1:20 PM
jpienaar accepted this revision.Jul 29 2022, 5:38 AM
This revision was automatically updated to reflect the committed changes.