This is an archive of the discontinued LLVM Phabricator instance.

Extension of "Implement IR versioning through post-parsing upgrade through OpAsmDialectInterface"
ClosedPublic

Authored by mfrancio on Feb 9 2023, 6:58 AM.

Details

Summary

[mlir] Implements IR versioning capability

A dialect can opt-in to handle versioning through the BytecodeDialectInterface. Few hooks are exposed to the dialect to allow managing a version encoded into the bytecode file. The version is loaded lazily and allows to retrieve the version information while parsing the input IR, and gives an opportunity to each dialect for which a version is present to perform IR upgrades post-parsing through the upgradeFromVersion method. Custom Attribute and Type encodings can also be upgraded according to the dialect version using readAttribute and readType methods.

There is no restriction on what kind of information a dialect is allowed to encode to model its versioning. Currently, versioning is supported only for bytecode formats.

Diff Detail

Event Timeline

mfrancio created this revision.Feb 9 2023, 6:58 AM
Herald added a project: Restricted Project. · View Herald TranscriptFeb 9 2023, 6:58 AM
mfrancio requested review of this revision.Feb 9 2023, 6:58 AM
myhsu edited reviewers, added: myhsu; removed: mshockwave.Feb 9 2023, 8:52 AM
myhsu added a subscriber: myhsu.

My Phab handle is different from my Discourse account

jpienaar added inline comments.
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1466

I'd prefer upgrade of the in memory structure to not be inside the reader. We already have a way to parse without verification, this upgrade is of the in memory structure which can be done separate. In here I'd prefer only upgrades related to parsing/before it gets to memory. This could be done at the top level entry point though, but outside of the parsing guts feels.

mlir/lib/IR/AsmPrinter.cpp
3091 ↗(On Diff #496112)

When this was discussed we talked about needing to have builtin attr version be treated specially (else one can't parse its version to know how to parse integerattr even).

mlir/test/lib/Dialect/Test/TestDialect.cpp
176

Note: error messages should follow LLVM convention and be a sentence fragment (start lower case, no trailing punctuation)

1675

I may have missed where this is used.

myhsu added a comment.Feb 9 2023, 3:54 PM

I believe some tests are missing like those related to bytecode. Also could you attach diff with more context as instructed here.

mlir/lib/Bytecode/Writer/BytecodeWriter.cpp
504

format: add braces

mfrancio updated this revision to Diff 498417.Feb 17 2023, 9:53 AM
mfrancio added a reviewer: jpienaar.

Revised diff according to the comments received:

  • Adds a new built-in "VersionAttr";
  • Decouples VersionAttr from other built-in Attributes, so that they can grow independently;
  • Leaves complete freedom to each dialect on how to manage versioning and how to encode its version into the VersionAttr;
  • Exposes a couple of hooks to optionally print/parse the dialect version as custom string.
mfrancio marked 4 inline comments as done.Feb 17 2023, 9:59 AM
mfrancio added inline comments.
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1466

I considered this, but I found a little bit confusing the need to carry over the version at which the IR was parsed into the top level entry point - I actually like a lot the fact that the version stays within the parsing, so that only the current version of the dialect exists at the entry point level.

mlir/lib/IR/AsmPrinter.cpp
3091 ↗(On Diff #496112)

I added a new built in attribute called VersionAttr.

mlir/test/lib/Dialect/Test/TestDialect.cpp
1675

Yes, indeed I forgot to upload the corresponding mlir file.

mfrancio edited the summary of this revision. (Show Details)Feb 17 2023, 10:08 AM
mfrancio marked 2 inline comments as done.Feb 17 2023, 10:41 AM

I don't understand why we need a builtin VersionAttr at all?

I don't understand why we need a builtin VersionAttr at all?

Nevermind, this is just lifetime management, doesn't seem unreasonable.

rriddle requested changes to this revision.Feb 17 2023, 11:04 AM

Haven't had time to dig in here, but adding an attribute doesn't feel right for version. Why is this necessary? As opposed to being something specific to the assembly format?

This revision now requires changes to proceed.Feb 17 2023, 11:04 AM

Haven't had time to dig in here, but adding an attribute doesn't feel right for version. Why is this necessary? As opposed to being something specific to the assembly format?

Quoting @jpienaar: "When this was discussed we talked about needing to have builtin attr version be treated specially (else one can't parse its version to know how to parse integerattr even)."

One of the comments I received in the first version was indeed to use a new builtin attr that could be treated specially and allow decoupling with the existing attributes. Also, we wish to encode anything on the version and leave freedom to each dialect to do whatever (essentially writing on/retrieving the buffer). It indeed felt natural to use a new attribute that handles that bag of bytes. I would be happy to revise as necessary if there is a better solution to do this.

Haven't had time to dig in here, but adding an attribute doesn't feel right for version. Why is this necessary? As opposed to being something specific to the assembly format?

I was thinking about how to manage the lifetime, but I think the attribute does not change much actually: a handle still need to be made available whether it is an attribute or not.
So we could design it with a data structure stored on the parser itself and so made available to the dialects during the parsing.
It wouldn't survive the parsing phase though: post-parsing you lose the information about the version producer.

Haven't had time to dig in here, but adding an attribute doesn't feel right for version. Why is this necessary? As opposed to being something specific to the assembly format?

I was thinking about how to manage the lifetime, but I think the attribute does not change much actually: a handle still need to be made available whether it is an attribute or not.
So we could design it with a data structure stored on the parser itself and so made available to the dialects during the parsing.
It wouldn't survive the parsing phase though: post-parsing you lose the information about the version producer.

In the current implementation you don't really need to have a lifetime that exceeds the parsing. However, this may change in the future and having a reserved attribute for doing this may come handy. Are there any other drawbacks I do not see for adding a VersionAttr? If there are, I would be more than happy to revise further with the idea of adding a parser data structure. Any additional feedback would be greatly appreciated.

mfrancio updated this revision to Diff 499217.Feb 21 2023, 9:57 AM
mfrancio edited the summary of this revision. (Show Details)

In the hope of reaching consensus, I am uploading a revised diff that removes the use of attributes entirely for Version and introduces a new dedicated AsmDialectVersionHandle to manage the lifetime of the buffer.

To summarize, the proposed approach:

  • Introduces a new AsmDialectVersionHandle to manage the lifetime of buffer representing dialect attribute info;
  • decouples versioning of a dialect to the rest of the mlir infrastructure, so each can be used and grow independently;
  • Leaves complete freedom to each dialect on how to manage versioning and how to encode its version into the Version Handle;
  • Exposes a couple of hooks to optionally print/parse the dialect version as custom string.

I hope you will find this interesting and compelling for the project.

Haven't had time to dig in here, but adding an attribute doesn't feel right for version. Why is this necessary? As opposed to being something specific to the assembly format?

Have we successfully addressed the concern here and is this ready to land?

I just skimmed through, I'd need some time to review this but I'm travelling right now and not sure if I'll get to it before the week end.

My intuition right now would be to implement it only for the Bytecode for now. The story there is more comprehensive than for the textual format, where we only offer the post-parsing upgrade and no control during parsing (and I'm not convinced we should encourage this).

Something that would be nice also is an example of how to use the version while parsing a type or an attribute to support an upgraded format (for example a new field that was added post version 1.40 or something like that).

mlir/include/mlir/IR/OpImplementation.h
483

Can you increase the amount of doc here: add context about what purpose it serves and some info on the context where it is used.

mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1501

Can you spell the type here?

1506

Should byteCodedialect.dialect be available here?

1510

Isn't dyn_cast working for dialect interfaces?

1767

Is going through the string name for the dialect the best way to resolve this? (I would think we have a dialect ID directly available? And using integer makes everything else more straighforward)

mlir/test/lib/Dialect/Test/TestDialect.cpp
63

We should use the bytecode encoding here for portability purpose at minima.
That is encore the two int as varint (and decode them when loading).

mlir/test/lib/Dialect/Test/TestOps.td
3171

Leftover?

In addition to the stuff mentioned, I'd also love to see top-level docs detailing versioning, how it's structured, and how to hook in.

mlir/test/lib/Dialect/Test/TestOps.td
3170–3171

Dead code?

mfrancio marked 2 inline comments as done.Feb 23 2023, 6:05 PM

In addition to the stuff mentioned, I'd also love to see top-level docs detailing versioning, how it's structured, and how to hook in.

Thanks for your feedback. I will also add some tests related to the byte code encoding itself, similarly to what already done for the other sections -- I've been holding off to doing it while trying to get the bulk of the code reviewed.

mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1501

definitely.

1506

Yes, it should, but it looks like you would have to handle a bunch of cases in the general case.

From BytecodeDialect.h:

/// The loaded dialect entry. This field is std::nullopt if we haven't
/// attempted to load, nullptr if we failed to load, otherwise the loaded
/// dialect.
std::optional<Dialect *> dialect;

I find getting the dialect from the context directly to be generally safer here.

1510

Yes, it does work - are there any issues in using this API though?

I'll change it anyway, since we could dyn_cast_or_null and remove the check for nullptr on the dialect.

1767

It is true that the bytecode holds an integer which references the string section, but I don't see an existing API to reference the string by idx.

I don't really see the "non-straightforward" part anyway - we parse a string with a clean API, and we use it as a hash to map to the version handle. Am I missing something?

mlir/test/lib/Dialect/Test/TestOps.td
3170–3171

yep. thanks for pointing it out!

mfrancio added inline comments.Feb 23 2023, 6:05 PM
mlir/include/mlir/IR/OpImplementation.h
483

Definitely.

mlir/test/lib/Dialect/Test/TestDialect.cpp
63

This is a very good point that I overlooked.

It looks like we have two separate problems here - one is the portability, and the other one is to apply some sort of compression (not really critical in my view, but nice to have).

For the purpose of the example, the first problem could be solved simply by using the helpers exposed by llvm under llvm/Support/Endian.h. For example, we could write/read the integers representing the version using

inline void write16le(void *P, uint16_t V) { write16<little>(P, V); }
inline uint16_t read16le(const void *P) { return read16<little>(P); }

For the second, using varInt is definitely a great idea. It would be great to reuse the same byte code emitters and readers but it looks like they are not really exposed outside the bytecode cpp files. What we could do is to expose the varInt portion of it as helpers under mlir/Support. I am open to doing it, but since this is just an example, is it really worth it?

Looking forward to hear your thoughts.

mfrancio updated this revision to Diff 500076.Feb 23 2023, 9:36 PM
mfrancio marked an inline comment as done.
mfrancio edited the summary of this revision. (Show Details)

Updates with respect to the previous diff:

  • Removes versioning capabilities from the textual format
  • Adds new tests specific to the bytecode format
  • Adds some documentation and addresses some comments

To summarize, the proposed approach:

  • Introduces a new AsmDialectVersionHandle to manage the lifetime of buffer representing dialect attribute info;
  • decouples versioning of a dialect to the rest of the mlir infrastructure, so each can be used and grow independently;
  • Leaves complete freedom to each dialect on how to manage versioning and how to encode its version into the Version Handle.

Looking forward to your feedback.

Going in the right direction, thanks for the update!

mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Efficiency: string manipulation isn't free.

That said it is pretty bounded here, we should have at most one version per dialect...

But stepping back: why aren't we emitting the version in the dialect section?
We could emit an varint for the version blob size, if it is zero that means there is no version attached to the dialect.
That seems like it could fit right before the op names.

mlir/test/lib/Dialect/Test/TestDialect.cpp
63

The bytecode primitive are exposed in the public header mlir/include/mlir/Bytecode/BytecodeImplementation.h.

Have a look at the dialect interface for manipulating types and attribute:

virtual Attribute readAttribute(DialectBytecodeReader &reader) const {

We should model the API here similarly: for a dialect writing a custom version blob should be no different than writing an attribute.

mfrancio added inline comments.Feb 24 2023, 8:15 AM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

The reason why I didn't do it is that it would break existing bytecodes and would require increasing the bytecode version (I am talking about mlir::bytecode::kVersion). I am open to this, but I don't really see the immediate need. It could always be done as part of a major update of the bytecode version itself.

mlir/test/lib/Dialect/Test/TestDialect.cpp
63

Sounds good, I'll take a look.

mehdi_amini added inline comments.Feb 24 2023, 8:30 AM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

In general I'm not in favor of taking detour when we know where we want to land (I don't see a problem with upgrading the bytecode as a breaking change at this point). There are a couple of things I intend to break there as well soon-ish.

mfrancio added inline comments.Feb 24 2023, 8:42 AM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Maybe this was already discussed in the past and I missed it, but isn't the bytecode version itself going to be backward compatible? Is there any interest in achieving this?

saksenadhruv added inline comments.Feb 24 2023, 9:41 AM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Yes, we actually are hoping to ship a serialization format with versioning soon, and would like bytecode to have some compatibility, or atleast a way to upgrade/downgrade when we break it in next couple of months.

What is the guidance on using bytecode for serialization and compatibility?

We are using versioning on our dialect but we need some underlying guarantees on the bytecode itself as well.

mehdi_amini added inline comments.Feb 24 2023, 9:45 AM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Yes we want it to be stable. From my point of view I am aware of 3 features I want to get before I'm comfortable with trying to claim that we reached the "stability" point.

  • Dialect Versioning (thanks you for driving this!)
  • Use-list order.
  • Lazy-loading ability.

(Some people may have other ideas, I'm not aware of any)

Then there is my work on "properties", but I suspect we can preserve backward compatibility on this (assuming the dialects themselves don't change of course).

mfrancio planned changes to this revision.Mar 2 2023, 5:48 PM
mfrancio added inline comments.
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Back to:

We could emit an varint for the version blob size, if it is zero that means there is no version attached to the dialect.
That seems like it could fit right before the op names.

If we are in agreement on the current proposal (the dialect provides a version handle which holds a buffer to be written to file), we can definitely emit this blob of data into the dialect section as a breaking change. Can you kindly confirm before I move forward with the change?

mlir/test/lib/Dialect/Test/TestDialect.cpp
63

I considered this, but I don't really see a way to model the API for reading and writing a version into the dialect section through what is exposed in BytecodeImplementation.h. That dialect interface seem to have very specific objectives that are tied to writing and reading custom attributes and types into their respective sections - what we need is an interface that allows the dialect to write into a custom buffer. We could model the API through this interface, but it would become something pretty close to EncodingEmitter implemented in mlir/lib/Bytecode/Writer/BytecodeWriter.cpp, line 64. Wouldn't it be just more convenient to expose something like this under Support?

It is true that writing a blob of data is no different than writing an attribute, but what changes here is the way this blob of data is created. For the attribute, its encoding is defined. But since we want to be independent from any existing attribute, and also completely defined by the user, I don't really see another convenient way of doing this other than exposing low level API to the user to write whatever encoding they need into their data blob that they wish to use to represent the version.

mehdi_amini added inline comments.Mar 5 2023, 12:30 PM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Yes I think we should just do that now if we need to, that said in https://reviews.llvm.org/D145328 I did the change in a backward compatible way.

mlir/test/lib/Dialect/Test/TestDialect.cpp
63

I considered this, but I don't really see a way to model the API for reading and writing a version into the dialect section through what is exposed in BytecodeImplementation.h. That dialect interface seem to have very specific objectives that are tied to writing and reading custom attributes and types into their respective sections

Right, sorry if I have the impression that this interface was "ready to be used" as-is here, I meant to point it as an example of an API that allows dialect author to access bytecode manipulation primitives.

what we need is an interface that allows the dialect to write into a custom buffer. We could model the API through this interface, but it would become something pretty close to EncodingEmitter implemented in mlir/lib/Bytecode/Writer/BytecodeWriter.cpp, line 64. Wouldn't it be just more convenient to expose something like this under Support?
It is true that writing a blob of data is no different than writing an attribute, but what changes here is the way this blob of data is created. For the attribute, its encoding is defined. But since we want to be independent from any existing attribute, and also completely defined by the user, I don't really see another convenient way of doing this other than exposing low level API to the user to write whatever encoding they need into their data blob that they wish to use to represent the version.

I started typing a long answer here, but felt like I was missing something so I sketched something here instead: https://reviews.llvm.org/D145328

(there is still a bug, and a I haven't regenerated the bytecode test file, but the interface is there!)

jpienaar added inline comments.Mar 5 2023, 9:02 PM
mlir/test/lib/Dialect/Test/TestDialect.cpp
63

I like the sketch.

mfrancio added inline comments.Mar 5 2023, 9:16 PM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Yes! This is exactly what I envisioned when I started implementing the first draft, but I didn't post it as I didn't want to rely on a bytecode version change. Nice to see this.

We could also opt to remove the version section explicitly and inline the read/write of size/bytes reusing the alignment of the parent dialect section (probably a bit more memory efficient), but this works.

mlir/test/lib/Dialect/Test/TestDialect.cpp
63

Thanks for the suggestion. This is very neat, I'll try to finalize it and regenerate the bytecode test files.

mehdi_amini added inline comments.Mar 6 2023, 1:24 AM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

The reason I used a section is that when we load the version section we haven't loaded the dialect yet so we don't have the interface.

mehdi_amini added inline comments.Mar 6 2023, 1:26 AM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Something to add still is an attribute in the test dialect that is serialized at v0.1 and read / upgraded during parsing of v0.2.
I suspect we're missing making the version available on the readAttribute API.

mfrancio updated this revision to Diff 502876.Mar 6 2023, 5:53 PM
mfrancio edited the summary of this revision. (Show Details)

Updates diff incorporating changes from https://reviews.llvm.org/D145328

Includes an example for upgrading an attribute that was written at v1 and it is read at v2 with a different encoding.

mfrancio added inline comments.Mar 6 2023, 6:00 PM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

I did the change! It is tested only for attributes, but I can easily extend it to types as well!

1767

The reason I used a section is that when we load the version section we haven't loaded the dialect yet so we don't have the interface.

I still don't see the reason. I think the section could just be inlined. You don't need the interface to read it (we would just hold the buffer). The interface is needed later to resolve the buffer and decode it... Unless I am missing something subtle :)

mfrancio updated this revision to Diff 502915.Mar 6 2023, 9:33 PM
mehdi_amini accepted this revision.Mar 6 2023, 9:47 PM

LGTM, but please wait for @rriddle to stamp it as well!

mlir/include/mlir/Bytecode/BytecodeImplementation.h
324

Can you add a simple doc?

mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Right, I guess I didn't find the method to do it!

Do you see how to emit the content of the versionEmitter differently than using emitSection? We need to emit the size and then the content. The logic in emitSection() has this logic:

// Push our current buffer and then merge the provided section body into
// ours.
appendResult(std::move(currentResult));
for (std::vector<uint8_t> &result : emitter.prevResultStorage)
  prevResultStorage.push_back(std::move(result));
llvm::append_range(prevResultList, emitter.prevResultList);
prevResultSize += emitter.prevResultSize;
appendResult(std::move(emitter.currentResult));

(knowing that the writeVersion interface can't do it because it needs to compute the size first before emitting the content)

mlir/test/lib/Dialect/Test/TestDialect.cpp
109

Would else be enough here?

Looking very close, thanks!

mlir/docs/BytecodeFormat.md
166–169

Why use a separate section? I would have expected to have this just be part of the op_name_group (which should be renamed at this point to dialect). We can store a bit with numOpNames or dialect to indicate if a version is present, and then optionally read it.

mlir/include/mlir/Bytecode/BytecodeImplementation.h
320–322

Why is it necessary for dialect authors to write the size? I would expect this could be automatically handled (e.g. via back-patching)?

mlir/include/mlir/IR/OpImplementation.h
1466

This change feels unrelated, can you revert?

Nice! I'm fine with delaying textual form.

mlir/docs/BytecodeFormat.md
166–169

I don't think it's documented here, can we have multiple op_name_groups for same dialect? With the naming change it feels like it's saying there will be only one per dialect.

+1 to bit and making this optionally specified if bit set. (I think section may be overloaded, I see this as proposed as just convenient way of grouping these two optional items together).

mlir/include/mlir/Bytecode/BytecodeImplementation.h
275

If we have a versioned dialect, it would seem we'd always have to use this method just in case (and if version is unspecified then there is just no upgrade).

mlir/lib/Bytecode/Reader/BytecodeReader.cpp
414

Why does DialectReader need to be thread through? I thought it was a rather cheap, stateless structure to create.

mfrancio planned changes to this revision.Mar 7 2023, 6:52 AM
mfrancio added inline comments.
mlir/docs/BytecodeFormat.md
166–169

Agreed, I'll plan for this change.

You shouldn't really need a bit - just printing the buffer is enough. A size zero means that no version is available.

From a quick look I don't think you can have multiple op_name_groups per dialect - those are indeed already grouped by dialect:

// Parse the operation names, which are grouped by dialect.
auto parseOpName = [&](BytecodeDialect *dialect) {
  StringRef opName;
  if (failed(stringReader.parseString(sectionReader, opName)))
    return failure();
  opNames.emplace_back(dialect, opName);
  return success();
};
while (!sectionReader.empty())
  if (failed(parseDialectGrouping(sectionReader, dialects, parseOpName)))
    return failure();

so changing the op_name_group to be dialect itself should be fine.

mlir/include/mlir/Bytecode/BytecodeImplementation.h
275

I feel it's anyway up to the dialect to decide what to do here. Having a fall-back seems convenient to me, but if it looks confusing or there is desire to push for the versioned implementation anyway we can emit an error similarly to the other reader.

320–322

This slipped - it is indeed not necessary. I'll update the comment.

mlir/include/mlir/IR/OpImplementation.h
1466

Sure!

mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

Exactly, I was thinking of consolidating this into a new method of reader and avoid the existence of a new dialect version section. I'll try!

mlir/test/lib/Dialect/Test/TestDialect.cpp
109

I think the comment below is misleading - the intent was to forbid reading a newer than current version. I'll revise this.

mehdi_amini added inline comments.Mar 7 2023, 7:21 AM
mlir/docs/BytecodeFormat.md
166–169

We could use a bit on the numOpNames to gate the existence of the version:

op_name_group {
  dialect: varint,
  numOpNamesAndIsVersionAvailable: varint, // (numOpNames << 1 | versionAvailable)
  version : dialect_version_section
  opNames: varint[]
}

That way we don't write a section when there is no version there.

mehdi_amini added inline comments.Mar 7 2023, 7:30 AM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
414

It is cheap, but needs to be created from things unavailable in this class, so you'd need to thread through more of other things here!

I was wondering (post review) if we should split the reader and writer commit parts, so that we could give a bit of time for bytecode consumers to get updated first (thinking of projects that span multiple repos). I mean, it is unstable at the moment, but wouldn't cause any additional churn.

mlir/docs/BytecodeFormat.md
166–169

They are grouped by dialect, but it just emplaces inside a vector. The emplacing results in the "flat" ID, so one can have multiple instances of this where the flat id can be small for the most common operations independent of dialect.

So we'd probably need to just verify that a version is only specified once for a dialect (we could allow multiple as long as the same but that seems undesirable from size poitn of view)

And yes what Mehdi suggested is what River also mentioned.

mlir/include/mlir/Bytecode/BytecodeImplementation.h
275

Yes I'm less worried about dialect than checking expectations for bytecode format parser (where it defers to dialect inside attribute/type parser its under full control of dialect).

mlir/lib/Bytecode/Reader/BytecodeReader.cpp
414

What else do you need to thread through? Dialect version?

mlir/lib/Bytecode/Reader/BytecodeReader.cpp
414

Look at the call sites:

DialectReader dialectReader(*this, stringReader, resourceReader, reader);
if (failed(entry.dialect->load(dialectReader, fileLoc.getContext())))
  return failure();

So stringReader, resourceReader are the extra I think?
(also some call sites already have the dialectReader available)

mfrancio updated this revision to Diff 504197.Mar 10 2023, 9:56 AM
mfrancio marked 30 inline comments as done.

Added a bit flag to detect if a dialect is versioned and trigger the read of the section.

mfrancio marked an inline comment as done.Mar 10 2023, 10:00 AM
mfrancio added inline comments.
mlir/docs/BytecodeFormat.md
166–169

Added the bit flag. It felt more natural doing it on the dialect name itself instead of going inside the dialect version grouping. Let me know if there are any concerns.

mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1767

I considered this again, but the only thing that would eventually "save" is really to print the var int of the section, so it felt not strictly necessary now that we have the bit flag.

rriddle accepted this revision.Mar 10 2023, 10:03 AM
rriddle added inline comments.
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1068

Can you drop the trivial braces here?

1408

I really thought we had a helper that read a varint and extracted a flag.

This revision is now accepted and ready to land.Mar 10 2023, 10:03 AM
mfrancio added inline comments.Mar 10 2023, 10:07 AM
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1408

Oh indeed, I didn't see it. Thanks!

mfrancio updated this revision to Diff 504212.Mar 10 2023, 10:27 AM
mfrancio marked 2 inline comments as done.
mfrancio added inline comments.
mlir/lib/Bytecode/Reader/BytecodeReader.cpp
1068

yep, thanks!

mehdi_amini accepted this revision.Mar 10 2023, 11:47 AM

LGTM, thanks for being so patient through the reviews!

Let's wait for @rriddle to give a final approval.

LGTM, thanks for being so patient through the reviews!

Let's wait for @rriddle to give a final approval.

Actually River reviewed already (I starting reviewing earlier and went out, figured I didn't hit "submit" before).

Do you have commit access or do you need help to land this?

mfrancio marked an inline comment as done.Mar 10 2023, 11:54 AM

Do you have commit access or do you need help to land this?

I don't have commit access, it's the first time I commit here! I was just waiting for the builds to complete before asking for help.

Thank you all for the feedback - it's been nice to collaborate!

Right now you have test failures. On my Mac locally I see:

Failed Tests (2):
  MLIR :: Bytecode/versioning/versioned_attr.mlir
  MLIR :: Bytecode/versioning/versioned_op.mlir
This revision was landed with ongoing or failed builds.Mar 10 2023, 2:29 PM
This revision was automatically updated to reflect the committed changes.