This is an archive of the discontinued LLVM Phabricator instance.

[mlir] Support for defining Types in tblgen
ClosedPublic

Authored by jdd on Aug 31 2020, 8:28 PM.

Details

Summary

Adds a TypeDef class to OpBase and backing generation code

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
jdd retitled this revision from [mlir] Support for defining Types in tblgen to [mlir] [InProgress] Support for defining Types in tblgen.Aug 31 2020, 8:35 PM
jdd updated this revision to Diff 289319.Sep 1 2020, 4:46 PM

Moved hash_value(float) to hashing.h, fixed clang-tidy warnings

jdd updated this revision to Diff 289321.Sep 1 2020, 4:49 PM

Restoring old revision

silvas added a subscriber: silvas.Sep 2 2020, 1:51 PM
silvas added inline comments.
llvm/lib/Support/Hashing.cpp
30 ↗(On Diff #289321)

Drive-by comment: this actually is not a correct hashing implementation. it can result in x == y yet have hash(x) != hash(y) and other violation of invariants. (consider negative 0 and NaN's)

silvas added inline comments.Sep 2 2020, 1:52 PM
llvm/lib/Support/Hashing.cpp
30 ↗(On Diff #289321)

Thanks! This all looks really really good. I'm extremely busy over the next few days, but I'll try to give it a run over this weekend. Happy to see this coming to fruition.

jdd added inline comments.Sep 11 2020, 2:56 PM
llvm/lib/Support/Hashing.cpp
30 ↗(On Diff #289321)

yeah, I'm not accounting for -0. NaN is "just" a performance problem for degenerate cases AFAICT.

I'll probably remove these. I only need them for a test which probably doesn't make sense so I'll modify the test to not use them.

jdd updated this revision to Diff 292115.Sep 15 2020, 11:10 PM
jdd retitled this revision from [mlir] [InProgress] Support for defining Types in tblgen to [mlir] Support for defining Types in tblgen.Sep 15 2020, 11:11 PM
jdd edited the summary of this revision. (Show Details)
jdd edited the summary of this revision. (Show Details)Sep 15 2020, 11:22 PM

Thanks for driving this!

Started making my way through it.

mlir/include/mlir/IR/OpBase.td
2355

nit: Please add proper punctuation to the end of comments.

2357

nit: I would swap the names here:

Dialect dialect = owningDialect;

2394

Please use the same name for this field as the other ODS classes; extraClassDeclaration.

2400

Can you add comments to these?

2427

nit: There are a few format issues in this code block.

mlir/include/mlir/TableGen/CodeGenHelpers.h
26 ↗(On Diff #292115)

nit: Only specify inline on methods if you have a good reason to.

41 ↗(On Diff #292115)

nit: Same comments on inline here and throughout.

41 ↗(On Diff #292115)

Seems like this could take the fully qualified cpp namespace string instead of a dialect.

mlir/include/mlir/TableGen/TypeDef.h
1

This comment block looks incorrect, can you look at other files for examples?

22

Are these necessary given that you include Record.h?

61

nit: Can you add a better comment here?

101

This comment and a few others look like they need to be updated.

111

This class needs some documentation.

129

nit: Use llvm::function_ref instead of std::function if you don't need to store the functor.

mlir/include/mlir/TableGen/TypeDefGenHelpers.h
2

Do we need this file for this commit? This looks like it is only used for the testing and I'd rather leave parsing/printer utilities separately.

mlir/lib/TableGen/TypeDef.cpp
23

dyn_cast may return null, use cast if you know that it is of the type you expect.

33

nit: Can you spell out auto here, the type isn't obvious.

65

nit: Add braces around the if here, the body isn't trivial.

66

nit: Cache the end iterator of for loops unless you rely on the value changing during the loop.

71

return membersDag ? membersDag->getNumArgs() : 0;?

85

The error message here seems off.

129

nit: Don't use else after return.

141

PrintFatalError? Same below.

jdd added a comment.Sep 16 2020, 12:22 PM

It looks like the test build didn't happen... Am I right? Did I do something wrong?

jdd added inline comments.Sep 16 2020, 12:42 PM
mlir/include/mlir/TableGen/CodeGenHelpers.h
41 ↗(On Diff #292115)

It could, but everything which uses it gets the namespace from the dialect so this enables maximum code sharing.

mlir/include/mlir/TableGen/TypeDefGenHelpers.h
2

It's not just for testing. The auto-generated parsers/printers use the templates in this file so whenever one is using auto-generated printers/parsers this file has to be #included. The alternative is to emit this file in the generated code, but I prefer this approach so as to not bloat the emit'ed code.

I get that this template-based design decision is likely to be controversial. This is one piece where I'm looking for high-level feedback.

jdd updated this revision to Diff 292678.Sep 17 2020, 5:43 PM

Changed 'memebers' to 'parameters' and addressed existing review comments

jdd marked 19 inline comments as done.Sep 17 2020, 5:46 PM

Addressed

jdd updated this revision to Diff 292932.Sep 18 2020, 6:41 PM

Adding ::get method to classes so we can keep storage types in the cpp file.

Also, modifying the custom allocator for ArrayRef parameters so as to not require types with default constructors.

jdd updated this revision to Diff 294166.Sep 24 2020, 2:26 PM

Doesn't cleanly apply to current master. Fixing.

rriddle added inline comments.Sep 28 2020, 12:29 PM
mlir/include/mlir/TableGen/TypeDefGenHelpers.h
2

Yeah, I'd prefer removing all of the parsing stuff from this revision. It is just going to hold back all of the other stuff, and is something that can be discussed separately.

Apologies for the delay, I've been OOO for the past few weeks. Will take another look in the next day or two.

jdd added inline comments.Sep 28 2020, 12:56 PM
mlir/include/mlir/TableGen/TypeDefGenHelpers.h
2

If the parsing stuff what's blocking this, then I agree. I'd like to keep the custom parser code in tblgen and the parsing dispatch. I think this is value and and is not controversial, right?

jdd updated this revision to Diff 294829.Sep 28 2020, 4:41 PM

Syncing w/ master & removing auto-parsing stuff

jdd updated this revision to Diff 294830.Sep 28 2020, 4:44 PM

I will never understand arc

jdd updated this revision to Diff 294833.Sep 28 2020, 4:47 PM

Let's try this again

jdd updated this revision to Diff 294848.Sep 28 2020, 6:10 PM

Fixing whitespace and clang-tidy issues

jdd updated this revision to Diff 294849.Sep 28 2020, 6:11 PM

Fixing the previous arc diff

jdd updated this revision to Diff 295073.Sep 29 2020, 11:54 AM

One more clang-tidy fix

jdd updated this revision to Diff 295632.Oct 1 2020, 11:16 AM

Changes based on Chris' comments

(https://github.com/circt/llvm/pull/3)

rriddle requested changes to this revision.Oct 1 2020, 8:03 PM

Sorry for the delay. Took another run through. Thanks.

mlir/include/mlir/IR/OpBase.td
2364

nit: Please add proper punctuation to the end of comments.

There are many cases within the revision still.

2374

nit: Phrase the comments as statements instead of questions.

2440

nit: Drop trivial braces, and use pre-increment.

2444

The formatting is still a bit off.

mlir/lib/TableGen/TypeDef.cpp
69

nit: Drop trivial braces here and throughout the revision.

130

Drop the newline here.

163

Don't use else after return.

164

Should this be a fatal error instead?

mlir/test/lib/Dialect/Test/TestTypes.cpp
20

Namespaces in source files should only really be used for class declarations. Everything else should be in the top-level scope.

36

nit: Return the error directly.

63

nit: Merge all of these together using ||

93

nit: Use llvm::interleaveComma here instead.

mlir/test/lib/Dialect/Test/TestTypes.h
27 ↗(On Diff #295632)

Please add comments for this class and it's usages.

28 ↗(On Diff #295632)

This public seems unnecessary.

rriddle added inline comments.Oct 1 2020, 8:03 PM
mlir/include/mlir/TableGen/TypeDef.h
132

Seems like TypeDef could just provide a range of it's parameters and callers of this could just you llvm::map_range instead.

mlir/test/lib/Dialect/Test/TestTypes.cpp
28

nit: Drop all of the mlir:: in this file, and throughout the revision.

101

The definitions of methods in header files should be fully qualified :: and not placed in a namespace in the .cpp.

mlir/tools/mlir-tblgen/OpDocGen.cpp
202 ↗(On Diff #295632)

nit: Prefer early return instead.

205 ↗(On Diff #295632)

nit: Unless the end loop iterator changes, cache it.

mlir/tools/mlir-tblgen/TypeDefGen.cpp
83

The name of this parameter doesn't seem to match what you are doing here.

85

Could you just format the parameters into a llvm::raw_string_ostream instead of constructing separate strings?

98

Same comment as above.

134

This looks like the indent is off.

186

Do we need to generate this method? Seems like the uses of it by the generator could just inline the string instead.

197

nit: Spell out auto here, also use a reference instead of by-value.

222

Seems like this would be something only generated in the source file and marked as static.

274

This looks over indented, should be 2 spaces not 4.

293

This looks misaligned.

311

nit: Spell out auto here and don't use a by-value iterator variable.

348

Should these be auto &instead?

373

Please remove all of the personal comments from the revision.

381

nit: Comments should be complete sentences. This applies throughout the revision.

420

nit: Same comment about auto here as above.

481

Stray comment?

505

Can we use a LogicalResult return instead?

513

Can we avoid generating print/parse on the types themselves? We should try to keep as much internal details hidden away as possible.

This revision now requires changes to proceed.Oct 1 2020, 8:03 PM
jdd added a comment.EditedOct 2 2020, 12:34 PM

As for declaring parse(...), print(...), and getMnemonic() in the header file: these are intended for cases where (for some reason) the dialect doesn't use the global parse/print dispatch method. The dialect could call each parse/print method in its Dialect::parseType/Dialect::printType methods, in concert with the getMnemonic(). I figure even if it's not often used, its just an extra 3 lines in each type declaration.

Do you agree or should I remove it?

mlir/include/mlir/IR/OpBase.td
2364

Ah. I thought proper punctuation was only necessary after full sentences.

jdd marked an inline comment as done.Oct 2 2020, 7:19 PM
jdd added inline comments.
mlir/test/lib/Dialect/Test/TestTypes.cpp
63

Nice!

93

That's handy

101

I've removed two from the header

mlir/tools/mlir-tblgen/TypeDefGen.cpp
222

The dialect calls these from its parseType and printType functions so we must declare them here.

jdd marked 32 inline comments as done.Oct 2 2020, 8:40 PM
jdd edited the summary of this revision. (Show Details)
jdd updated this revision to Diff 295953.Oct 2 2020, 8:43 PM
jdd edited the summary of this revision. (Show Details)

Revisions base on River's comments

jdd marked 2 inline comments as done.Oct 2 2020, 8:45 PM
jdd updated this revision to Diff 295955.Oct 2 2020, 8:46 PM

Missed a comment

jdd updated this revision to Diff 296316.Oct 5 2020, 3:30 PM

clang-format

Really great stuff, I'm glad you've been pushing this! Mostly nits left for the code, it's enough that we can just iterate further in tree. Can you document all of this cool new stuff? Either in docs/OpDefinitions or a new docs/TypeDefinitions?

mlir/include/mlir/TableGen/TypeDef.h
71

nit: Can we remove the llvm:: on these as well?

93

Is the llvm:: here necessary?

mlir/lib/TableGen/TypeDef.cpp
152

Please drop else after a return.

154

nit: Drop the != nullptr.

mlir/test/lib/Dialect/Test/TestTypeDefs.td
118

Please cache the end iterator of loops if the size doesn't change.

119

The formatting here seems off.

mlir/test/lib/Dialect/Test/TestTypes.cpp
36

nit: Drop trivial braces here. Also if you use braces on one if/else, please use it for all of them.

87

Please only use namespaces in source files for classes.

105

nit: Drop the newline here.

mlir/tools/mlir-tblgen/OpDocGen.cpp
273 ↗(On Diff #296316)

nit: Drop trivial braces here.

mlir/tools/mlir-tblgen/TypeDefGen.cpp
57

Please use braces on every else if they are used on one.

63

nit: auto -> const TypeDef &

82

llvm::interleaveComma?

99

Same here?

309

Can you stream the tgfmt directly into os?

433

Can you stream it into os directly?

jdd marked 16 inline comments as done.Oct 9 2020, 10:21 PM

I'll update the documentation as you requested. Do you mind if I do it in a subsequent commit? I have a feeling there's going to be some back-and-forth on the documentation and I don't want that editorial process to gate this commit.

Meanwhile, I've addressed all your comments. No matter how carefully I examine my code, I just keep missing your nits. I gotta learn to write code in the LLVM style. It's pretty much the complete opposite of how I'm used to writing code!

mlir/include/mlir/TableGen/TypeDef.h
71

Actually, no. It doesn't find Optional if I remove it. I'd have to add a 'using namespace' in the header, which is a no-no.

Why StringRef and ArrayRef work and nothing else in the llvm namespace is a mystery to me, though my understanding of C++ namespace scoping rules is doubtlessly lacking.

mlir/test/lib/Dialect/Test/TestTypes.cpp
87

Doesn't work in this case. Since FieldInfo is in the mlir namespace (with the rest of the Test dialect), these two functions have to be also. They do not have to be exposed outside of this file, so they're not declared in the header.

mlir/tools/mlir-tblgen/TypeDefGen.cpp
82

Ohhh. I'll do ya one better. Lemme know if the is too far deep into C++ trickery territory.

jdd updated this revision to Diff 297386.Oct 9 2020, 10:22 PM
jdd marked an inline comment as done.
  • Changes based on review
  • Remove llvm:: from the cpp file
jdd marked an inline comment as done.Oct 9 2020, 10:31 PM
jdd updated this revision to Diff 297438.Oct 10 2020, 7:16 PM
  • Clang-tidy fix
rriddle accepted this revision.Oct 13 2020, 11:45 AM
In D86904#2323208, @jdd wrote:

I'll update the documentation as you requested. Do you mind if I do it in a subsequent commit? I have a feeling there's going to be some back-and-forth on the documentation and I don't want that editorial process to gate this commit.

That's fine with me.

LGTM after fixing the remaining comments.

mlir/include/mlir/TableGen/TypeDef.h
71

Optional, like ArrayRef/StringRef/and many other things, is already re-exported into the MLIR namespace.

https://github.com/llvm/llvm-project/blob/3f2386de6325157324a0dc2ae00ef3db3a144563/mlir/include/mlir/Support/LLVM.h#L110

mlir/test/lib/Dialect/Test/TestTypes.cpp
87

You need to fully qualify the methods: https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions

You should very very rarely (most cases are bugs in the compiler, e.g. some templates in GCC 5) need to actually use a namespace in the .cpp to define a previously declared method.

mlir/tools/mlir-tblgen/TypeDefGen.cpp
27

nit: Can you please avoid using namespace llvm here?

50

nit: Add & unless it is trivially copyable, using the type instead of auto would it make it a bit clearer what is being iterated here.

73

I would prefer that these just be function_ref and boolean fields of the class instead.

74

This class needs to be wrapped in an anonymous namespace.

75

nit: Prefer having the members at the end of the class definition, not first.

97

Please do not use typedef, prefer using Foo = blah; directives instead.

This revision is now accepted and ready to land.Oct 13 2020, 11:45 AM
jdd marked 8 inline comments as done.Oct 13 2020, 2:10 PM
jdd added inline comments.
mlir/tools/mlir-tblgen/TypeDefGen.cpp
73

That kinda defeats the syntactic niceness. Will convert this to normal class w/ enum to determine the format. This will make the calling syntax longer but preserve the efficiency.

jdd updated this revision to Diff 297957.Oct 13 2020, 2:15 PM
jdd marked an inline comment as done.
  • Another round of changes based on feedback.
jdd updated this revision to Diff 297968.Oct 13 2020, 3:00 PM
  • Rebasing w/ conflicts
  • New branch, applying everything at once
  • Clang-(format|tidy) fixes
  • One more clang-tidy fix
  • Changes based on Chris' comments
  • Ending a sentence comment w/ a period
  • Revisions base on River's comments
  • Clang-tidy and some missed comments
  • Missed a comment
  • clang-format
  • Changes based on review
  • Remove llvm:: from the cpp file
  • Clang-tidy fix
  • Another round of changes based on feedback.
This revision was automatically updated to reflect the committed changes.
In D86904#2323208, @jdd wrote:

I'll update the documentation as you requested. Do you mind if I do it in a subsequent commit? I have a feeling there's going to be some back-and-forth on the documentation and I don't want that editorial process to gate this commit.

When you get to landing the doc, please update the newsletter on Discourse :)
This is a really cool feature.