This is an archive of the discontinued LLVM Phabricator instance.

[flang][fir] Upstream FIR dialect changes.
AbandonedPublic

Authored by schweitz on Jan 25 2021, 3:09 PM.

Details

Summary

FIR dialect changes.

  1. Use more of the newer tablegen support for dialects that's been arriving in the mlir/ tree. This reduces/eliminates some older hand-written code, etc.
  2. Improves documentation on some FIR ops.
  3. Adds new ops to support a generalization of Fortran "parallel" array expression semantics.
  4. Adds new ops to support leveraging of the Affine dialect in MLIR. This will allow Fortran codes to make use of the polyhedral model being built for TensorFlow, etc.
  5. Miscellaneous renaming of ops and types.
  6. Adds an opaque attribute for passing on-the-side data.
  7. Various bug fixes.
  8. Updates tests.

A history of all these sources, including all contributors, previous reviews and revisions, etc., can be found at https://github.com/flang-compiler/f18-llvm-project/

These diffs depend upon D95630.

Diff Detail

Event Timeline

schweitz created this revision.Jan 25 2021, 3:09 PM
schweitz requested review of this revision.Jan 25 2021, 3:09 PM

Thanks! It is exciting to start seeing the FIR change coming upstream!

The patch is not easy to review right now though (also it does not build), it seems like mixing a lot of unrelated things and it makes it not easy to understand what is tested and how. It would be much easier to review the canonicalization pattern change with its own test in isolation. Similarly for the addition of new ops, and the inliner interface for example. There is a non trivial amount of hand-written verifier diagnostics that don't seem tested either?

flang/include/flang/Optimizer/Dialect/FIRAttr.h
137

I'm not sure how it works? How is the context reconstructed on reparsing these?

Also I don't see any test using such attribute right now, can you provide one?

flang/include/flang/Optimizer/Dialect/FIRDialect.h
49

I don't find a definition for these methods right now?

68

Why is this needed? In general this indicates a misconfiguration of the pipeline and shouldn't be used.

(also this method is not used in this patch)

flang/include/flang/Optimizer/Dialect/FIROps.td
1107

It'd be nice to have at least one test that parses all the fields as an example, and have it in the description here in ODS as well.

1110–1111

That's quite some C++ code in ODS. In general for non trivial method please replace with a call to a function implemented in the .cpp file: it plays much nicer with IDEs and debuggers.
(same below for the new ops you're introducing)

1449

Is the "shape" keyword a valid syntax? I'm not sure I understand from the declarative syntax above

1833

Are there tests for all these diagnostics?

flang/lib/Optimizer/Dialect/FIROps.cpp
135

This method is a bit strange: it returns result here but I don't see it ever modified?

If you only ever returns ranges of Value held by an operation, then ValueRange seems appropriate instead of std::vector<Value>

609

It isn't clear why this is needed?

732

I'd rather read a comment that just describes what the pattern is doing on a high level / a snippet of the replacement.

schweitz added inline comments.Jan 25 2021, 5:15 PM
flang/include/flang/Optimizer/Dialect/FIRDialect.h
49
68

The test tools call this to register the dialects they'll use. If there is some preferred way of forcing that to happen, can you provide a pointer? Thanks.

mehdi_amini added inline comments.Jan 25 2021, 5:18 PM
flang/include/flang/Optimizer/Dialect/FIRDialect.h
49

OK, but this patch can't be built / tested at the moment in this state I think.

68

The dialect registered don't need to be loaded in the context, just inserting them should be enough for parsing any input containing these dialects.

If there are passes converting a dialect into another one, the destination dialect can be declared through https://mlir.llvm.org/docs/PassManagement/#dependent-dialects

schweitz added inline comments.Jan 26 2021, 8:39 AM
flang/include/flang/Optimizer/Dialect/FIRDialect.h
49

Per the feedback received on the flang community calls regarding the upstreaming plan, these diffs are meant to follow the proposed plan. The FIR "chunk" being broken down into a small series of linked reviews. In aggregate, the diffs compose a buildable and testable merge.

68

Thanks for the reference. It's not part of these diffs, but it is probably worth mentioning that the flang front-end is not structured as an MLIR pass.

mehdi_amini added inline comments.Jan 26 2021, 1:32 PM
flang/include/flang/Optimizer/Dialect/FIRDialect.h
49

Ah that wasn't clear to me what you were trying to do here.

I'm still not on-board with not landing these incrementally though. Many pieces in this patch can easily be split, tested, and landed independently of a large blob.

68

Ah right of course, but the loading (and so this function) should be located with the frontend components that will emit these dialects, not with the dialect itself.

Is the frontend really directly emitting all of the dialects above?

schweitz added inline comments.Jan 27 2021, 2:11 PM
flang/include/flang/Optimizer/Dialect/FIRAttr.h
137

This attribute allows us to attach a dictionary of on-the-side data structures to a ModuleOp. These data structures do not change the meaning of the FIR/MLIR representation but may be convenient in other ways.

For the sake of roundtripping, the exact nature of the objects being referenced is washed out. They effectively are optional attributes that roundtrip as "None" values.

We can certainly add a test.

flang/include/flang/Optimizer/Dialect/FIRDialect.h
68

This was a convenient place to put this list, which is shared in common between the bridge test tools.

We do use all of the dialects in the list.

flang/include/flang/Optimizer/Dialect/FIROps.td
1107

Agreed. That was the case, but as FIR has been developing and changing, some of these new options and ops should be added.

1110–1111

No problem.

1449

No, it is not. Thanks for pointing this out. Looks like a typo or 3 here.

1833

We are working on a set of negative diagnostic tests and will add them.

flang/lib/Optimizer/Dialect/FIROps.cpp
732

Sure. We can clean this up, make it clearer.

schweitz added inline comments.Jan 27 2021, 2:18 PM
flang/lib/Optimizer/Dialect/FIROps.cpp
135

Good points. We'll see if we can tweak this one.

609

Not sure what "this" refers to, but I assume you mean the check/test. This is older code--it may have been here since before MLIR was merged into the mono repo--that predates some of the improvements made to the autogenerated verification code in the last year or so. It may not longer be needed.

mehdi_amini added inline comments.Jan 27 2021, 2:44 PM
flang/lib/Optimizer/Dialect/FIROps.cpp
609

I meant that the only diff here (and on other lines in this patch) is adding mlir:: namespace prefix, however many other mlir Types aren't prefixed, so this seems unnecessary here unless there is some sort of ambiguity?

SouraVX added inline comments.
flang/lib/Optimizer/Dialect/FIRDialect.cpp
63–68

Is this intended ? some Types containing the prefix fir:: and some not ?

schweitz added inline comments.Jan 28 2021, 10:05 AM
flang/lib/Optimizer/Dialect/FIRDialect.cpp
63–68

Ah, right. The ones prefixed with fir:: have duplicate classes in other namespaces. So this future-proofs this code to make sure the correct classes are used.

schweitz updated this revision to Diff 319913.Jan 28 2021, 10:15 AM

(please ping when you get test coverage)

flang/include/flang/Optimizer/Dialect/FIRDialect.h
68

The list is fine, I'm mostly looking at the loading aspect: you could decouple this by having this function take a registry, and the caller be in charge of pre-loading or not (the frontend should pre-load, the opt testing should should not).
(Basically, keep the existing method as is, loading is a caller problem)

flang/include/flang/Optimizer/Dialect/FIRType.h
451

Can this all be defined in TableGen?

flang/test/Fir/fir-ops.fir
4

You're disabling the test here, what's happening?

flang/tools/tco/tco.cpp
64

I'm surprised you don't need this, the call to parseSourceFile right below should rely on dialects being registered?

schweitz updated this revision to Diff 320033.Jan 28 2021, 7:29 PM
schweitz added inline comments.Jan 28 2021, 7:34 PM
flang/test/Fir/fir-ops.fir
4
mehdi_amini added inline comments.Jan 28 2021, 9:40 PM
flang/test/Fir/fir-ops.fir
4

Please try to organize patches that are scoped and tested individually, otherwise it is impossible to review and understand how things are tested and play together.

As explained previously, the original 4 reviews (now 5) were subparts of the same comprehensive change set. That comprehensive patch is a coherent, logical set of changes to upstream the code generation pass that converts FIR to LLVM IR. It is complete, buildable, testable, comes with a raft of new tests, etc. It is big. Very big. Upstreaming being blocked for months has consequences.

This large change set was broken into multiple reviews in response to the feedback on how to proceed with the upstreaming process. Somewhat smaller chunks were suggested as potentially helpful. Again, this set of changes was therefore broken into 4 (now 5) logical subgroups.

It was then pointed out here in phabricator that the mutual dependence (circular references) between these subsections/reviews meant that each review did not build because of its dependences on the other subparts of the change set. That was true.

To address this issue, it required that all circular references be eliminated and that each subpart of these changes be dependent only on itself and its ancestors (not anything in the descendant changes). That was accomplished by adding a minimal number of "noise" changes to kneecap and/or scaffold out the subparts in a way that they would build without circular dependences. Of course, these process directed changes are cosmetic and confusing, which is why this was not originally presented that way.

If the reviewers are inclined, these sub-reviews can be abandoned and a single very large review, squashing the process noise, can replace them.

In the end, this change set (the 5 linked reviews) is one big pie, whether it is reviewed as a whole pie or by the bite.

As explained previously, the original 4 reviews (now 5) were subparts of the same comprehensive change set. That comprehensive patch is a coherent, logical set of changes to upstream the code generation pass that converts FIR to LLVM IR.

I see it differently here, I see a lot of independent changes that should be reviewed, restructured to be tested appropriately, and upstreamed independently in a sequence.
Each individual pass could be a single patch with its own test for example.

There is no other way to have a review of these changes right now.

schweitz abandoned this revision.Jan 29 2021, 5:41 PM