Page MenuHomePhabricator

[Flang] Changes to mangling code
ClosedPublic

Authored by kiranchandramohan on Apr 6 2021, 9:06 AM.

Details

Summary

Call static functions using the class name (fir::NameUniquer).
Add function for mangling derivedTypes.

All the name mangling functions that are ultimately called are
tested in unittests/Optimizer/InternalNamesTest.cpp.

Original Author: Eric Schweitz

#upstreaming #NFC mostly

Diff Detail

Event Timeline

kiranchandramohan requested review of this revision.Apr 6 2021, 9:06 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 6 2021, 9:06 AM
kiranchandramohan edited the summary of this revision. (Show Details)Apr 6 2021, 11:15 AM
kiranchandramohan added a reviewer: jeanPerier.
schweitz accepted this revision.Apr 6 2021, 7:50 PM
This revision is now accepted and ready to land.Apr 6 2021, 7:50 PM
jeanPerier accepted this revision.Apr 7 2021, 2:21 AM
This revision was landed with ongoing or failed builds.Apr 9 2021, 9:14 AM
This revision was automatically updated to reflect the committed changes.

For future reference might be worth separating independent patches and giving a more specific/descriptive title. The three lines in the description seem like separate patch descriptions.

Though the description mentions that the functions are all tested - yet it looks like this patch adds a new function/overload without any callers, so how is it tested?

Thanks @dblaikie for highlighting the issues in this patch.

For future reference might be worth separating independent patches and giving a more specific/descriptive title. The three lines in the description seem like separate patch descriptions.

OK I will separate them and improve the description in future.

Though the description mentions that the functions are all tested - yet it looks like this patch adds a new function/overload without any callers, so how is it tested?

Apologies for the misunderstanding here. What i meant is that the function that is ultimately called (fir::NameUniquer::doType) is already tested and this is a wrapper around that function. I will check today if I can add a unit
test for this function (assuming that is the recommendation here).

Thanks @dblaikie for highlighting the issues in this patch.

For future reference might be worth separating independent patches and giving a more specific/descriptive title. The three lines in the description seem like separate patch descriptions.

OK I will separate them and improve the description in future.

Thanks! No worries.

Though the description mentions that the functions are all tested - yet it looks like this patch adds a new function/overload without any callers, so how is it tested?

Apologies for the misunderstanding here. What i meant is that the function that is ultimately called (fir::NameUniquer::doType) is already tested and this is a wrapper around that function. I will check today if I can add a unit
test for this function (assuming that is the recommendation here).

Hmm - well, then there's the question of: What's the purpose of the new function? We don't usually add new code without usage (& then test the new code via that usage). But there are some times when it's suitable to implement new code with nothing but a unit test for coverage - usually only if that code is intended for API usage by clients outside the LLVM project.

What's the purpose/intended usage of this new code?

Though the description mentions that the functions are all tested - yet it looks like this patch adds a new function/overload without any callers, so how is it tested?

Apologies for the misunderstanding here. What i meant is that the function that is ultimately called (fir::NameUniquer::doType) is already tested and this is a wrapper around that function. I will check today if I can add a unit
test for this function (assuming that is the recommendation here).

Hmm - well, then there's the question of: What's the purpose of the new function? We don't usually add new code without usage (& then test the new code via that usage). But there are some times when it's suitable to implement new code with nothing but a unit test for coverage - usually only if that code is intended for API usage by clients outside the LLVM project.

What's the purpose/intended usage of this new code?

Names have to be unique at the FIR (MLIR IR for Flang) layer. So the usage of name mangling is when we lower from the parse-tree of Flang to FIR. The bridge code which does this lowering has not yet landed in llvm-project/flang. The approach for upstreaming this portion is to bring in all the support functions and classes and then introduce the bridge code and show the lowering and testing for each Fortran construct. (Note that at the FIR layer and lowering to LLVM IR the agreed approach is to upstream pass by pass).

Though the description mentions that the functions are all tested - yet it looks like this patch adds a new function/overload without any callers, so how is it tested?

Apologies for the misunderstanding here. What i meant is that the function that is ultimately called (fir::NameUniquer::doType) is already tested and this is a wrapper around that function. I will check today if I can add a unit
test for this function (assuming that is the recommendation here).

Hmm - well, then there's the question of: What's the purpose of the new function? We don't usually add new code without usage (& then test the new code via that usage). But there are some times when it's suitable to implement new code with nothing but a unit test for coverage - usually only if that code is intended for API usage by clients outside the LLVM project.

What's the purpose/intended usage of this new code?

Names have to be unique at the FIR (MLIR IR for Flang) layer. So the usage of name mangling is when we lower from the parse-tree of Flang to FIR. The bridge code which does this lowering has not yet landed in llvm-project/flang. The approach for upstreaming this portion is to bring in all the support functions and classes and then introduce the bridge code and show the lowering and testing for each Fortran construct.

That presents a challenge for testing - usually we'd implement partial support for a feature more end-to-end (ie: make it "work" but without all the functionality) then as functionality is added incrementally, so is corresponding testing.

But adding lots of implementation code that's basically "dead" until the final patches wire it up makes it hard to tell which parts are tested - it'll need to be significantly unit tested & be a bit out of sync with the way the rest of LLVM is authored and tested.

(Note that at the FIR layer and lowering to LLVM IR the agreed approach is to upstream pass by pass).

I'm not especially familiar with MLIR or Flang to properly parse this statement - could you rephrase/add more detail?

Though the description mentions that the functions are all tested - yet it looks like this patch adds a new function/overload without any callers, so how is it tested?

Apologies for the misunderstanding here. What i meant is that the function that is ultimately called (fir::NameUniquer::doType) is already tested and this is a wrapper around that function. I will check today if I can add a unit
test for this function (assuming that is the recommendation here).

Hmm - well, then there's the question of: What's the purpose of the new function? We don't usually add new code without usage (& then test the new code via that usage). But there are some times when it's suitable to implement new code with nothing but a unit test for coverage - usually only if that code is intended for API usage by clients outside the LLVM project.

What's the purpose/intended usage of this new code?

Names have to be unique at the FIR (MLIR IR for Flang) layer. So the usage of name mangling is when we lower from the parse-tree of Flang to FIR. The bridge code which does this lowering has not yet landed in llvm-project/flang. The approach for upstreaming this portion is to bring in all the support functions and classes and then introduce the bridge code and show the lowering and testing for each Fortran construct.

That presents a challenge for testing - usually we'd implement partial support for a feature more end-to-end (ie: make it "work" but without all the functionality) then as functionality is added incrementally, so is corresponding testing.

But adding lots of implementation code that's basically "dead" until the final patches wire it up makes it hard to tell which parts are tested - it'll need to be significantly unit tested & be a bit out of sync with the way the rest of LLVM is authored and tested.

I am not disagreeing with the two points that you suggest. The partial end-to-end support was suggested but did not get the flang community approval. I believe the difficulty included finding small patches which implement end to end functionality that can be upstreamed. For e.g. when i tried to upstream the parse-tree to FIR lowering (https://reviews.llvm.org/D79731) of an empty subroutine in May last year it was a huge patch and there was not much support for it.

(Note that at the FIR layer and lowering to LLVM IR the agreed approach is to upstream pass by pass).

I'm not especially familiar with MLIR or Flang to properly parse this statement - could you rephrase/add more detail?

I was just highlighting the difference in approach of upstreaming the parse tree -> MLIR lowering and the MLIR -> MLIR transformations/MLIR -> LLVM IR lowering.
There is a lot of support code required for the former (parse tree -> MLIR) due to the big difference in representation of the parse-tree and MLIR. And all this support code is written from scratch.
In the case of MLIR transformations or lowering to LLVM IR a lot of support code already exists in llvm-project and hence for these cases we can upstream in the traditional way and it is the agreed approach for upstreaming. This would be like upstreaming the lowering of each MLIR operation or upstreaming each pass which does a transformation on MLIR.

Though the description mentions that the functions are all tested - yet it looks like this patch adds a new function/overload without any callers, so how is it tested?

Apologies for the misunderstanding here. What i meant is that the function that is ultimately called (fir::NameUniquer::doType) is already tested and this is a wrapper around that function. I will check today if I can add a unit
test for this function (assuming that is the recommendation here).

Hmm - well, then there's the question of: What's the purpose of the new function? We don't usually add new code without usage (& then test the new code via that usage). But there are some times when it's suitable to implement new code with nothing but a unit test for coverage - usually only if that code is intended for API usage by clients outside the LLVM project.

What's the purpose/intended usage of this new code?

Names have to be unique at the FIR (MLIR IR for Flang) layer. So the usage of name mangling is when we lower from the parse-tree of Flang to FIR. The bridge code which does this lowering has not yet landed in llvm-project/flang. The approach for upstreaming this portion is to bring in all the support functions and classes and then introduce the bridge code and show the lowering and testing for each Fortran construct.

That presents a challenge for testing - usually we'd implement partial support for a feature more end-to-end (ie: make it "work" but without all the functionality) then as functionality is added incrementally, so is corresponding testing.

But adding lots of implementation code that's basically "dead" until the final patches wire it up makes it hard to tell which parts are tested - it'll need to be significantly unit tested & be a bit out of sync with the way the rest of LLVM is authored and tested.

I am not disagreeing with the two points that you suggest. The partial end-to-end support was suggested but did not get the flang community approval.

Got any links to records of the discussion there? (perhaps this is an awkward case of having flang commits going to the LLVM-commits list, but the flang community conversations aren't happening in llvm-dev or other llvm areas - though good that it catches some community norm differences that we'd like to keep closer to the established LLVM conventions)

It might be worth bringing up this aspect of this kind of incremental development in that conversation, perhaps it changes the weighting of alternatives to some degree.

I believe the difficulty included finding small patches which implement end to end functionality that can be upstreamed. For e.g. when i tried to upstream the parse-tree to FIR lowering (https://reviews.llvm.org/D79731) of an empty subroutine in May last year it was a huge patch and there was not much support for it.

Agreed, that seems like a rather large patch - though I wonder if it is as small as it could be (for instance, at a glance - I can't seem to find a use of the AbstractConverter (it's only derived by one type and doesn't seem to be used via references/pointers to the base type - so maybe FirConverter could exist without the extra abstraction as a first pass/minimizing of this patch and the base class could be added in whatever later patch needs that abstraction)

But, yeah - so long as things are tested as they're added, one way or another, it's not deeply problematic - whatever way flang folks would like, just a minor suggestion that it might be worth considering if a more functionally incremental approach would be possible/beneficial.

(Note that at the FIR layer and lowering to LLVM IR the agreed approach is to upstream pass by pass).

I'm not especially familiar with MLIR or Flang to properly parse this statement - could you rephrase/add more detail?

I was just highlighting the difference in approach of upstreaming the parse tree -> MLIR lowering and the MLIR -> MLIR transformations/MLIR -> LLVM IR lowering.
There is a lot of support code required for the former (parse tree -> MLIR) due to the big difference in representation of the parse-tree and MLIR. And all this support code is written from scratch.
In the case of MLIR transformations or lowering to LLVM IR a lot of support code already exists in llvm-project and hence for these cases we can upstream in the traditional way and it is the agreed approach for upstreaming. This would be like upstreaming the lowering of each MLIR operation or upstreaming each pass which does a transformation on MLIR.

Ah, thanks!

This comment was removed by mehdi_amini.

I am not disagreeing with the two points that you suggest. The partial end-to-end support was suggested but did not get the flang community approval.

Got any links to records of the discussion there? (perhaps this is an awkward case of having flang commits going to the LLVM-commits list, but the flang community conversations aren't happening in llvm-dev or other llvm areas - though good that it catches some community norm differences that we'd like to keep closer to the established LLVM conventions)

It might be worth bringing up this aspect of this kind of incremental development in that conversation, perhaps it changes the weighting of alternatives to some degree.

These discussions mostly happened during the flang technical/biweekly calls that we have.

I believe the difficulty included finding small patches which implement end to end functionality that can be upstreamed. For e.g. when i tried to upstream the parse-tree to FIR lowering (https://reviews.llvm.org/D79731) of an empty subroutine in May last year it was a huge patch and there was not much support for it.

Agreed, that seems like a rather large patch - though I wonder if it is as small as it could be (for instance, at a glance - I can't seem to find a use of the AbstractConverter (it's only derived by one type and doesn't seem to be used via references/pointers to the base type - so maybe FirConverter could exist without the extra abstraction as a first pass/minimizing of this patch and the base class could be added in whatever later patch needs that abstraction)

But, yeah - so long as things are tested as they're added, one way or another, it's not deeply problematic - whatever way flang folks would like, just a minor suggestion that it might be worth considering if a more functionally incremental approach would be possible/beneficial.

As I understand there is going to be some discussions going to happen in the next two weeks on trying to establish a better process for upstreaming the Flang code. May be there is enough support code upstream so that we can actually do the functionally incremental approach now. I will raise this during the discussions.

For this particular patch I will try to add some unittests.