This is an archive of the discontinued LLVM Phabricator instance.

[flang][OpenMP] Initial support the lowering of copyin clause
ClosedPublic

Authored by peixin on Jun 9 2022, 7:56 PM.

Details

Summary

This supports the lowering of copyin clause initially. The pointer,
allocatable, common block, polymorphic varaibles will be supported
later.

This also includes the following changes:

  1. Resolve the COPYIN clause and make the entity as host associated.
  1. Fix collectSymbolSet by adding one option to control collecting the symbol itself or ultimate symbol of it so that it can be used explicitly differentiate the host and associated variables in host-association.
  1. Add one helper function lookupOneLevelUpSymbol to differentiate the usage of host and associated variables explicitly. The previous lowering of firstprivate depends on the order of createHostAssociateVarClone and lookupSymbol of host symbol. With this fix, this dependence is removed.
  1. Reuse copyHostAssociateVar for copying operation of COPYIN clause.

Diff Detail

Event Timeline

peixin created this revision.Jun 9 2022, 7:56 PM
peixin requested review of this revision.Jun 9 2022, 7:56 PM
NimishMishra accepted this revision.Jun 15 2022, 8:25 AM

This LGTM. Explicit differentiation of host and associated variables in host-association seems ok to me.

A few comments.

flang/lib/Lower/Bridge.cpp
522–525

This piece of change is something that I use in the default clause lowering as well. I will reuse it from your patch here. Thanks for this

flang/test/Lower/OpenMP/copyin.f90
3

@kiranchandramohan commented on D125668 to include flang driver tests also. Do we start doing it with all of our patches, or was it a one off thing? Maybe @kiranchandramohan can suggest better.

175

I do not understand the double omp.threadprivate %[[VAL_0]] here. It also seems like VAL_2 is not used here after. Do we expect the separate section to have their own separate x7?

This revision is now accepted and ready to land.Jun 15 2022, 8:25 AM
peixin added inline comments.Jun 15 2022, 9:08 AM
flang/lib/Lower/Bridge.cpp
522–525

Please try it. You can also expand this function for default clause if necessary.

flang/test/Lower/OpenMP/copyin.f90
3

OK. I agree with @kiranchandramohan . For new patches, let us add two RUNs with bbc and flang-new. Maybe he will refactoring all the test cases when finishing the upstreaming.

175

The double omp.threadprivate %[[VAL_0]] is there since two section directives are there. I think this is the problem of not good organization of current constructs and clauses in OpenMP.cpp. It can be addressed by either refactoring OpenMP.cpp or removing the dead code in O3 somewhere.

> Do we expect the separate section to have their own separate x7?

I don't think so. The copyin clause is not allowed in sections construct. For this combined construct, copyin is for parallel construct. So, two x7 share one storage.

peixin updated this revision to Diff 437499.Jun 16 2022, 4:07 AM
peixin edited the summary of this revision. (Show Details)

Fix the double threadprivatization.

peixin added inline comments.Jun 16 2022, 4:10 AM
flang/lib/Lower/OpenMP.cpp
174

@NimishMishra Please notice that using original symbol (aka. associated symbol) may cause multiple privatization. I changed the collectSymbolSet so to make it more generous. Then you can use it for either host or associated symbols explicitly.

NimishMishra added inline comments.Jun 16 2022, 6:15 AM
flang/lib/Lower/OpenMP.cpp
174

That's great! Thank you.

schweitz added inline comments.Jun 17 2022, 10:10 AM
flang/include/flang/Lower/AbstractConverter.h
110

Is there a more descriptive term than "original" for these symbols? It's not entirely clear what is meant exactly.

peixin updated this revision to Diff 438102.Jun 17 2022, 10:53 PM
peixin edited the summary of this revision. (Show Details)

Address the comments from @schweitz .

peixin added inline comments.Jun 17 2022, 10:58 PM
flang/include/flang/Lower/AbstractConverter.h
110

I reverse the logic of the boolean variable and use the ultimate symbol by default so to make it be consistent with collectHostAssociatedVariables. Is it OK to you now?

No change requested: An alternative method would be to let the privatisation/threadprivatisation function return the host symbol and then use that for making the copy. This might need some rearrangement of the code but will not require the separate host lookup function. Have you considered this?

flang/lib/Lower/OpenMP.cpp
176

Nit: Is else required?

peixin updated this revision to Diff 438343.Jun 20 2022, 5:03 AM

Remove the else.

No change requested: An alternative method would be to let the privatisation/threadprivatisation function return the host symbol and then use that for making the copy. This might need some rearrangement of the code but will not require the separate host lookup function. Have you considered this?

Using the function return value or pointer argument storing the host symbol can solve the problem. But it is still kind of not readable. I add the helper function and use the current code for another purpose: to make the symbol mapping explicitly shown to the following developers. When I first read and write the code for private/firstprivate/threadprivate/copyin constructs/clauses, I am really confused to use the lookupSymbol since the symbol mapping is not shown directly. With both lookupHostSymbol and lookupSymbol functions, it is easier for the following developers who wants to reuse the copyHostAssociateVar to know what they are using. And after reading lookupHostSymbol and lookupSymbol, it is easy to know the symbols are organized into different symbol map stack. What do you think?

flang/lib/Lower/OpenMP.cpp
176

Thanks. Fixed.

No change requested: An alternative method would be to let the privatisation/threadprivatisation function return the host symbol and then use that for making the copy. This might need some rearrangement of the code but will not require the separate host lookup function. Have you considered this?

Using the function return value or pointer argument storing the host symbol can solve the problem. But it is still kind of not readable. I add the helper function and use the current code for another purpose: to make the symbol mapping explicitly shown to the following developers. When I first read and write the code for private/firstprivate/threadprivate/copyin constructs/clauses, I am really confused to use the lookupSymbol since the symbol mapping is not shown directly. With both lookupHostSymbol and lookupSymbol functions, it is easier for the following developers who wants to reuse the copyHostAssociateVar to know what they are using. And after reading lookupHostSymbol and lookupSymbol, it is easy to know the symbols are organized into different symbol map stack. What do you think?

When this was initially implemented we had separate bindings for the host symbol and the associated symbols. Subsequently, @schweitz refactored code to simplify the lowering and used the ultimate symbol always. This meant that the SymbolBox/ExtendedValue used are mapped with the same key value (the ultimate symbol) for both the host and the associated symbol.
I have the following issues/questions.
-> I don't know whether skipping a level of the map is desirable (or always correct) for accessing the host symbol's binding.
-> Also skipping a level does not necessarily mean we are getting a host value. Whereas if we use the hostSymbolBox that was used for privatising/threadprivatising, we are guaranteed that we have the host Symbol's binding.
-> Also, this function (lookupHostSymbol) has to be strictly used after creating a binding for the associated symbol.

-> I don't know whether skipping a level of the map is desirable (or always correct) for accessing the host symbol's binding.

@schweitz Is this OK with you?

-> Also skipping a level does not necessarily mean we are getting a host value. Whereas if we use the hostSymbolBox that was used for privatising/threadprivatising, we are guaranteed that we have the host Symbol's binding.

Yes, the prerequisite of getting a host value using lookupHostSymbol is calling it in the associated symbol's stack. Otherwise, it returns null symbol box, which is expected.

The previous copyHostAssociateVar actually do two things, create the host-associated value and copy host to associated value. I move the creation operation out since I want to use it for copyin clause. Then, hostSymbolBox cannot guarantee getting the host symbol's binding since the associated level symbol map stack already have the symbol binding.

-> Also, this function (lookupHostSymbol) has to be strictly used after creating a binding for the associated symbol.

No need. This function has nothing to do with the binding in the associated level. If calling this function in the associated level symbol map stack when there is no binding for the associated symbol, you can get the host value. The difference between lookupSymbol and lookupHostSymbol for this case (has not created the binding for the associated symbol) is that lookupSymbol does not find the symbol box in the associated level symbol map stack and lookupHostSymbol skip the associated level symbol map stack.

When this was initially implemented we had separate bindings for the host symbol and the associated symbols. Subsequently, @schweitz refactored code to simplify the lowering and used the ultimate symbol always. This meant that the SymbolBox/ExtendedValue used are mapped with the same key value (the ultimate symbol) for both the host and the associated symbol.

I think there is some difference in the host association between Fortran and OpenMP. In Fortran, it is taken as one argument of procedure. In OpenMP, we use the host and associated symbols in the same program unit.

-> Also, this function (lookupHostSymbol) has to be strictly used after creating a binding for the associated symbol.

No need. This function has nothing to do with the binding in the associated level. If calling this function in the associated level symbol map stack when there is no binding for the associated symbol, you can get the host value. The difference between lookupSymbol and lookupHostSymbol for this case (has not created the binding for the associated symbol) is that lookupSymbol does not find the symbol box in the associated level symbol map stack and lookupHostSymbol skip the associated level symbol map stack.

What I mean here is that if you skip one level, there is a possibility that you can pop out the map containing the entry for the host version.

Using the function return value or pointer argument storing the host symbol can solve the problem.

Sorry, I did not explain it correctly. Here, it should be one vector of the symbol box of the host symbol instead of the host symbol since the host symbol can be the common block (D127215). It would be like the following:

diff --git a/flang/lib/Lower/Bridge.cpp b/flang/lib/Lower/Bridge.cpp
index 6caea151c20e..c759276a2f06 100644
--- a/flang/lib/Lower/Bridge.cpp
+++ b/flang/lib/Lower/Bridge.cpp
@@ -422,14 +422,16 @@ public:
         llvm::None);
   }
 
-  bool createHostAssociateVarClone(
+  llvm::SmallVector<Fortran::lower::SymbolBox> createHostAssociateVarClone(
       const Fortran::semantics::Symbol &sym) override final {
+    llvm::SmallVector<Fortran::lower::SymbolBox> sbs;
     mlir::Location loc = genLocation(sym.name());
     mlir::Type symType = genType(sym);
     const auto *details = sym.detailsIf<Fortran::semantics::HostAssocDetails>();
     assert(details && "No host-association found");
     const Fortran::semantics::Symbol &hsym = details->symbol();
     Fortran::lower::SymbolBox hsb = lookupSymbol(hsym);
+    sbs.emplace_back(hsb);
 
     auto allocate = [&](llvm::ArrayRef<mlir::Value> shape,
                         llvm::ArrayRef<mlir::Value> typeParams) -> mlir::Value {
@@ -472,7 +474,8 @@ public:
           return fir::substBase(hexv, temp);
         });
 
-    return bindIfNewSymbol(sym, exv);
+    assert(bindIfNewSymbol(sym, exv) && "failed binding");
+    return sbs;
   }
 
   void

In addition, the implementation of copyHostAssociateVar for common block will become more complex. You can check the implementation of private clause for common block. The easy extension will handle it and the structure is clear.

What I mean here is that if you skip one level, there is a possibility that you can pop out the map containing the entry for the host version.

If using this function is in the associated level symbol map stack, there is no such possibility. If using this function is in the host level symbol map stack, it does not skip the host level and returns one null symbol box, for which case, you should use lookupSymbol.

Once creating one symbol-value binding for the host associated symbol, there is no helper function to get the host symbol value. One function supporting it can help us make the code simple and readable.

-> Also, this function (lookupHostSymbol) has to be strictly used after creating a binding for the associated symbol.

No need. This function has nothing to do with the binding in the associated level. If calling this function in the associated level symbol map stack when there is no binding for the associated symbol, you can get the host value. The difference between lookupSymbol and lookupHostSymbol for this case (has not created the binding for the associated symbol) is that lookupSymbol does not find the symbol box in the associated level symbol map stack and lookupHostSymbol skip the associated level symbol map stack.

What I mean here is that if you skip one level, there is a possibility that you can pop out the map containing the entry for the host version.

Sorry, I think I was wrong here. Since we always push before we enter an OpenMP region, i guess this issue i mentioned can never happen.

The general point I am trying to make is that it will be good to ensure that the symboxBox we used to create a clone is the same as the one that is used as the source of the copy for Firstprivate. If we can add an assert for this then that will be great.

The general point I am trying to make is that it will be good to ensure that the symboxBox we used to create a clone is the same as the one that is used as the source of the copy for Firstprivate. If we can add an assert for this then that will be great.

Yes, it is good to have it if possible. Unfortunately, we cannot have this assertion. If the symboxBox we used to create a clone is known, there is no need for the helper function getting the source of the copy for Firstprivate since we can use the SymbolBox directly. In addition, the symboxBox we used to create a clone is obtained from the loopupSymbol, which means that the binding for the associated symbol does not exist. In this case, loopupSymbol and loopupHostSymbol are totally the same, so there is no need for this assertion. Instead, one assertion is added to check the symbolBox obtained from loopupHostSymbol if something goes wrong.

All in all, binding symbol and looking up symbol always refer to ultimate symbol in different symbol map stack (https://github.com/llvm/llvm-project/blob/44e8a205f4cf747b920726428ee9e35c2ac3d706/flang/include/flang/Lower/SymbolMap.h#L338 , https://github.com/llvm/llvm-project/blob/44e8a205f4cf747b920726428ee9e35c2ac3d706/flang/lib/Lower/SymbolMap.cpp#L35). Therefore, if calling loopupSymbol for host-association case can get the symboxBox for the host symbol, calling loopupHostSymbol can get, too.

The general point I am trying to make is that it will be good to ensure that the symboxBox we used to create a clone is the same as the one that is used as the source of the copy for Firstprivate. If we can add an assert for this then that will be great.

Yes, it is good to have it if possible. Unfortunately, we cannot have this assertion. If the symboxBox we used to create a clone is known, there is no need for the helper function getting the source of the copy for Firstprivate since we can use the SymbolBox directly. In addition, the symboxBox we used to create a clone is obtained from the loopupSymbol, which means that the binding for the associated symbol does not exist. In this case, loopupSymbol and loopupHostSymbol are totally the same, so there is no need for this assertion. Instead, one assertion is added to check the symbolBox obtained from loopupHostSymbol if something goes wrong.

All in all, binding symbol and looking up symbol always refer to ultimate symbol in different symbol map stack (https://github.com/llvm/llvm-project/blob/44e8a205f4cf747b920726428ee9e35c2ac3d706/flang/include/flang/Lower/SymbolMap.h#L338 , https://github.com/llvm/llvm-project/blob/44e8a205f4cf747b920726428ee9e35c2ac3d706/flang/lib/Lower/SymbolMap.cpp#L35). Therefore, if calling loopupSymbol for host-association case can get the symboxBox for the host symbol, calling loopupHostSymbol can get, too.

OK. Can we add some kind of assert, like the following?
-> lookUpSymbol != lookUpHostSymbol
-> There exists a binding of the symbol while doing a shallow lookup.

peixin updated this revision to Diff 438894.Jun 21 2022, 7:48 PM

Add the assertions.

peixin added a comment.EditedJun 21 2022, 7:51 PM

-> lookUpSymbol != lookUpHostSymbol

Added this kind of check.

-> There exists a binding of the symbol while doing a shallow lookup.

Good point, shallowLookupSymbol is more strict. Fixed.

LGTM.

Have a few comments about renaming.

flang/include/flang/Lower/SymbolMap.h
307–308

Nit: would nonShallowLookupSymbol or nonLocalLookupSymbol be a better name? You can explain in the comment that this is typically used in OpenMP code for getting host variable bindings.

flang/lib/Lower/OpenMP.cpp
207

Nit: no auto.

209

Nit: use = in Lowering code.

flang/lib/Lower/SymbolMap.cpp
55

Nit: Would lookOneLevelUpSymbol be a better name? You can add a comment saying that this is generally used to get mappings of host symbols.

peixin updated this revision to Diff 439361.Jun 23 2022, 5:49 AM
peixin edited the summary of this revision. (Show Details)

Thanks @kiranchandramohan for the suggestions. loopupOneLevelUpSymbol is better and comments itself. Address the comments.

peixin updated this revision to Diff 439634.Jun 23 2022, 11:23 PM

Rebase before land.