[OpenMP][Flang][Semantics] Add semantics support for USE_DEVICE_PTR clause on OMP TARGET DATA directive.

[raghavendhra]

Summary

Initial support for USE_DEVICE_PTR clause on OMP TARGET DATA directive.

[clementval]

Are you planning to implement this check:

A list item that specifies a given variable may not appear in more than one use_device_ptr clause.

flang/test/Semantics/OpenMP/use_device_ptr.f90

22

New line here.

[raghavendhra]

In D148254#4284179, @clementval wrote:

Are you planning to implement this check:

A list item that specifies a given variable may not appear in more than one use_device_ptr clause.

Yes, in a future patch in which I am thinking to add more checks related to restrictions.

[kiranchandramohan]

Thanks for the patch. I have a few comments inline.

flang/lib/Semantics/resolve-directives.cpp

440

We probably need a debug-dump-symbols test to show that the SymbolFlag is applied.

510

From the standard:

Pointers that appear in a use_device_ptr clause are privatized and the device pointers to the corresponding list items in the device data environment are assigned into the private versions.

Since the pointer in the use_device_ptr is privatised, it is best to follow the representation for privatised variables i.e represent them with a new Symbol. It is probably good to add Symbol::Flag::OmpUseDevicePtr to ompFlagsRequireNewSymbol.

flang/test/Semantics/OpenMP/use_device_ptr.f90

16

If the location pointed to by b inside the region is different from the location pointed outside to by b then it is probably better to create a new symbol for b inside the region.

[raghavendhra]

Addressed review comments and rebased patch

[raghavendhra]

Addressed review comments and rebased

[kiranchandramohan]

At the moment a new symbol is not created because target data does not create a scope at the moment. We have to modify the OmpVisitor::NeedsScope function in resolve-names.cpp file to create a scope.

Please update the document (https://github.com/llvm/llvm-project/blob/main/flang/docs/OpenMP-semantics.md) to include that new scope is created for target data and new Symbol is created for use_device_pointer. Also include the reason.

flang/test/Semantics/OpenMP/use_device_ptr.f90

15

The test should check both the original symbol and the host-associated symbol.

[raghavendhra]

In D148254#4290092, @kiranchandramohan wrote:

Also include the reason.

Can you please point me to the location where I should be adding the reason. Should it be mentioned in flang/docs/OpenMP-semantics.md? If so under which section should I mention it.

[raghavendhra]

Addressed review comments and rebased

[kiranchandramohan]

LGTM. Thanks for the patch and for addressing the review comments.

In D148254#4297176, @raghavendhra wrote:

In D148254#4290092, @kiranchandramohan wrote:

Also include the reason.

Can you please point me to the location where I should be adding the reason. Should it be mentioned in If so under which section should I mention it.

Yes, in flang/docs/OpenMP-semantics.md. Please add as additional info in the sections where you made changes.

[raghavendhra]

Rebased and now ready to land.

flang/docs/OpenMP-semantics.md

</table>
       *      Discussed in “Module File Extensions for OpenMP” section
 
 
 ### New Symbol with new Scope
 
 For the following OpenMP regions:
 
 * `target` regions
+* `target data` regions
 * `teams` regions
 * `parallel` regions
 * `simd` regions
 * task generating regions (created by `task` or `taskloop` constructs)
 * worksharing regions
 (created by `do`, `sections`, `single`, or `workshare` constructs)
 
 A new `Scope` will be created
<table>
    </td>
    <td>Yes
    </td>
    <td>New Symbol
    </td>
    <td>OmpLastPrivate
    </td>
   </tr>
+  <tr>
+   <td>use_device_ptr
+   </td>
+   <td>Yes
+   </td>
+   <td>New Symbol
+   </td>
+   <td>OmpUseDevicePtr
+   </td>
+  </tr>
 </table>
 
 To determine the right data-sharing attribute,
 OpenMP defines that the data-sharing attributes
 of variables that are referenced in a construct can be
 _predetermined_, _explicitly determined_, or _implicitly determined_.
 
 #### Predetermined data-sharing attributes

 ### Data-mapping Attribute
 
 When encountering the _target data_ and _target_ directives,
 the data-mapping attributes of any variable referenced in a target region
 will be determined and represented as `Flag` in the `Symbol` object
 of the variable.
 No `Symbol` or `Scope` will be created.
 
+However, there are some exceptions for this, Pointers that appear in a
+use_device_ptr clause are privatized and the device pointers to the
+corresponding list items in the device data environment are assigned into the
+private versions so it is best to follow the representation for privatised
+variables i.e represent them with a new Symbol and `OmpUseDevicePtr` flag.
+
 The basic steps to determine the data-mapping attribute are:
 
 1. If _map_ clause is present,
 the data-mapping attribute is determined by the _map-type_
 on the clause and its corresponding `Flag` are listed below:
 
 <table>
   <tr>

flang/include/flang/Semantics/symbol.h

ENUM_CLASS(Flag,
       // OpenACC data-mapping attribute
       AccCopyIn, AccCopyOut, AccCreate, AccDelete, AccPresent,
       // OpenACC miscellaneous flags
       AccCommonBlock, AccThreadPrivate, AccReduction, AccNone, AccPreDetermined,
       // OpenMP data-sharing attribute
       OmpShared, OmpPrivate, OmpLinear, OmpFirstPrivate, OmpLastPrivate,
       // OpenMP data-mapping attribute
       OmpMapTo, OmpMapFrom, OmpMapAlloc, OmpMapRelease, OmpMapDelete,
+      OmpUseDevicePtr,
       // OpenMP data-copying attribute
       OmpCopyIn, OmpCopyPrivate,
       // OpenMP miscellaneous flags
       OmpCommonBlock, OmpReduction, OmpAligned, OmpNontemporal, OmpAllocate,
       OmpDeclarativeAllocateDirective, OmpExecutableAllocateDirective,
       OmpDeclareSimd, OmpDeclareTarget, OmpThreadprivate, OmpDeclareReduction,
       OmpFlushed, OmpCriticalLock, OmpIfSpecified, OmpNone, OmpPreDetermined);
   using Flags = common::EnumSet<Flag, Flag_enumSize>;

flang/lib/Parser/openmp-parsers.cpp

TYPE_PARSER(
     "SIZES" >> construct<OmpClause>(construct<OmpClause::Sizes>(
                    parenthesized(nonemptyList(scalarIntExpr)))) ||
     "THREADS" >> construct<OmpClause>(construct<OmpClause::Threads>()) ||
     "THREAD_LIMIT" >> construct<OmpClause>(construct<OmpClause::ThreadLimit>(
                           parenthesized(scalarIntExpr))) ||
     "TO" >> construct<OmpClause>(construct<OmpClause::To>(
                 parenthesized(Parser<OmpObjectList>{}))) ||
     "USE_DEVICE_PTR" >> construct<OmpClause>(construct<OmpClause::UseDevicePtr>(
-                            parenthesized(nonemptyList(name)))) ||
+                            parenthesized(Parser<OmpObjectList>{}))) ||
     "UNIFIED_ADDRESS" >>
         construct<OmpClause>(construct<OmpClause::UnifiedAddress>()) ||
     "UNIFIED_SHARED_MEMORY" >>
         construct<OmpClause>(construct<OmpClause::UnifiedSharedMemory>()) ||
     "UNIFORM" >> construct<OmpClause>(construct<OmpClause::Uniform>(
                      parenthesized(nonemptyList(name)))) ||
     "UNTIED" >> construct<OmpClause>(construct<OmpClause::Untied>()))
 

flang/lib/Semantics/check-omp-structure.cpp

for (auto it{block.begin()}; it != block.end(); ++it) {
     }
   }
 }
 
 const parser::OmpObjectList *OmpStructureChecker::GetOmpObjectList(
     const parser::OmpClause &clause) {
 
   // Clauses with OmpObjectList as its data member
-  using MemberObjectListClauses = std::tuple<parser::OmpClause::Copyprivate,
-      parser::OmpClause::Copyin, parser::OmpClause::Firstprivate,
-      parser::OmpClause::From, parser::OmpClause::Lastprivate,
-      parser::OmpClause::Link, parser::OmpClause::Private,
-      parser::OmpClause::Shared, parser::OmpClause::To>;
+  using MemberObjectListClauses =
+      std::tuple<parser::OmpClause::Copyprivate, parser::OmpClause::Copyin,
+          parser::OmpClause::Firstprivate, parser::OmpClause::From,
+          parser::OmpClause::Lastprivate, parser::OmpClause::Link,
+          parser::OmpClause::Private, parser::OmpClause::Shared,
+          parser::OmpClause::To, parser::OmpClause::UseDevicePtr>;
 
   // Clauses with OmpObjectList in the tuple
   using TupleObjectListClauses = std::tuple<parser::OmpClause::Allocate,
       parser::OmpClause::Map, parser::OmpClause::Reduction>;
 
   // TODO:: Generate the tuples using TableGen.
   // Handle other constructs with OmpObjectList such as OpenMPThreadprivate.
   return common::visit(

flang/lib/Semantics/resolve-directives.cpp

if (const auto *dependSink{
       for (const auto &dependSinkElement : dependSinkVec) {
         const auto &name{std::get<parser::Name>(dependSinkElement.t)};
         ResolveName(&name);
       }
     }
     return false;
   }
 
+  bool Pre(const parser::OmpClause::UseDevicePtr &x) {
+    ResolveOmpObjectList(x.v, Symbol::Flag::OmpUseDevicePtr);
+    return false;
+  }
+
   void Post(const parser::Name &);
 
   // Keep track of labels in the statements that causes jumps to target labels
   void Post(const parser::GotoStmt &gotoStmt) { CheckSourceLabel(gotoStmt.v); }
   void Post(const parser::ComputedGotoStmt &computedGotoStmt) {
     for (auto &label : std::get<std::list<parser::Label>>(computedGotoStmt.t)) {
       CheckSourceLabel(label);
     }
static constexpr Symbol::Flags dataSharingAttributeFlags{
       Symbol::Flag::OmpShared, Symbol::Flag::OmpPrivate,
       Symbol::Flag::OmpFirstPrivate, Symbol::Flag::OmpLastPrivate,
       Symbol::Flag::OmpReduction, Symbol::Flag::OmpLinear};
 
   static constexpr Symbol::Flags privateDataSharingAttributeFlags{
       Symbol::Flag::OmpPrivate, Symbol::Flag::OmpFirstPrivate,
       Symbol::Flag::OmpLastPrivate};
 
   static constexpr Symbol::Flags ompFlagsRequireNewSymbol{
       Symbol::Flag::OmpPrivate, Symbol::Flag::OmpLinear,
       Symbol::Flag::OmpFirstPrivate, Symbol::Flag::OmpLastPrivate,
       Symbol::Flag::OmpReduction, Symbol::Flag::OmpCriticalLock,
-      Symbol::Flag::OmpCopyIn};
+      Symbol::Flag::OmpCopyIn, Symbol::Flag::OmpUseDevicePtr};
 
   static constexpr Symbol::Flags ompFlagsRequireMark{
       Symbol::Flag::OmpThreadprivate};
 
   static constexpr Symbol::Flags dataCopyingAttributeFlags{
       Symbol::Flag::OmpCopyIn, Symbol::Flag::OmpCopyPrivate};
 
   std::vector<const parser::Name *> allocateNames_; // on one directive

flang/lib/Semantics/resolve-names.cpp

void Post(const parser::OmpEndSectionsDirective &) {
     messageHandler().set_currStmtSource(std::nullopt);
   }
 };
 
 bool OmpVisitor::NeedsScope(const parser::OpenMPBlockConstruct &x) {
   const auto &beginBlockDir{std::get<parser::OmpBeginBlockDirective>(x.t)};
   const auto &beginDir{std::get<parser::OmpBlockDirective>(beginBlockDir.t)};
   switch (beginDir.v) {
-  case llvm::omp::Directive::OMPD_target_data:
   case llvm::omp::Directive::OMPD_master:
   case llvm::omp::Directive::OMPD_ordered:
   case llvm::omp::Directive::OMPD_taskgroup:
     return false;
   default:
     return true;
   }
 }

flang/test/Semantics/OpenMP/use_device_ptr.f90

+! RUN: %flang -fc1 -fopenmp -fdebug-dump-symbols %s | FileCheck %s
+! OpenMP Version 5.0
+! 2.10.1 use_device_ptr clause
+! List items that appear in a use_device_ptr clause are converted into device
+! pointers to the corresponding list item in the device data environment.
+
+subroutine omp_target_data
+   use iso_c_binding
+   integer :: a(1024)
+   !CHECK: b size=8 offset=4096: ObjectEntity type: TYPE(c_ptr)
+   type(C_PTR) :: b
+   integer, pointer :: arrayB
+   a = 1
+   !$omp target data map(tofrom: a, arrayB) use_device_ptr(b)
+   !CHECK: b (OmpUseDevicePtr)
+      allocate(arrayB)
+      call c_f_pointer(b, arrayB)
+      a = arrayB
+   !$omp end target data
+end subroutine omp_target_data
+

llvm/include/llvm/Frontend/OpenMP/OMP.td

def OMPC_To : Clause<"to"> {
   let flangClass = "OmpObjectList";
 }
 def OMPC_From : Clause<"from"> {
   let clangClass = "OMPFromClause";
   let flangClass = "OmpObjectList";
 }
 def OMPC_UseDevicePtr : Clause<"use_device_ptr"> {
   let clangClass = "OMPUseDevicePtrClause";
-  let flangClass = "Name";
-  let isValueList = true;
+  let flangClass = "OmpObjectList";
 }
 def OMPC_IsDevicePtr : Clause<"is_device_ptr"> {
   let clangClass = "OMPIsDevicePtrClause";
   let flangClass = "Name";
   let isValueList = true;
 }
 def OMPC_HasDeviceAddr : Clause<"has_device_addr"> {
   let clangClass = "OMPHasDeviceAddrClause";