[OpenMP] Lower taskwait using OpenMP IR Builder

[rogfer01]

Summary

The code generation is exactly the same as it was.

But note that the special handling of untied tasks is still handled by emitUntiedSwitch in clang.

[jdoerfert]

Also for taskwait inside untied tasks we should be reusing the available global_tid instead of calling __kmpc_global_thread_num again.

FWIW, calling getThreadID(getIdent(SrcLocStr)) is the right thing to do here. We do use a cache in that method and at the end of the optimization pipeline, after inlining, etc. we can replace/merge omp_get_thread_id, __kmpc_global_thread_num, and global_tid arguments as a separate (more powerful) step.

llvm/lib/IR/OpenMPIRBuilder.cpp

230

I will modify my patch to allow an insertion point without block (in order to mimic Clang for now). Other than that in the emit vs. create thing, I think this is fine.

Nit: We both could actually cache getIdent(SrcLocStr) so we don't have to go through the map ;)

[rogfer01]

ChangeLog:

• Update patch against changes in D69785

[rogfer01]

ChangeLog:

• Enable the new code generation in codegen tests using taskwait

[jdoerfert]

I think this is fine, one comment though.

llvm/include/llvm/Frontend/OpenMPIRBuilder.h

81 ↗

(On Diff #231114)

If taskwait cannot cause control flow we don't need to return a new insert point, I think.

126 ↗

(On Diff #231114)

Copy & paste, also see my other comment.

[jdoerfert]

Also for taskwait inside untied tasks we should be reusing the available global_tid instead of calling __kmpc_global_thread_num again.

FWIW, We will just clean up in a later pass (under review). We have to do that anyway, adding some smarts here is barely worth it and I also removed similar logic from my patches.

[jdoerfert]

LGTM, assuming the comments are addressed (incl. the commit message)

[rogfer01]

In D69828#1761133, @jdoerfert wrote:

Also for taskwait inside untied tasks we should be reusing the available global_tid instead of calling __kmpc_global_thread_num again.

FWIW, We will just clean up in a later pass (under review). We have to do that anyway, adding some smarts here is barely worth it and I also removed similar logic from my patches.

Sure. I actually forgot to update the summary because you already mentioned this IIRC.

clang/lib/CodeGen/CGOpenMPRuntime.cpp

6356

This is to preserve exactly the same behaviour as before.

[rogfer01]

llvm/include/llvm/Frontend/OpenMPIRBuilder.h

81 ↗

(On Diff #231114)

Ah I see, we want a similar behaviour to that of IRBuilder in which the new IP is updated after something has been emitted.

Makes sense.

[rogfer01]

ChangeLog:

• Remove unnecessary return of the insertion point and update code accordingly.

[kiranchandramohan]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

258 ↗

(On Diff #231207)

Is the emit function required here? We could have this in the create function itself right?

[rogfer01]

ChangeLog:

• Rebase

[rogfer01]

ChangeLog:

• Rebase

[rogfer01]

ChangeLog:

• Rebase

clang/lib/CodeGen/CGOpenMPRuntime.cpp

return Address(
           createRuntimeFunction(OMPRTL__kmpc_task_reduction_get_th_data), Args),
       SharedLVal.getAlignment());
 }
 
 void CGOpenMPRuntime::emitTaskwaitCall(CodeGenFunction &CGF,
                                        SourceLocation Loc) {
   if (!CGF.HaveInsertPoint())
     return;
-  // Build call kmp_int32 __kmpc_omp_taskwait(ident_t *loc, kmp_int32
-  // global_tid);
-  llvm::Value *Args[] = {emitUpdateLocation(CGF, Loc), getThreadID(CGF, Loc)};
-  // Ignore return result until untied tasks are supported.
-  CGF.EmitRuntimeCall(createRuntimeFunction(OMPRTL__kmpc_omp_taskwait), Args);
+
+  CGF.CGM.getOpenMPIRBuilder().emitOMPTaskwait({CGF.Builder.saveIP()});
+
   if (auto *Region = dyn_cast_or_null<CGOpenMPRegionInfo>(CGF.CapturedStmtInfo))
     Region->emitUntiedSwitch(CGF);
 }
 
 void CGOpenMPRuntime::emitInlinedDirective(CodeGenFunction &CGF,
                                            OpenMPDirectiveKind InnerKind,
                                            const RegionCodeGenTy &CodeGen,
                                            bool HasCancel) {
   if (!CGF.HaveInsertPoint())
     return;

llvm/include/llvm/IR/OpenMPIRBuilder.h

#include "llvm/IR/OpenMPKinds.def"
   ///
   /// \param Loc The location where the barrier directive was encountered.
   /// \param DK The kind of directive that caused the barrier.
   /// \param CheckCancelFlag Flag to indicate a cancel barrier return value
   ///                        should be checked and acted upon.
   void emitOMPBarrier(const LocationDescription &Loc, DirektiveKind DK,
                       bool CheckCancelFlag = true);
 
+  /// Generator for '#omp taskwait'
+  ///
+  /// \param Loc The location where the taskwait directive was encountered.
+  ///
+  void emitOMPTaskwait(const LocationDescription& Loc);
+
   ///}
 
 private:
   /// Return the function declaration for the runtime function with \p FnID.
   Function *getRuntimeFunction(RTLFnKind FnID);
 
   /// Return the (LLVM-IR) string describing the source location \p LocStr.
   Constant *getSrcLocStr(std::string LocStr);

llvm/include/llvm/IR/OpenMPKinds.def

 #endif
 
 #define __OMP_RTL(Name, IsVarArg, ReturnType, ...)                             \
   OMP_RTL(OMPRTL_##Name, #Name, IsVarArg, ReturnType, __VA_ARGS__)
 
 __OMP_RTL(__kmpc_barrier, false, Void, IdentPtr, Int32)
 __OMP_RTL(__kmpc_cancel_barrier, false, Int32, IdentPtr, Int32)
 __OMP_RTL(__kmpc_global_thread_num, false, Int32, IdentPtr)
+__OMP_RTL(__kmpc_omp_taskwait, false, Int32, IdentPtr, Int32)
 
 #undef __OMP_RTL
 #undef OMP_RTL
 
 #ifndef OMP_RTL_ATTRS
 #define OMP_RTL_ATTRS(Enum, FnAttrSet, RetAttrSet, ArgAttrSets)
 #endif
 

llvm/lib/IR/OpenMPIRBuilder.cpp

void OpenMPIRBuilder::emitBarrierImpl(const LocationDescription &Loc,
 
   if (UseCancelBarrier && CheckCancelFlag) {
     Value *Cmp = Builder.CreateIsNotNull(Result);
     // TODO Reimplement part of llvm::SplitBlockAndInsertIfThen in a helper and
     // use it here.
     (void)Cmp;
   }
 }
+
+void OpenMPIRBuilder::emitOMPTaskwait(const LocationDescription &Loc) {
+  assert(Loc.IP.getBlock() && "No insertion point provided!");
+  Builder.restoreIP(Loc.IP);
+
+  // Build call kmp_int32 __kmpc_omp_taskwait(ident_t *loc, kmp_int32
+  // global_tid);
+  Constant *SrcLocStr = getSrcLocStr(Loc);
+  Value *Args[] = {getIdent(SrcLocStr), getThreadID(getIdent(SrcLocStr))};
+
+  // Ignore return result until untied tasks are supported.
+  Builder.CreateCall(getRuntimeFunction(OMPRTL___kmpc_omp_taskwait), Args);
+}