[OpenMP] Clang Support for taskwait nowait clause

[koops]

Summary

Support for taskwait nowait clause with placeholder for runtime changes.

[tianshilei1992]

clang/lib/CodeGen/CGOpenMPRuntime.cpp

6133

better to use consistent name as declaration. how about HasNowaitClause?

llvm/include/llvm/Frontend/OpenMP/OMPIRBuilder.h

638

I'd keep the style consistent. Here there is a default argument bool HaveNoTaskWaitClause = false but no for emitTaskwaitCall. It's better to use default argument (potentially save a couple of changes) for both, or no default for both.

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

1253

Is the OpenMPVer necessary as an argument? Do we have alternative way to tell the version? If not, I think it makes more sense to take it as a data member of OpenMPIRBuilder instead of passing as a function argument.

[ABataev]

clang/lib/CodeGen/CGOpenMPRuntime.cpp

6133

I would recommend to make it a member of OMPTaskDataTy

[tianshilei1992]

clang/lib/CodeGen/CGOpenMPRuntime.cpp

6133

Yeah, that sounds better.

[koops]

In the latest revision that was submitted the following changes were made:

1. Changed HaveNoTaskWaitClause to HasNowaitClause.
2. HasNowaitClause is now a data member of OMPTaskDataTy.
3. OpenMPVer is now a data member of OpenMPIRBuilder.

[ABataev]

clang/lib/CodeGen/CGStmtOpenMP.cpp

5215

Data.HasNowaitClause = S.hasClausesOfKind<OMPNowaitClause>();

[koops]

Contains changes suggested for:

Comment at: clang/lib/CodeGen/CGStmtOpenMP.cpp:5215
+ if (S.hasClausesOfKind<OMPNowaitClause>()) Data.HasNowaitClause = true;
Replaced with:
Data.HasNowaitClause = S.hasClausesOfKind<OMPNowaitClause>();

[koops]

I seem to have made changes for all the feedback which were provided. Can someone please look at the changes I have uploaded?

[ABataev]

clang/lib/CodeGen/CGStmtOpenMP.cpp

5215–5216

No need for if here, just

Data.HasNowaitClause = S.hasClausesOfKind<OMPNowaitClause>();

llvm/include/llvm/Frontend/OpenMP/OMPIRBuilder.h

1418

int OpenMPVersion = 0?

1421

int getOpenMPVersion() const {return OpenMPVersion; }
Plus, do you really need an OpenMP version check here? Just call the new function always

[koops]

The current patch has the following changes:

1. "if" statement removed for Data.HasNowaitClause = S.hasClausesOfKind<OMPNowaitClause>();
2. OpenMPVer replaced with OpenMPVersion.

[koops]

Reply

llvm/include/llvm/Frontend/OpenMP/OMPIRBuilder.h

1421

The new function __kmpc_omp_taskwait_deps_51, in openmp/runtime/src/kmp_taskdeps.cpp, is a placeholder for taking into account "nowait" clause from Spec 5.1. The scheduling of tasks might change during runtime due to "nowait" clause. Hence a check for OpenMP version is needed.

[koops]

Changes for the build failure due to git-clang-format. Some minor changes are also because of git pull.

[jdoerfert]

clang/lib/CodeGen/CGOpenMPRuntime.cpp

6180

Can we avoid the duplication, always make the array 7 long, and use the ArrayRef substring capability to pass in either 6 or 7 elements?

Conceptually something like this:

Fill all
emitRuntimeCall(..., ArrayRef(DepWaitTaskArgs, OpenMP >= 51 ? 7: 6)

On second thought, why do we need to ever emit the old runtime call at all? Always emit the new one.

6199

Same as above.

llvm/include/llvm/Frontend/OpenMP/OMPIRBuilder.h

80–82

Not sure a default is wise here. Users should pick. And we might want to have a "latest" constant somewhere, just set to 99.

638

The clause is not NoTaskwait, it's nowait, right? Why not HasNowaitClause?

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

1272

Same as above, new version only is sufficient.

2307

Unrelated and not even sure which one we should use nowadays. Please split this off.

openmp/runtime/src/kmp.h

3984

No default arguments here please.

openmp/runtime/src/kmp_taskdeps.cpp

890

Implement the old version by calling the new version with no_wait false. I assume you copied the original code, this should have indicated a problem (wrt. code duplication).

[koops]

Johannes Doerfert,
Most of your comments are related to avoiding having the kmpc_omp_taskwait_51( ) , in the runtime, and instead introduce a "nowait" related parameter in the kmpc_omp_taskwait( ) itself. I have explained earlier that kmpc_omp_taskwait_51( ) is a placeholder for introducing new code, at a later stage. The scheduling of tasks might change in the runtime due to "nowait" clause. I have done this with the sole purpose of having backward compatibility. If you feel that is not needed then I will remove the new code and modify the Code Generation also according to your comments.

Thank you for your detailed comments.

[jdoerfert]

In D131830#3874169, @koops wrote:

Johannes Doerfert,
Most of your comments are related to avoiding having the kmpc_omp_taskwait_51( ) , in the runtime, and instead introduce a "nowait" related parameter in the kmpc_omp_taskwait( ) itself. I have explained earlier that kmpc_omp_taskwait_51( ) is a placeholder for introducing new code, at a later stage. The scheduling of tasks might change in the runtime due to "nowait" clause. I have done this with the sole purpose of having backward compatibility. If you feel that is not needed then I will remove the new code and modify the Code Generation also according to your comments.

My comments are not about avoiding kmpc_omp_taskwait_51 in the runtime. On the contrary. My comments say: Always use kmpc_omp_taskwait_51 and eliminate the old kmpc_omp_taskwait. Do not generate calls to kmpc_omp_taskwait but to kmpc_omp_taskwait_51 instead. Implement kmpc_omp_taskwait with a call to kmpc_omp_taskwait_51. Doing both will remove most of the new code and we still get the same benefits.
I hope this is clearer now.

[koops]

Changes for the suggestion : "Always use kmpc_omp_taskwait_51 and eliminate the old kmpc_omp_taskwait".

[koops]

Can someone please review my latest changes?

[ABataev]

clang/lib/CodeGen/CGOpenMPRuntime.cpp

5087

Use CGF.Int32Ty

6149

Use CGF.Int32Ty

[koops]

Changes for "Use CGF.Int32Ty" instead of CGF.IntTy.

[ABataev]

LG

[koops]

No Changes. Pulling in recent changes to trigger a rebuild.

[koops]

No changes. Pulling in recent changes to trigger a rebuild.

[ABataev]

Do you need to commit it?

[koops]

Yes, I don't have commit rights. I have tried to reproduce the pre-merge check failures on RedHat and Ubuntu but, the tests don't fail.

[ABataev]

Ok, I'll commit it.

[mstorsjo]

openmp/runtime/src/dllexports

348

This is missing a similar export of __kmpc_omp_taskwait_51, which causes a bunch of tests to fail if running them on Windows now. (Also, where do the numbers for these exports get assigned?)

[koops]

Addition of __kmpc_omp_taskwait_51 in dllexports.

[koops]

Martin Storsjö (or anybody who works on windows) I have uploaded a new patch which has __kmpc_omp_taskwait_51 in dllexports. Can you please check the patch on windows?

[koops]

git pull & reloading the earlier patch.

[mstorsjo]

In D131830#3985449, @koops wrote:

Martin Storsjö (or anybody who works on windows) I have uploaded a new patch which has __kmpc_omp_taskwait_51 in dllexports. Can you please check the patch on windows?

Thanks, now this seems to work fine for me! (There seems to be another unrelated regression at the same time though, regarding the symbol __kmpc_fork_call_if.)

[koops]

(There seems to be another unrelated regression at the same time though, regarding the symbol __kmpc_fork_call_if.)

I do not know about this regression. I have not touched this function.

[mstorsjo]

In D131830#3986436, @koops wrote:

(There seems to be another unrelated regression at the same time though, regarding the symbol __kmpc_fork_call_if.)

I do not know about this regression. I have not touched this function.

Yeah, that one should be separate, I guess it stems from D138495.

This updated patch should be ok with me to recommit.

[protze.joachim]

Why do we need __kmpc_omp_taskwait_51 ? Taskwait nowait only makes sense with dependencies. It should only result in additional nodes/edges in the task dependency graph without any code to execute. There is not taskwait (aka task barrier) in this case.

The taskwait construct has the restriction in the spec:

The nowait clause may only appear on a taskwait directive if the depend clause is present.

[nikic]

clang/test/OpenMP/taskwait_depend_nowait_codegen.cpp

2

Please do not add any new tests using -no-opaque-pointers.

[koops]

Taking care of :

1. "The nowait clause may only appear on a taskwait directive if the depend clause is present.".
2. "Please do not add any new tests using -no-opaque-pointers".
3. Added a new test to expect failure for point 1.

[ABataev]

LG

clang/lib/CodeGen/CGOpenMPRuntime.h

struct OMPTaskDataTy final {
   llvm::PointerIntPair<llvm::Value *, 1, bool> Schedule;
   llvm::PointerIntPair<llvm::Value *, 1, bool> Priority;
   llvm::Value *Reductions = nullptr;
   unsigned NumberOfParts = 0;
   bool Tied = true;
   bool Nogroup = false;
   bool IsReductionWithTaskMod = false;
   bool IsWorksharingReduction = false;
+  bool HasNowaitClause = false;
 };
 
 /// Class intended to support codegen of all kind of the reduction clauses.
 class ReductionCodeGen {
 private:
   /// Data required for codegen of reduction clauses.
   struct ReductionData {
     /// Reference to the item shared between tasks to reduce into.

clang/lib/CodeGen/CGOpenMPRuntime.cpp

static FieldDecl *addFieldToRecordDecl(ASTContext &C, DeclContext *DC,
   Field->setAccess(AS_public);
   DC->addDecl(Field);
   return Field;
 }
 
 CGOpenMPRuntime::CGOpenMPRuntime(CodeGenModule &CGM, StringRef FirstSeparator,
                                  StringRef Separator)
     : CGM(CGM), FirstSeparator(FirstSeparator), Separator(Separator),
-      OMPBuilder(CGM.getModule()), OffloadEntriesInfoManager(CGM) {
+      OMPBuilder(CGM.getModule(), CGM.getLangOpts().OpenMP),
+      OffloadEntriesInfoManager(CGM) {
   KmpCriticalNameTy = llvm::ArrayType::get(CGM.Int32Ty, /*NumElements*/ 8);
 
   // Initialize Types used in OpenMPIRBuilder from OMPKinds.def
   OMPBuilder.initialize();
   loadOffloadInfoMetadata();
 }
 
 void CGOpenMPRuntime::clear() {
void CGOpenMPRuntime::emitTaskCall(CodeGenFunction &CGF, SourceLocation Loc,
   if (!Data.Dependences.empty()) {
     DepWaitTaskArgs[0] = UpLoc;
     DepWaitTaskArgs[1] = ThreadID;
     DepWaitTaskArgs[2] = NumOfElements;
     DepWaitTaskArgs[3] = DependenciesArray.getPointer();
     DepWaitTaskArgs[4] = CGF.Builder.getInt32(0);
     DepWaitTaskArgs[5] = llvm::ConstantPointerNull::get(CGF.VoidPtrTy);
   }
   auto &M = CGM.getModule();
   auto &&ElseCodeGen = [this, &M, &TaskArgs, ThreadID, NewTaskNewTaskTTy,
                         TaskEntry, &Data, &DepWaitTaskArgs,
                         Loc](CodeGenFunction &CGF, PrePostActionTy &) {
     CodeGenFunction::RunCleanupsScope LocalScope(CGF);
     // Build void __kmpc_omp_wait_deps(ident_t *, kmp_int32 gtid,
     // kmp_int32 ndeps, kmp_depend_info_t *dep_list, kmp_int32
     // ndeps_noalias, kmp_depend_info_t *noalias_dep_list); if dependence info
     // is specified.
return Address(
           OMPBuilder.getOrCreateRuntimeFunction(
               CGM.getModule(), OMPRTL___kmpc_task_reduction_get_th_data),
           Args),
       CGF.Int8Ty, SharedLVal.getAlignment());
 }
 
 void CGOpenMPRuntime::emitTaskwaitCall(CodeGenFunction &CGF, SourceLocation Loc,
                                        const OMPTaskDataTy &Data) {
   if (!CGF.HaveInsertPoint())
     return;
 
   if (CGF.CGM.getLangOpts().OpenMPIRBuilder && Data.Dependences.empty()) {
     // TODO: Need to support taskwait with dependences in the OpenMPIRBuilder.
-    OMPBuilder.createTaskwait(CGF.Builder);
+    OMPBuilder.createTaskwait(CGF.Builder, Data.HasNowaitClause);
   } else {
     llvm::Value *ThreadID = getThreadID(CGF, Loc);
     llvm::Value *UpLoc = emitUpdateLocation(CGF, Loc);
     auto &M = CGM.getModule();
     Address DependenciesArray = Address::invalid();
     llvm::Value *NumOfElements;
     std::tie(NumOfElements, DependenciesArray) =
         emitDependClause(CGF, Data.Dependences, Loc);
-    llvm::Value *DepWaitTaskArgs[6];
     if (!Data.Dependences.empty()) {
+      if (CGM.getLangOpts().OpenMP >= 51) {
+        llvm::Value *DepWaitTaskArgs[7];
         DepWaitTaskArgs[0] = UpLoc;
         DepWaitTaskArgs[1] = ThreadID;
         DepWaitTaskArgs[2] = NumOfElements;
         DepWaitTaskArgs[3] = DependenciesArray.getPointer();
         DepWaitTaskArgs[4] = CGF.Builder.getInt32(0);
         DepWaitTaskArgs[5] = llvm::ConstantPointerNull::get(CGF.VoidPtrTy);
+        DepWaitTaskArgs[6] =
+            llvm::ConstantInt::get(CGF.IntTy, Data.HasNowaitClause);
 
         CodeGenFunction::RunCleanupsScope LocalScope(CGF);
-
+        CGF.EmitRuntimeCall(OMPBuilder.getOrCreateRuntimeFunction(
+                                M, OMPRTL___kmpc_omp_taskwait_deps_51),
+                            DepWaitTaskArgs);
+      } else {
         // Build void __kmpc_omp_wait_deps(ident_t *, kmp_int32 gtid,
         // kmp_int32 ndeps, kmp_depend_info_t *dep_list, kmp_int32
-      // ndeps_noalias, kmp_depend_info_t *noalias_dep_list); if dependence info
-      // is specified.
-      CGF.EmitRuntimeCall(
-          OMPBuilder.getOrCreateRuntimeFunction(M, OMPRTL___kmpc_omp_wait_deps),
+        // ndeps_noalias, kmp_depend_info_t *noalias_dep_list); if dependence
+        // info is specified.
+        llvm::Value *DepWaitTaskArgs[6];
+        DepWaitTaskArgs[0] = UpLoc;
+        DepWaitTaskArgs[1] = ThreadID;
+        DepWaitTaskArgs[2] = NumOfElements;
+        DepWaitTaskArgs[3] = DependenciesArray.getPointer();
+        DepWaitTaskArgs[4] = CGF.Builder.getInt32(0);
+        DepWaitTaskArgs[5] = llvm::ConstantPointerNull::get(CGF.VoidPtrTy);
+
+        CodeGenFunction::RunCleanupsScope LocalScope(CGF);
+        CGF.EmitRuntimeCall(OMPBuilder.getOrCreateRuntimeFunction(
+                                M, OMPRTL___kmpc_omp_wait_deps),
                             DepWaitTaskArgs);
+      }
 
     } else {
 
       // Build call kmp_int32 __kmpc_omp_taskwait(ident_t *loc, kmp_int32
       // global_tid);
-      llvm::Value *Args[] = {UpLoc, ThreadID};
+      llvm::Value *HasNowaitClauseVal =
+          llvm::ConstantInt::get(CGF.IntTy, Data.HasNowaitClause);
+      if (CGM.getLangOpts().OpenMP >= 51) {
+        llvm::Value *Args[3] = {UpLoc, ThreadID, HasNowaitClauseVal};
         // Ignore return result until untied tasks are supported.
-      CGF.EmitRuntimeCall(
-          OMPBuilder.getOrCreateRuntimeFunction(M, OMPRTL___kmpc_omp_taskwait),
+        CGF.EmitRuntimeCall(OMPBuilder.getOrCreateRuntimeFunction(
+                                M, OMPRTL___kmpc_omp_taskwait_51),
+                            Args);
+      } else {
+        llvm::Value *Args[2] = {UpLoc, ThreadID};
+        // Ignore return result until untied tasks are supported.
+        CGF.EmitRuntimeCall(OMPBuilder.getOrCreateRuntimeFunction(
+                                M, OMPRTL___kmpc_omp_taskwait),
                             Args);
       }
     }
+  }
 
   if (auto *Region = dyn_cast_or_null<CGOpenMPRegionInfo>(CGF.CapturedStmtInfo))
     Region->emitUntiedSwitch(CGF);
 }
 
 void CGOpenMPRuntime::emitInlinedDirective(CodeGenFunction &CGF,
                                            OpenMPDirectiveKind InnerKind,
                                            const RegionCodeGenTy &CodeGen,

clang/lib/CodeGen/CGStmtOpenMP.cpp

 void CodeGenFunction::EmitOMPBarrierDirective(const OMPBarrierDirective &S) {
   CGM.getOpenMPRuntime().emitBarrierCall(*this, S.getBeginLoc(), OMPD_barrier);
 }
 
 void CodeGenFunction::EmitOMPTaskwaitDirective(const OMPTaskwaitDirective &S) {
   OMPTaskDataTy Data;
   // Build list of dependences
   buildDependences(S, Data);
+  Data.HasNowaitClause = S.hasClausesOfKind<OMPNowaitClause>();
   CGM.getOpenMPRuntime().emitTaskwaitCall(*this, S.getBeginLoc(), Data);
 }
 
 bool isSupportedByOpenMPIRBuilder(const OMPTaskgroupDirective &T) {
   return T.clauses().empty();
 }
 
 void CodeGenFunction::EmitOMPTaskgroupDirective(
     const OMPTaskgroupDirective &S) {

clang/test/OpenMP/task_codegen.cpp

 // CHECK2-51-NEXT:    call void @_ZN1SC1Ev(%struct.S* noundef nonnull align 4 dereferenceable(4) [[REF_TMP_I]])
 // CHECK2-51-NEXT:    [[TMP50:%.*]] = bitcast %struct.S* [[TMP17]] to i8*
 // CHECK2-51-NEXT:    [[TMP51:%.*]] = bitcast %struct.S* [[REF_TMP_I]] to i8*
 // CHECK2-51-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 [[TMP50]], i8* align 4 [[TMP51]], i64 4, i1 false)
 // CHECK2-51-NEXT:    call void @_ZN1SD1Ev(%struct.S* noundef nonnull align 4 dereferenceable(4) [[REF_TMP_I]]) #[[ATTR4]]
 // CHECK2-51-NEXT:    [[A9_I:%.*]] = getelementptr inbounds [[STRUCT_S]], %struct.S* [[TMP19]], i32 0, i32 0
 // CHECK2-51-NEXT:    store i32 10, i32* [[A9_I]], align 4
 // CHECK2-51-NEXT:    [[TMP52:%.*]] = load i32, i32* [[DOTGLOBAL_TID__ADDR_I]], align 4, !noalias !122
-// CHECK2-51-NEXT:    [[TMP53:%.*]] = call i32 @__kmpc_omp_taskwait(%struct.ident_t* @[[GLOB1]], i32 [[TMP52]])
+// CHECK2-51-NEXT:    [[TMP53:%.*]] = call i32 @__kmpc_omp_taskwait_51(%struct.ident_t* @[[GLOB1]], i32 [[TMP52]], i32 0)
 // CHECK2-51-NEXT:    [[TMP54:%.*]] = load i32*, i32** [[DOTPART_ID__ADDR_I]], align 8, !noalias !122
 // CHECK2-51-NEXT:    store i32 5, i32* [[TMP54]], align 4
 // CHECK2-51-NEXT:    [[TMP55:%.*]] = load i32, i32* [[DOTGLOBAL_TID__ADDR_I]], align 4, !noalias !122
 // CHECK2-51-NEXT:    [[TMP56:%.*]] = load i8*, i8** [[DOTTASK_T__ADDR_I]], align 8, !noalias !122
 // CHECK2-51-NEXT:    [[TMP57:%.*]] = call i32 @__kmpc_omp_task(%struct.ident_t* @[[GLOB1]], i32 [[TMP55]], i8* [[TMP56]])
 // CHECK2-51-NEXT:    br label [[DOTOMP_OUTLINED__19_EXIT]]
 // CHECK2-51:       .untied.jmp.10.i:
 // CHECK2-51-NEXT:    call void @_ZN1SD1Ev(%struct.S* noundef nonnull align 4 dereferenceable(4) [[TMP19]]) #[[ATTR4]]

 // CHECK3-51-NEXT:    call void @_ZN1SC1Ev(%struct.S* noundef nonnull align 4 dereferenceable(4) [[REF_TMP_I]])
 // CHECK3-51-NEXT:    [[TMP38:%.*]] = bitcast %struct.S* [[S1_I]] to i8*
 // CHECK3-51-NEXT:    [[TMP39:%.*]] = bitcast %struct.S* [[REF_TMP_I]] to i8*
 // CHECK3-51-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 [[TMP38]], i8* align 4 [[TMP39]], i64 4, i1 false), !noalias !122
 // CHECK3-51-NEXT:    call void @_ZN1SD1Ev(%struct.S* noundef nonnull align 4 dereferenceable(4) [[REF_TMP_I]]) #[[ATTR4]]
 // CHECK3-51-NEXT:    [[A12_I:%.*]] = getelementptr inbounds [[STRUCT_S]], %struct.S* [[S2_I]], i32 0, i32 0
 // CHECK3-51-NEXT:    store i32 10, i32* [[A12_I]], align 4, !noalias !122
 // CHECK3-51-NEXT:    [[OMP_GLOBAL_THREAD_NUM13_I:%.*]] = call i32 @__kmpc_global_thread_num(%struct.ident_t* @[[GLOB1]])
-// CHECK3-51-NEXT:    [[TMP40:%.*]] = call i32 @__kmpc_omp_taskwait(%struct.ident_t* @[[GLOB1]], i32 [[OMP_GLOBAL_THREAD_NUM13_I]])
+// CHECK3-51-NEXT:    [[TMP40:%.*]] = call i32 @__kmpc_omp_taskwait_51(%struct.ident_t* @[[GLOB1]], i32 [[OMP_GLOBAL_THREAD_NUM13_I]], i32 0)
 // CHECK3-51-NEXT:    [[TMP41:%.*]] = load i32*, i32** [[DOTPART_ID__ADDR_I]], align 8, !noalias !122
 // CHECK3-51-NEXT:    store i32 4, i32* [[TMP41]], align 4
 // CHECK3-51-NEXT:    [[OMP_GLOBAL_THREAD_NUM14_I:%.*]] = call i32 @__kmpc_global_thread_num(%struct.ident_t* @[[GLOB23]])
 // CHECK3-51-NEXT:    [[TMP42:%.*]] = load i8*, i8** [[DOTTASK_T__ADDR_I]], align 8, !noalias !122
 // CHECK3-51-NEXT:    [[TMP43:%.*]] = call i32 @__kmpc_omp_task(%struct.ident_t* @[[GLOB1]], i32 [[OMP_GLOBAL_THREAD_NUM14_I]], i8* [[TMP42]])
 // CHECK3-51-NEXT:    br label [[DOTOMP_OUTLINED__19_EXIT]]
 // CHECK3-51:       .untied.jmp.15.i:
 // CHECK3-51-NEXT:    call void @_ZN1SD1Ev(%struct.S* noundef nonnull align 4 dereferenceable(4) [[S2_I]]) #[[ATTR4]]

 // CHECK4-51-NEXT:    call void @_ZN1SC1Ev(%struct.S* noundef nonnull align 4 dereferenceable(4) [[REF_TMP_I]])
 // CHECK4-51-NEXT:    [[TMP38:%.*]] = bitcast %struct.S* [[S1_I]] to i8*
 // CHECK4-51-NEXT:    [[TMP39:%.*]] = bitcast %struct.S* [[REF_TMP_I]] to i8*
 // CHECK4-51-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 [[TMP38]], i8* align 4 [[TMP39]], i64 4, i1 false), !noalias !112
 // CHECK4-51-NEXT:    call void @_ZN1SD1Ev(%struct.S* noundef nonnull align 4 dereferenceable(4) [[REF_TMP_I]]) #[[ATTR4]]
 // CHECK4-51-NEXT:    [[A12_I:%.*]] = getelementptr inbounds [[STRUCT_S]], %struct.S* [[S2_I]], i32 0, i32 0
 // CHECK4-51-NEXT:    store i32 10, i32* [[A12_I]], align 4, !noalias !112
 // CHECK4-51-NEXT:    [[OMP_GLOBAL_THREAD_NUM13_I:%.*]] = call i32 @__kmpc_global_thread_num(%struct.ident_t* @[[GLOB1]])
-// CHECK4-51-NEXT:    [[TMP40:%.*]] = call i32 @__kmpc_omp_taskwait(%struct.ident_t* @[[GLOB1]], i32 [[OMP_GLOBAL_THREAD_NUM13_I]])
+// CHECK4-51-NEXT:    [[TMP40:%.*]] = call i32 @__kmpc_omp_taskwait_51(%struct.ident_t* @[[GLOB1]], i32 [[OMP_GLOBAL_THREAD_NUM13_I]], i32 0)
 // CHECK4-51-NEXT:    [[TMP41:%.*]] = load i32*, i32** [[DOTPART_ID__ADDR_I]], align 8, !noalias !112
 // CHECK4-51-NEXT:    store i32 4, i32* [[TMP41]], align 4
 // CHECK4-51-NEXT:    [[OMP_GLOBAL_THREAD_NUM14_I:%.*]] = call i32 @__kmpc_global_thread_num(%struct.ident_t* @[[GLOB21]])
 // CHECK4-51-NEXT:    [[TMP42:%.*]] = load i8*, i8** [[DOTTASK_T__ADDR_I]], align 8, !noalias !112
 // CHECK4-51-NEXT:    [[TMP43:%.*]] = call i32 @__kmpc_omp_task(%struct.ident_t* @[[GLOB1]], i32 [[OMP_GLOBAL_THREAD_NUM14_I]], i8* [[TMP42]])
 // CHECK4-51-NEXT:    br label [[DOTOMP_OUTLINED__17_EXIT]]
 // CHECK4-51:       .untied.jmp.15.i:
 // CHECK4-51-NEXT:    call void @_ZN1SD1Ev(%struct.S* noundef nonnull align 4 dereferenceable(4) [[S2_I]]) #[[ATTR4]]

clang/test/OpenMP/taskwait_ast_print.cpp

-// RUN: %clang_cc1 -verify -fopenmp -fopenmp-version=50 -ast-print %s | FileCheck %s
-// RUN: %clang_cc1 -fopenmp -fopenmp-version=50 -x c++ -std=c++11 -emit-pch -o %t %s
-// RUN: %clang_cc1 -fopenmp -fopenmp-version=50 -std=c++11 -include-pch %t -fsyntax-only -verify %s -ast-print | FileCheck %s
-
-// RUN: %clang_cc1 -verify -fopenmp-simd -fopenmp-version=50 -ast-print %s | FileCheck %s
-// RUN: %clang_cc1 -fopenmp-simd -fopenmp-version=50 -x c++ -std=c++11 -emit-pch -o %t %s
-// RUN: %clang_cc1 -fopenmp-simd -fopenmp-version=50 -std=c++11 -include-pch %t -fsyntax-only -verify %s -ast-print | FileCheck %s
+// RUN: %clang_cc1 -verify -fopenmp -fopenmp-version=51 -ast-print %s | FileCheck %s
+// RUN: %clang_cc1 -fopenmp -fopenmp-version=51 -x c++ -std=c++11 -emit-pch -o %t %s
+// RUN: %clang_cc1 -fopenmp -fopenmp-version=51 -std=c++11 -include-pch %t -fsyntax-only -verify %s -ast-print | FileCheck %s
+
+// RUN: %clang_cc1 -verify -fopenmp-simd -fopenmp-version=51 -ast-print %s | FileCheck %s
+// RUN: %clang_cc1 -fopenmp-simd -fopenmp-version=51 -x c++ -std=c++11 -emit-pch -o %t %s
+// RUN: %clang_cc1 -fopenmp-simd -fopenmp-version=51 -std=c++11 -include-pch %t -fsyntax-only -verify %s -ast-print | FileCheck %s
 // expected-no-diagnostics
 
 #ifndef HEADER
 #define HEADER
 
 void foo() {}
 
 template <class T>
 T tmain(T argc) {
   static T a;
 #pragma omp taskwait
 #pragma omp taskwait depend(in:a, argc)
   return a + argc;
 }
+
+template <class T>
+T nmain(T argc) {
+  static T a;
+#pragma omp taskwait
+#pragma omp taskwait nowait
+  return a + argc;
+}
+
+template <class T>
+T ndmain(T argc) {
+  static T a;
+#pragma omp taskwait
+#pragma omp taskwait depend(in:a, argc) nowait
+  return a + argc;
+}
+
 // CHECK:      static T a;
 // CHECK-NEXT: #pragma omp taskwait{{$}}
 // CHECK-NEXT: #pragma omp taskwait depend(in : a,argc){{$}}
 // CHECK:      static int a;
 // CHECK-NEXT: #pragma omp taskwait
 // CHECK-NEXT: #pragma omp taskwait depend(in : a,argc){{$}}
 // CHECK:      static char a;
 // CHECK-NEXT: #pragma omp taskwait
 // CHECK-NEXT: #pragma omp taskwait depend(in : a,argc){{$}}
 
+// CHECK:      static T a;
+// CHECK-NEXT: #pragma omp taskwait{{$}}
+// CHECK-NEXT: #pragma omp taskwait nowait{{$}}
+// CHECK:      static int a;
+// CHECK-NEXT: #pragma omp taskwait
+// CHECK-NEXT: #pragma omp taskwait nowait{{$}}
+// CHECK:      static char a;
+// CHECK-NEXT: #pragma omp taskwait
+// CHECK-NEXT: #pragma omp taskwait nowait{{$}}
+
+// CHECK:      static T a;
+// CHECK-NEXT: #pragma omp taskwait{{$}}
+// CHECK-NEXT: #pragma omp taskwait depend(in : a,argc) nowait{{$}}
+// CHECK:      static int a;
+// CHECK-NEXT: #pragma omp taskwait
+// CHECK-NEXT: #pragma omp taskwait depend(in : a,argc) nowait{{$}}
+// CHECK:      static char a;
+// CHECK-NEXT: #pragma omp taskwait
+// CHECK-NEXT: #pragma omp taskwait depend(in : a,argc) nowait{{$}}
+
 int main(int argc, char **argv) {
   static int a;
 // CHECK: static int a;
 #pragma omp taskwait
 #pragma omp taskwait depend(out:a, argc)
   // CHECK-NEXT: #pragma omp taskwait
   // CHECK-NEXT: #pragma omp taskwait depend(out : a,argc)
-  return tmain(argc) + tmain(argv[0][0]) + a;
+  return tmain(argc) + tmain(argv[0][0]) + a + nmain(argc) + nmain(argv[0][0]) + ndmain(argc) + ndmain(argv[0][0]);
 }
 
 #endif

clang/test/OpenMP/taskwait_depend_nowait_codegen.cpp

+// RUN: %clang_cc1 -no-opaque-pointers -verify -triple x86_64-apple-darwin10 -fopenmp -fopenmp-version=51 -x c++ -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp -fopenmp-version=51 -x c++ -triple x86_64-apple-darwin10 -emit-pch -o %t %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp -fopenmp-version=51 -x c++ -triple x86_64-apple-darwin10 -include-pch %t -verify %s -emit-llvm -o - | FileCheck %s
+
+// RUN: %clang_cc1 -no-opaque-pointers -verify -triple x86_64-apple-darwin10 -fopenmp-simd -fopenmp-version=51 -x c++ -emit-llvm %s -o - | FileCheck --check-prefix SIMD-ONLY0 %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp-simd -fopenmp-version=51 -x c++ -triple x86_64-apple-darwin10 -emit-pch -o %t %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp-simd -fopenmp-version=51 -x c++ -triple x86_64-apple-darwin10 -include-pch %t -verify %s -emit-llvm -o - | FileCheck --check-prefix SIMD-ONLY0 %s
+// SIMD-ONLY0-NOT: {{__kmpc|__tgt}}
+// expected-no-diagnostics
+#ifndef HEADER
+#define HEADER
+
+void foo() {}
+
+template <class T>
+T tmain(T &argc) {
+  static T a;
+  #pragma omp taskwait depend(in:argc)
+  return a + argc;
+}
+
+template <class T>
+T nmain(T &argc) {
+  static T a;
+  #pragma omp taskwait depend(in:argc) nowait
+  return a + argc;
+}
+
+int main(int argc, char **argv) {
+  int n = 0;
+  #pragma omp task shared(n,argc) depend(out:n)
+     n = argc;
+  return tmain(n) + nmain(n);
+}
+
+// CHECK-LABEL: @main
+// CHECK: [[GTID:%.+]] = call i32 @__kmpc_global_thread_num(%{{.+}}* @{{.+}})
+// CHECK: [[ALLOC:%.+]] = call i8* @__kmpc_omp_task_alloc(%{{.+}}* @{{.+}}, i32 [[GTID]], i32 1, i64 40, i64 16, i32 (i32, i8*)* bitcast (i32 (i32, %{{.+}}*)* @{{.+}} to i32 (i32, i8*)*))
+// CHECK: %{{.+}} = call i32 @__kmpc_omp_task_with_deps(%{{.+}}* @{{.+}}, i32 [[GTID]], i8* [[ALLOC]], i32 1, i8* %{{[0-9]*}}, i32 0, i8* null)
+
+// CHECK-LABEL: define {{.*}} @{{.*}}tmain{{.*}}
+// CHECK: [[GTID:%.+]] = call i32 @__kmpc_global_thread_num(%{{.+}}* @{{.+}})
+// CHECK: call void @__kmpc_omp_taskwait_deps_51(%{{.+}}* @{{.+}}, i32 [[GTID]], i32 1, i8* %{{.}}, i32 0, i8* null, i32 0)
+
+// CHECK-LABEL: define {{.*}} @{{.*}}nmain{{.*}}
+// CHECK: [[GTID:%.+]] = call i32 @__kmpc_global_thread_num(%{{.+}}* @{{.+}})
+// CHECK: call void @__kmpc_omp_taskwait_deps_51(%{{.+}}* @{{.+}}, i32 [[GTID]], i32 1, i8* %{{.}}, i32 0, i8* null, i32 1)
+
+
+#endif

clang/test/OpenMP/taskwait_nowait_codegen.cpp

+// RUN: %clang_cc1 -no-opaque-pointers -verify -triple x86_64-apple-darwin10 -fopenmp -fopenmp-version=51 -x c++ -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp -fopenmp-version=51 -x c++ -triple x86_64-apple-darwin10 -emit-pch -o %t %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp -fopenmp-version=51 -x c++ -triple x86_64-apple-darwin10 -include-pch %t -verify %s -emit-llvm -o - | FileCheck %s
+//
+// RUN: %clang_cc1 -no-opaque-pointers -verify -triple x86_64-apple-darwin10 -fopenmp -fopenmp-version=51 -fopenmp-enable-irbuilder -x c++ -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp -fopenmp-version=51 -fopenmp-enable-irbuilder -x c++ -triple x86_64-apple-darwin10 -emit-pch -o %t %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp -fopenmp-version=51 -fopenmp-enable-irbuilder -x c++ -triple x86_64-apple-darwin10 -include-pch %t -verify %s -emit-llvm -o - | FileCheck %s
+
+// RUN: %clang_cc1 -no-opaque-pointers -verify -triple x86_64-apple-darwin10 -fopenmp-simd -fopenmp-version=51 -x c++ -emit-llvm %s -o - | FileCheck --check-prefix SIMD-ONLY0 %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp-simd -fopenmp-version=51 -x c++ -triple x86_64-apple-darwin10 -emit-pch -o %t %s
+// RUN: %clang_cc1 -no-opaque-pointers -fopenmp-simd -fopenmp-version=51 -x c++ -triple x86_64-apple-darwin10 -include-pch %t -verify %s -emit-llvm -o - | FileCheck --check-prefix SIMD-ONLY0 %s
+// SIMD-ONLY0-NOT: {{__kmpc|__tgt}}
+// expected-no-diagnostics
+#ifndef HEADER
+#define HEADER
+
+void foo() {}
+
+template <class T>
+T tmain(T argc) {
+  static T a;
+#pragma omp taskwait
+  return a + argc;
+}
+
+template <class T>
+T no_wait(T argc) {
+  static T a;
+#pragma omp taskwait nowait
+  return a + argc;
+}
+
+int main(int argc, char **argv) {
+#pragma omp taskwait
+  return tmain(argc)+no_wait(argc);
+}
+
+// CHECK-LABEL: @main
+// CHECK: [[GTID:%.+]] = call i32 @__kmpc_global_thread_num(%{{.+}}* @{{.+}})
+// CHECK: call i32 @__kmpc_omp_taskwait_51(%{{.+}}* @{{.+}}, i32 [[GTID]], i32 0)
+
+// CHECK-LABEL: define {{.*}} @{{.*}}tmain{{.*}}
+// CHECK: [[GTID:%.+]] = call i32 @__kmpc_global_thread_num(%{{.+}}* @{{.+}})
+// CHECK: call i32 @__kmpc_omp_taskwait_51(%{{.+}}* @{{.+}}, i32 [[GTID]], i32 0)
+//
+// CHECK-LABEL: define {{.*}} @{{.*}}no_wait{{.*}}
+// CHECK: [[GTID:%.+]] = call i32 @__kmpc_global_thread_num(%{{.+}}* @{{.+}})
+// CHECK: call i32 @__kmpc_omp_taskwait_51(%{{.+}}* @{{.+}}, i32 [[GTID]], i32 1)
+
+
+#endif

llvm/include/llvm/Frontend/OpenMP/OMP.td

def OMP_Critical : Directive<"critical"> {
   let allowedClauses = [
     VersionedClause<OMPC_Hint>
   ];
 }
 def OMP_TaskYield : Directive<"taskyield"> {}
 def OMP_Barrier : Directive<"barrier"> {}
 def OMP_TaskWait : Directive<"taskwait"> {
   let allowedClauses = [
-    VersionedClause<OMPC_Depend, 50>
+    VersionedClause<OMPC_Depend, 50>,
+    VersionedClause<OMPC_NoWait, 51>
   ];
 }
 def OMP_TaskGroup : Directive<"taskgroup"> {
   let allowedClauses = [
     VersionedClause<OMPC_TaskReduction, 50>,
     VersionedClause<OMPC_Allocate, 50>
   ];
 }

llvm/include/llvm/Frontend/OpenMP/OMPIRBuilder.h

 
 /// An interface to create LLVM-IR for OpenMP directives.
 ///
 /// Each OpenMP directive has a corresponding public generator method.
 class OpenMPIRBuilder {
 public:
   /// Create a new OpenMPIRBuilder operating on the given module \p M. This will
   /// not have an effect on \p M (see initialize).
-  OpenMPIRBuilder(Module &M) : M(M), Builder(M.getContext()) {}
+  OpenMPIRBuilder(Module &M, int OpenMPVersion = 40)
+    : M(M), Builder(M.getContext()), OpenMPVersion(OpenMPVersion) {
+  }
   ~OpenMPIRBuilder();
 
   /// Initialize the internal state, this will put structures types and
   /// potentially other helpers into the underlying module. Must be called
   /// before any other method and only once!
   void initialize();
 
   /// Finalize the underlying module, e.g., by outlining regions.
public:
   /// Generator for '#omp flush'
   ///
   /// \param Loc The location where the flush directive was encountered
   void createFlush(const LocationDescription &Loc);
 
   /// Generator for '#omp taskwait'
   ///
   /// \param Loc The location where the taskwait directive was encountered.
-  void createTaskwait(const LocationDescription &Loc);
+  void createTaskwait(const LocationDescription &Loc,
+                      bool HasNoTaskwaitClause = false);
 
   /// Generator for '#omp taskyield'
   ///
   /// \param Loc The location where the taskyield directive was encountered.
   void createTaskyield(const LocationDescription &Loc);
 
   /// Generator for `#omp task`
   ///
bool isLastFinalizationInfoCancellable(omp::Directive DK) {
     return !FinalizationStack.empty() &&
            FinalizationStack.back().IsCancellable &&
            FinalizationStack.back().DK == DK;
   }
 
   /// Generate a taskwait runtime call.
   ///
   /// \param Loc The location at which the request originated and is fulfilled.
-  void emitTaskwaitImpl(const LocationDescription &Loc);
+  void emitTaskwaitImpl(const LocationDescription &Loc,
+                        bool HasNowaitClause = false);
 
   /// Generate a taskyield runtime call.
   ///
   /// \param Loc The location at which the request originated and is fulfilled.
   void emitTaskyieldImpl(const LocationDescription &Loc);
 
   /// Return the current thread ID.
   ///
emitAtomicUpdate(InsertPointTy AllocaIP, Value *X, Type *XElemTy, Value *Expr,
                    bool IsXBinopExpr);
 
   /// Emit the binary op. described by \p RMWOp, using \p Src1 and \p Src2 .
   ///
   /// \Return The instruction
   Value *emitRMWOpAsInstruction(Value *Src1, Value *Src2,
                                 AtomicRMWInst::BinOp RMWOp);
 
+  int OpenMPVersion;
 public:
+  /// Methods for OpenMPVersion
+  int getOpenMPVersion() { return OpenMPVersion; }
+
   /// a struct to pack relevant information while generating atomic Ops
   struct AtomicOpValue {
     Value *Var = nullptr;
     Type *ElemTy = nullptr;
     bool IsSigned = false;
     bool IsVolatile = false;
   };
 

llvm/include/llvm/Frontend/OpenMP/OMPKinds.def

 __OMP_RTL(__kmpc_barrier, false, Void, IdentPtr, Int32)
 __OMP_RTL(__kmpc_cancel, false, Int32, IdentPtr, Int32, Int32)
 __OMP_RTL(__kmpc_cancel_barrier, false, Int32, IdentPtr, Int32)
 __OMP_RTL(__kmpc_flush, false, Void, IdentPtr)
 __OMP_RTL(__kmpc_global_thread_num, false, Int32, IdentPtr)
 __OMP_RTL(__kmpc_get_hardware_thread_id_in_block, false, Int32, )
 __OMP_RTL(__kmpc_fork_call, true, Void, IdentPtr, Int32, ParallelTaskPtr)
 __OMP_RTL(__kmpc_omp_taskwait, false, Int32, IdentPtr, Int32)
+__OMP_RTL(__kmpc_omp_taskwait_51, false, Int32, IdentPtr, Int32, Int32)
 __OMP_RTL(__kmpc_omp_taskyield, false, Int32, IdentPtr, Int32, /* Int */ Int32)
 __OMP_RTL(__kmpc_push_num_threads, false, Void, IdentPtr, Int32,
           /* Int */ Int32)
 __OMP_RTL(__kmpc_push_proc_bind, false, Void, IdentPtr, Int32, /* Int */ Int32)
 __OMP_RTL(__kmpc_omp_reg_task_with_affinity, false, Int32, IdentPtr, Int32,
           /* kmp_task_t */ VoidPtr, Int32,
           /* kmp_task_affinity_info_t */ VoidPtr)
 
__OMP_RTL(__kmpc_task_reduction_get_th_data, false, VoidPtr, Int32, VoidPtr,
           VoidPtr)
 __OMP_RTL(__kmpc_task_reduction_init, false, VoidPtr, Int32, Int32, VoidPtr)
 __OMP_RTL(__kmpc_task_reduction_modifier_init, false, VoidPtr, VoidPtr, Int32,
           Int32, Int32, VoidPtr)
 __OMP_RTL(__kmpc_proxy_task_completed_ooo, false, Void, VoidPtr)
 
 __OMP_RTL(__kmpc_omp_wait_deps, false, Void, IdentPtr, Int32, Int32,
           /* kmp_depend_info_t */ VoidPtr, Int32, VoidPtr)
+__OMP_RTL(__kmpc_omp_taskwait_deps_51, false, Void, IdentPtr, Int32, Int32,
+          /* kmp_depend_info_t */ VoidPtr, Int32, VoidPtr, Int32)
 __OMP_RTL(__kmpc_cancellationpoint, false, Int32, IdentPtr, Int32, Int32)
 
 __OMP_RTL(__kmpc_fork_teams, true, Void, IdentPtr, Int32, ParallelTaskPtr)
 __OMP_RTL(__kmpc_push_num_teams, false, Void, IdentPtr, Int32, Int32, Int32)
 
 __OMP_RTL(__kmpc_copyprivate, false, Void, IdentPtr, Int32, SizeTy, VoidPtr,
           CopyFunctionPtr, Int32)
 __OMP_RTL(__kmpc_threadprivate_cached, false, VoidPtr, IdentPtr, Int32, VoidPtr,

 __OMP_RTL_ATTRS(__kmpc_flush, BarrierAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs))
 __OMP_RTL_ATTRS(__kmpc_global_thread_num, GetterAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs))
 __OMP_RTL_ATTRS(__kmpc_fork_call, ForkAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs, AttributeSet(), ReadOnlyPtrAttrs))
 __OMP_RTL_ATTRS(__kmpc_omp_taskwait, BarrierAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs))
+__OMP_RTL_ATTRS(__kmpc_omp_taskwait_51, BarrierAttrs, AttributeSet(),
+                ParamAttrs(ReadOnlyPtrAttrs))
 __OMP_RTL_ATTRS(__kmpc_omp_taskyield, InaccessibleArgOnlyAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs))
 __OMP_RTL_ATTRS(__kmpc_push_num_threads, InaccessibleArgOnlyAttrs,
                 AttributeSet(), ParamAttrs(ReadOnlyPtrAttrs))
 __OMP_RTL_ATTRS(__kmpc_push_proc_bind, InaccessibleArgOnlyAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs))
 __OMP_RTL_ATTRS(__kmpc_serialized_parallel, InaccessibleArgOnlyAttrs,
                 AttributeSet(), ParamAttrs(ReadOnlyPtrAttrs))

 __OMP_RTL_ATTRS(__kmpc_task_reduction_modifier_init, DefaultAttrs,
                 ReturnPtrAttrs, ParamAttrs())
 __OMP_RTL_ATTRS(__kmpc_proxy_task_completed_ooo, DefaultAttrs, AttributeSet(),
                 ParamAttrs())
 
 __OMP_RTL_ATTRS(__kmpc_omp_wait_deps, BarrierAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs, AttributeSet(), AttributeSet(),
                            ReadOnlyPtrAttrs))
+__OMP_RTL_ATTRS(__kmpc_omp_taskwait_deps_51, BarrierAttrs, AttributeSet(),
+                ParamAttrs(ReadOnlyPtrAttrs, AttributeSet(), AttributeSet(),
+                           ReadOnlyPtrAttrs))
 __OMP_RTL_ATTRS(__kmpc_cancellationpoint, DefaultAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs))
 
 __OMP_RTL_ATTRS(__kmpc_fork_teams, ForkAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs, AttributeSet(), ReadOnlyPtrAttrs))
 __OMP_RTL_ATTRS(__kmpc_push_num_teams, InaccessibleArgOnlyAttrs, AttributeSet(),
                 ParamAttrs(ReadOnlyPtrAttrs))
 

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

 }
 
 void OpenMPIRBuilder::createFlush(const LocationDescription &Loc) {
   if (!updateToLocation(Loc))
     return;
   emitFlush(Loc);
 }
 
-void OpenMPIRBuilder::emitTaskwaitImpl(const LocationDescription &Loc) {
+void OpenMPIRBuilder::emitTaskwaitImpl(const LocationDescription &Loc,
+                                       bool HasNowaitClause) {
   // Build call kmp_int32 __kmpc_omp_taskwait(ident_t *loc, kmp_int32
   // global_tid);
   uint32_t SrcLocStrSize;
   Constant *SrcLocStr = getOrCreateSrcLocStr(Loc, SrcLocStrSize);
   Value *Ident = getOrCreateIdent(SrcLocStr, SrcLocStrSize);
+  if (OpenMPVersion >= 51) {
+    Value *NowaitClauseValue = ConstantInt::get(Int32, HasNowaitClause);
+    Value *Args[] = {Ident, getOrCreateThreadID(Ident), NowaitClauseValue};
+
+    // Ignore return result until untied tasks are supported.
+    Builder.CreateCall(
+        getOrCreateRuntimeFunctionPtr(OMPRTL___kmpc_omp_taskwait_51), Args);
+  } else {
     Value *Args[] = {Ident, getOrCreateThreadID(Ident)};
 
     // Ignore return result until untied tasks are supported.
-  Builder.CreateCall(getOrCreateRuntimeFunctionPtr(OMPRTL___kmpc_omp_taskwait),
-                     Args);
+    Builder.CreateCall(
+        getOrCreateRuntimeFunctionPtr(OMPRTL___kmpc_omp_taskwait), Args);
+  }
 }
 
-void OpenMPIRBuilder::createTaskwait(const LocationDescription &Loc) {
+void OpenMPIRBuilder::createTaskwait(const LocationDescription &Loc,
+                                     bool HasNowaitClause) {
   if (!updateToLocation(Loc))
     return;
-  emitTaskwaitImpl(Loc);
+  emitTaskwaitImpl(Loc, HasNowaitClause);
 }
 
 void OpenMPIRBuilder::emitTaskyieldImpl(const LocationDescription &Loc) {
   // Build call __kmpc_omp_taskyield(loc, thread_id, 0);
   uint32_t SrcLocStrSize;
   Constant *SrcLocStr = getOrCreateSrcLocStr(Loc, SrcLocStrSize);
   Value *Ident = getOrCreateIdent(SrcLocStr, SrcLocStrSize);
   Constant *I32Null = ConstantInt::getNullValue(Int32);
OpenMPIRBuilder::InsertPointTy OpenMPIRBuilder::applyWorkshareLoop(
   case OMPScheduleType::BaseAuto:
   case OMPScheduleType::BaseGreedy:
   case OMPScheduleType::BaseBalanced:
   case OMPScheduleType::BaseSteal:
   case OMPScheduleType::BaseGuidedSimd:
   case OMPScheduleType::BaseRuntimeSimd:
     assert(!ChunkSize &&
            "schedule type does not support user-defined chunk sizes");
-    [[fallthrough]];
+    LLVM_FALLTHROUGH;
   case OMPScheduleType::BaseDynamicChunked:
   case OMPScheduleType::BaseGuidedChunked:
   case OMPScheduleType::BaseGuidedIterativeChunked:
   case OMPScheduleType::BaseGuidedAnalyticalChunked:
   case OMPScheduleType::BaseStaticBalancedChunked:
     return applyDynamicWorkshareLoop(DL, CLI, AllocaIP, EffectiveScheduleType,
                                      NeedsBarrier, ChunkSize);
 

openmp/runtime/src/dllexports

 %ifndef stub
         __kmpc_push_proc_bind               237
         __kmpc_taskgroup                    238
         __kmpc_end_taskgroup                239
         __kmpc_push_num_teams               240
         __kmpc_fork_teams                   241
         __kmpc_omp_task_with_deps           242
         __kmpc_omp_wait_deps                243
+        __kmpc_omp_taskwait_deps_51         288
         __kmpc_cancel                       244
         __kmpc_cancellationpoint            245
         __kmpc_cancel_barrier               246
         __kmpc_dist_for_static_init_4       247
         __kmpc_dist_for_static_init_4u      248
         __kmpc_dist_for_static_init_8       249
         __kmpc_dist_for_static_init_8u      250
         __kmpc_dist_dispatch_init_4         251

openmp/runtime/src/kmp.h

KMP_EXPORT kmp_task_t *__kmpc_omp_target_task_alloc(
     size_t sizeof_shareds, kmp_routine_entry_t task_entry, kmp_int64 device_id);
 KMP_EXPORT void __kmpc_omp_task_begin_if0(ident_t *loc_ref, kmp_int32 gtid,
                                           kmp_task_t *task);
 KMP_EXPORT void __kmpc_omp_task_complete_if0(ident_t *loc_ref, kmp_int32 gtid,
                                              kmp_task_t *task);
 KMP_EXPORT kmp_int32 __kmpc_omp_task_parts(ident_t *loc_ref, kmp_int32 gtid,
                                            kmp_task_t *new_task);
 KMP_EXPORT kmp_int32 __kmpc_omp_taskwait(ident_t *loc_ref, kmp_int32 gtid);
+/* __kmpc_omp_taskwait_51 : Function for OpenMP 5.1 nowait clause.
+ *                          Placeholder for taskwait with nowait clause.*/
+KMP_EXPORT kmp_int32 __kmpc_omp_taskwait_51(ident_t *loc_ref, kmp_int32 gtid,
+                                            kmp_int32 have_no_wait);
 
 KMP_EXPORT kmp_int32 __kmpc_omp_taskyield(ident_t *loc_ref, kmp_int32 gtid,
                                           int end_part);
 
 #if TASK_UNUSED
 void __kmpc_omp_task_begin(ident_t *loc_ref, kmp_int32 gtid, kmp_task_t *task);
 void __kmpc_omp_task_complete(ident_t *loc_ref, kmp_int32 gtid,
                               kmp_task_t *task);
 #endif // TASK_UNUSED
 
 /* ------------------------------------------------------------------------ */
 
 KMP_EXPORT void __kmpc_taskgroup(ident_t *loc, int gtid);
 KMP_EXPORT void __kmpc_end_taskgroup(ident_t *loc, int gtid);
 
 KMP_EXPORT kmp_int32 __kmpc_omp_task_with_deps(
     ident_t *loc_ref, kmp_int32 gtid, kmp_task_t *new_task, kmp_int32 ndeps,
     kmp_depend_info_t *dep_list, kmp_int32 ndeps_noalias,
     kmp_depend_info_t *noalias_dep_list);
 KMP_EXPORT void __kmpc_omp_wait_deps(ident_t *loc_ref, kmp_int32 gtid,
                                      kmp_int32 ndeps,
                                      kmp_depend_info_t *dep_list,
                                      kmp_int32 ndeps_noalias,
                                      kmp_depend_info_t *noalias_dep_list);
+/* __kmpc_omp_taskwait_deps_51 : Function for OpenMP 5.1 nowait clause.
+ *                               Placeholder for taskwait with nowait clause.*/
+KMP_EXPORT void __kmpc_omp_taskwait_deps_51(ident_t *loc_ref, kmp_int32 gtid,
+                                            kmp_int32 ndeps,
+                                            kmp_depend_info_t *dep_list,
+                                            kmp_int32 ndeps_noalias,
+                                            kmp_depend_info_t *noalias_dep_list,
+                                            kmp_int32 have_no_wait = false);
+
 extern kmp_int32 __kmp_omp_task(kmp_int32 gtid, kmp_task_t *new_task,
                                 bool serialize_immediate);
 
 KMP_EXPORT kmp_int32 __kmpc_cancel(ident_t *loc_ref, kmp_int32 gtid,
                                    kmp_int32 cncl_kind);
 KMP_EXPORT kmp_int32 __kmpc_cancellationpoint(ident_t *loc_ref, kmp_int32 gtid,
                                               kmp_int32 cncl_kind);
 KMP_EXPORT kmp_int32 __kmpc_cancel_barrier(ident_t *loc_ref, kmp_int32 gtid);

openmp/runtime/src/kmp_taskdeps.cpp

#endif /* OMPT_SUPPORT */
   }
 
 #if OMPT_SUPPORT
   __ompt_taskwait_dep_finish(current_task, taskwait_task_data);
 #endif /* OMPT_SUPPORT */
   KA_TRACE(10, ("__kmpc_omp_wait_deps(exit): T#%d finished waiting : loc=%p\n",
                 gtid, loc_ref));
 }
+
+/* __kmpc_omp_taskwait_deps_51 : Function for OpenMP 5.1 nowait clause.
+ *                               Placeholder for taskwait with nowait clause.
+ *                               The code is a copy of
+ *                               __kmpc_omp_taskwait_deps.*/
+void __kmpc_omp_taskwait_deps_51(ident_t *loc_ref, kmp_int32 gtid,
+                                 kmp_int32 ndeps, kmp_depend_info_t *dep_list,
+                                 kmp_int32 ndeps_noalias,
+                                 kmp_depend_info_t *noalias_dep_list,
+                                 kmp_int32 have_no_wait) {
+  KA_TRACE(10, ("__kmpc_omp_taskwait_deps(enter): T#%d loc=%p nowait#%d\n",
+                gtid, loc_ref, have_no_wait));
+  if (ndeps == 0 && ndeps_noalias == 0) {
+    KA_TRACE(10, ("__kmpc_omp_taskwait_deps(exit): T#%d has no dependences to "
+                  "wait upon : loc=%p\n",
+                  gtid, loc_ref));
+    return;
+  }
+  __kmp_assert_valid_gtid(gtid);
+  kmp_info_t *thread = __kmp_threads[gtid];
+  kmp_taskdata_t *current_task = thread->th.th_current_task;
+
+#if OMPT_SUPPORT
+  // this function represents a taskwait construct with depend clause
+  // We signal 4 events:
+  //  - creation of the taskwait task
+  //  - dependences of the taskwait task
+  //  - schedule and finish of the taskwait task
+  ompt_data_t *taskwait_task_data = &thread->th.ompt_thread_info.task_data;
+  KMP_ASSERT(taskwait_task_data->ptr == NULL);
+  if (ompt_enabled.enabled) {
+    if (!current_task->ompt_task_info.frame.enter_frame.ptr)
+      current_task->ompt_task_info.frame.enter_frame.ptr =
+          OMPT_GET_FRAME_ADDRESS(0);
+    if (ompt_enabled.ompt_callback_task_create) {
+      ompt_callbacks.ompt_callback(ompt_callback_task_create)(
+          &(current_task->ompt_task_info.task_data),
+          &(current_task->ompt_task_info.frame), taskwait_task_data,
+          ompt_task_taskwait | ompt_task_undeferred | ompt_task_mergeable, 1,
+          OMPT_LOAD_OR_GET_RETURN_ADDRESS(gtid));
+    }
+  }
+
+#if OMPT_OPTIONAL
+  /* OMPT grab all dependences if requested by the tool */
+  if (ndeps + ndeps_noalias > 0 && ompt_enabled.ompt_callback_dependences) {
+    kmp_int32 i;
+
+    int ompt_ndeps = ndeps + ndeps_noalias;
+    ompt_dependence_t *ompt_deps = (ompt_dependence_t *)KMP_OMPT_DEPS_ALLOC(
+        thread, (ndeps + ndeps_noalias) * sizeof(ompt_dependence_t));
+
+    KMP_ASSERT(ompt_deps != NULL);
+
+    for (i = 0; i < ndeps; i++) {
+      ompt_deps[i].variable.ptr = (void *)dep_list[i].base_addr;
+      if (dep_list[i].flags.in && dep_list[i].flags.out)
+        ompt_deps[i].dependence_type = ompt_dependence_type_inout;
+      else if (dep_list[i].flags.out)
+        ompt_deps[i].dependence_type = ompt_dependence_type_out;
+      else if (dep_list[i].flags.in)
+        ompt_deps[i].dependence_type = ompt_dependence_type_in;
+      else if (dep_list[i].flags.mtx)
+        ompt_deps[ndeps + i].dependence_type =
+            ompt_dependence_type_mutexinoutset;
+      else if (dep_list[i].flags.set)
+        ompt_deps[ndeps + i].dependence_type = ompt_dependence_type_inoutset;
+    }
+    for (i = 0; i < ndeps_noalias; i++) {
+      ompt_deps[ndeps + i].variable.ptr = (void *)noalias_dep_list[i].base_addr;
+      if (noalias_dep_list[i].flags.in && noalias_dep_list[i].flags.out)
+        ompt_deps[ndeps + i].dependence_type = ompt_dependence_type_inout;
+      else if (noalias_dep_list[i].flags.out)
+        ompt_deps[ndeps + i].dependence_type = ompt_dependence_type_out;
+      else if (noalias_dep_list[i].flags.in)
+        ompt_deps[ndeps + i].dependence_type = ompt_dependence_type_in;
+      else if (noalias_dep_list[i].flags.mtx)
+        ompt_deps[ndeps + i].dependence_type =
+            ompt_dependence_type_mutexinoutset;
+      else if (noalias_dep_list[i].flags.set)
+        ompt_deps[ndeps + i].dependence_type = ompt_dependence_type_inoutset;
+    }
+    ompt_callbacks.ompt_callback(ompt_callback_dependences)(
+        taskwait_task_data, ompt_deps, ompt_ndeps);
+    /* We can now free the allocated memory for the dependences */
+    /* For OMPD we might want to delay the free until end of this function */
+    KMP_OMPT_DEPS_FREE(thread, ompt_deps);
+    ompt_deps = NULL;
+  }
+#endif /* OMPT_OPTIONAL */
+#endif /* OMPT_SUPPORT */
+
+  // We can return immediately as:
+  // - dependences are not computed in serial teams (except with proxy tasks)
+  // - if the dephash is not yet created it means we have nothing to wait for
+  bool ignore = current_task->td_flags.team_serial ||
+                current_task->td_flags.tasking_ser ||
+                current_task->td_flags.final;
+  ignore =
+      ignore && thread->th.th_task_team != NULL &&
+      thread->th.th_task_team->tt.tt_found_proxy_tasks == FALSE &&
+      thread->th.th_task_team->tt.tt_hidden_helper_task_encountered == FALSE;
+  ignore = ignore || current_task->td_dephash == NULL;
+
+  if (ignore) {
+    KA_TRACE(10, ("__kmpc_omp_taskwait_deps(exit): T#%d has no blocking "
+                  "dependences : loc=%p\n",
+                  gtid, loc_ref));
+#if OMPT_SUPPORT
+    __ompt_taskwait_dep_finish(current_task, taskwait_task_data);
+#endif /* OMPT_SUPPORT */
+    return;
+  }
+
+  kmp_depnode_t node = {0};
+  __kmp_init_node(&node);
+
+  if (!__kmp_check_deps(gtid, &node, NULL, &current_task->td_dephash,
+                        DEP_BARRIER, ndeps, dep_list, ndeps_noalias,
+                        noalias_dep_list)) {
+    KA_TRACE(10, ("__kmpc_omp_taskwait_deps(exit): T#%d has no blocking "
+                  "dependences : loc=%p\n",
+                  gtid, loc_ref));
+#if OMPT_SUPPORT
+    __ompt_taskwait_dep_finish(current_task, taskwait_task_data);
+#endif /* OMPT_SUPPORT */
+    return;
+  }
+
+  int thread_finished = FALSE;
+  kmp_flag_32<false, false> flag(
+      (std::atomic<kmp_uint32> *)&node.dn.npredecessors, 0U);
+  while (node.dn.npredecessors > 0) {
+    flag.execute_tasks(thread, gtid, FALSE,
+                       &thread_finished USE_ITT_BUILD_ARG(NULL),
+                       __kmp_task_stealing_constraint);
+  }
+
+#if OMPT_SUPPORT
+  __ompt_taskwait_dep_finish(current_task, taskwait_task_data);
+#endif /* OMPT_SUPPORT */
+  KA_TRACE(10, ("__kmpc_omp_taskwait_deps(exit): T#%d finished waiting : loc=%p\
+                \n",
+                gtid, loc_ref));
+
+}

openmp/runtime/src/kmp_tasking.cpp

#if OMPT_SUPPORT
   }
 #endif
   return res;
 }
 
 template <bool ompt>
 static kmp_int32 __kmpc_omp_taskwait_template(ident_t *loc_ref, kmp_int32 gtid,
                                               void *frame_address,
-                                              void *return_address) {
+                                              void *return_address,
+                                              kmp_int32 have_no_wait) {
   kmp_taskdata_t *taskdata = nullptr;
   kmp_info_t *thread;
   int thread_finished = FALSE;
   KMP_SET_THREAD_STATE_BLOCK(TASKWAIT);
 
   KA_TRACE(10, ("__kmpc_omp_taskwait(enter): T#%d loc=%p\n", gtid, loc_ref));
   KMP_DEBUG_ASSERT(gtid >= 0);
 
#endif // OMPT_SUPPORT && OMPT_OPTIONAL
 
   return TASK_CURRENT_NOT_QUEUED;
 }
 
 #if OMPT_SUPPORT && OMPT_OPTIONAL
 OMPT_NOINLINE
 static kmp_int32 __kmpc_omp_taskwait_ompt(ident_t *loc_ref, kmp_int32 gtid,
                                           void *frame_address,
-                                          void *return_address) {
+                                          void *return_address,
+                                          kmp_int32 have_no_wait) {
   return __kmpc_omp_taskwait_template<true>(loc_ref, gtid, frame_address,
-                                            return_address);
+                                            return_address, have_no_wait);
 }
 #endif // OMPT_SUPPORT && OMPT_OPTIONAL
 
 // __kmpc_omp_taskwait: Wait until all tasks generated by the current task are
 // complete
 kmp_int32 __kmpc_omp_taskwait(ident_t *loc_ref, kmp_int32 gtid) {
 #if OMPT_SUPPORT && OMPT_OPTIONAL
   if (UNLIKELY(ompt_enabled.enabled)) {
     OMPT_STORE_RETURN_ADDRESS(gtid);
     return __kmpc_omp_taskwait_ompt(loc_ref, gtid, OMPT_GET_FRAME_ADDRESS(0),
-                                    OMPT_LOAD_RETURN_ADDRESS(gtid));
+                                    OMPT_LOAD_RETURN_ADDRESS(gtid), 0);
+  }
+#endif
+  return __kmpc_omp_taskwait_template<false>(loc_ref, gtid, NULL, NULL, 0);
+}
+
+/* __kmpc_omp_taskwait_51 : Function for OpenMP 5.1 nowait clause.
+ *                          Placeholder for taskwait with nowait clause.
+ *                          The code is a copy of __kmpc_omp_taskwait.*/
+kmp_int32 __kmpc_omp_taskwait_51(ident_t *loc_ref, kmp_int32 gtid,
+                                 kmp_int32 have_no_wait) {
+#if OMPT_SUPPORT && OMPT_OPTIONAL
+  if (UNLIKELY(ompt_enabled.enabled)) {
+    OMPT_STORE_RETURN_ADDRESS(gtid);
+    return __kmpc_omp_taskwait_ompt(loc_ref, gtid, OMPT_GET_FRAME_ADDRESS(0),
+                                    OMPT_LOAD_RETURN_ADDRESS(gtid),
+                                    have_no_wait);
   }
 #endif
-  return __kmpc_omp_taskwait_template<false>(loc_ref, gtid, NULL, NULL);
+  return __kmpc_omp_taskwait_template<false>(loc_ref, gtid, NULL, NULL,
+                                             have_no_wait);
 }
 
 // __kmpc_omp_taskyield: switch to a different task
 kmp_int32 __kmpc_omp_taskyield(ident_t *loc_ref, kmp_int32 gtid, int end_part) {
   kmp_taskdata_t *taskdata = NULL;
   kmp_info_t *thread;
   int thread_finished = FALSE;