Fix OMPT support for task frames for parallel regions and parallel regions + loops

[jmellorcrummey]

Summary

This patch makes it possible for a performance tool that uses call stack unwinding to map implementation-level call stacks from master and worker threads into a unified global view. There are several components to this patch.

include/*/ompt.h.var

• Add a new enumeration type that indicates whether the code for a master task for a parallel region is invoked by the user program or the runtime system
• Change the signature for OMPT parallel begin/end callbacks to indicate whether the master task will be invoked by the program or the runtime system. This enables a performance tool using call stack unwinding to handle these two cases differently. For this case, profiler that uses call stack unwinding needs to know that the call path prefix for the master task may differ from those available within the begin/end callbacks if the program invokes tha master.

kmp.h

• Change the signature for __kmp_join_call to take an additional parameter indicating the fork_context type. This is needed to supply the OMPT parallel end callback with information about whether the compiler or the runtime invoked the master task for a parallel region.

kmp_csupport.c

• Ensure that the OMPT task frame field reenter_runtime_frame is properly set and cleared before and after calls to fork and join threads for a parallel region.
• Adjust the code for the new signature for __kmp_join_call.
• Adjust the OMPT parallel begin callback invocations to carry the extra parameter indicating whether the program or the runtime invokes the master task for a parallel region.

kmp_gsupport.c

• Apply all of the analogous changes described for kmp_csupport.c for the GOMP interface
• Add OMPT support for the GOMP combined parallel region + loop API to maintain the OMPT task frame field reenter_runtime_frame.

kmp_runtime.c:

• Use the new information passed by __kmp_join_call to adjust the OMPT parallel end callback invocations to carry the extra parameter indicating whether the program or the runtime invokes the master task for a parallel region.

ompt_internal.h:

• Use the flavor of the parallel region API (GNU or Intel) to determine who invokes the master task.

[Hahnfeld]

Besides the #if 0 around the assert only some minor issues about indention. I don't know how rigid we should be with this...

runtime/src/kmp_gsupport.c

493

Indention

522

Looks like 3 space indention while there is 4 space indention around...

546

Indention

554

Indention

runtime/src/kmp_runtime.c

2094

This probably shouldn't be here, it's part of another issue...

[jmellorcrummey]

runtime/src/kmp_gsupport.c

554

The indentation anomalies came from working with vi, which used tabs rather than spaces. I can update the diff if this warrants it.

runtime/src/kmp_runtime.c

2094

I don't quite know how to handle this. The runtime works though I have test codes that cause the assert to trip. In my development version, I patched out the assert pending patches from Intel. Is leaving in the broken assert preferable? Should I delete the assert entirely and let Intel restore a variant of it later after they have altered the runtime to ensure that the condition is true?

[AndreyChurbanov]

runtime/src/kmp_runtime.c

2094

John, I think that either #if0 or temporarily removing the assertion should work fine. But please do this as a separate patch, because it is indeed not related to the OMPT changes. We will then restore the assertion with the patch that fixes the reason of it.

[jmellorcrummey]

Address concerns about indentation.

Remove #if 0 around a problematic assertion, which is a separate issue from the purpose of this patch.

[Hahnfeld]

LGTM (if I'm allowed to say...)

No regressions with test suite and finally fixes some of the errors with GCC and ompt_get_task_frame.

runtime/src/include/30/ompt.h.var

#define FOREACH_OMPT_STATE(macro)                                                               \
     /* misc (112..127) */                                                                       \
     macro (ompt_state_undefined, 0x70)       /* undefined thread state */
 
 
 #define FOREACH_OMPT_EVENT(macro)                                                                               \
                                                                                                                 \
     /*--- Mandatory Events ---*/                                                                                \
     macro (ompt_event_parallel_begin,           ompt_new_parallel_callback_t,   1) /* parallel begin */         \
-    macro (ompt_event_parallel_end,             ompt_parallel_callback_t,       2) /* parallel end */           \
+    macro (ompt_event_parallel_end,             ompt_end_parallel_callback_t,   2) /* parallel end */           \
                                                                                                                 \
     macro (ompt_event_task_begin,               ompt_new_task_callback_t,       3) /* task begin */             \
     macro (ompt_event_task_end,                 ompt_task_callback_t,           4) /* task destroy */           \
                                                                                                                 \
     macro (ompt_event_thread_begin,             ompt_thread_type_callback_t,    5) /* thread begin */           \
     macro (ompt_event_thread_end,               ompt_thread_type_callback_t,    6) /* thread end */             \
                                                                                                                 \
     macro (ompt_event_control,                  ompt_control_callback_t,        7) /* support control calls */  \

 );
 
 typedef enum {
     ompt_thread_initial = 1, // start the enumeration at 1
     ompt_thread_worker  = 2,
     ompt_thread_other   = 3
 } ompt_thread_type_t;
 
+typedef enum {
+    ompt_invoker_program = 0,         /* program invokes master task  */
+    ompt_invoker_runtime = 1          /* runtime invokes master task  */
+} ompt_invoker_t;
+
 typedef void (*ompt_thread_type_callback_t) (
     ompt_thread_type_t thread_type,   /* type of thread               */
     ompt_thread_id_t thread_id        /* ID of thread                 */
 );
 
 typedef void (*ompt_wait_callback_t) (
     ompt_wait_id_t wait_id            /* wait id                      */
 );
typedef void (*ompt_new_workshare_callback_t) (
     void *workshare_function          /* pointer to outlined function */
 );
 
 typedef void (*ompt_new_parallel_callback_t) (
     ompt_task_id_t parent_task_id,    /* id of parent task            */
     ompt_frame_t *parent_task_frame,  /* frame data of parent task    */
     ompt_parallel_id_t parallel_id,   /* id of parallel region        */
     uint32_t requested_team_size,     /* number of threads in team    */
-    void *parallel_function           /* pointer to outlined function */
+    void *parallel_function,          /* pointer to outlined function */
+    ompt_invoker_t invoker            /* who invokes master task?     */
+);
+
+typedef void (*ompt_end_parallel_callback_t) (
+    ompt_parallel_id_t parallel_id,   /* id of parallel region       */
+    ompt_task_id_t task_id,           /* id of task                  */
+    ompt_invoker_t invoker            /* who invokes master task?    */ 
 );
 
 /* tasks */
 typedef void (*ompt_task_callback_t) (
     ompt_task_id_t task_id            /* id of task                   */
 );
 
 typedef void (*ompt_task_switch_callback_t) (

runtime/src/include/40/ompt.h.var

#define FOREACH_OMPT_STATE(macro)                                                               \
     /* misc (112..127) */                                                                       \
     macro (ompt_state_undefined, 0x70)       /* undefined thread state */
 
 
 #define FOREACH_OMPT_EVENT(macro)                                                                               \
                                                                                                                 \
     /*--- Mandatory Events ---*/                                                                                \
     macro (ompt_event_parallel_begin,           ompt_new_parallel_callback_t,   1) /* parallel begin */         \
-    macro (ompt_event_parallel_end,             ompt_parallel_callback_t,       2) /* parallel end */           \
+    macro (ompt_event_parallel_end,             ompt_end_parallel_callback_t,   2) /* parallel end */           \
                                                                                                                 \
     macro (ompt_event_task_begin,               ompt_new_task_callback_t,       3) /* task begin */             \
     macro (ompt_event_task_end,                 ompt_task_callback_t,           4) /* task destroy */           \
                                                                                                                 \
     macro (ompt_event_thread_begin,             ompt_thread_type_callback_t,    5) /* thread begin */           \
     macro (ompt_event_thread_end,               ompt_thread_type_callback_t,    6) /* thread end */             \
                                                                                                                 \
     macro (ompt_event_control,                  ompt_control_callback_t,        7) /* support control calls */  \

 );
 
 typedef enum {
     ompt_thread_initial = 1, // start the enumeration at 1
     ompt_thread_worker  = 2,
     ompt_thread_other   = 3
 } ompt_thread_type_t;
 
+typedef enum {
+    ompt_invoker_program = 0,         /* program invokes master task  */
+    ompt_invoker_runtime = 1          /* runtime invokes master task  */
+} ompt_invoker_t;
+
 typedef void (*ompt_thread_type_callback_t) (
     ompt_thread_type_t thread_type,   /* type of thread               */
     ompt_thread_id_t thread_id        /* ID of thread                 */
 );
 
 typedef void (*ompt_wait_callback_t) (
     ompt_wait_id_t wait_id            /* wait id                      */
 );
typedef void (*ompt_new_workshare_callback_t) (
     void *workshare_function          /* pointer to outlined function */
 );
 
 typedef void (*ompt_new_parallel_callback_t) (
     ompt_task_id_t parent_task_id,    /* id of parent task            */
     ompt_frame_t *parent_task_frame,  /* frame data of parent task    */
     ompt_parallel_id_t parallel_id,   /* id of parallel region        */
     uint32_t requested_team_size,     /* number of threads in team    */
-    void *parallel_function           /* pointer to outlined function */
+    void *parallel_function,          /* pointer to outlined function */
+    ompt_invoker_t invoker            /* who invokes master task?     */
+);
+
+typedef void (*ompt_end_parallel_callback_t) (
+    ompt_parallel_id_t parallel_id,   /* id of parallel region       */
+    ompt_task_id_t task_id,           /* id of task                  */
+    ompt_invoker_t invoker            /* who invokes master task?    */ 
 );
 
 /* tasks */
 typedef void (*ompt_task_callback_t) (
     ompt_task_id_t task_id            /* id of task                   */
 );
 
 typedef void (*ompt_task_switch_callback_t) (

runtime/src/include/41/ompt.h.var

#define FOREACH_OMPT_STATE(macro)                                                               \
     /* misc (112..127) */                                                                       \
     macro (ompt_state_undefined, 0x70)       /* undefined thread state */
 
 
 #define FOREACH_OMPT_EVENT(macro)                                                                               \
                                                                                                                 \
     /*--- Mandatory Events ---*/                                                                                \
     macro (ompt_event_parallel_begin,           ompt_new_parallel_callback_t,   1) /* parallel begin */         \
-    macro (ompt_event_parallel_end,             ompt_parallel_callback_t,       2) /* parallel end */           \
+    macro (ompt_event_parallel_end,             ompt_end_parallel_callback_t,   2) /* parallel end */           \
                                                                                                                 \
     macro (ompt_event_task_begin,               ompt_new_task_callback_t,       3) /* task begin */             \
     macro (ompt_event_task_end,                 ompt_task_callback_t,           4) /* task destroy */           \
                                                                                                                 \
     macro (ompt_event_thread_begin,             ompt_thread_type_callback_t,    5) /* thread begin */           \
     macro (ompt_event_thread_end,               ompt_thread_type_callback_t,    6) /* thread end */             \
                                                                                                                 \
     macro (ompt_event_control,                  ompt_control_callback_t,        7) /* support control calls */  \

 );
 
 typedef enum {
     ompt_thread_initial = 1, // start the enumeration at 1
     ompt_thread_worker  = 2,
     ompt_thread_other   = 3
 } ompt_thread_type_t;
 
+typedef enum {
+    ompt_invoker_program = 0,         /* program invokes master task  */
+    ompt_invoker_runtime = 1          /* runtime invokes master task  */
+} ompt_invoker_t;
+
 typedef void (*ompt_thread_type_callback_t) (
     ompt_thread_type_t thread_type,   /* type of thread               */
     ompt_thread_id_t thread_id        /* ID of thread                 */
 );
 
 typedef void (*ompt_wait_callback_t) (
     ompt_wait_id_t wait_id            /* wait id                      */
 );
typedef void (*ompt_new_workshare_callback_t) (
     void *workshare_function          /* pointer to outlined function */
 );
 
 typedef void (*ompt_new_parallel_callback_t) (
     ompt_task_id_t parent_task_id,    /* id of parent task            */
     ompt_frame_t *parent_task_frame,  /* frame data of parent task    */
     ompt_parallel_id_t parallel_id,   /* id of parallel region        */
     uint32_t requested_team_size,     /* number of threads in team    */
-    void *parallel_function           /* pointer to outlined function */
+    void *parallel_function,          /* pointer to outlined function */
+    ompt_invoker_t invoker            /* who invokes master task?    */ 
+);
+
+typedef void (*ompt_end_parallel_callback_t) (
+    ompt_parallel_id_t parallel_id,   /* id of parallel region       */
+    ompt_task_id_t task_id,           /* id of task                  */
+    ompt_invoker_t invoker            /* who invokes master task?    */ 
 );
 
 /* tasks */
 typedef void (*ompt_task_callback_t) (
     ompt_task_id_t task_id            /* id of task                   */
 );
 
 typedef void (*ompt_task_switch_callback_t) (

runtime/src/kmp.h

 /* TODO: revert workaround for Intel(R) 64 tracker #96 */
 #if (KMP_ARCH_ARM || KMP_ARCH_X86_64 || KMP_ARCH_AARCH64) && KMP_OS_LINUX
                              va_list *ap
 #else
                              va_list ap
 #endif
                              );
 
-extern void __kmp_join_call( ident_t *loc, int gtid
+extern void __kmp_join_call( ident_t *loc, int gtid, enum fork_context_e fork_context
 #if OMP_40_ENABLED
                            , int exit_teams = 0
 #endif
                            );
 
 extern void __kmp_serialized_parallel(ident_t *id, kmp_int32 gtid);
 extern void __kmp_internal_fork( ident_t *id, int gtid, kmp_team_t *team );
 extern void __kmp_internal_join( ident_t *id, int gtid, kmp_team_t *team );

runtime/src/kmp_csupport.c

__kmpc_fork_call(ident_t *loc, kmp_int32 argc, kmpc_micro microtask, ...)
   KMP_COUNT_BLOCK(OMP_PARALLEL);
   int         gtid = __kmp_entry_gtid();
   // maybe to save thr_state is enough here
   {
     va_list     ap;
     va_start(   ap, microtask );
 
 #if OMPT_SUPPORT
+    int tid = __kmp_tid_from_gtid( gtid );
     kmp_info_t *master_th = __kmp_threads[ gtid ];
     kmp_team_t *parent_team = master_th->th.th_team;
-    int tid = __kmp_tid_from_gtid( gtid );
+    if (ompt_status & ompt_status_track) {
        parent_team->t.t_implicit_task_taskdata[tid].
            ompt_task_info.frame.reenter_runtime_frame = __builtin_frame_address(0);
+    }
 #endif
 
 #if INCLUDE_SSC_MARKS
     SSC_MARK_FORKING();
 #endif
     __kmp_fork_call( loc, gtid, fork_context_intel,
             argc,
 #if OMPT_SUPPORT
             VOLATILE_CAST(void *) microtask,      // "unwrapped" task
 #endif
             VOLATILE_CAST(microtask_t) microtask, // "wrapped" task
             VOLATILE_CAST(launch_t)    __kmp_invoke_task_func,
 /* TODO: revert workaround for Intel(R) 64 tracker #96 */
 #if (KMP_ARCH_X86_64 || KMP_ARCH_ARM || KMP_ARCH_AARCH64) && KMP_OS_LINUX
             &ap
 #else
             ap
 #endif
             );
 #if INCLUDE_SSC_MARKS
     SSC_MARK_JOINING();
 #endif
-    __kmp_join_call( loc, gtid );
+    __kmp_join_call( loc, gtid, fork_context_intel );
 
     va_end( ap );
 
 #if OMPT_SUPPORT
     if (ompt_status & ompt_status_track) {
         parent_team->t.t_implicit_task_taskdata[tid].
             ompt_task_info.frame.reenter_runtime_frame = 0;
     }
__kmpc_fork_teams(ident_t *loc, kmp_int32 argc, kmpc_micro microtask, ...)
     kmp_info_t *this_thr = __kmp_threads[ gtid ];
     va_list     ap;
     va_start(   ap, microtask );
 
     // remember teams entry point and nesting level
     this_thr->th.th_teams_microtask = microtask;
     this_thr->th.th_teams_level = this_thr->th.th_team->t.t_level; // AC: can be >0 on host
 
+#if OMPT_SUPPORT
+    kmp_team_t *parent_team = this_thr->th.th_team;
+    int tid = __kmp_tid_from_gtid( gtid );
+    if (ompt_status & ompt_status_track) {
+        parent_team->t.t_implicit_task_taskdata[tid].
+           ompt_task_info.frame.reenter_runtime_frame = __builtin_frame_address(0);
+    }
+#endif
+
     // check if __kmpc_push_num_teams called, set default number of teams otherwise
     if ( this_thr->th.th_teams_size.nteams == 0 ) {
         __kmp_push_num_teams( loc, gtid, 0, 0 );
     }
     KMP_DEBUG_ASSERT(this_thr->th.th_set_nproc >= 1);
     KMP_DEBUG_ASSERT(this_thr->th.th_teams_size.nteams >= 1);
     KMP_DEBUG_ASSERT(this_thr->th.th_teams_size.nth >= 1);
 
     __kmp_fork_call( loc, gtid, fork_context_intel,
             argc,
 #if OMPT_SUPPORT
             VOLATILE_CAST(void *) microtask,               // "unwrapped" task
 #endif
             VOLATILE_CAST(microtask_t) __kmp_teams_master, // "wrapped" task
             VOLATILE_CAST(launch_t)    __kmp_invoke_teams_master,
 #if (KMP_ARCH_X86_64 || KMP_ARCH_ARM || KMP_ARCH_AARCH64) && KMP_OS_LINUX
             &ap
 #else
             ap
 #endif
             );
-    __kmp_join_call( loc, gtid );
+    __kmp_join_call( loc, gtid, fork_context_intel );
+
+#if OMPT_SUPPORT
+    if (ompt_status & ompt_status_track) {
+        parent_team->t.t_implicit_task_taskdata[tid].
+           ompt_task_info.frame.reenter_runtime_frame = NULL;
+    }
+#endif
+
     this_thr->th.th_teams_microtask = NULL;
     this_thr->th.th_teams_level = 0;
     *(kmp_int64*)(&this_thr->th.th_teams_size) = 0L;
     va_end( ap );
 }
 #endif /* OMP_40_ENABLED */
 
 

runtime/src/kmp_gsupport.c

if (ompt_status & ompt_status_track) {
         ompt_frame->exit_runtime_frame = NULL;
 
         // parallel region callback
         if ((ompt_status == ompt_status_track_callback) &&
             ompt_callbacks.ompt_callback(ompt_event_parallel_begin)) {
             int team_size = 1;
             ompt_callbacks.ompt_callback(ompt_event_parallel_begin)(
                 ompt_task_id, ompt_frame, ompt_parallel_id,
-                team_size, (void *) task);
+                team_size, (void *) task,
+                OMPT_INVOKER(fork_context_gnu));
         }
 
         // set up lightweight task
         ompt_lw_taskteam_t *lwt = (ompt_lw_taskteam_t *)
             __kmp_allocate(sizeof(ompt_lw_taskteam_t));
         __ompt_lw_taskteam_init(lwt, thr, gtid, (void *) task, ompt_parallel_id);
         lwt->ompt_task_info.task_id = my_ompt_task_id;
         lwt->ompt_task_info.frame.exit_runtime_frame = 0;

 #if OMPT_SUPPORT
     ompt_parallel_id_t parallel_id;
     ompt_frame_t *ompt_frame = NULL;
 
     if (ompt_status & ompt_status_track) {
         ompt_team_info_t *team_info = __ompt_get_teaminfo(0, NULL);
         parallel_id = team_info->parallel_id;
 
+  // Record that we re-entered the runtime system in the implicit
+  // task frame representing the parallel region. 
         ompt_frame = __ompt_get_task_frame_internal(0);
-        ompt_frame->exit_runtime_frame = __builtin_frame_address(0);
+        ompt_frame->reenter_runtime_frame = __builtin_frame_address(0);
 
 #if OMPT_TRACE
         if ((ompt_status == ompt_status_track_callback) &&
             ompt_callbacks.ompt_callback(ompt_event_implicit_task_end)) {
             ompt_task_info_t *task_info = __ompt_get_taskinfo(0);
             ompt_callbacks.ompt_callback(ompt_event_implicit_task_end)(
                 parallel_id, task_info->task_id);
         }
 #endif
 
         // unlink if necessary. no-op if there is not a lightweight task.
         ompt_lw_taskteam_t *lwt = __ompt_lw_taskteam_unlink(thr);
         // GOMP allocates/frees lwt since it can't be kept on the stack
-        if (lwt) __kmp_free(lwt);
+        if (lwt) { 
+           __kmp_free(lwt);
+      
+#if OMPT_SUPPORT
+           if (ompt_status & ompt_status_track) {
+        // Since a lightweight task was destroyed, make sure that the
+        // remaining deepest task knows the stack frame where the runtime 
+        // was reentered.
+              ompt_frame = __ompt_get_task_frame_internal(0);
+              ompt_frame->reenter_runtime_frame = __builtin_frame_address(0);
+           }
+#endif
+        }
+
     }
 #endif
 
     if (! __kmp_threads[gtid]->th.th_team->t.t_serialized) {
         kmp_info_t *thr = __kmp_threads[gtid];
         __kmp_run_after_invoked_task(gtid, __kmp_tid_from_gtid(gtid), thr,
           thr->th.th_team);
-        __kmp_join_call(&loc, gtid);
+
+#if OMPT_SUPPORT
+        if (ompt_status & ompt_status_track) {
+          // Set reenter frame in parent task, which will become current task
+          // in the midst of join. This is needed before the end_parallel callback.
+          ompt_frame = __ompt_get_task_frame_internal(1);
+          ompt_frame->reenter_runtime_frame = __builtin_frame_address(0);
+        }
+#endif
+
+        __kmp_join_call(&loc, gtid, fork_context_gnu);
+
+#if OMPT_SUPPORT
+        if (ompt_status & ompt_status_track) {
+          ompt_frame->reenter_runtime_frame = NULL;
+        }
+#endif
     }
     else {
         __kmpc_end_serialized_parallel(&loc, gtid);
 
 #if OMPT_SUPPORT
         if (ompt_status & ompt_status_track) {
+      // Record that we re-entered the runtime system in the frame that 
+      // created the parallel region.
+            ompt_frame->reenter_runtime_frame = __builtin_frame_address(0);
+
             if ((ompt_status == ompt_status_track_callback) &&
                 ompt_callbacks.ompt_callback(ompt_event_parallel_end)) {
                 ompt_task_info_t *task_info = __ompt_get_taskinfo(0);
                 ompt_callbacks.ompt_callback(ompt_event_parallel_end)(
-                    parallel_id, task_info->task_id);
+                    parallel_id, task_info->task_id, 
+                    OMPT_INVOKER(fork_context_gnu));
             }
 
+            ompt_frame->reenter_runtime_frame = NULL;
+
             thr->th.ompt_thread_info.state =
                 (((thr->th.th_team)->t.t_serialized) ?
                 ompt_state_work_serial : ompt_state_work_parallel);
         }
 #endif
 
     }
 }

 
 
 //
 // Combined parallel / loop worksharing constructs
 //
 // There are no ull versions (yet).
 //
 
-#define PARALLEL_LOOP_START(func, schedule) \
+#define PARALLEL_LOOP_START(func, schedule, ompt_pre, ompt_post) \
     void func (void (*task) (void *), void *data, unsigned num_threads,      \
       long lb, long ub, long str, long chunk_sz)                             \
     {                                                                        \
         int gtid = __kmp_entry_gtid();                                       \
         MKLOC(loc, #func);                                                   \
         KA_TRACE(20, ( #func ": T#%d, lb 0x%lx, ub 0x%lx, str 0x%lx, chunk_sz 0x%lx\n",        \
           gtid, lb, ub, str, chunk_sz ));                                    \
                                                                              \
+        ompt_pre();                                                          \
+                                                                             \
         if (__kmpc_ok_to_fork(&loc) && (num_threads != 1)) {                 \
             if (num_threads != 0) {                                          \
                 __kmp_push_num_threads(&loc, gtid, num_threads);             \
             }                                                                \
             __kmp_GOMP_fork_call(&loc, gtid, task,                           \
               (microtask_t)__kmp_GOMP_parallel_microtask_wrapper, 9,         \
               task, data, num_threads, &loc, (schedule), lb,                 \
               (str > 0) ? (ub - 1) : (ub + 1), str, chunk_sz);               \
         }                                                                    \
         else {                                                               \
             __kmp_GOMP_serialized_parallel(&loc, gtid, task);                \
         }                                                                    \
                                                                              \
         KMP_DISPATCH_INIT(&loc, gtid, (schedule), lb,                        \
           (str > 0) ? (ub - 1) : (ub + 1), str, chunk_sz,                    \
           (schedule) != kmp_sch_static);                                     \
                                                                              \
+        ompt_post();                                                         \
+                                                                             \
         KA_TRACE(20, ( #func " exit: T#%d\n", gtid));                        \
     }
 
 
-PARALLEL_LOOP_START(xexpand(KMP_API_NAME_GOMP_PARALLEL_LOOP_STATIC_START), kmp_sch_static)
-PARALLEL_LOOP_START(xexpand(KMP_API_NAME_GOMP_PARALLEL_LOOP_DYNAMIC_START), kmp_sch_dynamic_chunked)
-PARALLEL_LOOP_START(xexpand(KMP_API_NAME_GOMP_PARALLEL_LOOP_GUIDED_START), kmp_sch_guided_chunked)
-PARALLEL_LOOP_START(xexpand(KMP_API_NAME_GOMP_PARALLEL_LOOP_RUNTIME_START), kmp_sch_runtime)
+
+#if OMPT_SUPPORT
+
+#define OMPT_LOOP_PRE() \
+    ompt_frame_t *parent_frame; \
+    if (ompt_status & ompt_status_track) { \
+        parent_frame = __ompt_get_task_frame_internal(0); \
+        parent_frame->reenter_runtime_frame = __builtin_frame_address(0); \
+    }
+
+
+#define OMPT_LOOP_POST() \
+    if (ompt_status & ompt_status_track) { \
+        parent_frame->reenter_runtime_frame = NULL; \
+    }
+
+#else
+
+#define OMPT_LOOP_PRE() 
+
+#define OMPT_LOOP_POST()
+
+#endif
+
+
+PARALLEL_LOOP_START(xexpand(KMP_API_NAME_GOMP_PARALLEL_LOOP_STATIC_START), 
+                    kmp_sch_static, OMPT_LOOP_PRE, OMPT_LOOP_POST)
+PARALLEL_LOOP_START(xexpand(KMP_API_NAME_GOMP_PARALLEL_LOOP_DYNAMIC_START), 
+                    kmp_sch_dynamic_chunked, OMPT_LOOP_PRE, OMPT_LOOP_POST)
+PARALLEL_LOOP_START(xexpand(KMP_API_NAME_GOMP_PARALLEL_LOOP_GUIDED_START), 
+                    kmp_sch_guided_chunked, OMPT_LOOP_PRE, OMPT_LOOP_POST)
+PARALLEL_LOOP_START(xexpand(KMP_API_NAME_GOMP_PARALLEL_LOOP_RUNTIME_START), 
+                    kmp_sch_runtime, OMPT_LOOP_PRE, OMPT_LOOP_POST)
 
 
 //
 // Tasking constructs
 //
 
 void
 xexpand(KMP_API_NAME_GOMP_TASK)(void (*func)(void *), void *data, void (*copy_func)(void *, void *),

runtime/src/kmp_runtime.c

 
 #if OMPT_SUPPORT
     if ((ompt_status == ompt_status_track_callback) &&
         ompt_callbacks.ompt_callback(ompt_event_parallel_begin)) {
         int team_size = master_set_numthreads;
 
         ompt_callbacks.ompt_callback(ompt_event_parallel_begin)(
             ompt_task_id, ompt_frame, ompt_parallel_id,
-            team_size, unwrapped_task);
+            team_size, unwrapped_task, OMPT_INVOKER(call_context));
     }
 #endif
 
     master_th->th.th_ident = loc;
 
 #if OMP_40_ENABLED
     if ( master_th->th.th_teams_microtask &&
          ap && microtask != (microtask_t)__kmp_teams_master && level == teams_level ) {
#if OMPT_TRACE
                 __ompt_lw_taskteam_unlink(master_th);
                 // reset clear the task id only after unlinking the task
                 lw_taskteam.ompt_task_info.task_id = ompt_task_id_none;
 #endif
 
                 if ((ompt_status == ompt_status_track_callback) &&
                     ompt_callbacks.ompt_callback(ompt_event_parallel_end)) {
                     ompt_callbacks.ompt_callback(ompt_event_parallel_end)(
-                        ompt_parallel_id, ompt_task_id);
+                        ompt_parallel_id, ompt_task_id,
+                        OMPT_INVOKER(call_context));
                 }
                 master_th->th.ompt_thread_info.state = ompt_state_overhead;
             }
 #endif
             return TRUE;
         }
 
         parent_team->t.t_pkfn  = microtask;
#endif
 
                     __ompt_lw_taskteam_unlink(master_th);
                     // reset clear the task id only after unlinking the task
                     lw_taskteam.ompt_task_info.task_id = ompt_task_id_none;
 
                     if ((ompt_status == ompt_status_track_callback) &&
                         ompt_callbacks.ompt_callback(ompt_event_parallel_end)) {
                         ompt_callbacks.ompt_callback(ompt_event_parallel_end)(
-                            ompt_parallel_id, ompt_task_id);
+                            ompt_parallel_id, ompt_task_id,
+                            OMPT_INVOKER(call_context));
                     }
                     master_th->th.ompt_thread_info.state = ompt_state_overhead;
                 }
 #endif
             } else if ( microtask == (microtask_t)__kmp_teams_master ) {
                 KMP_DEBUG_ASSERT( master_th->th.th_team == master_th->th.th_serial_team );
                 team = master_th->th.th_team;
                 //team->t.t_pkfn = microtask;
#endif
 
                     __ompt_lw_taskteam_unlink(master_th);
                     // reset clear the task id only after unlinking the task
                     lw_taskteam.ompt_task_info.task_id = ompt_task_id_none;
 
                     if ((ompt_status == ompt_status_track_callback) &&
                         ompt_callbacks.ompt_callback(ompt_event_parallel_end)) {
                         ompt_callbacks.ompt_callback(ompt_event_parallel_end)(
-                            ompt_parallel_id, ompt_task_id);
+                            ompt_parallel_id, ompt_task_id,
+                            OMPT_INVOKER(call_context));
                     }
                     master_th->th.ompt_thread_info.state = ompt_state_overhead;
                 }
 #endif
 #if OMP_40_ENABLED
             }
 #endif /* OMP_40_ENABLED */
         }

 #endif /* OMP_40_ENABLED */
     team->t.t_sched      = get__sched_2(parent_team, master_tid); // set master's schedule as new run-time schedule
 
     // Update the floating point rounding in the team if required.
     propagateFPControl(team);
 
     if ( __kmp_tasking_mode != tskm_immediate_exec ) {
         // Set master's task team to team's task team. Unless this is hot team, it should be NULL.
+#if 0
         KMP_DEBUG_ASSERT(master_th->th.th_task_team == parent_team->t.t_task_team[master_th->th.th_task_state]);
+#endif
         KA_TRACE( 20, ( "__kmp_fork_call: Master T#%d pushing task_team %p / team %p, new task_team %p / team %p\n",
                       __kmp_gtid_from_thread( master_th ), master_th->th.th_task_team,
                       parent_team, team->t.t_task_team[master_th->th.th_task_state], team ) );
         if (level) {
             // Take a memo of master's task_state
             KMP_DEBUG_ASSERT(master_th->th.th_task_state_memo_stack);
             if (master_th->th.th_task_state_top >= master_th->th.th_task_state_stack_sz) { // increase size
                 kmp_uint8 *old_stack, *new_stack = (kmp_uint8 *) __kmp_allocate( 2*master_th->th.th_task_state_stack_sz );
__kmp_join_restore_state(
     thread->th.ompt_thread_info.state = ((team->t.t_serialized) ?
         ompt_state_work_serial : ompt_state_work_parallel);
 }
 
 static inline void
 __kmp_join_ompt(
     kmp_info_t *thread,
     kmp_team_t *team,
-    ompt_parallel_id_t parallel_id)
+    ompt_parallel_id_t parallel_id,
+    fork_context_e fork_context)
 {
     if (ompt_callbacks.ompt_callback(ompt_event_parallel_end)) {
         ompt_task_info_t *task_info = __ompt_get_taskinfo(0);
         ompt_callbacks.ompt_callback(ompt_event_parallel_end)(
-            parallel_id, task_info->task_id);
+            parallel_id, task_info->task_id, OMPT_INVOKER(fork_context));
     }
 
     __kmp_join_restore_state(thread,team);
 }
 #endif
 
 void
-__kmp_join_call(ident_t *loc, int gtid
+__kmp_join_call(ident_t *loc, int gtid, enum fork_context_e fork_context
 #if OMP_40_ENABLED
                , int exit_teams
 #endif /* OMP_40_ENABLED */
 )
 {
     KMP_TIME_BLOCK(KMP_join_call);
     kmp_team_t     *team;
     kmp_team_t     *parent_team;
#endif
                     // Synchronize thread's task state
                     other_threads[i]->th.th_task_state = master_th->th.th_task_state;
                 }
             }
         }
 
 #if OMPT_SUPPORT
         if (ompt_status == ompt_status_track_callback) {
-            __kmp_join_ompt(master_th, parent_team, parallel_id);
+            __kmp_join_ompt(master_th, parent_team, parallel_id, fork_context);
         }
 #endif
 
         return;
     }
 #endif /* OMP_40_ENABLED */
 
     /* do cleanup and restore the parent team */
#endif /* OMP_40_ENABLED */
      // TODO: GEH - cannot do this assertion because root thread not set up as executing
      // KMP_ASSERT( master_th->th.th_current_task->td_flags.executing == 0 );
      master_th->th.th_current_task->td_flags.executing = 1;
 
     __kmp_release_bootstrap_lock( &__kmp_forkjoin_lock );
 
 #if OMPT_SUPPORT
     if (ompt_status == ompt_status_track_callback) {
-        __kmp_join_ompt(master_th, parent_team, parallel_id);
+        __kmp_join_ompt(master_th, parent_team, parallel_id, fork_context);
     }
 #endif
 
     KMP_MB();
     KA_TRACE( 20, ("__kmp_join_call: exit T#%d\n", gtid ));
 }
 
 /* ------------------------------------------------------------------------ */
#if OMPT_SUPPORT
             (void *)thr->th.th_teams_microtask,      // "unwrapped" task
 #endif
             (microtask_t)thr->th.th_teams_microtask, // "wrapped" task
             VOLATILE_CAST(launch_t) __kmp_invoke_task_func,
             NULL );
 #if INCLUDE_SSC_MARKS
     SSC_MARK_JOINING();
 #endif
-    __kmp_join_call( loc, gtid, 1 ); // AC: last parameter "1" eliminates join barrier which won't work because
-                                     // worker threads are in a fork barrier waiting for more parallel regions
+    
+    // AC: last parameter "1" eliminates join barrier which won't work because
+    __kmp_join_call( loc, gtid, fork_context_intel, 1 ); 
 }
 
 int
 __kmp_invoke_teams_master( int gtid )
 {
     kmp_info_t  *this_thr = __kmp_threads[ gtid ];
     kmp_team_t  *team     = this_thr->th.th_team;
     #if KMP_DEBUG

runtime/src/ompt-internal.h

 #ifndef __OMPT_INTERNAL_H__
 #define __OMPT_INTERNAL_H__
 
 #include "ompt.h"
 #include "ompt-event-specific.h"
 
 #define OMPT_VERSION 1
 
 #define _OMP_EXTERN extern "C"
 
+#define OMPT_INVOKER(x) \
+  ((x == fork_context_gnu) ? ompt_invoker_program : ompt_invoker_runtime)
 
 
 #define ompt_callback(e) e ## _callback
 
 /* track and track_callback share a bit so that one can test whether either is
  * set by anding a bit.
  */
 typedef enum {