[OpenMP] RISCV64 port

[fpallares]

Summary

This is a port of libomp for the RISC-V 64-bit Linux target.

We have tested this port on a HiFive Unleashed development board using a downstream LLVM that has support for the missing bits in upstream. As of now, all tests are passing, including OMPT.

[Hahnfeld]

Most of the changes look pretty straight forward to me, but I currently don't have time to read through the assembly implementation of __kmp_invoke_microtask. I left two comments inline to discuss compressed vs uncompressed encodings for testing OMPT.

runtime/test/ompt/callback.h

170 ↗

(On Diff #192441)

Should this also handle the case that the nop is uncompressed (this should be 4 bytes)?

172–173 ↗

(On Diff #192441)

Did you try to find out why the compiler inserts a jump?
And, as above, should be also allow the instruction to be 4 bytes long?

[fpallares]

Update print_possible_return_addresses macro in runtime/test/ompt/callback.h to handle uncompressed instructions, as pointed out by @Hahnfeld.

[fpallares]

Thanks for the quick review @Hahnfeld, I've just replied to your inline comments and updated the patch!

runtime/test/ompt/callback.h

170 ↗

(On Diff #192441)

As Linux requires riscv64gc we assumed compressed instructions would be enabled, but certainly it is not necessarily the case (riscv64g should be supported). Good catch!

172–173 ↗

(On Diff #192441)

The unnecessary jump to the immediate successor block is removed by FastIsel in the targets that enable it. For now RISCV64 does not.

[Hahnfeld]

Sorry for the delay...

runtime/src/z_Linux_asm.S

1720–1722 ↗

(On Diff #192598)

I don't think we need this, it's now handled at all call-sites of __kmp_invoke_microtask. I'll post a patch to resolve any inconsistencies.

AFAICT removing this should reduce some of the (conditional) complexity of the stack layout.

1759–1760 ↗

(On Diff #192598)

I'd propose to make them negative, the implementation for x86_64 does so as well.

1802–1805 ↗

(On Diff #192598)

For example, this can go away.

1874–1878 ↗

(On Diff #192598)

Not needed, as said above.

[fpallares]

Remove reset of exit_frame pointer for OMPT since it is now done at call-site and fix return address computation macro used for some tests (as the generated code has changed).

[fpallares]

Thanks @Hahnfeld for the review. I've just updated the patch.

[fpallares]

Now that D60456 has landed we believe this need not be WIP anymore.

[Hahnfeld]

The changes look good to me, thanks for your patience!

[fpallares]

I've tested this patch with current upstream LLVM and Clang on a HiFive Unleashed board and all tests are passing, including OMPT. I don't have commit access yet, @Hahnfeld would you mind commiting this for me? Thank you!

[Hahnfeld]

In D59880#1598584, @fpallares wrote:

I've tested this patch with current upstream LLVM and Clang on a HiFive Unleashed board and all tests are passing, including OMPT. I don't have commit access yet, @Hahnfeld would you mind commiting this for me? Thank you!

Sure, not problem. The commit should be on its way through the repositories.

openmp/README.rst

 
 **OPENMP_FILECHECK_EXECUTABLE** = ``/path/to/FileCheck``
   Specify full path to ``FileCheck`` executable for running tests.  The default
   is to search the ``PATH`` and the directory in **OPENMP_LLVM_TOOLS_DIR**.
 
 Options for ``libomp``
 ----------------------
 
-**LIBOMP_ARCH** = ``aarch64|arm|i386|mic|mips|mips64|ppc64|ppc64le|x86_64``
+**LIBOMP_ARCH** = ``aarch64|arm|i386|mic|mips|mips64|ppc64|ppc64le|x86_64|riscv64``
   The default value for this option is chosen based on probing the compiler for
   architecture macros (e.g., is ``__x86_64__`` predefined by compiler?).
 
 **LIBOMP_MIC_ARCH** = ``knc|knf``
   Intel(R) Many Integrated Core Architecture (Intel(R) MIC Architecture) to
   build for.  This value is ignored if **LIBOMP_ARCH** does not equal ``mic``.
 
 **LIBOMP_LIB_TYPE** = ``normal|profile|stubs``
**LIBOMP_USE_ADAPTIVE_LOCKS** = ``ON|OFF``
   by default for IA-32 architecture and Intel(R) 64 architecture.
 
 **LIBOMP_USE_INTERNODE_ALIGNMENT** = ``OFF|ON``
   Align certain data structures on 4096-byte.  This option is useful on
   multi-node systems where a small ``CACHE_LINE`` setting leads to false sharing.
 
 **LIBOMP_OMPT_SUPPORT** = ``ON|OFF``
   Include support for the OpenMP Tools Interface (OMPT).
-  This option is supported and ``ON`` by default for x86, x86_64, AArch64, and
-  PPC64 on Linux* and macOS*.
+  This option is supported and ``ON`` by default for x86, x86_64, AArch64,
+  PPC64 and RISCV64 on Linux* and macOS*.
   This option is ``OFF`` if this feature is not supported for the platform.
 
 **LIBOMP_OMPT_OPTIONAL** = ``ON|OFF``
   Include support for optional OMPT functionality.  This option is ignored if
   **LIBOMP_OMPT_SUPPORT** is ``OFF``.
 
 **LIBOMP_STATS** = ``OFF|ON``
   Include stats-gathering code.

openmp/runtime/CMakeLists.txt

 include(LibompHandleFlags)
 include(LibompDefinitions)
 
 # Determine the target architecture
 if(${OPENMP_STANDALONE_BUILD})
   # If adding a new architecture, take a look at cmake/LibompGetArchitecture.cmake
   libomp_get_architecture(LIBOMP_DETECTED_ARCH)
   set(LIBOMP_ARCH ${LIBOMP_DETECTED_ARCH} CACHE STRING
-    "The architecture to build for (x86_64/i386/arm/ppc64/ppc64le/aarch64/mic/mips/mips64).")
+    "The architecture to build for (x86_64/i386/arm/ppc64/ppc64le/aarch64/mic/mips/mips64/riscv64).")
   # Should assertions be enabled?  They are on by default.
   set(LIBOMP_ENABLE_ASSERTIONS TRUE CACHE BOOL
     "enable assertions?")
 else() # Part of LLVM build
   # Determine the native architecture from LLVM.
   string(TOLOWER "${LLVM_TARGET_ARCH}" LIBOMP_NATIVE_ARCH)
   if( LIBOMP_NATIVE_ARCH STREQUAL "host" )
     string(REGEX MATCH "^[^-]*" LIBOMP_NATIVE_ARCH ${LLVM_HOST_TRIPLE})
else() # Part of LLVM build
   elseif(LIBOMP_NATIVE_ARCH MATCHES "powerpc")
     set(LIBOMP_ARCH ppc64)
   elseif(LIBOMP_NATIVE_ARCH MATCHES "aarch64")
     set(LIBOMP_ARCH aarch64)
   elseif(LIBOMP_NATIVE_ARCH MATCHES "arm64")
     set(LIBOMP_ARCH aarch64)
   elseif(LIBOMP_NATIVE_ARCH MATCHES "arm")
     set(LIBOMP_ARCH arm)
+  elseif(LIBOMP_NATIVE_ARCH MATCHES "riscv64")
+    set(LIBOMP_ARCH riscv64)
   else()
     # last ditch effort
     libomp_get_architecture(LIBOMP_ARCH)
   endif ()
   set(LIBOMP_ENABLE_ASSERTIONS ${LLVM_ENABLE_ASSERTIONS})
 endif()
-libomp_check_variable(LIBOMP_ARCH 32e x86_64 32 i386 arm ppc64 ppc64le aarch64 mic mips mips64)
+libomp_check_variable(LIBOMP_ARCH 32e x86_64 32 i386 arm ppc64 ppc64le aarch64 mic mips mips64 riscv64)
 
 set(LIBOMP_LIB_TYPE normal CACHE STRING
   "Performance,Profiling,Stubs library (normal/profile/stubs)")
 libomp_check_variable(LIBOMP_LIB_TYPE normal profile stubs)
 # Set the OpenMP Year and Month assiociated with version
 set(LIBOMP_OMP_YEAR_MONTH 201611)
 set(LIBOMP_MIC_ARCH knc CACHE STRING
   "Intel(R) Many Integrated Core Architecture (Intel(R) MIC Architecture) (knf/knc).  Ignored if not Intel(R) MIC Architecture build.")

 set(ARM FALSE)
 set(AARCH64 FALSE)
 set(PPC64BE FALSE)
 set(PPC64LE FALSE)
 set(PPC64 FALSE)
 set(MIC FALSE)
 set(MIPS64 FALSE)
 set(MIPS FALSE)
+set(RISCV64 FALSE)
 if("${LIBOMP_ARCH}" STREQUAL "i386" OR "${LIBOMP_ARCH}" STREQUAL "32")    # IA-32 architecture
   set(IA32 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "x86_64" OR "${LIBOMP_ARCH}" STREQUAL "32e") # Intel(R) 64 architecture
   set(INTEL64 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "arm") # ARM architecture
   set(ARM TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "ppc64") # PPC64BE architecture
   set(PPC64BE TRUE)
   set(PPC64 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "ppc64le") # PPC64LE architecture
   set(PPC64LE TRUE)
   set(PPC64 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "aarch64") # AARCH64 architecture
   set(AARCH64 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "mic") # Intel(R) Many Integrated Core Architecture
   set(MIC TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "mips") # MIPS architecture
     set(MIPS TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "mips64") # MIPS64 architecture
     set(MIPS64 TRUE)
+  elseif("${LIBOMP_ARCH}" STREQUAL "riscv64") # RISCV64 architecture
+    set(RISCV64 TRUE)
 endif()
 
 # Set some flags based on build_type
 set(RELEASE_BUILD FALSE)
 set(DEBUG_BUILD FALSE)
 set(RELWITHDEBINFO_BUILD FALSE)
 set(MINSIZEREL_BUILD FALSE)
 string(TOLOWER "${CMAKE_BUILD_TYPE}" libomp_build_type_lowercase)

openmp/runtime/README.txt

 * IA-32 architecture
 * Intel(R) 64 architecture
 * Intel(R) Many Integrated Core Architecture
 * ARM* architecture
 * Aarch64 (64-bit ARM) architecture
 * IBM(R) Power architecture (big endian)
 * IBM(R) Power architecture (little endian)
 * MIPS and MIPS64 architecture
+* RISCV64 architecture
 
 Supported RTL Build Configurations
 ==================================
 
 Supported Architectures: IA-32 architecture, Intel(R) 64, and
 Intel(R) Many Integrated Core Architecture
 
               ----------------------------------------------

openmp/runtime/cmake/LibompGetArchitecture.cmake

set(detect_arch_src_txt "
     #elif defined(__powerpc64__) && defined(__LITTLE_ENDIAN__)
       #error ARCHITECTURE=ppc64le
     #elif defined(__powerpc64__)
       #error ARCHITECTURE=ppc64
     #elif defined(__mips__) && defined(__mips64)
       #error ARCHITECTURE=mips64
     #elif defined(__mips__) && !defined(__mips64)
       #error ARCHITECTURE=mips
+    #elif defined(__riscv) && __riscv_xlen == 64
+      #error ARCHITECTURE=riscv64
     #else
       #error ARCHITECTURE=UnknownArchitecture
     #endif
   ")
   # Write out ${detect_arch_src_txt} to a file within the cmake/ subdirectory
   file(WRITE "${CMAKE_CURRENT_BINARY_DIR}/libomp_detect_arch.c" ${detect_arch_src_txt})
 
   # Try to compile using the C Compiler.  It will always error out with an #error directive, so store error output to ${local_architecture}

openmp/runtime/cmake/LibompMicroTests.cmake

elseif(${ARM})
       libomp_append(libomp_expected_library_deps libffi.so.5)
       libomp_append(libomp_expected_library_deps ld-linux-armhf.so.3)
     elseif(${PPC64})
       libomp_append(libomp_expected_library_deps libc.so.6)
       libomp_append(libomp_expected_library_deps ld64.so.1)
     elseif(${MIPS} OR ${MIPS64})
       libomp_append(libomp_expected_library_deps libc.so.6)
       libomp_append(libomp_expected_library_deps ld.so.1)
+    elseif(${RISCV64})
+      libomp_append(libomp_expected_library_deps libc.so.6)
+      libomp_append(libomp_expected_library_deps ld.so.1)
     endif()
     libomp_append(libomp_expected_library_deps libpthread.so.0 IF_FALSE STUBS_LIBRARY)
     libomp_append(libomp_expected_library_deps libhwloc.so.5 LIBOMP_USE_HWLOC)
   endif()
   libomp_append(libomp_expected_library_deps libstdc++.so.6 LIBOMP_USE_STDCPPLIB)
   libomp_append(libomp_expected_library_deps libm.so.6 LIBOMP_STATS)
 endif()
 # Perl script expects comma separated list

openmp/runtime/cmake/LibompUtils.cmake

function(libomp_get_legal_arch return_arch_string)
   elseif(${PPC64LE})
     set(${return_arch_string} "PPC64LE" PARENT_SCOPE)
   elseif(${AARCH64})
     set(${return_arch_string} "AARCH64" PARENT_SCOPE)
   elseif(${MIPS})
     set(${return_arch_string} "MIPS" PARENT_SCOPE)
   elseif(${MIPS64})
     set(${return_arch_string} "MIPS64" PARENT_SCOPE)
+  elseif(${RISCV64})
+    set(${return_arch_string} "RISCV64" PARENT_SCOPE)
   else()
     set(${return_arch_string} "${LIBOMP_ARCH}" PARENT_SCOPE)
     libomp_warning_say("libomp_get_legal_arch(): Warning: Unknown architecture: Using ${LIBOMP_ARCH}")
   endif()
 endfunction()
 
 # void libomp_check_variable(string var, ...);
 # - runs through all values checking if ${var} == value

openmp/runtime/cmake/config-ix.cmake

if(NOT LIBOMP_HAVE___BUILTIN_FRAME_ADDRESS)
   set(LIBOMP_HAVE_OMPT_SUPPORT FALSE)
 else()
   if( # hardware architecture supported?
      ((LIBOMP_ARCH STREQUAL x86_64) OR
       (LIBOMP_ARCH STREQUAL i386) OR
 #      (LIBOMP_ARCH STREQUAL arm) OR
       (LIBOMP_ARCH STREQUAL aarch64) OR
       (LIBOMP_ARCH STREQUAL ppc64le) OR
-      (LIBOMP_ARCH STREQUAL ppc64))
+      (LIBOMP_ARCH STREQUAL ppc64) OR
+      (LIBOMP_ARCH STREQUAL riscv64))
      AND # OS supported?
      ((WIN32 AND LIBOMP_HAVE_PSAPI) OR APPLE OR (NOT WIN32 AND LIBOMP_HAVE_WEAK_ATTRIBUTE)))
     set(LIBOMP_HAVE_OMPT_SUPPORT TRUE)
   else()
     set(LIBOMP_HAVE_OMPT_SUPPORT FALSE)
   endif()
 endif()
 

openmp/runtime/src/kmp_csupport.cpp

#if KMP_COMPILER_ICC
     _mm_mfence();
 #elif KMP_COMPILER_MSVC
     MemoryBarrier();
 #else
     __sync_synchronize();
 #endif // KMP_COMPILER_ICC
   }
 #endif // KMP_MIC
-#elif (KMP_ARCH_ARM || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS || KMP_ARCH_MIPS64)
+#elif (KMP_ARCH_ARM || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS || KMP_ARCH_MIPS64 || \
+       KMP_ARCH_RISCV64)
 // Nothing to see here move along
 #elif KMP_ARCH_PPC64
 // Nothing needed here (we have a real MB above).
 #if KMP_OS_CNK
   // The flushing thread needs to yield here; this prevents a
   // busy-waiting thread from saturating the pipeline. flush is
   // often used in loops like this:
   // while (!flag) {

openmp/runtime/src/kmp_os.h

 #define KMP_INT32_SPEC "d"
 #define KMP_UINT32_SPEC "u"
 #define KMP_INT64_SPEC "lld"
 #define KMP_UINT64_SPEC "llu"
 #endif /* KMP_OS_UNIX */
 
 #if KMP_ARCH_X86 || KMP_ARCH_ARM || KMP_ARCH_MIPS
 #define KMP_SIZE_T_SPEC KMP_UINT32_SPEC
-#elif KMP_ARCH_X86_64 || KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64
+#elif KMP_ARCH_X86_64 || KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 ||                 \
+    KMP_ARCH_MIPS64 || KMP_ARCH_RISCV64
 #define KMP_SIZE_T_SPEC KMP_UINT64_SPEC
 #else
 #error "Can't determine size_t printf format specifier."
 #endif
 
 #if KMP_ARCH_X86
 #define KMP_SIZE_T_MAX (0xFFFFFFFF)
 #else

 #define KMP_IMB() asm("nop")
 #else
 #define KMP_MB() /* _asm{ nop } */
 #define KMP_IMB() /* _asm{ nop } */
 #endif
 #endif /* KMP_OS_WINDOWS */
 
 #if KMP_ARCH_PPC64 || KMP_ARCH_ARM || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS ||     \
-    KMP_ARCH_MIPS64
+    KMP_ARCH_MIPS64 || KMP_ARCH_RISCV64
 #define KMP_MB() __sync_synchronize()
 #endif
 
 #ifndef KMP_MB
 #define KMP_MB() /* nothing to do */
 #endif
 
 #ifndef KMP_IMB

openmp/runtime/src/kmp_platform.h

 #define KMP_ARCH_X86 0
 #define KMP_ARCH_X86_64 0
 #define KMP_ARCH_AARCH64 0
 #define KMP_ARCH_PPC64_BE 0
 #define KMP_ARCH_PPC64_LE 0
 #define KMP_ARCH_PPC64 (KMP_ARCH_PPC64_LE || KMP_ARCH_PPC64_BE)
 #define KMP_ARCH_MIPS 0
 #define KMP_ARCH_MIPS64 0
+#define KMP_ARCH_RISCV64 0
 
 #if KMP_OS_WINDOWS
 #if defined(_M_AMD64) || defined(__x86_64)
 #undef KMP_ARCH_X86_64
 #define KMP_ARCH_X86_64 1
 #else
 #undef KMP_ARCH_X86
 #define KMP_ARCH_X86 1

 #elif defined __mips__
 #if defined __mips64
 #undef KMP_ARCH_MIPS64
 #define KMP_ARCH_MIPS64 1
 #else
 #undef KMP_ARCH_MIPS
 #define KMP_ARCH_MIPS 1
 #endif
+#elif defined __riscv && __riscv_xlen == 64
+#undef KMP_ARCH_RISCV64
+#define KMP_ARCH_RISCV64 1
 #endif
 #endif
 
 #if defined(__ARM_ARCH_7__) || defined(__ARM_ARCH_7R__) ||                     \
     defined(__ARM_ARCH_7A__)
 #define KMP_ARCH_ARMV7 1
 #endif
 

 
 // Platforms which support Intel(R) Many Integrated Core Architecture
 #define KMP_MIC_SUPPORTED                                                      \
   ((KMP_ARCH_X86 || KMP_ARCH_X86_64) && (KMP_OS_LINUX || KMP_OS_WINDOWS))
 
 // TODO: Fixme - This is clever, but really fugly
 #if (1 !=                                                                      \
      KMP_ARCH_X86 + KMP_ARCH_X86_64 + KMP_ARCH_ARM + KMP_ARCH_PPC64 +          \
-         KMP_ARCH_AARCH64 + KMP_ARCH_MIPS + KMP_ARCH_MIPS64)
+     KMP_ARCH_AARCH64 + KMP_ARCH_MIPS + KMP_ARCH_MIPS64 + KMP_ARCH_RISCV64)
 #error Unknown or unsupported architecture
 #endif
 
 #endif // KMP_PLATFORM_H

openmp/runtime/src/kmp_runtime.cpp

#define FAST_REDUCTION_TREE_METHOD_GENERATED ((reduce_data) && (reduce_func))
   if (team_size == 1) {
 
     retval = empty_reduce_block;
 
   } else {
 
     int atomic_available = FAST_REDUCTION_ATOMIC_METHOD_GENERATED;
 
-#if KMP_ARCH_X86_64 || KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64
+#if KMP_ARCH_X86_64 || KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 ||                   \
+    KMP_ARCH_MIPS64 || KMP_ARCH_RISCV64
 
 #if KMP_OS_LINUX || KMP_OS_DRAGONFLY || KMP_OS_FREEBSD || KMP_OS_NETBSD ||     \
     KMP_OS_OPENBSD || KMP_OS_WINDOWS || KMP_OS_DARWIN || KMP_OS_HURD
 
     int teamsize_cutoff = 4;
 
 #if KMP_MIC_SUPPORTED
     if (__kmp_mic_type != non_mic) {

openmp/runtime/src/thirdparty/ittnotify/ittnotify_config.h

 #ifndef ITT_ARCH_MIPS
 #  define ITT_ARCH_MIPS  6
 #endif /* ITT_ARCH_MIPS */
 
 #ifndef ITT_ARCH_MIPS64
 #  define ITT_ARCH_MIPS64  6
 #endif /* ITT_ARCH_MIPS64 */
 
+#ifndef ITT_ARCH_RISCV64
+#  define ITT_ARCH_RISCV64  7
+#endif /* ITT_ARCH_RISCV64 */
+
 #ifndef ITT_ARCH
 #  if defined _M_IX86 || defined __i386__
 #    define ITT_ARCH ITT_ARCH_IA32
 #  elif defined _M_X64 || defined _M_AMD64 || defined __x86_64__
 #    define ITT_ARCH ITT_ARCH_IA32E
 #  elif defined _M_IA64 || defined __ia64__
 #    define ITT_ARCH ITT_ARCH_IA64
 #  elif defined _M_ARM || defined __arm__
 #    define ITT_ARCH ITT_ARCH_ARM
 #  elif defined __powerpc64__
 #    define ITT_ARCH ITT_ARCH_PPC64
 #  elif defined __aarch64__
 #    define ITT_ARCH ITT_ARCH_AARCH64
 #  elif defined __mips__ && !defined __mips64
 #    define ITT_ARCH ITT_ARCH_MIPS
 #  elif defined __mips__ && defined __mips64
 #    define ITT_ARCH ITT_ARCH_MIPS64
+#  elif defined __riscv && __riscv_xlen == 64
+#    define ITT_ARCH ITT_ARCH_RISCV64
 #  endif
 #endif
 
 #ifdef __cplusplus
 #  define ITT_EXTERN_C extern "C"
 #  define ITT_EXTERN_C_BEGIN extern "C" {
 #  define ITT_EXTERN_C_END }
 #else

 {
     long result;
     __asm__ __volatile__("lock\nxadd %0,%1"
                           : "=r"(result),"=m"(*(volatile int*)ptr)
                           : "0"(addend), "m"(*(volatile int*)ptr)
                           : "memory");
     return result;
 }
-#elif ITT_ARCH==ITT_ARCH_ARM || ITT_ARCH==ITT_ARCH_PPC64 || ITT_ARCH==ITT_ARCH_AARCH64 || ITT_ARCH==ITT_ARCH_MIPS ||  ITT_ARCH==ITT_ARCH_MIPS64
+#elif ITT_ARCH == ITT_ARCH_ARM || ITT_ARCH == ITT_ARCH_PPC64 ||                \
+    ITT_ARCH == ITT_ARCH_AARCH64 || ITT_ARCH == ITT_ARCH_MIPS ||               \
+    ITT_ARCH == ITT_ARCH_MIPS64 || ITT_ARCH == ITT_ARCH_RISCV64
 #define __TBB_machine_fetchadd4(addr, val) __sync_fetch_and_add(addr, val)
 #endif /* ITT_ARCH==ITT_ARCH_IA64 */
 #ifndef ITT_SIMPLE_INIT
 ITT_INLINE long
 __itt_interlocked_increment(volatile long* ptr) ITT_INLINE_ATTRIBUTE;
 ITT_INLINE long __itt_interlocked_increment(volatile long* ptr)
 {
     return __TBB_machine_fetchadd4(ptr, 1) + 1L;

openmp/runtime/src/z_Linux_asm.S

 .Lfunc_end0:
   .size __kmp_invoke_microtask, .Lfunc_end0-.Lfunc_begin0
   .cfi_endproc
 
 // -- End  __kmp_invoke_microtask
 
 #endif /* KMP_ARCH_PPC64 */
 
+#if KMP_ARCH_RISCV64
+
+//------------------------------------------------------------------------
+//
+// typedef void (*microtask_t)(int *gtid, int *tid, ...);
+//
+// int __kmp_invoke_microtask(microtask_t pkfn, int gtid, int tid, int argc,
+//                            void *p_argv[]
+// #if OMPT_SUPPORT
+//                            ,
+//                            void **exit_frame_ptr
+// #endif
+//                            ) {
+// #if OMPT_SUPPORT
+//   *exit_frame_ptr = OMPT_GET_FRAME_ADDRESS(0);
+// #endif
+//
+//   (*pkfn)(&gtid, &tid, argv[0], ...);
+//
+//   return 1;
+// }
+//
+// Parameters:
+//   a0: pkfn
+//   a1: gtid
+//   a2: tid
+//   a3: argc
+//   a4: p_argv
+//   a5: exit_frame_ptr
+//
+// Locals:
+//   __gtid: gtid param pushed on stack so can pass &gtid to pkfn
+//   __tid: tid param pushed on stack so can pass &tid to pkfn
+//
+// Temp. registers:
+//
+//  t0: used to calculate the dynamic stack size / used to hold pkfn address
+//  t1: used as temporary for stack placement calculation
+//  t2: used as temporary for stack arguments
+//  t3: used as temporary for number of remaining pkfn parms
+//  t4: used to traverse p_argv array
+//
+// return: a0 (always 1/TRUE)
+//
+
+__gtid = -20
+__tid = -24
+
+// -- Begin __kmp_invoke_microtask
+// mark_begin;
+  .text
+  .globl  __kmp_invoke_microtask
+  .p2align  1
+  .type __kmp_invoke_microtask,@function
+__kmp_invoke_microtask:
+  .cfi_startproc
+
+  // First, save ra and fp
+  addi  sp, sp, -16
+  sd  ra, 8(sp)
+  sd  fp, 0(sp)
+  addi  fp, sp, 16
+  .cfi_def_cfa  fp, 0
+  .cfi_offset ra, -8
+  .cfi_offset fp, -16
+
+  // Compute the dynamic stack size:
+  //
+  // - We need 8 bytes for storing 'gtid' and 'tid', so we can pass them by
+  //   reference
+  // - We need 8 bytes for each argument that cannot be passed to the 'pkfn'
+  //   function by register. Given that we have 8 of such registers (a[0-7])
+  //   and two + 'argc' arguments (consider &gtid and &tid), we need to
+  //   reserve max(0, argc - 6)*8 extra bytes
+  //
+  // The total number of bytes is then max(0, argc - 6)*8 + 8
+
+  // Compute max(0, argc - 6) using the following bithack:
+  // max(0, x) = x - (x & (x >> 31)), where x := argc - 6
+  // Source: http://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
+  addi  t0, a3, -6
+  srai  t1, t0, 31
+  and t1, t0, t1
+  sub t0, t0, t1
+
+  addi  t0, t0, 1
+
+  slli  t0, t0, 3
+  sub sp, sp, t0
+
+  // Align the stack to 16 bytes
+  andi  sp, sp, -16
+
+  mv  t0, a0
+  mv  t3, a3
+  mv  t4, a4
+
+#if OMPT_SUPPORT
+  // Save frame pointer into exit_frame
+  sd  fp, 0(a5)
+#endif
+
+  // Prepare arguments for the pkfn function (first 8 using a0-a7 registers)
+
+  sw  a1, __gtid(fp)
+  sw  a2, __tid(fp)
+
+  addi  a0, fp, __gtid
+  addi  a1, fp, __tid
+
+  beqz  t3, .L_kmp_3
+  ld  a2, 0(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a3, 8(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a4, 16(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a5, 24(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a6, 32(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a7, 40(t4)
+
+  // Prepare any additional argument passed through the stack
+  addi  t4, t4, 48
+  mv  t1, sp
+  j .L_kmp_2
+.L_kmp_1:
+  ld  t2, 0(t4)
+  sd  t2, 0(t1)
+  addi  t4, t4, 8
+  addi  t1, t1, 8
+.L_kmp_2:
+  addi  t3, t3, -1
+  bnez  t3, .L_kmp_1
+
+.L_kmp_3:
+  // Call pkfn function
+  jalr  t0
+
+  // Restore stack and return
+
+  addi  a0, zero, 1
+
+  addi  sp, fp, -16
+  ld  fp, 0(sp)
+  ld  ra, 8(sp)
+  addi  sp, sp, 16
+  ret
+.Lfunc_end0:
+  .size __kmp_invoke_microtask, .Lfunc_end0-__kmp_invoke_microtask
+  .cfi_endproc
+
+// -- End  __kmp_invoke_microtask
+
+#endif /* KMP_ARCH_RISCV64 */
+
 #if KMP_ARCH_ARM || KMP_ARCH_MIPS
     .data
     .comm .gomp_critical_user_,32,8
     .data
     .align 4
     .global __kmp_unnamed_critical_addr
 __kmp_unnamed_critical_addr:
     .4byte .gomp_critical_user_
     .size __kmp_unnamed_critical_addr,4
 #endif /* KMP_ARCH_ARM */
 
-#if KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64
+#if KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64 || KMP_ARCH_RISCV64
     .data
     .comm .gomp_critical_user_,32,8
     .data
     .align 8
     .global __kmp_unnamed_critical_addr
 __kmp_unnamed_critical_addr:
     .8byte .gomp_critical_user_
     .size __kmp_unnamed_critical_addr,8
-#endif /* KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 */
+#endif /* KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64 ||
+          KMP_ARCH_RISCV64 */
 
 #if KMP_OS_LINUX
 # if KMP_ARCH_ARM
 .section .note.GNU-stack,"",%progbits
 # else
 .section .note.GNU-stack,"",@progbits
 # endif
 #endif

openmp/runtime/src/z_Linux_util.cpp

 
 } // __kmp_get_load_balance
 
 #endif // KMP_OS_DARWIN
 
 #endif // USE_LOAD_BALANCE
 
 #if !(KMP_ARCH_X86 || KMP_ARCH_X86_64 || KMP_MIC ||                            \
-      ((KMP_OS_LINUX || KMP_OS_DARWIN) && KMP_ARCH_AARCH64) || KMP_ARCH_PPC64)
+      ((KMP_OS_LINUX || KMP_OS_DARWIN) && KMP_ARCH_AARCH64) ||                 \
+      KMP_ARCH_PPC64 || KMP_ARCH_RISCV64)
 
 // we really only need the case with 1 argument, because CLANG always build
 // a struct of pointers to shared variables referenced in the outlined function
 int __kmp_invoke_microtask(microtask_t pkfn, int gtid, int tid, int argc,
                            void *p_argv[]
 #if OMPT_SUPPORT
                            ,
                            void **exit_frame_ptr

openmp/runtime/test/ompt/callback.h

#define print_possible_return_addresses(addr) \
   printf("%" PRIu64 ": current_address=%p or %p\n", ompt_get_thread_data()->value, \
          ((char *)addr) - 8, ((char *)addr) - 12)
 #elif KMP_ARCH_AARCH64
 // On AArch64 the NOP instruction is 4 bytes long, can be followed by inserted
 // store instruction (another 4 bytes long).
 #define print_possible_return_addresses(addr) \
   printf("%" PRIu64 ": current_address=%p or %p\n", ompt_get_thread_data()->value, \
          ((char *)addr) - 4, ((char *)addr) - 8)
+#elif KMP_ARCH_RISCV64
+#if __riscv_compressed
+// On RV64GC the C.NOP instruction is 2 byte long. In addition, the compiler
+// inserts a J instruction (targeting the successor basic block), which
+// accounts for another 4 bytes. Finally, an additional J instruction may
+// appear (adding 4 more bytes) when the C.NOP is referenced elsewhere (ie.
+// another branch).
+#define print_possible_return_addresses(addr) \
+  printf("%" PRIu64 ": current_address=%p or %p\n", \
+         ompt_get_thread_data()->value, ((char *)addr) - 6, ((char *)addr) - 10)
+#else
+// On RV64G the NOP instruction is 4 byte long. In addition, the compiler
+// inserts a J instruction (targeting the successor basic block), which
+// accounts for another 4 bytes. Finally, an additional J instruction may
+// appear (adding 4 more bytes) when the NOP is referenced elsewhere (ie.
+// another branch).
+#define print_possible_return_addresses(addr) \
+  printf("%" PRIu64 ": current_address=%p or %p\n", \
+         ompt_get_thread_data()->value, ((char *)addr) - 8, ((char *)addr) - 12)
+#endif
 #else
 #error Unsupported target architecture, cannot determine address offset!
 #endif
 
 
 // This macro performs a somewhat similar job to print_current_address(), except
 // that it discards a certain number of nibbles from the address and only prints
 // the most significant bits / nibbles. This can be used for cases where the

openmp/runtime/tools/lib/Platform.pm

if ( defined( $arch ) ) {
         } elsif ( $arch =~ m{\Aaarch64} ) {
                 $arch = "aarch64";
         } elsif ( $arch =~ m{\Amic} ) {
             $arch = "mic";
         } elsif ( $arch =~ m{\Amips64} ) {
             $arch = "mips64";
         } elsif ( $arch =~ m{\Amips} ) {
             $arch = "mips";
+        } elsif ( $arch =~ m{\Ariscv64} ) {
+            $arch = "riscv64";
         } else {
             $arch = undef;
         }; # if
     }; # if
     return $arch;
 }; # sub canon_arch
 
 # Canonize Intel(R) Many Integrated Core Architecture name.
sub canon_mic_arch($) {
     my %legal = (
         "32"  => "IA-32 architecture",
         "32e" => "Intel(R) 64",
         "arm" => "ARM",
         "aarch64" => "AArch64",
         "mic" => "Intel(R) Many Integrated Core Architecture",
         "mips" => "MIPS",
         "mips64" => "MIPS64",
+        "riscv64" => "RISC-V (64-bit)",
     );
 
     sub legal_arch($) {
         my ( $arch ) = @_;
         $arch = canon_arch( $arch );
         if ( defined( $arch ) ) {
             $arch = $legal{ $arch };
         }; # if
# Detect host arch.
     } elsif ( $hardware_platform eq "ppc64" ) {
         $_host_arch = "ppc64";
     } elsif ( $hardware_platform eq "aarch64" ) {
         $_host_arch = "aarch64";
     } elsif ( $hardware_platform eq "mips64" ) {
         $_host_arch = "mips64";
     } elsif ( $hardware_platform eq "mips" ) {
         $_host_arch = "mips";
+    } elsif ( $hardware_platform eq "riscv64" ) {
+        $_host_arch = "riscv64";
     } else {
         die "Unsupported host hardware platform: \"$hardware_platform\"; stopped";
     }; # if
 }
 
 # Detect host OS.
 {
     my $operating_system = Uname::operating_system();

 =head2 Functions.
 
 =over
 
 =item B<canon_arch( $arch )>
 
 Input string is an architecture name to canonize. The function recognizes many variants, for example:
 C<32e>, C<Intel64>, C<Intel(R) 64>, etc. Returned string is a canononized architecture name,
-one of: C<32>, C<32e>, C<64>, C<arm>, C<ppc64le>, C<ppc64>, C<mic>, C<mips>, C<mips64>, or C<undef> is input string is not recognized.
+one of: C<32>, C<32e>, C<64>, C<arm>, C<ppc64le>, C<ppc64>, C<mic>, C<mips>, C<mips64>, C<riscv64> or C<undef> is input string is not recognized.
 
 =item B<legal_arch( $arch )>
 
 Input string is architecture name. The function recognizes the same variants as C<arch_canon()> does.
 Returned string is a name approved by Intel Legal, one of: C<IA-32 architecture>, C<Intel(R) 64>
 or C<undef> if input string is not recognized.
 
 =item B<arch_opt( $arch )>

openmp/runtime/tools/lib/Uname.pm

if ( 0 ) {
     } elsif ( $values{ machine } =~ m{\Appc64\z} ) {
         $values{ hardware_platform } = "ppc64";
     } elsif ( $values{ machine } =~ m{\Aaarch64\z} ) {
         $values{ hardware_platform } = "aarch64";
     } elsif ( $values{ machine } =~ m{\Amips64\z} ) {
         $values{ hardware_platform } = "mips64";
     } elsif ( $values{ machine } =~ m{\Amips\z} ) {
         $values{ hardware_platform } = "mips";
+    } elsif ( $values{ machine } =~ m{\Ariscv64\z} ) {
+        $values{ hardware_platform } = "riscv64";
     } else {
         die "Unsupported machine (\"$values{ machine }\") returned by POSIX::uname(); stopped";
     }; # if
 } elsif ( $^O eq "darwin" ) {
     if ( 0 ) {
     } elsif ( $values{ machine } eq "x86" or $values{ machine } eq "i386" ) {
         $values{ hardware_platform } =
             sub {

openmp/www/README.txt

 * IA-32 architecture
 * Intel(R) 64 architecture
 * Intel(R) Many Integrated Core Architecture
 * ARM* architecture
 * Aarch64 (64-bit ARM) architecture
 * IBM(R) Power architecture (big endian)
 * IBM(R) Power architecture (little endian)
 * MIPS and MIPS64 architectures
+* RISC-V 64 bit architecture
 
 Supported RTL Build Configurations
 ==================================
 
 Supported Architectures: IA-32 architecture, Intel(R) 64, and
 Intel(R) Many Integrated Core Architecture
 
               ----------------------------------------------

openmp/www/index.html

<ul>
       <li>ARM&reg;&nbsp; architecture processors</li>
       <li>PowerPC&trade;&nbsp; processors</li>
       <li>32 and 64 bit X86
         processors when compiled with clang, with the Intel compiler
         or with gcc, and also the Intel&reg;&nbsp;Xeon Phi&trade; product family, when compiled with
         the Intel compiler.
       </li>
       <li>MIPS and MIPS64</li>
+      <li>RISC-V 64-bit</li>
     </ul>
     Ports to other architectures and operating systems are welcome.
   </p>
 
   <p>A full OS and architecture compatibility matrix is in
     <a href="README.txt">README.txt</a>
   </p>