[OpenMP] RISCV64 port

[fpallares]

Summary

This is a port of libomp for the RISC-V 64-bit Linux target.

We have tested this port on a HiFive Unleashed development board using a downstream LLVM that has support for the missing bits in upstream. As of now, all tests are passing, including OMPT.

[Hahnfeld]

Most of the changes look pretty straight forward to me, but I currently don't have time to read through the assembly implementation of __kmp_invoke_microtask. I left two comments inline to discuss compressed vs uncompressed encodings for testing OMPT.

runtime/test/ompt/callback.h

170

Should this also handle the case that the nop is uncompressed (this should be 4 bytes)?

172–173

Did you try to find out why the compiler inserts a jump?
And, as above, should be also allow the instruction to be 4 bytes long?

[fpallares]

Update print_possible_return_addresses macro in runtime/test/ompt/callback.h to handle uncompressed instructions, as pointed out by @Hahnfeld.

[fpallares]

Thanks for the quick review @Hahnfeld, I've just replied to your inline comments and updated the patch!

runtime/test/ompt/callback.h

170

As Linux requires riscv64gc we assumed compressed instructions would be enabled, but certainly it is not necessarily the case (riscv64g should be supported). Good catch!

172–173

The unnecessary jump to the immediate successor block is removed by FastIsel in the targets that enable it. For now RISCV64 does not.

[Hahnfeld]

Sorry for the delay...

runtime/src/z_Linux_asm.S

1720–1722

I don't think we need this, it's now handled at all call-sites of __kmp_invoke_microtask. I'll post a patch to resolve any inconsistencies.

AFAICT removing this should reduce some of the (conditional) complexity of the stack layout.

1759–1760

I'd propose to make them negative, the implementation for x86_64 does so as well.

1802–1805

For example, this can go away.

1874–1878

Not needed, as said above.

[fpallares]

Remove reset of exit_frame pointer for OMPT since it is now done at call-site and fix return address computation macro used for some tests (as the generated code has changed).

[fpallares]

Thanks @Hahnfeld for the review. I've just updated the patch.

[fpallares]

Now that D60456 has landed we believe this need not be WIP anymore.

[Hahnfeld]

The changes look good to me, thanks for your patience!

[fpallares]

I've tested this patch with current upstream LLVM and Clang on a HiFive Unleashed board and all tests are passing, including OMPT. I don't have commit access yet, @Hahnfeld would you mind commiting this for me? Thank you!

[Hahnfeld]

In D59880#1598584, @fpallares wrote:

I've tested this patch with current upstream LLVM and Clang on a HiFive Unleashed board and all tests are passing, including OMPT. I don't have commit access yet, @Hahnfeld would you mind commiting this for me? Thank you!

Sure, not problem. The commit should be on its way through the repositories.

README.rst

 
 **OPENMP_FILECHECK_EXECUTABLE** = ``/path/to/FileCheck``
   Specify full path to ``FileCheck`` executable for running tests.  The default
   is to search the ``PATH`` and the directory in **OPENMP_LLVM_TOOLS_DIR**.
 
 Options for ``libomp``
 ----------------------
 
-**LIBOMP_ARCH** = ``aarch64|arm|i386|mic|mips|mips64|ppc64|ppc64le|x86_64``
+**LIBOMP_ARCH** = ``aarch64|arm|i386|mic|mips|mips64|ppc64|ppc64le|x86_64|riscv64``
   The default value for this option is chosen based on probing the compiler for
   architecture macros (e.g., is ``__x86_64__`` predefined by compiler?).
 
 **LIBOMP_MIC_ARCH** = ``knc|knf``
   Intel(R) Many Integrated Core Architecture (Intel(R) MIC Architecture) to
   build for.  This value is ignored if **LIBOMP_ARCH** does not equal ``mic``.
 
 **LIBOMP_OMP_VERSION** = ``50|45|40|30``
**LIBOMP_USE_ADAPTIVE_LOCKS** = ``ON|OFF``
   Extensions (Intel(R) TSX).  This feature is x86 specific and turned ``ON``
   by default for IA-32 architecture and Intel(R) 64 architecture.
 
 **LIBOMP_USE_INTERNODE_ALIGNMENT** = ``OFF|ON``
   Align certain data structures on 4096-byte.  This option is useful on
   multi-node systems where a small ``CACHE_LINE`` setting leads to false sharing.
 
 **LIBOMP_OMPT_SUPPORT** = ``ON|OFF``
-  Include support for the OpenMP Tools Interface (OMPT). 
-  This option is supported and ``ON`` by default for x86, x86_64, AArch64, and 
-  PPC64 on Linux* and macOS*.
+  Include support for the OpenMP Tools Interface (OMPT).
+  This option is supported and ``ON`` by default for x86, x86_64, AArch64,
+  PPC64 and RISCV64 on Linux* and macOS*.
   This option is ``OFF`` if this feature is not supported for the platform.
 
 **LIBOMP_OMPT_OPTIONAL** = ``ON|OFF``
   Include support for optional OMPT functionality.  This option is ignored if
   **LIBOMP_OMPT_SUPPORT** is ``OFF``.
 
 **LIBOMP_STATS** = ``OFF|ON``
   Include stats-gathering code.

runtime/CMakeLists.txt

 include(LibompHandleFlags)
 include(LibompDefinitions)
 
 # Determine the target architecture
 if(${OPENMP_STANDALONE_BUILD})
   # If adding a new architecture, take a look at cmake/LibompGetArchitecture.cmake
   libomp_get_architecture(LIBOMP_DETECTED_ARCH)
   set(LIBOMP_ARCH ${LIBOMP_DETECTED_ARCH} CACHE STRING
-    "The architecture to build for (x86_64/i386/arm/ppc64/ppc64le/aarch64/mic/mips/mips64).")
+    "The architecture to build for (x86_64/i386/arm/ppc64/ppc64le/aarch64/mic/mips/mips64/riscv64).")
   # Should assertions be enabled?  They are on by default.
   set(LIBOMP_ENABLE_ASSERTIONS TRUE CACHE BOOL
     "enable assertions?")
 else() # Part of LLVM build
   # Determine the native architecture from LLVM.
   string(TOLOWER "${LLVM_TARGET_ARCH}" LIBOMP_NATIVE_ARCH)
   if( LIBOMP_NATIVE_ARCH STREQUAL "host" )
     string(REGEX MATCH "^[^-]*" LIBOMP_NATIVE_ARCH ${LLVM_HOST_TRIPLE})
else() # Part of LLVM build
   elseif(LIBOMP_NATIVE_ARCH MATCHES "powerpc")
     set(LIBOMP_ARCH ppc64)
   elseif(LIBOMP_NATIVE_ARCH MATCHES "aarch64")
     set(LIBOMP_ARCH aarch64)
   elseif(LIBOMP_NATIVE_ARCH MATCHES "arm64")
     set(LIBOMP_ARCH aarch64)
   elseif(LIBOMP_NATIVE_ARCH MATCHES "arm")
     set(LIBOMP_ARCH arm)
+  elseif(LIBOMP_NATIVE_ARCH MATCHES "riscv64")
+    set(LIBOMP_ARCH riscv64)
   else()
     # last ditch effort
     libomp_get_architecture(LIBOMP_ARCH)
   endif ()
   set(LIBOMP_ENABLE_ASSERTIONS ${LLVM_ENABLE_ASSERTIONS})
 endif()
-libomp_check_variable(LIBOMP_ARCH 32e x86_64 32 i386 arm ppc64 ppc64le aarch64 mic mips mips64)
+libomp_check_variable(LIBOMP_ARCH 32e x86_64 32 i386 arm ppc64 ppc64le aarch64 mic mips mips64 riscv64)
 
 set(LIBOMP_LIB_TYPE normal CACHE STRING
   "Performance,Profiling,Stubs library (normal/profile/stubs)")
 libomp_check_variable(LIBOMP_LIB_TYPE normal profile stubs)
 set(LIBOMP_OMP_VERSION 50 CACHE STRING
   "The OpenMP version (50/45/40/30)")
 libomp_check_variable(LIBOMP_OMP_VERSION 50 45 40 30)
 # Set the OpenMP Year and Month assiociated with version

 set(ARM FALSE)
 set(AARCH64 FALSE)
 set(PPC64BE FALSE)
 set(PPC64LE FALSE)
 set(PPC64 FALSE)
 set(MIC FALSE)
 set(MIPS64 FALSE)
 set(MIPS FALSE)
+set(RISCV64 FALSE)
 if("${LIBOMP_ARCH}" STREQUAL "i386" OR "${LIBOMP_ARCH}" STREQUAL "32")    # IA-32 architecture
   set(IA32 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "x86_64" OR "${LIBOMP_ARCH}" STREQUAL "32e") # Intel(R) 64 architecture
   set(INTEL64 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "arm") # ARM architecture
   set(ARM TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "ppc64") # PPC64BE architecture
   set(PPC64BE TRUE)
   set(PPC64 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "ppc64le") # PPC64LE architecture
   set(PPC64LE TRUE)
   set(PPC64 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "aarch64") # AARCH64 architecture
   set(AARCH64 TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "mic") # Intel(R) Many Integrated Core Architecture
   set(MIC TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "mips") # MIPS architecture
     set(MIPS TRUE)
 elseif("${LIBOMP_ARCH}" STREQUAL "mips64") # MIPS64 architecture
     set(MIPS64 TRUE)
+  elseif("${LIBOMP_ARCH}" STREQUAL "riscv64") # RISCV64 architecture
+    set(RISCV64 TRUE)
 endif()
 
 # Set some flags based on build_type
 set(RELEASE_BUILD FALSE)
 set(DEBUG_BUILD FALSE)
 set(RELWITHDEBINFO_BUILD FALSE)
 set(MINSIZEREL_BUILD FALSE)
 string(TOLOWER "${CMAKE_BUILD_TYPE}" libomp_build_type_lowercase)

runtime/README.txt

 * IA-32 architecture
 * Intel(R) 64 architecture
 * Intel(R) Many Integrated Core Architecture
 * ARM* architecture
 * Aarch64 (64-bit ARM) architecture
 * IBM(R) Power architecture (big endian)
 * IBM(R) Power architecture (little endian)
 * MIPS and MIPS64 architecture
+* RISCV64 architecture
 
 Supported RTL Build Configurations
 ==================================
 
 Supported Architectures: IA-32 architecture, Intel(R) 64, and
 Intel(R) Many Integrated Core Architecture
 
               ----------------------------------------------

runtime/cmake/LibompGetArchitecture.cmake

set(detect_arch_src_txt "
     #elif defined(__powerpc64__) && defined(__LITTLE_ENDIAN__)
       #error ARCHITECTURE=ppc64le
     #elif defined(__powerpc64__)
       #error ARCHITECTURE=ppc64
     #elif defined(__mips__) && defined(__mips64)
       #error ARCHITECTURE=mips64
     #elif defined(__mips__) && !defined(__mips64)
       #error ARCHITECTURE=mips
+    #elif defined(__riscv) && __riscv_xlen == 64
+      #error ARCHITECTURE=riscv64
     #else
       #error ARCHITECTURE=UnknownArchitecture
     #endif
   ")
   # Write out ${detect_arch_src_txt} to a file within the cmake/ subdirectory
   file(WRITE "${CMAKE_CURRENT_BINARY_DIR}/libomp_detect_arch.c" ${detect_arch_src_txt})
 
   # Try to compile using the C Compiler.  It will always error out with an #error directive, so store error output to ${local_architecture}

runtime/cmake/LibompMicroTests.cmake

elseif(${ARM})
       libomp_append(libomp_expected_library_deps libffi.so.5)
       libomp_append(libomp_expected_library_deps ld-linux-armhf.so.3)
     elseif(${PPC64})
       libomp_append(libomp_expected_library_deps libc.so.6)
       libomp_append(libomp_expected_library_deps ld64.so.1)
     elseif(${MIPS} OR ${MIPS64})
       libomp_append(libomp_expected_library_deps libc.so.6)
       libomp_append(libomp_expected_library_deps ld.so.1)
+    elseif(${RISCV64})
+      libomp_append(libomp_expected_library_deps libc.so.6)
+      libomp_append(libomp_expected_library_deps ld.so.1)
     endif()
     libomp_append(libomp_expected_library_deps libpthread.so.0 IF_FALSE STUBS_LIBRARY)
     libomp_append(libomp_expected_library_deps libhwloc.so.5 LIBOMP_USE_HWLOC)
   endif()
   libomp_append(libomp_expected_library_deps libstdc++.so.6 LIBOMP_USE_STDCPPLIB)
   libomp_append(libomp_expected_library_deps libm.so.6 LIBOMP_STATS)
 endif()
 # Perl script expects comma separated list

runtime/cmake/LibompUtils.cmake

function(libomp_get_legal_arch return_arch_string)
   elseif(${PPC64LE})
     set(${return_arch_string} "PPC64LE" PARENT_SCOPE)
   elseif(${AARCH64})
     set(${return_arch_string} "AARCH64" PARENT_SCOPE)
   elseif(${MIPS})
     set(${return_arch_string} "MIPS" PARENT_SCOPE)
   elseif(${MIPS64})
     set(${return_arch_string} "MIPS64" PARENT_SCOPE)
+  elseif(${RISCV64})
+    set(${return_arch_string} "RISCV64" PARENT_SCOPE)
   else()
     set(${return_arch_string} "${LIBOMP_ARCH}" PARENT_SCOPE)
     libomp_warning_say("libomp_get_legal_arch(): Warning: Unknown architecture: Using ${LIBOMP_ARCH}")
   endif()
 endfunction()
 
 # void libomp_check_variable(string var, ...);
 # - runs through all values checking if ${var} == value

runtime/cmake/config-ix.cmake

if(NOT LIBOMP_HAVE___BUILTIN_FRAME_ADDRESS)
   set(LIBOMP_HAVE_OMPT_SUPPORT FALSE)
 else()
   if( # hardware architecture supported?
      ((LIBOMP_ARCH STREQUAL x86_64) OR
       (LIBOMP_ARCH STREQUAL i386) OR
 #      (LIBOMP_ARCH STREQUAL arm) OR
       (LIBOMP_ARCH STREQUAL aarch64) OR
       (LIBOMP_ARCH STREQUAL ppc64le) OR
-      (LIBOMP_ARCH STREQUAL ppc64))
+      (LIBOMP_ARCH STREQUAL ppc64) OR
+      (LIBOMP_ARCH STREQUAL riscv64))
      AND # OS supported?
      ((WIN32 AND LIBOMP_HAVE_PSAPI) OR APPLE OR (NOT WIN32 AND LIBOMP_HAVE_WEAK_ATTRIBUTE)))
     set(LIBOMP_HAVE_OMPT_SUPPORT TRUE)
   else()
     set(LIBOMP_HAVE_OMPT_SUPPORT FALSE)
   endif()
 endif()
 

runtime/src/kmp_csupport.cpp

#if KMP_COMPILER_ICC
     _mm_mfence();
 #elif KMP_COMPILER_MSVC
     MemoryBarrier();
 #else
     __sync_synchronize();
 #endif // KMP_COMPILER_ICC
   }
 #endif // KMP_MIC
-#elif (KMP_ARCH_ARM || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS || KMP_ARCH_MIPS64)
+#elif (KMP_ARCH_ARM || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS || KMP_ARCH_MIPS64 || \
+       KMP_ARCH_RISCV64)
 // Nothing to see here move along
 #elif KMP_ARCH_PPC64
 // Nothing needed here (we have a real MB above).
 #if KMP_OS_CNK
   // The flushing thread needs to yield here; this prevents a
   // busy-waiting thread from saturating the pipeline. flush is
   // often used in loops like this:
   // while (!flag) {

runtime/src/kmp_os.h

 #define KMP_INT32_SPEC "d"
 #define KMP_UINT32_SPEC "u"
 #define KMP_INT64_SPEC "lld"
 #define KMP_UINT64_SPEC "llu"
 #endif /* KMP_OS_UNIX */
 
 #if KMP_ARCH_X86 || KMP_ARCH_ARM || KMP_ARCH_MIPS
 #define KMP_SIZE_T_SPEC KMP_UINT32_SPEC
-#elif KMP_ARCH_X86_64 || KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64
+#elif KMP_ARCH_X86_64 || KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 ||                 \
+    KMP_ARCH_MIPS64 || KMP_ARCH_RISCV64
 #define KMP_SIZE_T_SPEC KMP_UINT64_SPEC
 #else
 #error "Can't determine size_t printf format specifier."
 #endif
 
 #if KMP_ARCH_X86
 #define KMP_SIZE_T_MAX (0xFFFFFFFF)
 #else

 #define KMP_IMB() asm("nop")
 #else
 #define KMP_MB() /* _asm{ nop } */
 #define KMP_IMB() /* _asm{ nop } */
 #endif
 #endif /* KMP_OS_WINDOWS */
 
 #if KMP_ARCH_PPC64 || KMP_ARCH_ARM || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS ||     \
-    KMP_ARCH_MIPS64
+    KMP_ARCH_MIPS64 || KMP_ARCH_RISCV64
 #define KMP_MB() __sync_synchronize()
 #endif
 
 #ifndef KMP_MB
 #define KMP_MB() /* nothing to do */
 #endif
 
 #ifndef KMP_IMB

runtime/src/kmp_platform.h

 #define KMP_ARCH_X86 0
 #define KMP_ARCH_X86_64 0
 #define KMP_ARCH_AARCH64 0
 #define KMP_ARCH_PPC64_BE 0
 #define KMP_ARCH_PPC64_LE 0
 #define KMP_ARCH_PPC64 (KMP_ARCH_PPC64_LE || KMP_ARCH_PPC64_BE)
 #define KMP_ARCH_MIPS 0
 #define KMP_ARCH_MIPS64 0
+#define KMP_ARCH_RISCV64 0
 
 #if KMP_OS_WINDOWS
 #if defined(_M_AMD64) || defined(__x86_64)
 #undef KMP_ARCH_X86_64
 #define KMP_ARCH_X86_64 1
 #else
 #undef KMP_ARCH_X86
 #define KMP_ARCH_X86 1

 #elif defined __mips__
 #if defined __mips64
 #undef KMP_ARCH_MIPS64
 #define KMP_ARCH_MIPS64 1
 #else
 #undef KMP_ARCH_MIPS
 #define KMP_ARCH_MIPS 1
 #endif
+#elif defined __riscv && __riscv_xlen == 64
+#undef KMP_ARCH_RISCV64
+#define KMP_ARCH_RISCV64 1
 #endif
 #endif
 
 #if defined(__ARM_ARCH_7__) || defined(__ARM_ARCH_7R__) ||                     \
     defined(__ARM_ARCH_7A__)
 #define KMP_ARCH_ARMV7 1
 #endif
 

 
 // Platforms which support Intel(R) Many Integrated Core Architecture
 #define KMP_MIC_SUPPORTED                                                      \
   ((KMP_ARCH_X86 || KMP_ARCH_X86_64) && (KMP_OS_LINUX || KMP_OS_WINDOWS))
 
 // TODO: Fixme - This is clever, but really fugly
 #if (1 !=                                                                      \
      KMP_ARCH_X86 + KMP_ARCH_X86_64 + KMP_ARCH_ARM + KMP_ARCH_PPC64 +          \
-         KMP_ARCH_AARCH64 + KMP_ARCH_MIPS + KMP_ARCH_MIPS64)
+     KMP_ARCH_AARCH64 + KMP_ARCH_MIPS + KMP_ARCH_MIPS64 + KMP_ARCH_RISCV64)
 #error Unknown or unsupported architecture
 #endif
 
 #endif // KMP_PLATFORM_H

runtime/src/kmp_runtime.cpp

#define FAST_REDUCTION_TREE_METHOD_GENERATED ((reduce_data) && (reduce_func))
   if (team_size == 1) {
 
     retval = empty_reduce_block;
 
   } else {
 
     int atomic_available = FAST_REDUCTION_ATOMIC_METHOD_GENERATED;
 
-#if KMP_ARCH_X86_64 || KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64
+#if KMP_ARCH_X86_64 || KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 ||                   \
+    KMP_ARCH_MIPS64 || KMP_ARCH_RISCV64
 
 #if KMP_OS_LINUX || KMP_OS_DRAGONFLY || KMP_OS_FREEBSD || KMP_OS_NETBSD ||     \
     KMP_OS_OPENBSD || KMP_OS_WINDOWS || KMP_OS_DARWIN || KMP_OS_HURD
 
     int teamsize_cutoff = 4;
 
 #if KMP_MIC_SUPPORTED
     if (__kmp_mic_type != non_mic) {

runtime/src/thirdparty/ittnotify/ittnotify_config.h

 #ifndef ITT_ARCH_MIPS
 #  define ITT_ARCH_MIPS  6
 #endif /* ITT_ARCH_MIPS */
 
 #ifndef ITT_ARCH_MIPS64
 #  define ITT_ARCH_MIPS64  6
 #endif /* ITT_ARCH_MIPS64 */
 
+#ifndef ITT_ARCH_RISCV64
+#  define ITT_ARCH_RISCV64  7
+#endif /* ITT_ARCH_RISCV64 */
+
 #ifndef ITT_ARCH
 #  if defined _M_IX86 || defined __i386__
 #    define ITT_ARCH ITT_ARCH_IA32
 #  elif defined _M_X64 || defined _M_AMD64 || defined __x86_64__
 #    define ITT_ARCH ITT_ARCH_IA32E
 #  elif defined _M_IA64 || defined __ia64__
 #    define ITT_ARCH ITT_ARCH_IA64
 #  elif defined _M_ARM || defined __arm__
 #    define ITT_ARCH ITT_ARCH_ARM
 #  elif defined __powerpc64__
 #    define ITT_ARCH ITT_ARCH_PPC64
 #  elif defined __aarch64__
 #    define ITT_ARCH ITT_ARCH_AARCH64
 #  elif defined __mips__ && !defined __mips64
 #    define ITT_ARCH ITT_ARCH_MIPS
 #  elif defined __mips__ && defined __mips64
 #    define ITT_ARCH ITT_ARCH_MIPS64
+#  elif defined __riscv && __riscv_xlen == 64
+#    define ITT_ARCH ITT_ARCH_RISCV64
 #  endif
 #endif
 
 #ifdef __cplusplus
 #  define ITT_EXTERN_C extern "C"
 #  define ITT_EXTERN_C_BEGIN extern "C" {
 #  define ITT_EXTERN_C_END }
 #else

 {
     long result;
     __asm__ __volatile__("lock\nxadd %0,%1"
                           : "=r"(result),"=m"(*(volatile int*)ptr)
                           : "0"(addend), "m"(*(volatile int*)ptr)
                           : "memory");
     return result;
 }
-#elif ITT_ARCH==ITT_ARCH_ARM || ITT_ARCH==ITT_ARCH_PPC64 || ITT_ARCH==ITT_ARCH_AARCH64 || ITT_ARCH==ITT_ARCH_MIPS ||  ITT_ARCH==ITT_ARCH_MIPS64
+#elif ITT_ARCH == ITT_ARCH_ARM || ITT_ARCH == ITT_ARCH_PPC64 ||                \
+    ITT_ARCH == ITT_ARCH_AARCH64 || ITT_ARCH == ITT_ARCH_MIPS ||               \
+    ITT_ARCH == ITT_ARCH_MIPS64 || ITT_ARCH == ITT_ARCH_RISCV64
 #define __TBB_machine_fetchadd4(addr, val) __sync_fetch_and_add(addr, val)
 #endif /* ITT_ARCH==ITT_ARCH_IA64 */
 #ifndef ITT_SIMPLE_INIT
 ITT_INLINE long
 __itt_interlocked_increment(volatile long* ptr) ITT_INLINE_ATTRIBUTE;
 ITT_INLINE long __itt_interlocked_increment(volatile long* ptr)
 {
     return __TBB_machine_fetchadd4(ptr, 1) + 1L;

runtime/src/z_Linux_asm.S

 .Lfunc_end0:
   .size __kmp_invoke_microtask, .Lfunc_end0-.Lfunc_begin0
   .cfi_endproc
 
 // -- End  __kmp_invoke_microtask
 
 #endif /* KMP_ARCH_PPC64 */
 
+#if KMP_ARCH_RISCV64
+
+//------------------------------------------------------------------------
+//
+// typedef void (*microtask_t)(int *gtid, int *tid, ...);
+//
+// int __kmp_invoke_microtask(microtask_t pkfn, int gtid, int tid, int argc,
+//                            void *p_argv[]
+// #if OMPT_SUPPORT
+//                            ,
+//                            void **exit_frame_ptr
+// #endif
+//                            ) {
+// #if OMPT_SUPPORT
+//   *exit_frame_ptr = OMPT_GET_FRAME_ADDRESS(0);
+// #endif
+//
+//   (*pkfn)(&gtid, &tid, argv[0], ...);
+//
+// #if OMPT_SUPPORT
+//   *exit_frame_ptr = 0;
+// #endif
+//
+//   return 1;
+// }
+//
+// Parameters:
+//   a0: pkfn
+//   a1: gtid
+//   a2: tid
+//   a3: argc
+//   a4: p_argv
+//   a5: exit_frame_ptr
+//
+// Locals:
+//   __exit_frame: exit_frame_ptr param pushed on stack so it can be restored
+//                 after the call to pkfn
+//   __gtid: gtid param pushed on stack so can pass &gtid to pkfn
+//   __tid: tid param pushed on stack so can pass &tid to pkfn
+//
+// Temp. registers:
+//
+//  t0: used to calculate the dynamic stack size / used to hold pkfn address
+//  t1: used as temporary for stack placement calculation
+//  t2: used as temporary for stack arguments
+//  t3: used as temporary for number of remaining pkfn parms
+//  t4: used to traverse p_argv array
+//
+// return: a0 (always 1/TRUE)
+//
+
+// When OMPT is to be used we need to reserve 8 extra bytes on the stack to
+// save a5, so the offsets from the fp defined here are shifted 8 bytes
+#if OMPT_SUPPORT
+__exit_frame = 24
+__gtid = 28
+__tid = 32
+#else
+__gtid = 20
+__tid = 24
+#endif
+
+// -- Begin __kmp_invoke_microtask
+// mark_begin;
+  .text
+  .globl  __kmp_invoke_microtask
+  .p2align  1
+  .type __kmp_invoke_microtask,@function
+__kmp_invoke_microtask:
+  .cfi_startproc
+
+  // First, save ra and fp
+  addi  sp, sp, -16
+  sd  ra, 8(sp)
+  sd  fp, 0(sp)
+  addi  fp, sp, 16
+  .cfi_def_cfa  fp, 0
+  .cfi_offset ra, -8
+  .cfi_offset fp, -16
+
+  // Compute the dynamic stack size:
+  //
+  // - We need 8 bytes for storing 'gtid' and 'tid', so we can pass them by
+  //   reference
+  // - We need 8 bytes for each argument that cannot be passed to the 'pkfn'
+  //   function by register. Given that we have 8 of such registers (a[0-7])
+  //   and two + 'argc' arguments (consider &gtid and &tid), we need to
+  //   reserve max(0, argc - 6)*8 extra bytes
+  // - If OMPT is enabled we need 8 extra bytes for saving a5 register
+  // (caller-saved) before the function call
+  //
+  // The total number of bytes is then max(0, argc - 6)*8 + 8 + OMPT? 8:0
+
+  // Compute max(0, argc - 6) using the following bithack:
+  // max(0, x) = x - (x & (x >> 31)), where x := argc - 6
+  // Source: http://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
+  addi  t0, a3, -6
+  srai  t1, t0, 31
+  and t1, t0, t1
+  sub t0, t0, t1
+
+#if OMPT_SUPPORT
+  // Reserve 8 extra bytes for saving a5 register when OMPT is enabled
+  addi  t0, t0, 2
+#else
+  addi  t0, t0, 1
+#endif
+
+  slli  t0, t0, 3
+  sub sp, sp, t0
+
+  // Align the stack to 16 bytes
+  andi  sp, sp, -16
+
+  mv  t0, a0
+  mv  t3, a3
+  mv  t4, a4
+
+#if OMPT_SUPPORT
+  // Save frame pointer into exit_frame
+  sd  fp, 0(a5)
+  // Save exit_frame address in the stack
+  sd  a5, -__exit_frame(fp)
+#endif
+
+  // Prepare arguments for the pkfn function (first 8 using a0-a7 registers)
+
+  sw  a1, -__gtid(fp)
+  sw  a2, -__tid(fp)
+
+  addi  a0, fp, -__gtid
+  addi  a1, fp, -__tid
+
+  beqz  t3, .L_kmp_3
+  ld  a2, 0(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a3, 8(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a4, 16(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a5, 24(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a6, 32(t4)
+
+  addi  t3, t3, -1
+  beqz  t3, .L_kmp_3
+  ld  a7, 40(t4)
+
+  // Prepare any additional argument passed through the stack
+  addi  t4, t4, 48
+  mv  t1, sp
+  j .L_kmp_2
+.L_kmp_1:
+  ld  t2, 0(t4)
+  sd  t2, 0(t1)
+  addi  t4, t4, 8
+  addi  t1, t1, 8
+.L_kmp_2:
+  addi  t3, t3, -1
+  bnez  t3, .L_kmp_1
+
+.L_kmp_3:
+  // Call pkfn function
+  jalr  t0
+
+#if OMPT_SUPPORT
+  // Restore exit_frame address and clear exit_frame
+  ld  t0, -__exit_frame(fp)
+  sd  zero, 0(t0)
+#endif
+
+  // Restore stack and return
+
+  addi  a0, zero, 1
+
+  addi  sp, fp, -16
+  ld  fp, 0(sp)
+  ld  ra, 8(sp)
+  addi  sp, sp, 16
+  ret
+.Lfunc_end0:
+  .size __kmp_invoke_microtask, .Lfunc_end0-__kmp_invoke_microtask
+  .cfi_endproc
+
+// -- End  __kmp_invoke_microtask
+
+#endif /* KMP_ARCH_RISCV64 */
+
 #if KMP_ARCH_ARM || KMP_ARCH_MIPS
     .data
     .comm .gomp_critical_user_,32,8
     .data
     .align 4
     .global __kmp_unnamed_critical_addr
 __kmp_unnamed_critical_addr:
     .4byte .gomp_critical_user_
     .size __kmp_unnamed_critical_addr,4
 #endif /* KMP_ARCH_ARM */
 
-#if KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64
+#if KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64 || KMP_ARCH_RISCV64
     .data
     .comm .gomp_critical_user_,32,8
     .data
     .align 8
     .global __kmp_unnamed_critical_addr
 __kmp_unnamed_critical_addr:
     .8byte .gomp_critical_user_
     .size __kmp_unnamed_critical_addr,8
-#endif /* KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 */
+#endif /* KMP_ARCH_PPC64 || KMP_ARCH_AARCH64 || KMP_ARCH_MIPS64 ||
+          KMP_ARCH_RISCV64 */
 
 #if KMP_OS_LINUX
 # if KMP_ARCH_ARM
 .section .note.GNU-stack,"",%progbits
 # else
 .section .note.GNU-stack,"",@progbits
 # endif
 #endif

runtime/src/z_Linux_util.cpp

 
 } // __kmp_get_load_balance
 
 #endif // KMP_OS_DARWIN
 
 #endif // USE_LOAD_BALANCE
 
 #if !(KMP_ARCH_X86 || KMP_ARCH_X86_64 || KMP_MIC ||                            \
-      ((KMP_OS_LINUX || KMP_OS_DARWIN) && KMP_ARCH_AARCH64) || KMP_ARCH_PPC64)
+      ((KMP_OS_LINUX || KMP_OS_DARWIN) && KMP_ARCH_AARCH64) ||                 \
+      KMP_ARCH_PPC64 || KMP_ARCH_RISCV64)
 
 // we really only need the case with 1 argument, because CLANG always build
 // a struct of pointers to shared variables referenced in the outlined function
 int __kmp_invoke_microtask(microtask_t pkfn, int gtid, int tid, int argc,
                            void *p_argv[]
 #if OMPT_SUPPORT
                            ,
                            void **exit_frame_ptr

runtime/test/ompt/callback.h

#define print_possible_return_addresses(addr) \
   printf("%" PRIu64 ": current_address=%p or %p\n", ompt_get_thread_data()->value, \
          ((char *)addr) - 8, ((char *)addr) - 12)
 #elif KMP_ARCH_AARCH64
 // On AArch64 the NOP instruction is 4 bytes long, can be followed by inserted
 // store instruction (another 4 bytes long).
 #define print_possible_return_addresses(addr) \
   printf("%" PRIu64 ": current_address=%p or %p\n", ompt_get_thread_data()->value, \
          ((char *)addr) - 4, ((char *)addr) - 8)
+#elif KMP_ARCH_RISCV64
+// On RV64C the C.NOP instruction is 2 byte long. In addition, the compiler
+// inserts a data-movement instruction for non-void runtime functions which
+// accounts for another 2 or 4 bytes. Finally, a C.J instruction may appear
+// (adding 2 more bytes).
+#define print_possible_return_addresses(addr) \
+  printf("%" PRIu64 ": current_address=%p or %p or %p or %p\n", \
+         ompt_get_thread_data()->value, ((char *)addr) - 2, ((char *)addr) - 4, \
+         ((char *)addr) - 6, ((char *)addr) - 8)
 #else
 #error Unsupported target architecture, cannot determine address offset!
 #endif
 
 
 // This macro performs a somewhat similar job to print_current_address(), except
 // that it discards a certain number of nibbles from the address and only prints
 // the most significant bits / nibbles. This can be used for cases where the

runtime/tools/lib/Platform.pm

if ( defined( $arch ) ) {
         } elsif ( $arch =~ m{\Aaarch64} ) {
                 $arch = "aarch64";
         } elsif ( $arch =~ m{\Amic} ) {
             $arch = "mic";
         } elsif ( $arch =~ m{\Amips64} ) {
             $arch = "mips64";
         } elsif ( $arch =~ m{\Amips} ) {
             $arch = "mips";
+        } elsif ( $arch =~ m{\Ariscv64} ) {
+            $arch = "riscv64";
         } else {
             $arch = undef;
         }; # if
     }; # if
     return $arch;
 }; # sub canon_arch
 
 # Canonize Intel(R) Many Integrated Core Architecture name.
sub canon_mic_arch($) {
     my %legal = (
         "32"  => "IA-32 architecture",
         "32e" => "Intel(R) 64",
         "arm" => "ARM",
         "aarch64" => "AArch64",
         "mic" => "Intel(R) Many Integrated Core Architecture",
         "mips" => "MIPS",
         "mips64" => "MIPS64",
+        "riscv64" => "RISC-V (64-bit)",
     );
 
     sub legal_arch($) {
         my ( $arch ) = @_;
         $arch = canon_arch( $arch );
         if ( defined( $arch ) ) {
             $arch = $legal{ $arch };
         }; # if
# Detect host arch.
     } elsif ( $hardware_platform eq "ppc64" ) {
         $_host_arch = "ppc64";
     } elsif ( $hardware_platform eq "aarch64" ) {
         $_host_arch = "aarch64";
     } elsif ( $hardware_platform eq "mips64" ) {
         $_host_arch = "mips64";
     } elsif ( $hardware_platform eq "mips" ) {
         $_host_arch = "mips";
+    } elsif ( $hardware_platform eq "riscv64" ) {
+        $_host_arch = "riscv64";
     } else {
         die "Unsupported host hardware platform: \"$hardware_platform\"; stopped";
     }; # if
 }
 
 # Detect host OS.
 {
     my $operating_system = Uname::operating_system();

 =head2 Functions.
 
 =over
 
 =item B<canon_arch( $arch )>
 
 Input string is an architecture name to canonize. The function recognizes many variants, for example:
 C<32e>, C<Intel64>, C<Intel(R) 64>, etc. Returned string is a canononized architecture name,
-one of: C<32>, C<32e>, C<64>, C<arm>, C<ppc64le>, C<ppc64>, C<mic>, C<mips>, C<mips64>, or C<undef> is input string is not recognized.
+one of: C<32>, C<32e>, C<64>, C<arm>, C<ppc64le>, C<ppc64>, C<mic>, C<mips>, C<mips64>, C<riscv64> or C<undef> is input string is not recognized.
 
 =item B<legal_arch( $arch )>
 
 Input string is architecture name. The function recognizes the same variants as C<arch_canon()> does.
 Returned string is a name approved by Intel Legal, one of: C<IA-32 architecture>, C<Intel(R) 64>
 or C<undef> if input string is not recognized.
 
 =item B<arch_opt( $arch )>

runtime/tools/lib/Uname.pm

if ( 0 ) {
     } elsif ( $values{ machine } =~ m{\Appc64\z} ) {
         $values{ hardware_platform } = "ppc64";
     } elsif ( $values{ machine } =~ m{\Aaarch64\z} ) {
         $values{ hardware_platform } = "aarch64";
     } elsif ( $values{ machine } =~ m{\Amips64\z} ) {
         $values{ hardware_platform } = "mips64";
     } elsif ( $values{ machine } =~ m{\Amips\z} ) {
         $values{ hardware_platform } = "mips";
+    } elsif ( $values{ machine } =~ m{\Ariscv64\z} ) {
+        $values{ hardware_platform } = "riscv64";
     } else {
         die "Unsupported machine (\"$values{ machine }\") returned by POSIX::uname(); stopped";
     }; # if
 } elsif ( $^O eq "darwin" ) {
     if ( 0 ) {
     } elsif ( $values{ machine } eq "x86" or $values{ machine } eq "i386" ) {
         $values{ hardware_platform } =
             sub {

www/README.txt

 * IA-32 architecture
 * Intel(R) 64 architecture
 * Intel(R) Many Integrated Core Architecture
 * ARM* architecture
 * Aarch64 (64-bit ARM) architecture
 * IBM(R) Power architecture (big endian)
 * IBM(R) Power architecture (little endian)
 * MIPS and MIPS64 architectures
+* RISC-V 64 bit architecture
 
 Supported RTL Build Configurations
 ==================================
 
 Supported Architectures: IA-32 architecture, Intel(R) 64, and
 Intel(R) Many Integrated Core Architecture
 
               ----------------------------------------------

www/index.html

<ul>
       <li>ARM&reg;&nbsp; architecture processors</li>
       <li>PowerPC&trade;&nbsp; processors</li>
       <li>32 and 64 bit X86
         processors when compiled with clang, with the Intel compiler
         or with gcc, and also the Intel&reg;&nbsp;Xeon Phi&trade; product family, when compiled with
         the Intel compiler.
       </li>
       <li>MIPS and MIPS64</li>
+      <li>RISC-V 64-bit</li>
     </ul>
     Ports to other architectures and operating systems are welcome.
   </p>
 
   <p>A full OS and architecture compatibility matrix is in
     <a href="README.txt">README.txt</a>
   </p>