[OpenMP] Test unified shared memory tests only on systems that support it.

[Meinersbur]

Summary

Add a REQUIRES: unified_shared_memory option to tests that use #pragma omp requires unified_shared_memory.

For CUDA, the feature tag is derived from LIBOMPTARGET_DEP_CUDA_ARCH which itself is derived using cuda_select_nvcc_arch_flags. The latter determines which compute capability the GPU in the system supports. To ensure that this is the CUDA arch being used, we could also set the -Xopenmp-target -march= flag.
In the absence of an NVIDIA GPU, LIBOMPTARGET_DEP_CUDA_ARCH will be 35. That is, in that case we are assuming unified_shared_memory is not available. CUDA plugin testing could be disabled entirely in this case, but this currently depends on LIBOMPTARGET_CAN_LINK_LIBCUDA OR LIBOMPTARGET_FORCE_DLOPEN_LIBCUDA, not on whether the hardware is actually available.

For all other targets, nothing changes and we are assuming unified shared memory is available. This might need refinement if not the case.

This tries to fix the OpenMP Offloading Buildbot that, although brand-new, only has a Pascal-generation (sm_61) GPU installed. Hence, tests that require unified shared memory are currently failing. I wish I had known in advance.

[tianshilei1992]

LGTM

[protze.joachim]

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake. Please add checks there to identify unified-shared-memory support.

[Meinersbur]

In D101498#2724916, @protze.joachim wrote:

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake. Please add checks there to identify unified-shared-memory support.

Could you please suggest a mechanism that tests support for unified virtual memory in CUDA? According to https://developer.nvidia.com/blog/beyond-gpu-memory-limits-unified-memory-pascal/, the underlaying page migration mechanism is supported since Pascal, making me wonder why clang insists on a Volta-generation GPU for #pragma omp requires unified_shared_memory.

[Meinersbur]

In D101498#2724916, @protze.joachim wrote:

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake.

The file does not contain target-specific features, but "Set up testing infrastructure" (According to the comment in the top-level CMakeLists.txt). At the point when it is invoked, libomptarget has not been processed yet, making it impossible to determine target-dependent features.

[protze.joachim]

In D101498#2727812, @Meinersbur wrote:

In D101498#2724916, @protze.joachim wrote:

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake.

The file does not contain target-specific features, but "Set up testing infrastructure" (According to the comment in the top-level CMakeLists.txt). At the point when it is invoked, libomptarget has not been processed yet, making it impossible to determine target-dependent features.

Ok, I'd still prefer that we keep the logic to determine the feature support in CMake, where we have access to all CMake variables (without passing them into lit one by one).

Kelvin reported a different list of tests as unexpectedly passing in D101326. His list includes api.c.
On my system (Tesla V100/sm_70), close_manual.c succeeds, while the others don't.

For me it looks like REQUIRES: unified_shared_memory might not be sharp enough to mark those tests.
Probably we should go with UNSUPPORTED: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62 and set the value for the architecture available in the system.

openmp/libomptarget/test/unified_shared_memory/close_enter_exit.c

4–5

If the requires works, this line should go away

openmp/libomptarget/test/unified_shared_memory/close_modifier.c

1–2

If the requires works, this line should go away

openmp/libomptarget/test/unified_shared_memory/shared_update.c

2–3

If the requires works, this line should go away

[Meinersbur]

In D101498#2729100, @protze.joachim wrote:

In D101498#2727812, @Meinersbur wrote:

In D101498#2724916, @protze.joachim wrote:

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake.

The file does not contain target-specific features, but "Set up testing infrastructure" (According to the comment in the top-level CMakeLists.txt). At the point when it is invoked, libomptarget has not been processed yet, making it impossible to determine target-dependent features.

The add_openmp_testsuite call in the dest directory tests at the moment has no differentiation between different targets. To you intend a

if (CURRENT_TARGET STREQUAL "nvptx64-nvidia-cuda")

in there?
Alternatively, add_openmp_testsuite could be called from the plugins themselves where these feature tests occur anyway (my preferred option).

However, I don't see a gain, we still have to pass a new cmake variable to lit; E.g. UNSUPPORTED suggestion still requires LIBOMPTARGET_DEP_CUDA_ARCH.

For me it looks like REQUIRES: unified_shared_memory might not be sharp enough to mark those tests.
Probably we should go with UNSUPPORTED: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62 and set the value for the architecture available in the system.

Not disagreeing that an additional sm_* items in the feature list would not be useful, but REQUIRES: unified_shared_memory is much more descriptive and does not require updating each and every test file if it changes (either because new target without USM are added or patches such as D101595).

[Meinersbur]

@protze.joachim ping

I suggest to commit this for now; a refactoring patch can be applied on top of it

[protze.joachim]

Ok, let's see what people with different test systems report.

[Meinersbur]

• Remove XFAIL lines that are there because of USM

[JonChesterfield]

@ronlieb do you know how amdgcn can test if USM is supported? Wonder if that's tied up in target id

openmp/libomptarget/test/lit.cfg

 for feature in config.test_compiler_features:
     config.available_features.add(feature)
 
 if config.libomptarget_debug:
   config.available_features.add('libomptarget-debug')
 
 config.available_features.add(config.libomptarget_current_target)
 
+# Determine whether the test system supports unified memory.
+# For CUDA, this is the case with compute capability 70 (Volta) or higher.
+# For all other targets, we currently assume it is.
+supports_unified_shared_memory = True
+if config.libomptarget_current_target.startswith('nvptx'):
+  try:
+    cuda_arch = int(config.cuda_test_arch)
+    if cuda_arch < 70:
+      supports_unified_shared_memory = False
+  except ValueError:
+    # If the architecture is invalid, assume it is supported.
+    supports_unified_shared_memory = True
+if supports_unified_shared_memory:
+   config.available_features.add('unified_shared_memory')
+
 # Setup environment to find dynamic library at runtime
 if config.operating_system == 'Windows':
     append_dynamic_library_path('PATH', config.library_dir, ";")
     append_dynamic_library_path('PATH', config.omp_host_rtl_directory, ";")
 elif config.operating_system == 'Darwin':
     append_dynamic_library_path('DYLD_LIBRARY_PATH', config.library_dir, ":")
     append_dynamic_library_path('DYLD_LIBRARY_PATH', \
         config.omp_host_rtl_directory, ";")

openmp/libomptarget/test/lit.site.cfg.in

 @AUTO_GEN_COMMENT@
 
 config.test_c_compiler = "@OPENMP_TEST_C_COMPILER@"
 config.test_cxx_compiler = "@OPENMP_TEST_CXX_COMPILER@"
 config.test_compiler_features = @OPENMP_TEST_COMPILER_FEATURES@
 config.test_openmp_flags = "@OPENMP_TEST_OPENMP_FLAGS@"
 config.test_extra_flags = "@OPENMP_TEST_FLAGS@"
 config.cuda_path = "@CUDA_TOOLKIT_ROOT_DIR@"
 config.cuda_libdir = "@CUDA_LIBDIR@"
+config.cuda_test_arch = "@LIBOMPTARGET_DEP_CUDA_ARCH@"
 config.libomptarget_obj_root = "@CMAKE_CURRENT_BINARY_DIR@/@CURRENT_TARGET@"
 config.library_dir = "@LIBOMPTARGET_LIBRARY_DIR@"
 config.omp_header_directory = "@LIBOMPTARGET_OPENMP_HEADER_FOLDER@"
 config.omp_host_rtl_directory = "@LIBOMPTARGET_OPENMP_HOST_RTL_FOLDER@"
 config.operating_system = "@CMAKE_SYSTEM_NAME@"
 config.libomptarget_all_targets = "@LIBOMPTARGET_ALL_TARGETS@".split()
 config.libomptarget_current_target = "@CURRENT_TARGET@"
 config.libomptarget_filecheck = "@OPENMP_FILECHECK_EXECUTABLE@"
 config.libomptarget_not = "@OPENMP_NOT_EXECUTABLE@"
 config.libomptarget_debug = @LIBOMPTARGET_DEBUG@
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@CMAKE_CURRENT_SOURCE_DIR@/lit.cfg")

openmp/libomptarget/test/mapping/present/unified_shared_memory.c

 // RUN: %libomptarget-compile-generic -fopenmp-version=51
 // RUN: %libomptarget-run-generic 2>&1 \
 // RUN: | %fcheck-generic
 
+// REQUIRES: unified_shared_memory
+
 
 #include <stdio.h>
 
 // The runtime considers unified shared memory to be always present.
 #pragma omp requires unified_shared_memory
 
 int main() {
   int i;

openmp/libomptarget/test/unified_shared_memory/close_enter_exit.c

 // RUN: %libomptarget-compile-run-and-check-generic
 
+// REQUIRES: unified_shared_memory
 // UNSUPPORTED: clang-6, clang-7, clang-8, clang-9
-// XFAIL: nvptx64-nvidia-cuda
 
 // Fails on amdgcn with error: GPU Memory Error
 // XFAIL: amdgcn-amd-amdhsa
 
 #include <omp.h>
 #include <stdio.h>
 
 #pragma omp requires unified_shared_memory
 

openmp/libomptarget/test/unified_shared_memory/close_manual.c

 // RUN: %libomptarget-compile-run-and-check-generic
 
+// REQUIRES: unified_shared_memory
+
 #include <omp.h>
 #include <stdio.h>
 
 // ---------------------------------------------------------------------------
 // Various definitions copied from OpenMP RTL
 
 extern void __tgt_register_requires(int64_t);
 

openmp/libomptarget/test/unified_shared_memory/close_modifier.c

 // RUN: %libomptarget-compile-run-and-check-generic
-// XFAIL: nvptx64-nvidia-cuda
 
+// REQUIRES: unified_shared_memory
 // UNSUPPORTED: clang-6, clang-7, clang-8, clang-9
 
 // amdgcn does not have printf definition
 // XFAIL: amdgcn-amd-amdhsa
 
 #include <omp.h>
 #include <stdio.h>
 

openmp/libomptarget/test/unified_shared_memory/shared_update.c

 // RUN: %libomptarget-compile-run-and-check-generic
-// XFAIL: nvptx64-nvidia-cuda
+
+// REQUIRES: unified_shared_memory
 
 // amdgcn does not have printf definition
 // XFAIL: amdgcn-amd-amdhsa
 
 #include <stdio.h>
 #include <omp.h>
 
 // ---------------------------------------------------------------------------