Page MenuHomePhabricator

[OpenMP] Test unified shared memory tests only on systems that support it.
ClosedPublic

Authored by Meinersbur on Apr 28 2021, 3:55 PM.

Details

Summary

Add a REQUIRES: unified_shared_memory option to tests that use #pragma omp requires unified_shared_memory.

For CUDA, the feature tag is derived from LIBOMPTARGET_DEP_CUDA_ARCH which itself is derived using cuda_select_nvcc_arch_flags. The latter determines which compute capability the GPU in the system supports. To ensure that this is the CUDA arch being used, we could also set the -Xopenmp-target -march= flag.
In the absence of an NVIDIA GPU, LIBOMPTARGET_DEP_CUDA_ARCH will be 35. That is, in that case we are assuming unified_shared_memory is not available. CUDA plugin testing could be disabled entirely in this case, but this currently depends on LIBOMPTARGET_CAN_LINK_LIBCUDA OR LIBOMPTARGET_FORCE_DLOPEN_LIBCUDA, not on whether the hardware is actually available.

For all other targets, nothing changes and we are assuming unified shared memory is available. This might need refinement if not the case.

This tries to fix the OpenMP Offloading Buildbot that, although brand-new, only has a Pascal-generation (sm_61) GPU installed. Hence, tests that require unified shared memory are currently failing. I wish I had known in advance.

Diff Detail

Event Timeline

Meinersbur created this revision.Apr 28 2021, 3:55 PM
Meinersbur requested review of this revision.Apr 28 2021, 3:55 PM
This revision is now accepted and ready to land.Apr 28 2021, 5:08 PM
protze.joachim requested changes to this revision.Apr 29 2021, 12:37 AM

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake. Please add checks there to identify unified-shared-memory support.

This revision now requires changes to proceed.Apr 29 2021, 12:37 AM

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake. Please add checks there to identify unified-shared-memory support.

Could you please suggest a mechanism that tests support for unified virtual memory in CUDA? According to https://developer.nvidia.com/blog/beyond-gpu-memory-limits-unified-memory-pascal/, the underlaying page migration mechanism is supported since Pascal, making me wonder why clang insists on a Volta-generation GPU for #pragma omp requires unified_shared_memory.

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake.

The file does not contain target-specific features, but "Set up testing infrastructure" (According to the comment in the top-level CMakeLists.txt). At the point when it is invoked, libomptarget has not been processed yet, making it impossible to determine target-dependent features.

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake.

The file does not contain target-specific features, but "Set up testing infrastructure" (According to the comment in the top-level CMakeLists.txt). At the point when it is invoked, libomptarget has not been processed yet, making it impossible to determine target-dependent features.

Ok, I'd still prefer that we keep the logic to determine the feature support in CMake, where we have access to all CMake variables (without passing them into lit one by one).

Kelvin reported a different list of tests as unexpectedly passing in D101326. His list includes api.c.
On my system (Tesla V100/sm_70), close_manual.c succeeds, while the others don't.

For me it looks like REQUIRES: unified_shared_memory might not be sharp enough to mark those tests.
Probably we should go with UNSUPPORTED: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62 and set the value for the architecture available in the system.

openmp/libomptarget/test/unified_shared_memory/close_enter_exit.c
4–5

If the requires works, this line should go away

openmp/libomptarget/test/unified_shared_memory/close_modifier.c
1–2

If the requires works, this line should go away

openmp/libomptarget/test/unified_shared_memory/shared_update.c
2–3

If the requires works, this line should go away

The logic to identify whether certain features are supported for testing is in openmp/cmake/OpenMPTesting.cmake.

The file does not contain target-specific features, but "Set up testing infrastructure" (According to the comment in the top-level CMakeLists.txt). At the point when it is invoked, libomptarget has not been processed yet, making it impossible to determine target-dependent features.

The add_openmp_testsuite call in the dest directory tests at the moment has no differentiation between different targets. To you intend a

if (CURRENT_TARGET STREQUAL "nvptx64-nvidia-cuda")

in there?
Alternatively, add_openmp_testsuite could be called from the plugins themselves where these feature tests occur anyway (my preferred option).

However, I don't see a gain, we still have to pass a new cmake variable to lit; E.g. UNSUPPORTED suggestion still requires LIBOMPTARGET_DEP_CUDA_ARCH.

For me it looks like REQUIRES: unified_shared_memory might not be sharp enough to mark those tests.
Probably we should go with UNSUPPORTED: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62 and set the value for the architecture available in the system.

Not disagreeing that an additional sm_* items in the feature list would not be useful, but REQUIRES: unified_shared_memory is much more descriptive and does not require updating each and every test file if it changes (either because new target without USM are added or patches such as D101595).

@protze.joachim ping

I suggest to commit this for now; a refactoring patch can be applied on top of it

protze.joachim accepted this revision.May 11 2021, 12:43 AM

Ok, let's see what people with different test systems report.

This revision is now accepted and ready to land.May 11 2021, 12:43 AM
Meinersbur edited the summary of this revision. (Show Details)
  • Remove XFAIL lines that are there because of USM
This revision was landed with ongoing or failed builds.May 13 2021, 9:08 AM
This revision was automatically updated to reflect the committed changes.

@ronlieb do you know how amdgcn can test if USM is supported? Wonder if that's tied up in target id