[OpenMP] Fix nvidia-cuda-toolkit detection on Debian/Ubuntu

[jdenny]

Summary

The OpenMP runtime's cmake scripts do not correctly locate the
libdevice that the Debian/Ubuntu package nvidia-cuda-toolkit currently
includes, at least on my Ubuntu 18.04.1 installation. This patch
fixes that for me.

This problem was discussed at length in D55269. D40453 added a
similar adjustment in clang, but reviewers of D55269 concluded that,
for the OpenMP runtime, the right place to address this problem is in
cmake's CUDA support. However, it was also suggested we could add a
workaround to OpenMP's cmake scripts now. This patch contains such a
workaround, which I've tried to design so that it will have no harmful
effect if cmake improves in the future.

nvidia-cuda-toolkit also needs improvements because its intended
monolithic CUDA tree shim, /usr/lib/cuda, has many empty directories,
such as bin. I reported that at:

https://bugs.launchpad.net/ubuntu/+source/nvidia-cuda-toolkit/+bug/1808999

[jdenny]

• Fix a comment typo.

• Update summary with nvidia-cuda-toolkit bug report I just filed.

[grokos]

Since a similar solution was adopted for clang, I think we should let this one land. After all, it's a matter of consistency between the two projects.

openmp/libomptarget/cmake/Modules/LibomptargetGetDependencies.cmake

116 ↗

(On Diff #178681)

There's a convention that cmake variables are capitalized. Also, cmake variables local to libomptarget should start with the prefix LIBOMPTARGET_. So this variable's name should be changed to LIBOMPTARGET_CUDA_TOOLKIT_ROOT_DIR_PRESET ("preset" makes more sense than "old").

172 ↗

(On Diff #178681)

LIBOMPTARGET_CUDA_LIBDEVICE_SUBDIR

[jdenny]

openmp/libomptarget/cmake/Modules/LibomptargetGetDependencies.cmake

116 ↗

(On Diff #178681)

Thanks for the review. Sure, I'll make those changes.

However, I had thought that non-cached or local variables are usually lowercase. Is there a distinction here I'm not seeing, or is that just a different style than you prefer here?

185 ↗

(On Diff #178681)

Should candidate_dir be LIBOMPTARGET_CANDIDATE_DIR?

[jdenny]

Apply variable renames suggested by grokos.

[jdenny]

Ping?

[grokos]

LGTM.

openmp/libomptarget/cmake/Modules/LibomptargetGetDependencies.cmake

116 ↗

(On Diff #178681)

True, local variables are lowercase, but non-cached variables are still capitalized. For LIBOMPTARGET_CUDA_TOOLKIT_ROOT_DIR_PRESET and LIBOMPTARGET_CUDA_LIBDEVICE_SUBDIR I prefer the capitalized version because these variables may be used outside this file in the future (although right now they are only used locally).

185 ↗

(On Diff #178681)

It's OK, it's only local.

[jdenny]

openmp/libomptarget/cmake/Modules/LibomptargetGetDependencies.cmake

116 ↗

(On Diff #178681)

So your decision to capitalize in this case is based on your expectations about how usage could evolve in the future. Conversely, the candidate_dir from the other comment could be considered non-local, but your expectation is that it won't ever be used that way. Does all that sound right? That's all fine. I just want to be sure I understand. Thanks.

[grokos]

openmp/libomptarget/cmake/Modules/LibomptargetGetDependencies.cmake

116 ↗

(On Diff #178681)

Yes, that's pretty much it :)

[jdenny]

openmp/libomptarget/cmake/Modules/LibomptargetGetDependencies.cmake

116 ↗

(On Diff #178681)

Thanks, and thanks for the review!

openmp/trunk/libomptarget/cmake/Modules/LibomptargetGetDependencies.cmake

 
 mark_as_advanced(
   LIBOMPTARGET_DEP_LIBFFI_INCLUDE_DIRS 
   LIBOMPTARGET_DEP_LIBFFI_LIBRARIES)
   
 ################################################################################
 # Looking for CUDA...
 ################################################################################
+if (CUDA_TOOLKIT_ROOT_DIR)
+  set(LIBOMPTARGET_CUDA_TOOLKIT_ROOT_DIR_PRESET TRUE)
+endif()
 find_package(CUDA QUIET)
 
 set(LIBOMPTARGET_DEP_CUDA_FOUND ${CUDA_FOUND})
 set(LIBOMPTARGET_DEP_CUDA_INCLUDE_DIRS ${CUDA_INCLUDE_DIRS})
 
 mark_as_advanced(
   LIBOMPTARGET_DEP_CUDA_FOUND 
   LIBOMPTARGET_DEP_CUDA_INCLUDE_DIRS)

 endif()
 
 find_package_handle_standard_args(
   LIBOMPTARGET_DEP_CUDA_DRIVER
   DEFAULT_MSG
   LIBOMPTARGET_DEP_CUDA_DRIVER_LIBRARIES)
 
 mark_as_advanced(LIBOMPTARGET_DEP_CUDA_DRIVER_LIBRARIES)
+
+################################################################################
+# Looking for CUDA libdevice subdirectory
+#
+# Special case for Debian/Ubuntu to have nvidia-cuda-toolkit work
+# out of the box. More info on http://bugs.debian.org/882505
+################################################################################
+
+set(LIBOMPTARGET_CUDA_LIBDEVICE_SUBDIR nvvm/libdevice)
+
+# Don't alter CUDA_TOOLKIT_ROOT_DIR if the user specified it, if a value was
+# already cached for it, or if it already has libdevice.  Otherwise, on
+# Debian/Ubuntu, look where the nvidia-cuda-toolkit package normally installs
+# libdevice.
+if (NOT LIBOMPTARGET_CUDA_TOOLKIT_ROOT_DIR_PRESET AND
+    NOT EXISTS
+      "${CUDA_TOOLKIT_ROOT_DIR}/${LIBOMPTARGET_CUDA_LIBDEVICE_SUBDIR}")
+  find_program(LSB_RELEASE lsb_release)
+  if (LSB_RELEASE)
+    execute_process(COMMAND ${LSB_RELEASE} -is
+      OUTPUT_VARIABLE LSB_RELEASE_ID
+      OUTPUT_STRIP_TRAILING_WHITESPACE)
+    set(candidate_dir /usr/lib/cuda)
+    if ((LSB_RELEASE_ID STREQUAL "Debian" OR LSB_RELEASE_ID STREQUAL "Ubuntu")
+        AND EXISTS "${candidate_dir}/${LIBOMPTARGET_CUDA_LIBDEVICE_SUBDIR}")
+      set(CUDA_TOOLKIT_ROOT_DIR "${candidate_dir}" CACHE PATH
+          "Toolkit location." FORCE)
+    endif()
+  endif()
+endif()