[libc] Enable running libc unit tests on AMDGPU

[jhuber6]

Summary

The previous patches added the necessary support for global constructors
used to register tests. This patch enables the AMDGPU target to build
and run the unit tests on the GPU. Currently this only tests the ctype
tests, but adding more should be straightforward from here on.

[sivachandra]

libc/cmake/modules/LLVMLibCCheckCpuFeatures.cmake

47

Why was this moved?

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Do we ever want to run unit tests the GPU? If no, then I think the right thing to do is to convert the unit test targets to add_libc_test targets which will then do the right thing.

libc/test/src/CMakeLists.txt

13

Why do you need this? If you don't include math tests in entrypoints.txt, this will not become active anyway, correct?

[jhuber6]

libc/cmake/modules/LLVMLibCCheckCpuFeatures.cmake

47

Previously it prevented the above functions from being defined which are queried in the test setup.

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

I mostly did this because it was much less noise than converting everything to the new format. Since I figured that would be dependent on the rest of the libc project contributors to transition to the new test framework. This was the easy stopgap.

libc/test/src/CMakeLists.txt

13

It was active when I tested it, I don't have any math entrypoints in the GPU entrypoint file. It built fine, but it spammed like 30 of these warnings about MPFR into the terminal so I just wanted to silence that.

[sivachandra]

libc/cmake/modules/LLVMLibCCheckCpuFeatures.cmake

47

Ah! This is not ideal but I think doing it correctly is beyond the scope of this patch.

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

What we should be doing is, when adding an entrypoint to the GPU's entrypoints.txt, convert the entrypoint's unit tests to add_libc_test.

libc/test/src/CMakeLists.txt

12

If it is just this annoying warning, you can modify this to:

message(VERBOSE "Math test ...")

Then, that warning will only be displayed if --log-level is set to VERBOSE or lower.

[jhuber6]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Alright, this would require reordering the add_subdirectory() calls in the test CMake and exiting early for GPU builds.

[sivachandra]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

The goal was that it should not require such reordering. If you are requiring it, then it is a bug somewhere. Can you share the error messages you are seeing?

[jhuber6]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Various ones, right now I see the build fail because I don't have stdio configured so we get undefined references to _IOBUF. Also some others fail because stdio.h goes to the host version which has the traditional failure mode.

[sivachandra]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Can you give repro instructions that I can use to see what you are seeing?

[jhuber6]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Try configuring CMake with with -DLLVM_ENABLE_RUNTIMES=libc -DLIBC_GPU_ARCHITECTURES=gfx90a -DLIBC_GPU_TEST_ARCHITECTURE=gfx90a. That should allow you to build the tests. Then remove the guard at line libc/test/CMakeLists.txt:25 below. You will need lld.

[jhuber6]

Changing to specifically disable tests for the targets we have in the
entrypoints but don't support hermetic tests. These can be removed as more of
the test suite is brought over to the new format.

[jhuber6]

Update to support more entrypoints. This gets rid of the string conversion
entrypoints until we can figure out how to make the errno asserts work. Also I
had to somewhat hack around weird compiler behaviour. It's not ideal but since
it still tests the functionality I think it should be sufficient for now as I
prepare a bug report.

[sivachandra]

libc/config/gpu/entrypoints.txt

71 ↗

(On Diff #519232)

You can keep the entrypoints but skip the tests with a detailed comment. Up to you.

libc/test/src/__support/File/CMakeLists.txt

4 ↗

(On Diff #519232)

May the actual reason is that we don't yet support files on the GPU? If yes, then may be just say that?

libc/test/src/__support/blockstore_test.cpp

95 ↗

(On Diff #519232)

Without the context of this patch, the word "these" becomes vague. Can you may be say:

Combing this test with the above test makes the AMDGPU backend
generate code which hangs.
TODO: Investigate why the AMDGPU backend is generating such code.

libc/test/src/stdio/CMakeLists.txt

10

Those tests which depend on entrypoints will not be included. I think you want to return() on line 326?

[jhuber6]

Addressing comments.

libc/cmake/modules/LLVMLibCCheckCpuFeatures.cmake

 # ------------------------------------------------------------------------------
 # Cpu features definition and flags
 # ------------------------------------------------------------------------------
 
 # Initialize ALL_CPU_FEATURES as empty list.
 set(ALL_CPU_FEATURES "")
 
-if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
-  return()
-endif()
-
 if(${LIBC_TARGET_ARCHITECTURE_IS_X86})
   set(ALL_CPU_FEATURES SSE2 SSE4_2 AVX AVX2 AVX512F AVX512BW FMA)
   set(LIBC_COMPILE_OPTIONS_NATIVE -march=native)
 elseif(${LIBC_TARGET_ARCHITECTURE_IS_AARCH64})
   set(LIBC_COMPILE_OPTIONS_NATIVE -mcpu=native)
 endif()
 
 # Making sure ALL_CPU_FEATURES is sorted.
function(_intersection output_var list1 list2)
   foreach(element IN LISTS list1)
     if("${list2}" MATCHES "(^|;)${element}(;|$)")
       list(APPEND tmp "${element}")
     endif()
   endforeach()
   set(${output_var} ${tmp} PARENT_SCOPE)
 endfunction()
 
+if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+  return()
+endif()
+
 set(AVAILABLE_CPU_FEATURES "")
 if(LIBC_CROSSBUILD)
   # If we are doing a cross build, we will just assume that all CPU features
   # are available.
   set(AVAILABLE_CPU_FEATURES ${ALL_CPU_FEATURES})
 else()
   # Try compile a C file to check if flag is supported.
   set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)

libc/test/CMakeLists.txt

 )
 
 if(LIBC_TARGET_ARCHITECTURE_IS_GPU AND
    (NOT TARGET libc.utils.gpu.loader OR NOT TARGET libc.startup.gpu.crt1))
   message(WARNING "Cannot build libc GPU tests, missing loader implementation")
   return()
 endif()
 
-if(NOT LIBC_TARGET_ARCHITECTURE_IS_GPU)
+if(NOT LIBC_GPU_TARGET_ARCHITECTURE_IS_NVPTX)
   add_subdirectory(src)
   add_subdirectory(utils)
 endif()
 
 if(LLVM_LIBC_FULL_BUILD AND NOT LIBC_TARGET_OS_IS_BAREMETAL)
   add_subdirectory(IntegrationTest)
 endif()
 

libc/test/src/CMakeLists.txt

 function(add_fp_unittest name)
   cmake_parse_arguments(
     "MATH_UNITTEST"
     "NEED_MPFR" # Optional arguments
     "" # Single value arguments
     "LINK_LIBRARIES" # Multi-value arguments
     ${ARGN}
   )
 
   if(MATH_UNITTEST_NEED_MPFR)
     if(NOT LIBC_TESTS_CAN_USE_MPFR)
-      message("WARNING: Math test ${name} will be skipped as MPFR library is not available.")
+      message(VERBOSE "Math test ${name} will be skipped as MPFR library is not available.")
       return()
     endif()
   endif()
 
   if(MATH_UNITTEST_NEED_MPFR)
     list(APPEND MATH_UNITTEST_LINK_LIBRARIES libcMPFRWrapper libc_math_test_utils -lmpfr -lgmp)
   endif()
   list(APPEND MATH_UNITTEST_LINK_LIBRARIES LibcFPTestHelpers)
 

libc/test/src/__support/CMakeLists.txt

+if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+  return()
+endif()
+
 add_custom_target(libc_support_unittests)
 
 add_libc_unittest(
   blockstore_test
   SUITE
     libc_support_unittests
   SRCS
     blockstore_test.cpp

libc/test/src/errno/CMakeLists.txt

-if(NOT LLVM_LIBC_FULL_BUILD)
+if(NOT LLVM_LIBC_FULL_BUILD OR LIBC_TARGET_ARCHITECTURE_IS_GPU)
   return()
 endif()
 
 add_custom_target(libc_errno_unittests)
 
 add_libc_unittest(
   errno_test
   SUITE
     libc_errno_unittests
   SRCS
     errno_test.cpp
   DEPENDS
     libc.src.errno.errno
 )

libc/test/src/stdio/CMakeLists.txt

+if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+  return()
+endif()
+
 add_custom_target(libc_stdio_unittests)
 
 add_libc_unittest(
   fileop_test
   SUITE
     libc_stdio_unittests
   SRCS
     fileop_test.cpp
   DEPENDS
     libc.include.stdio
     libc.src.errno.errno
     libc.src.stdio.clearerr
     libc.src.stdio.fclose
     libc.src.stdio.feof

libc/test/src/stdlib/CMakeLists.txt

+if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+  return()
+endif()
+
 add_custom_target(libc_stdlib_unittests)
 
 add_libc_unittest(
   atof_test
   SUITE
     libc_stdlib_unittests
   SRCS
     atof_test.cpp

libc/test/src/string/CMakeLists.txt

+if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+  return()
+endif()
+
 add_custom_target(libc_string_unittests)
 
 add_subdirectory(memory_utils)
 
 add_libc_unittest(
   bcopy_test
   SUITE
     libc_string_unittests