[libc] Enable running libc unit tests on AMDGPU

[jhuber6]

Summary

The previous patches added the necessary support for global constructors
used to register tests. This patch enables the AMDGPU target to build
and run the unit tests on the GPU. Currently this only tests the ctype
tests, but adding more should be straightforward from here on.

[sivachandra]

libc/cmake/modules/LLVMLibCCheckCpuFeatures.cmake

51

Why was this moved?

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Do we ever want to run unit tests the GPU? If no, then I think the right thing to do is to convert the unit test targets to add_libc_test targets which will then do the right thing.

libc/test/src/CMakeLists.txt

13

Why do you need this? If you don't include math tests in entrypoints.txt, this will not become active anyway, correct?

[jhuber6]

libc/cmake/modules/LLVMLibCCheckCpuFeatures.cmake

51

Previously it prevented the above functions from being defined which are queried in the test setup.

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

I mostly did this because it was much less noise than converting everything to the new format. Since I figured that would be dependent on the rest of the libc project contributors to transition to the new test framework. This was the easy stopgap.

libc/test/src/CMakeLists.txt

13

It was active when I tested it, I don't have any math entrypoints in the GPU entrypoint file. It built fine, but it spammed like 30 of these warnings about MPFR into the terminal so I just wanted to silence that.

[sivachandra]

libc/cmake/modules/LLVMLibCCheckCpuFeatures.cmake

51

Ah! This is not ideal but I think doing it correctly is beyond the scope of this patch.

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

What we should be doing is, when adding an entrypoint to the GPU's entrypoints.txt, convert the entrypoint's unit tests to add_libc_test.

libc/test/src/CMakeLists.txt

12

If it is just this annoying warning, you can modify this to:

message(VERBOSE "Math test ...")

Then, that warning will only be displayed if --log-level is set to VERBOSE or lower.

[jhuber6]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Alright, this would require reordering the add_subdirectory() calls in the test CMake and exiting early for GPU builds.

[sivachandra]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

The goal was that it should not require such reordering. If you are requiring it, then it is a bug somewhere. Can you share the error messages you are seeing?

[jhuber6]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Various ones, right now I see the build fail because I don't have stdio configured so we get undefined references to _IOBUF. Also some others fail because stdio.h goes to the host version which has the traditional failure mode.

[sivachandra]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Can you give repro instructions that I can use to see what you are seeing?

[jhuber6]

libc/cmake/modules/LLVMLibCTestRules.cmake

74 ↗

(On Diff #518164)

Try configuring CMake with with -DLLVM_ENABLE_RUNTIMES=libc -DLIBC_GPU_ARCHITECTURES=gfx90a -DLIBC_GPU_TEST_ARCHITECTURE=gfx90a. That should allow you to build the tests. Then remove the guard at line libc/test/CMakeLists.txt:25 below. You will need lld.

[jhuber6]

Changing to specifically disable tests for the targets we have in the
entrypoints but don't support hermetic tests. These can be removed as more of
the test suite is brought over to the new format.

[jhuber6]

Update to support more entrypoints. This gets rid of the string conversion
entrypoints until we can figure out how to make the errno asserts work. Also I
had to somewhat hack around weird compiler behaviour. It's not ideal but since
it still tests the functionality I think it should be sufficient for now as I
prepare a bug report.

[sivachandra]

libc/config/gpu/entrypoints.txt

71

You can keep the entrypoints but skip the tests with a detailed comment. Up to you.

libc/test/src/__support/File/CMakeLists.txt

4

May the actual reason is that we don't yet support files on the GPU? If yes, then may be just say that?

libc/test/src/__support/blockstore_test.cpp

95

Without the context of this patch, the word "these" becomes vague. Can you may be say:

Combing this test with the above test makes the AMDGPU backend
generate code which hangs.
TODO: Investigate why the AMDGPU backend is generating such code.

libc/test/src/stdio/CMakeLists.txt

6

Those tests which depend on entrypoints will not be included. I think you want to return() on line 326?

[jhuber6]

Addressing comments.

libc/cmake/modules/LLVMLibCCheckCpuFeatures.cmake

 # ------------------------------------------------------------------------------
 # Cpu features definition and flags
 # ------------------------------------------------------------------------------
 
 # Initialize ALL_CPU_FEATURES as empty list.
 set(ALL_CPU_FEATURES "")
 
-if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
-  return()
-endif()
-
 if(${LIBC_TARGET_ARCHITECTURE_IS_X86})
   set(ALL_CPU_FEATURES SSE2 SSE4_2 AVX AVX2 AVX512F AVX512BW FMA)
   set(LIBC_COMPILE_OPTIONS_NATIVE -march=native)
 elseif(${LIBC_TARGET_ARCHITECTURE_IS_AARCH64})
   set(LIBC_COMPILE_OPTIONS_NATIVE -mcpu=native)
 endif()
 
 # Making sure ALL_CPU_FEATURES is sorted.
 list(SORT ALL_CPU_FEATURES)
 
 # Function to check whether the target CPU supports the provided set of features.
 # Usage:
 # cpu_supports(
 #   <output variable>
 #   <list of cpu features>
 # )
 function(cpu_supports output_var features)
+  if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+    unset(${output_var} PARENT_SCOPE)
+    return()
+  endif()
   _intersection(var "${LIBC_CPU_FEATURES}" "${features}")
   if("${var}" STREQUAL "${features}")
     set(${output_var} TRUE PARENT_SCOPE)
   else()
     unset(${output_var} PARENT_SCOPE)
   endif()
 endfunction()
 
 # ------------------------------------------------------------------------------
 # Internal helpers and utilities.
 # ------------------------------------------------------------------------------
 
 # Computes the intersection between two lists.
 function(_intersection output_var list1 list2)
   foreach(element IN LISTS list1)
     if("${list2}" MATCHES "(^|;)${element}(;|$)")
       list(APPEND tmp "${element}")
     endif()
   endforeach()
   set(${output_var} ${tmp} PARENT_SCOPE)
 endfunction()
 
 set(AVAILABLE_CPU_FEATURES "")
 if(LIBC_CROSSBUILD)
   # If we are doing a cross build, we will just assume that all CPU features
   # are available.
   set(AVAILABLE_CPU_FEATURES ${ALL_CPU_FEATURES})
 else()
   # Try compile a C file to check if flag is supported.
   set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)
   foreach(feature IN LISTS ALL_CPU_FEATURES)

libc/config/gpu/entrypoints.txt

set(TARGET_LIBC_ENTRYPOINTS
     libc.src.string.strrchr
     libc.src.string.strspn
     libc.src.string.strstr
     libc.src.string.strtok
     libc.src.string.strtok_r
 
     # stdlib.h entrypoints
     libc.src.stdlib.abs
-    libc.src.stdlib.atoi
-    libc.src.stdlib.atof
-    libc.src.stdlib.atol
-    libc.src.stdlib.atoll
     libc.src.stdlib.labs
     libc.src.stdlib.llabs
-    libc.src.stdlib.strtod
-    libc.src.stdlib.strtof
-    libc.src.stdlib.strtol
-    libc.src.stdlib.strtold
-    libc.src.stdlib.strtoll
-    libc.src.stdlib.strtoul
-    libc.src.stdlib.strtoull
 
     # stdlib.h entrypoints
     libc.src.stdlib._Exit
     libc.src.stdlib.atexit
     libc.src.stdlib.exit
 
     # Only implemented in the test suite
     libc.src.stdlib.malloc

libc/docs/gpu/support.rst

 
 stdlib.h
 --------
 
 =============  =========  ============
 Function Name  Available  RPC Required
 =============  =========  ============
 abs            |check|
-atoi           |check|
-atof           |check|
-atol           |check|
-atoll          |check|
+atoi           
+atof           
+atol           
+atoll          
 labs           |check|
 llabs          |check|
-strtod         |check|
-strtof         |check|
-strtol         |check|
-strtold        |check|
-strtoll        |check|
-strtoul        |check|
-strtoull       |check|
+strtod         
+strtof         
+strtol         
+strtold        
+strtoll        
+strtoul        
+strtoull       
 =============  =========  ============

libc/test/CMakeLists.txt

 )
 
 if(LIBC_TARGET_ARCHITECTURE_IS_GPU AND
    (NOT TARGET libc.utils.gpu.loader OR NOT TARGET libc.startup.gpu.crt1))
   message(WARNING "Cannot build libc GPU tests, missing loader implementation")
   return()
 endif()
 
-if(NOT LIBC_TARGET_ARCHITECTURE_IS_GPU)
+if(NOT LIBC_GPU_TARGET_ARCHITECTURE_IS_NVPTX)
   add_subdirectory(src)
   add_subdirectory(utils)
 endif()
 
 if(LLVM_LIBC_FULL_BUILD AND NOT LIBC_TARGET_OS_IS_BAREMETAL)
   add_subdirectory(IntegrationTest)
 endif()
 

libc/test/UnitTest/CMakeLists.txt

 )
 
 foreach(lib LibcFPTestHelpers LibcFPExceptionHelpers LibcMemoryHelpers 
             LibcPrintfHelpers LibcScanfHelpers)
   target_include_directories(${lib} PUBLIC ${LIBC_SOURCE_DIR})
   target_compile_options(${lib} PRIVATE -fno-exceptions -fno-rtti)
   target_link_libraries(${lib} LibcUnitTest)
 endforeach()
+
+# The GPU needs these flags applied to override the system triple.
+if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+  foreach(lib LibcMemoryHelpers)
+    target_include_directories(${lib} PRIVATE ${LIBC_BUILD_DIR}/include)
+    target_compile_options(${lib}
+        PRIVATE ${LIBC_HERMETIC_TEST_COMPILE_OPTIONS} -ffreestanding -nostdlib -nostdlib++)
+  endforeach()
+endif()

libc/test/src/CMakeLists.txt

 function(add_fp_unittest name)
   cmake_parse_arguments(
     "MATH_UNITTEST"
     "NEED_MPFR;HERMETIC_TEST_ONLY" # Optional arguments
     "" # Single value arguments
     "LINK_LIBRARIES" # Multi-value arguments
     ${ARGN}
   )
 
   if(MATH_UNITTEST_NEED_MPFR)
     if(NOT LIBC_TESTS_CAN_USE_MPFR)
-      message("WARNING: Math test ${name} will be skipped as MPFR library is not available.")
+      message(VERBOSE "Math test ${name} will be skipped as MPFR library is not available.")
       return()
     endif()
   endif()
 
   if(MATH_UNITTEST_HERMETIC_TEST_ONLY)
     set(test_type HERMETIC_TEST_ONLY)
   endif()
   if(MATH_UNITTEST_NEED_MPFR)
     if(MATH_UNITTEST_HERMETIC_TEST_ONLY)

libc/test/src/__support/CMakeLists.txt

SUITE
     libc-support-tests
   SRCS
     integer_to_string_test.cpp
   DEPENDS
     libc.src.__support.integer_to_string
     libc.src.__support.CPP.string_view
 )
 
+# The GPU does not support varargs currently.
+if(NOT LIBC_TARGET_ARCHITECTURE_IS_GPU)
   add_libc_test(
     arg_list_test
     SUITE
       libc-support-tests
     SRCS
       arg_list_test.cpp
     DEPENDS
       libc.src.__support.arg_list
   )
+endif()
 
 add_libc_test(
   uint_test
   SUITE
     libc-support-tests
   SRCS
     uint_test.cpp
   DEPENDS

libc/test/src/__support/File/CMakeLists.txt

-if(NOT (TARGET libc.src.__support.threads.mutex))
+if(NOT (TARGET libc.src.__support.threads.mutex) OR LIBC_TARGET_ARCHITECTURE_IS_GPU)
   # Not all platforms have a mutex implementation. If mutex is unvailable,
-  # we just skip everything about files.
+  # we just skip everything about files. The GPU does not currently support
+  # files as well.
   return()
 endif()
 
 add_libc_test(
   file_test
   SUITE
     libc-support-tests
   SRCS

libc/test/src/__support/blockstore_test.cpp

 TEST_F(LlvmLibcBlockStoreTest, PopulateAndIterateReverse8) {
   populate_and_iterate<4, 8, true>();
 }
 
 TEST_F(LlvmLibcBlockStoreTest, PopulateAndIterateReverse10) {
   populate_and_iterate<4, 10, true>();
 }
 
-TEST_F(LlvmLibcBlockStoreTest, Back) {
-  back_test<false>();
-  back_test<true>();
-}
+TEST_F(LlvmLibcBlockStoreTest, Back) { back_test<false>(); }
+
+// FIXME: Combing this test with the above test makes the AMDGPU backend
+// generate code which hangs. This should be fixed in the clang compiler.
+TEST_F(LlvmLibcBlockStoreTest, BackReverse) { back_test<true>(); }
 
 TEST_F(LlvmLibcBlockStoreTest, Empty) {
   empty_test<false>();
   empty_test<true>();
 }

libc/test/src/errno/CMakeLists.txt

-if(NOT LLVM_LIBC_FULL_BUILD)
+if(NOT LLVM_LIBC_FULL_BUILD OR LIBC_TARGET_ARCHITECTURE_IS_GPU)
   return()
 endif()
 
 add_custom_target(libc_errno_unittests)
 
 add_libc_unittest(
   errno_test
   SUITE
     libc_errno_unittests
   SRCS
     errno_test.cpp
   DEPENDS
     libc.src.errno.errno
 )

libc/test/src/stdio/CMakeLists.txt

 add_custom_target(libc_stdio_unittests)
 
 add_libc_unittest(
   fileop_test
   SUITE
     libc_stdio_unittests
   SRCS
     fileop_test.cpp
   DEPENDS
     libc.include.stdio
     libc.src.errno.errno
     libc.src.stdio.clearerr
     libc.src.stdio.fclose
     libc.src.stdio.feof
DEPENDS
     libc.src.stdio.fopen
     libc.src.stdio.fread
     libc.src.stdio.fseek
     libc.src.stdio.ftell
     libc.src.stdio.fwrite
     libc.src.stdio.setvbuf
 )
 
+if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
+  return()
+endif()
+
 add_subdirectory(printf_core)
 add_subdirectory(scanf_core)
 add_subdirectory(testdata)

libc/test/src/string/StrchrTest.h

void findsNullTerminator() {
 
     // Should return null terminator.
     ASSERT_STREQ(Func(src, '\0'), "");
     // Source string should not change.
     ASSERT_STREQ(src, "abcde");
   }
 
   void findsLastBehindFirstNullTerminator() {
-    const char src[6] = {'a', 'a', '\0', 'b', '\0', 'c'};
+    static const char src[6] = {'a', 'a', '\0', 'b', '\0', 'c'};
     // 'b' is behind a null terminator, so should not be found.
     ASSERT_STREQ(Func(src, 'b'), nullptr);
     // Same goes for 'c'.
     ASSERT_STREQ(Func(src, 'c'), nullptr);
 
     // Should find the second of the two a's.
     ASSERT_STREQ(Func(src, 'a'), "a");
   }