[libc] Enable integration tests targeting the GPU

[jhuber6]

Summary

This patch enables integration tests running on the GPU. This uses the
RPC interface implemented in D145913 to compile the necessary
dependencies for the integration test object. We can then use this to
compile the objects for the GPU directly and execute them using the AMD
HSA loader combined with its RPC server. For example, the compiler is
performing the following actions to execute the integration tests.

$ clang++ --target=amdgcn-amd-amdhsa -mcpu=gfx1030 -nostdlib -flto -ffreestanding \
    crt1.o io.o quick_exit.o test.o rpc_client.o args_test.o -o image
$ ./amdhsa_loader image 1 2 5
args_test.cpp:24: Expected 'my_streq(argv[3], "3")' to be true, but is false

This currently only works with a single threaded client implementation
running on AMDGPU. Further work will implement multiple clients for AMD
and the ability to run on NVPTX as well.

Depends on D145913

[sivachandra]

The libc infra components LGTM.

libc/test/integration/startup/gpu/args_test.cpp

1

I suppose you are not using the linux one because you don't have envp on the gpu? Or, is it future work? Also, unrelated to this change, what about global contructor/destructor on the GPUs?

[jhuber6]

libc/test/integration/startup/gpu/args_test.cpp

1

I could copy the env over as well. Could do that in a follow-up patch

[jhuber6]

We should only set the special libc.utils.gpu.loader target once for the target under test.

[lntue]

libc/cmake/modules/LLVMLibCTestRules.cmake

415–416

Nit: Maybe create a list for future extension like:

set(INTEGRATION_TARGETS linux gpu)
if (NOT (${LIBC_TARGET_OS} IN_LIST INTEGRATION_TARGETS))
...

[jhuber6]

Addressing comments

[sivachandra]

libc/test/integration/startup/gpu/args_test.cpp

1

If you copy the envp over, will the env vars checks pass?

[jhuber6]

libc/test/integration/startup/gpu/args_test.cpp

1

Presumably, I'd just need to give it the same treatment that I gave for argv.

[sivachandra]

OK from the libc structuring point of view.

libc/cmake/modules/LLVMLibCTestRules.cmake

415–416

Nit: supported_platforms may be, to avoid confusion with the "target cpu" or "build target".

libc/test/integration/startup/gpu/args_test.cpp

1

You can do it as a follow up of course. At that point, can we merge the linux and gpu tests?

[jhuber6]

libc/test/integration/startup/gpu/args_test.cpp

1

Yeah, we could do that. I'm wondering if I should add some more basic tests for the GPU right now. I'd like to be able to test the functionality of the entrypoints, but that's beyond the scope of these integration tests for now I believe.

[JonChesterfield]

Global ctor/dtor is tricky. I have a suspicion hip and openmp have done different things there. When to run the dtor is a challenge, and how many threads the ctors run with may have implementation divergence.

[jhuber6]

In D146256#4202584, @JonChesterfield wrote:

Global ctor/dtor is tricky. I have a suspicion hip and openmp have done different things there. When to run the dtor is a challenge, and how many threads the ctors run with may have implementation divergence.

I guess in the context of the loader launching the kernel, a device constructor would just be a kernel called before the main and a destructor is after. The problem with HIP and OpenMP is that they want to do host stuff as well, so the ordering is weird since we need to run the "device" constructors after main has been called. It's certainly possible to implement, but it would need to be exported as a list of kernels to call, which is somewhat reinventing OpenMP and HIP.

[JonChesterfield]

Libc could walk the ctors array before main and the dtors after, as normal function pointers. We might even be able to persuade hip or openmp to lower their ctor/dtors as function pointers in the same arrays, where they happen to launch a special kernel that walks at least one of them.

[sivachandra]

In D146256#4202873, @JonChesterfield wrote:

Libc could walk the ctors array before main and the dtors after, as normal function pointers. We might even be able to persuade hip or openmp to lower their ctor/dtors as function pointers in the same arrays, where they happen to launch a special kernel that walks at least one of them.

On linux/ELF platforms, the libc already runs through the function pointers in .init_array and .fini_arry`: https://github.com/llvm/llvm-project/blob/main/libc/startup/linux/x86_64/start.cpp#L111

If the GPU side of things can follow the same pattern, then it should be straight forward to extend the Linux ways to GPUs.

libc/CMakeLists.txt

if(LLVM_LIBC_FULL_BUILD)
   add_subdirectory(startup)
 endif()
 
 # The lib and test directories are added at the very end as tests
 # and libraries potentially draw from the components present in all
 # of the other directories.
 # TODO: Add testing support for the libc GPU target.
 add_subdirectory(lib)
-if(LLVM_INCLUDE_TESTS AND NOT LIBC_TARGET_ARCHITECTURE_IS_GPU)
+if(LLVM_INCLUDE_TESTS)
   add_subdirectory(test)
   add_subdirectory(fuzzing)
 endif()
 
 if(LIBC_INCLUDE_BENCHMARKS)
   add_subdirectory(benchmarks)
 endif()
 

libc/cmake/modules/LLVMLibCTestRules.cmake

 #     ENV <list of environment variables to set before running the test>
 #     COMPILE_OPTIONS <list of special compile options for this target>
 #   )
 #
 # The DEPENDS list can be empty. If not empty, it should be a list of
 # targets added with add_entrypoint_object or add_object_library.
 function(add_integration_test test_name)
   get_fq_target_name(${test_name} fq_target_name)
-  if(NOT (${LIBC_TARGET_OS} STREQUAL "linux"))
+  if(NOT (${LIBC_TARGET_OS} STREQUAL "linux") AND
+     NOT (${LIBC_TARGET_OS} STREQUAL "gpu"))
     message(STATUS "Skipping ${fq_target_name} as it is not available on ${LIBC_TARGET_OS}.")
     return()
   endif()
   cmake_parse_arguments(
     "INTEGRATION_TEST"
     "" # No optional arguments
     "SUITE" # Single value arguments
     "SRCS;HDRS;DEPENDS;ARGS;ENV;COMPILE_OPTIONS" # Multi-value arguments
function(add_integration_test test_name)
 
   get_fq_target_name(${test_name}.libc fq_libc_target_name)
 
   get_fq_deps_list(fq_deps_list ${INTEGRATION_TEST_DEPENDS})
   list(APPEND fq_deps_list
       # All integration tests use the operating system's startup object and need
       # to inherit the same dependencies.
       libc.startup.${LIBC_TARGET_OS}.crt1
+      libc.test.IntegrationTest.test
       # We always add the memory functions objects. This is because the
       # compiler's codegen can emit calls to the C memory functions.
       libc.src.string.bcmp
       libc.src.string.bzero
       libc.src.string.memcmp
       libc.src.string.memcpy
       libc.src.string.memmove
       libc.src.string.memset
target_include_directories(
     ${fq_build_target_name}
     PRIVATE
       ${LIBC_SOURCE_DIR}
       ${LIBC_BUILD_DIR}
       ${LIBC_BUILD_DIR}/include
   )
   target_compile_options(${fq_build_target_name}
                          PRIVATE -fpie -ffreestanding ${INTEGRATION_TEST_COMPILE_OPTIONS})
+  # The GPU build requires overriding the default CMake triple and architecture.
+  if(LIBC_GPU_TARGET_ARCHITECTURE_IS_AMDGPU)
+    target_compile_options(${fq_build_target_name} PRIVATE
+                           -mcpu=${LIBC_GPU_TARGET_ARCHITECTURE} -emit-llvm
+                           --target=${LIBC_GPU_TARGET_TRIPLE})
+  elseif(LIBC_GPU_TARGET_ARCHITECTURE_IS_NVPTX)
+    target_compile_options(${fq_build_target_name} PRIVATE
+                           -march=${LIBC_GPU_TARGET_ARCHITECTURE}
+                           --target=${LIBC_GPU_TARGET_TRIPLE})
+  endif()
+
   target_link_options(${fq_build_target_name} PRIVATE -nostdlib -static)
   target_link_libraries(${fq_build_target_name} ${fq_target_name}.__libc__
+                        libc.startup.${LIBC_TARGET_OS}.crt1
                         libc.test.IntegrationTest.test)
   add_dependencies(${fq_build_target_name}
                    libc.test.IntegrationTest.test
                    ${INTEGRATION_TEST_DEPENDS})
 
+  if(TARGET libc.utils.gpu.loader)
+    get_target_property(gpu_loader_exe libc.utils.gpu.loader "EXECUTABLE")
+  endif()
+
   add_custom_target(
     ${fq_target_name}
-    COMMAND ${INTEGRATION_TEST_ENV} $<TARGET_FILE:${fq_build_target_name}> ${INTEGRATION_TEST_ARGS}
+    COMMAND ${INTEGRATION_TEST_ENV}
+            $<$<BOOL:${LIBC_TARGET_ARCHITECTURE_IS_GPU}>:${gpu_loader_exe}>
+            $<TARGET_FILE:${fq_build_target_name}> ${INTEGRATION_TEST_ARGS}
     COMMENT "Running integration test ${fq_target_name}"
   )
   add_dependencies(${INTEGRATION_TEST_SUITE} ${fq_target_name})
 endfunction(add_integration_test)

libc/startup/gpu/amdgpu/CMakeLists.txt

COMPILE_OPTIONS
     -mcpu=${LIBC_GPU_TARGET_ARCHITECTURE}
     -emit-llvm # AMDGPU's intermediate object file format is bitcode.
     --target=${LIBC_GPU_TARGET_TRIPLE}
   NO_GPU_BUNDLE # Compile this file directly without special GPU handling.
 )
 get_fq_target_name(crt1 fq_name)
 
 # Ensure that clang uses the correct linker for this object type.
-target_link_libraries(${fq_name} PUBLIC 
-                      "--target=${LIBC_GPU_TARGET_TRIPLE}" "-flto")
+target_link_libraries(
+  ${fq_name}
+  PUBLIC
+  "-mcpu=${LIBC_GPU_TARGET_ARCHITECTURE}"
+  "--target=${LIBC_GPU_TARGET_TRIPLE}"
+  "-flto"
+)

libc/test/CMakeLists.txt

-add_subdirectory(UnitTest)
+add_custom_target(check-libc)
+add_custom_target(libc-unit-tests)
+add_dependencies(check-libc libc-unit-tests)
 
-if(LLVM_LIBC_FULL_BUILD AND NOT
-  (LIBC_TARGET_ARCHITECTURE_IS_GPU OR LIBC_TARGET_OS_IS_BAREMETAL))
-  add_subdirectory(IntegrationTest)
-endif()
+add_custom_target(exhaustive-check-libc)
+add_custom_target(libc-long-running-tests)
 
 add_header_library(
   errno_setter_matcher
   HDRS
     ErrnoSetterMatcher.h
   DEPENDS
     libc.src.errno.errno
 )
 
-add_custom_target(check-libc)
-add_custom_target(libc-unit-tests)
-add_dependencies(check-libc libc-unit-tests)
-
-add_custom_target(exhaustive-check-libc)
-add_custom_target(libc-long-running-tests)
+if(NOT TARGET libc.utils.gpu.loader OR NOT TARGET libc.startup.gpu.amdgpu.crt1)
+  message(WARNING "Cannot build libc GPU tests, missing loader implementation")
+  return()
+endif()
 
+if(NOT LIBC_TARGET_ARCHITECTURE_IS_GPU)
+  add_subdirectory(UnitTest)
   add_subdirectory(src)
   add_subdirectory(utils)
+endif()
+
+if(LLVM_LIBC_FULL_BUILD AND NOT LIBC_TARGET_OS_IS_BAREMETAL)
+  add_subdirectory(IntegrationTest)
+endif()
 
 if(NOT LLVM_LIBC_FULL_BUILD)
   return()
 endif()
 
-if(NOT ${LIBC_TARGET_OS} STREQUAL "linux")
-  # Integration tests are currently only available for linux.
+if(NOT ${LIBC_TARGET_OS} STREQUAL "linux" AND
+   NOT ${LIBC_TARGET_OS} STREQUAL "gpu")
+  # Integration tests are currently only available for linux and the GPU.
   return()
 endif()
 add_subdirectory(integration)

libc/test/IntegrationTest/CMakeLists.txt

+if(LIBC_GPU_TARGET_ARCHITECTURE_IS_AMDGPU)
+  set(TEST_COMPILE_FLAGS
+    -mcpu=${LIBC_GPU_TARGET_ARCHITECTURE}
+    -emit-llvm # AMDGPU's intermediate object file format is bitcode.
+    --target=${LIBC_GPU_TARGET_TRIPLE}
+  )
+elseif(LIBC_GPU_TARGET_ARCHITECTURE_IS_NVPTX)
+  set(TEST_COMPILE_FLAGS
+    -march=${LIBC_GPU_TARGET_ARCHITECTURE}
+    --target=${LIBC_GPU_TARGET_TRIPLE}
+  )
+endif()
+
 add_object_library(
   test
   SRCS
     test.cpp
+  COMPILE_OPTIONS
+    ${TEST_COMPILE_FLAGS}
   HDRS
     test.h
   DEPENDS
     libc.src.__support.OSUtil.osutil
+  NO_GPU_BUNDLE # Compile this file directly without special GPU handling.
 )

libc/test/integration/startup/gpu/CMakeLists.txt

+add_custom_target(libc-startup-tests)
+add_dependencies(libc-integration-tests libc-startup-tests)
+
+add_integration_test(
+  startup_args_test
+  SUITE libc-startup-tests
+  SRCS
+    args_test.cpp
+  ARGS
+    1 2 3
+)

libc/test/integration/startup/gpu/args_test.cpp

+//===-- Loader test to check args to main ---------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "test/IntegrationTest/test.h"
+
+static bool my_streq(const char *lhs, const char *rhs) {
+  const char *l, *r;
+  for (l = lhs, r = rhs; *l != '\0' && *r != '\0'; ++l, ++r)
+    if (*l != *r)
+      return false;
+
+  return *l == '\0' && *r == '\0';
+}
+
+TEST_MAIN(int argc, char **argv) {
+  ASSERT_TRUE(argc == 4);
+  ASSERT_TRUE(my_streq(argv[1], "1"));
+  ASSERT_TRUE(my_streq(argv[2], "2"));
+  ASSERT_TRUE(my_streq(argv[3], "3"));
+
+  return 0;
+}

libc/utils/gpu/loader/CMakeLists.txt

 add_library(gpu_loader OBJECT Main.cpp)
 target_include_directories(gpu_loader PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
 
 find_package(hsa-runtime64 QUIET 1.2.0 HINTS ${CMAKE_INSTALL_PREFIX} PATHS /opt/rocm)
 if(hsa-runtime64_FOUND)
   add_subdirectory(amdgpu)
+else()
+  message(STATUS "Skipping HSA loader for gpu target, no HSA was detected")
+endif()
+
+# Add a custom target to be used for testing.
+if(TARGET amdhsa_loader)
+  add_custom_target(libc.utils.gpu.loader)
+  add_dependencies(libc.utils.gpu.loader amdhsa_loader)
+  set_target_properties(
+    libc.utils.gpu.loader
+    PROPERTIES
+      EXECUTABLE "$<TARGET_FILE_NAME:amdhsa_loader>"
+  )
 endif()