[libc] Add rule named `add_libc_hermetic_test` which adds a hermetic test.

[sivachandra]

Summary

A convenience wrapper name add_libc_test is also added which adds both
a unit test and a hermetic test. The ctype tests have been switched over
to use add_libc_test.

[sivachandra]

I am sharing this early so that other can try it out. It is not in the shape I want to land so will be sending out more updates.

[jhuber6]

libc/cmake/modules/LLVMLibCTestRules.cmake

629

This is a typo I just fixed upstream.

679

Small change here just today that adds LOADER_ARGS to support calling tests with multiple threads on the GPU.

[jhuber6]

Thanks a lot for this work, I think it's getting us pretty close. Getting this to build for the GPU required adding the UnitTest and src directories in the CMake. Some other issues I've had when trying to build for the GPU.

1. Including stdlib.h in src/__support/CPP/string.h causes errors when they include the system headers. Presumably this is because we aren't providing this as a header in the GPU build yet?
2. The calls to ::free and ::realloc in src/__support/CPP/string.h did not compile when I tried, they were undefined in the module.
3. I got an LLVM crash when compiling the whole project targeting AMDGPU in LTO mode. This is the recommended way as AMDGPU doesn't have a true "ABI". Turning off LTO hides the crash but is a little more iffy on whether or not it's correct since there's no defined ABI for cross-TU calls.
4. My build was not picking up libc.src.__support.OSUtil.osutil in the final test for some reason, added it to the hermetic test and it worked.
5. We do not have __cxa_atexit defined
6. The UnitTest libraries need to have the explicit --target= and -mcpu= options passed in all cases to match the source.
7. After hacking around all of the above, the test built and runs with a "No tests run." message. I'm assuming this is because we are not handling the global constructors.

I'd appreciate some pointers on how to fix some of the above on the GPU side, I'm not as intimately familiar with the libc internals for registering constructors / exit functions.

libc/cmake/modules/LLVMLibCTestRules.cmake

568

Optional LOADER_ARGS here needed for the GPU.

679

libc/test/UnitTest/CMakeLists.txt

14

The GPU support will require overloading the triple and architecture for each one of these. So the above should work applied as compile options for each add_library under here.

[sivachandra]

Pass hermetic test compile args to the hermetic test framework also.

[sivachandra]

Add doc string for LOADER_ARGS.

[sivachandra]

In D148756#4283369, @jhuber6 wrote:

Thanks a lot for this work, I think it's getting us pretty close. Getting this to build for the GPU required adding the UnitTest and src directories in the CMake. Some other issues I've had when trying to build for the GPU.

1. Including stdlib.h in src/__support/CPP/string.h causes errors when they include the system headers. Presumably this is because we aren't providing this as a header in the GPU build yet?

I think so.

2. The calls to ::free and ::realloc in src/__support/CPP/string.h did not compile when I tried, they were undefined in the module.

Did not compile or did they produce linker errors? The allocator functions come from the hermetic test framework, just like with the integration test framework.

3. I got an LLVM crash when compiling the whole project targeting AMDGPU in LTO mode. This is the recommended way as AMDGPU doesn't have a true "ABI". Turning off LTO hides the crash but is a little more iffy on whether or not it's correct since there's no defined ABI for cross-TU calls.

Is this related to the libc in anyway?

4. My build was not picking up libc.src.__support.OSUtil.osutil in the final test for some reason, added it to the hermetic test and it worked.

Likely a missing dep somewhere? But, this patch does add OSUtil.osutil as a dep.

5. We do not have __cxa_atexit defined

This comes from the atexit implementation. This should be enabled for GPUs in some mannger.

6. The UnitTest libraries need to have the explicit --target= and -mcpu= options passed in all cases to match the source.

I have updated this patch to get all the relevant compile options.

7. After hacking around all of the above, the test built and runs with a "No tests run." message. I'm assuming this is because we are not handling the global constructors.

I'd appreciate some pointers on how to fix some of the above on the GPU side, I'm not as intimately familiar with the libc internals for registering constructors / exit functions.

Likely. I am happy to help/talk about how this works with normal ELF CPU binaries.

[jhuber6]

In D148756#4291622, @sivachandra wrote:

In D148756#4283369, @jhuber6 wrote:

Thanks a lot for this work, I think it's getting us pretty close. Getting this to build for the GPU required adding the UnitTest and src directories in the CMake. Some other issues I've had when trying to build for the GPU.

1. Including stdlib.h in src/__support/CPP/string.h causes errors when they include the system headers. Presumably this is because we aren't providing this as a header in the GPU build yet?

I think so.

I should probably enable a "malloc" that just allocates from a fixed size pool like we do in the integration tests, better than nothing.

2. The calls to ::free and ::realloc in src/__support/CPP/string.h did not compile when I tried, they were undefined in the module.

Did not compile or did they produce linker errors? The allocator functions come from the hermetic test framework, just like with the integration test framework.

Didn't compile, it was a C++ thing.

3. I got an LLVM crash when compiling the whole project targeting AMDGPU in LTO mode. This is the recommended way as AMDGPU doesn't have a true "ABI". Turning off LTO hides the crash but is a little more iffy on whether or not it's correct since there's no defined ABI for cross-TU calls.

Is this related to the libc in anyway?

Probably not, just noting that it'll be a roadblock. Once this support is in fully I'll create a reproducer and see if I can get it fixed.

4. My build was not picking up libc.src.__support.OSUtil.osutil in the final test for some reason, added it to the hermetic test and it worked.

Likely a missing dep somewhere? But, this patch does add OSUtil.osutil as a dep.

It used to be in the add_integration_test but that wasn't copied over to the hermetic test when I looked at it.

5. We do not have __cxa_atexit defined

This comes from the atexit implementation. This should be enabled for GPUs in some mannger.

Alright, I'll need to make a version that eschews the mutexes most likely.

7. After hacking around all of the above, the test built and runs with a "No tests run." message. I'm assuming this is because we are not handling the global constructors.

I'd appreciate some pointers on how to fix some of the above on the GPU side, I'm not as intimately familiar with the libc internals for registering constructors / exit functions.

Likely. I am happy to help/talk about how this works with normal ELF CPU binaries.

That sounds good, we can set up a time if you're free.

[sivachandra]

In D148756#4292265, @jhuber6 wrote:

I should probably enable a "malloc" that just allocates from a fixed size pool like we do in the integration tests, better than nothing.

This patch adds a HermeticTestUtils.cpp which does that.

2. The calls to ::free and ::realloc in src/__support/CPP/string.h did not compile when I tried, they were undefined in the module.

Did not compile or did they produce linker errors? The allocator functions come from the hermetic test framework, just like with the integration test framework.

Didn't compile, it was a C++ thing.

May be this: https://reviews.llvm.org/D149075

Alright, I'll need to make a version that eschews the mutexes most likely.

We need pass-through mutexes for other contexts also. So, we should have a more generic no-mutex solution.

[jhuber6]

No objections from me. Still a lot of work required to get it working on the GPU, but this is a very good step. Thanks.

libc/cmake/modules/LLVMLibCTestRules.cmake

648

Leftover?

[sivachandra]

libc/cmake/modules/LLVMLibCTestRules.cmake

648

Oops. I missed this comment. I have a few follow up changes so I will fix it in those patches.

libc/CMakeLists.txt

 
 option(LLVM_LIBC_FULL_BUILD "Build and test LLVM libc as if it is the full libc" OFF)
 option(LLVM_LIBC_IMPLEMENTATION_DEFINED_TEST_BEHAVIOR "Build LLVM libc tests assuming our implementation-defined behavior" ON)
 option(LLVM_LIBC_ENABLE_LINTING "Enables linting of libc source files" OFF)
 
 option(LIBC_GPU_BUILD "Build libc for the GPU. All CPU build options will be ignored." OFF)
 set(LIBC_TARGET_TRIPLE "" CACHE STRING "The target triple for the libc build.")
 
+set(LIBC_ENABLE_UNITTESTS ON)
+set(LIBC_ENABLE_HERMETIC_TESTS ON)
+
 # Defines LIBC_TARGET_ARCHITECTURE and associated macros.
 include(LLVMLibCArchitectures)
 
 if(LIBC_TARGET_ARCHITECTURE_IS_GPU)
   include(prepare_libc_gpu_build)
+  set(LIBC_ENABLE_UNITTESTS OFF)
 endif()
 
 include(LLVMLibCCheckMPFR)
 
 if(LLVM_LIBC_CLANG_TIDY)
   set(LLVM_LIBC_ENABLE_LINTING ON)
 endif()
 

libc/cmake/modules/LLVMLibCTestRules.cmake

 #    )
 function(create_libc_unittest fq_target_name)
   if(NOT LLVM_INCLUDE_TESTS)
     return()
   endif()
 
   cmake_parse_arguments(
     "LIBC_UNITTEST"
-    "NO_RUN_POSTBUILD;NO_LIBC_UNITTEST_TEST_MAIN" # Optional arguments
+    "NO_RUN_POSTBUILD" # Optional arguments
     "SUITE;CXX_STANDARD" # Single value arguments
     "SRCS;HDRS;DEPENDS;COMPILE_OPTIONS;LINK_LIBRARIES;FLAGS" # Multi-value arguments
     ${ARGN}
   )
   if(NOT LIBC_UNITTEST_SRCS)
     message(FATAL_ERROR "'add_libc_unittest' target requires a SRCS list of .cpp "
                         "files.")
   endif()
set_target_properties(${fq_build_target_name}
     PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
 
   add_dependencies(
     ${fq_build_target_name}
     ${fq_deps_list}
   )
 
   # LibcUnitTest should not depend on anything in LINK_LIBRARIES.
-  if(NO_LIBC_UNITTEST_TEST_MAIN)
-    list(APPEND link_libraries LibcUnitTest)
-  else()
-    list(APPEND link_libraries LibcUnitTest LibcUnitTestMain)
-  endif()
+  list(APPEND link_libraries LibcTestMain LibcUnitTest)
 
   target_link_libraries(${fq_build_target_name} PRIVATE ${link_libraries})
 
   if(NOT LIBC_UNITTEST_NO_RUN_POSTBUILD)
     add_custom_target(
       ${fq_target_name}
       COMMAND $<TARGET_FILE:${fq_build_target_name}>
       COMMENT "Running unit test ${fq_target_name}"
function(add_libc_fuzzer target_name)
   add_dependencies(libc-fuzzer ${fq_target_name})
 
   target_compile_options(${fq_target_name}
     PRIVATE
     ${LIBC_FUZZER_COMPILE_OPTIONS})
 
 endfunction(add_libc_fuzzer)
 
+# DEPRECATED: Use add_hermetic_test instead.
+#
 # Rule to add an integration test. An integration test is like a unit test
 # but does not use the system libc. Not even the startup objects from the
 # system libc are linked in to the final executable. The final exe is fully
 # statically linked. The libc that the final exe links to consists of only
 # the object files of the DEPENDS targets.
 # 
 # Usage:
 #   add_integration_test(
function(add_integration_test test_name)
   add_custom_target(
     ${fq_target_name}
     COMMAND ${test_cmd}
     COMMAND_EXPAND_LISTS
     COMMENT "Running integration test ${fq_target_name}"
   )
   add_dependencies(${INTEGRATION_TEST_SUITE} ${fq_target_name})
 endfunction(add_integration_test)
+
+set(LIBC_HERMETIC_TEST_COMPILE_OPTIONS ${LIBC_COMPILE_OPTIONS_DEFAULT}
+    -fpie -ffreestanding -fno-exceptions -fno-rtti)
+# The GPU build requires overriding the default CMake triple and architecture.
+if(LIBC_GPU_TARGET_ARCHITECTURE_IS_AMDGPU)
+  list(APPEND LIBC_HERMETIC_TEST_COMPILE_OPTIONS
+       -mcpu=${LIBC_GPU_TARGET_ARCHITECTURE} -flto --target=${LIBC_GPU_TARGET_TRIPLE})
+elseif(LIBC_GPU_TARGET_ARCHITECTURE_IS_NVPTX)
+  get_nvptx_compile_options(nvptx_options ${LIBC_GPU_TARGET_ARCHITECTURE})
+  list(APPEND ${nvptx_options} --target=${LIBC_GPU_TARGET_TRIPLE})
+endif()
+
+# Rule to add a hermetic test. A hermetic test is one whose executable is fully
+# statically linked and consists of pieces drawn only from LLVM's libc. Nothing,
+# including the startup objects, come from the system libc.
+#
+# Usage:
+#   add_libc_hermetic_test(
+#     <target name>
+#     SUITE <the suite to which the test should belong>
+#     SRCS <src1.cpp> [src2.cpp ...]
+#     HDRS [hdr1.cpp ...]
+#     DEPENDS <list of entrypoint or other object targets>
+#     ARGS <list of command line arguments to be passed to the test>
+#     ENV <list of environment variables to set before running the test>
+#     COMPILE_OPTIONS <list of special compile options for the test>
+#     LOADER_ARGS <list of special args to loaders (like the GPU loader)>
+#   )
+function(add_libc_hermetic_test test_name)
+  if(NOT TARGET libc.startup.${LIBC_TARGET_OS}.crt1)
+    message(VERBOSE "Skipping ${fq_target_name} as it is not available on ${LIBC_TARGET_OS}.")
+    return()
+  endif()
+  cmake_parse_arguments(
+    "HERMETIC_TEST"
+    "" # No optional arguments
+    "SUITE" # Single value arguments
+    "SRCS;HDRS;DEPENDS;ARGS;ENV;COMPILE_OPTIONS;LOADER_ARGS" # Multi-value arguments
+    ${ARGN}
+  )
+
+  if(NOT HERMETIC_TEST_SUITE)
+    message(FATAL_ERROR "SUITE not specified for ${fq_target_name}")
+  endif()
+  if(NOT HERMETIC_TEST_SRCS)
+    message(FATAL_ERROR "The SRCS list for add_integration_test is missing.")
+  endif()
+
+  get_fq_target_name(${test_name} fq_target_name)
+  get_fq_target_name(${test_name}.libc fq_libc_target_name)
+
+  get_fq_deps_list(fq_deps_list ${HERMETIC_TEST_DEPENDS})
+  list(APPEND fq_deps_list
+      # Hermetic tests use the platform's startup object. So, their deps also
+      # have to be collected.
+      libc.startup.${LIBC_TARGET_OS}.crt1
+      # We always add the memory functions objects. This is because the
+      # compiler's codegen can emit calls to the C memory functions.
+      libc.src.string.bcmp
+      libc.src.string.bzero
+      libc.src.string.memcmp
+      libc.src.string.memcpy
+      libc.src.string.memmove
+      libc.src.string.memset
+  )
+  list(REMOVE_DUPLICATES fq_deps_list)
+
+  # TODO: Instead of gathering internal object files from entrypoints,
+  # collect the object files with public names of entrypoints.
+  get_object_files_for_test(
+      link_object_files skipped_entrypoints_list ${fq_deps_list})
+  if(skipped_entrypoints_list)
+    set(msg "Skipping unittest ${fq_target_name} as it has missing deps: "
+            "${skipped_entrypoints_list}.")
+    message(STATUS ${msg})
+    return()
+  endif()
+  list(REMOVE_DUPLICATES link_object_files)
+
+  # Make a library of all deps
+  add_library(
+    ${fq_target_name}.__libc__
+    STATIC
+    EXCLUDE_FROM_ALL
+    ${link_object_files}
+  )
+  set_target_properties(${fq_target_name}.__libc__
+      PROPERTIES ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})
+  set_target_properties(${fq_target_name}.__libc__
+      PROPERTIES ARCHIVE_OUTPUT_NAME ${fq_target_name}.libc)
+
+  set(fq_build_target_name ${fq_target_name}.__build__)
+  add_executable(
+    ${fq_build_target_name}
+    EXCLUDE_FROM_ALL
+    # The NVIDIA 'nvlink' linker does not currently support static libraries.
+    $<$<BOOL:${LIBC_GPU_TARGET_ARCHITECTURE_IS_NVPTX}>:${link_object_files}>
+    ${HERMETIC_TEST_SRCS}
+    ${HERMETIC_TEST_HDRS}
+  )
+  set_target_properties(${fq_build_target_name}
+    PROPERTIES
+      RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+      #OUTPUT_NAME ${fq_target_name}
+  )
+  target_include_directories(
+    ${fq_build_target_name}
+    PRIVATE
+      ${LIBC_SOURCE_DIR}
+      ${LIBC_BUILD_DIR}
+      ${LIBC_BUILD_DIR}/include
+  )
+  target_compile_options(${fq_build_target_name}
+      PRIVATE ${LIBC_HERMETIC_TEST_COMPILE_OPTIONS} ${HERMETIC_TEST_COMPILE_OPTIONS})
+
+  target_link_options(${fq_build_target_name} PRIVATE -nostdlib -static)
+  target_link_libraries(
+    ${fq_build_target_name}
+    libc.startup.${LIBC_TARGET_OS}.crt1
+    LibcTestMain LibcHermeticTest
+    # The NVIDIA 'nvlink' linker does not currently support static libraries.
+    $<$<NOT:$<BOOL:${LIBC_GPU_TARGET_ARCHITECTURE_IS_NVPTX}>>:${fq_target_name}.__libc__>)
+  add_dependencies(${fq_build_target_name}
+                   LibcHermeticTest
+                   ${HERMETIC_TEST_DEPENDS})
+
+  # Tests on the GPU require an external loader utility to launch the kernel.
+  if(TARGET libc.utils.gpu.loader)
+    add_dependencies(${fq_build_target_name} libc.utils.gpu.loader)
+    get_target_property(gpu_loader_exe libc.utils.gpu.loader "EXECUTABLE")
+  endif()
+
+  set(test_cmd ${HERMETIC_TEST_ENV}
+      $<$<BOOL:${LIBC_TARGET_ARCHITECTURE_IS_GPU}>:${gpu_loader_exe}> ${HERMETIC_TEST_LOADER_ARGS}
+      $<TARGET_FILE:${fq_build_target_name}> ${HERMETIC_TEST_ARGS})
+  add_custom_target(
+    ${fq_target_name}
+    COMMAND ${test_cmd}
+    COMMAND_EXPAND_LISTS
+    COMMENT "Running hermetic test ${fq_target_name}"
+  )
+
+  add_dependencies(${HERMETIC_TEST_SUITE} ${fq_target_name})
+  add_dependencies(libc-hermetic-tests ${fq_target_name})
+endfunction(add_libc_hermetic_test)
+
+# A convenience function to add both a unit test as well as a hermetic test.
+function(add_libc_test test_name)
+  if(LIBC_ENABLE_UNITTESTS)
+    add_libc_unittest(${test_name}.__unit__ ${ARGN})
+  endif()
+  if(LIBC_ENABLE_HERMETIC_TESTS)
+    add_libc_hermetic_test(${test_name}.__hermetic__ ${ARGN})
+  endif()
+endfunction(add_libc_test)

libc/test/CMakeLists.txt

 add_custom_target(check-libc)
 add_custom_target(libc-unit-tests)
-add_dependencies(check-libc libc-unit-tests)
+add_custom_target(libc-hermetic-tests)
+add_dependencies(check-libc libc-unit-tests libc-hermetic-tests)
 
 add_custom_target(exhaustive-check-libc)
 add_custom_target(libc-long-running-tests)
 
 if(NOT LIBC_TARGET_ARCHITECTURE_IS_GPU)
   add_subdirectory(UnitTest)
 endif()
 

libc/test/UnitTest/CMakeLists.txt

-set(libc_unit_test_srcs
-  ExecuteFunction.h
+set(libc_test_srcs_common
   Test.h
   LibcTest.cpp
   LibcTest.h
   TestLogger.cpp
   TestLogger.h
 )
 
+set(libc_death_test_srcs ExecuteFunction.h)
 if(${LIBC_TARGET_OS} STREQUAL "linux")
-  list(APPEND libc_unit_test_srcs ExecuteFunctionUnix.cpp)
+  list(APPEND libc_death_test_srcs 
+       LibcDeathTestExecutors.cpp ExecuteFunctionUnix.cpp)
 endif()
 
 add_library(
   LibcUnitTest
-  ${libc_unit_test_srcs}
+  ${libc_test_srcs_common}
+  ${libc_death_test_srcs}
 )
 
-target_include_directories(LibcUnitTest PUBLIC ${LIBC_SOURCE_DIR})
-add_dependencies(
-  LibcUnitTest
+add_library(
+  LibcHermeticTest
+  ${libc_test_srcs_common}
+  HermeticTestUtils.cpp
+)
+
+foreach(lib LibcUnitTest LibcHermeticTest)
+  target_include_directories(${lib} PUBLIC ${LIBC_SOURCE_DIR})
+  target_compile_options(${lib} PRIVATE ${LIBC_HERMETIC_TEST_COMPILE_OPTIONS}
+                         -fno-exceptions -fno-rtti)
+  add_dependencies(${lib}
     libc.src.__support.CPP.string
     libc.src.__support.CPP.string_view
     libc.src.__support.CPP.type_traits
     libc.src.__support.OSUtil.osutil
-  libc.src.__support.uint128)
+    libc.src.__support.uint128
+  )
+endforeach()
+
+target_compile_options(LibcHermeticTest PRIVATE -ffreestanding -nostdlib -nostdlib++)
 
 add_library(
-  LibcUnitTestMain
+  LibcTestMain
   LibcTestMain.cpp
 )
 
-target_include_directories(LibcUnitTestMain PUBLIC ${LIBC_SOURCE_DIR})
-add_dependencies(LibcUnitTestMain LibcUnitTest)
-target_link_libraries(LibcUnitTestMain PUBLIC LibcUnitTest)
+target_include_directories(LibcTestMain PUBLIC ${LIBC_SOURCE_DIR})
+target_compile_options(LibcTestMain PRIVATE -fno-exceptions -fno-rtti)
+add_dependencies(LibcTestMain LibcUnitTest)
 
 add_header_library(
   string_utils
   HDRS
   StringUtils.h
   DEPENDS
     libc.src.__support.CPP.string
     libc.src.__support.CPP.type_traits

libc/test/UnitTest/HermeticTestUtils.cpp

+//===-- Implementation of libc death test executors -----------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include <stddef.h>
+#include <stdint.h>
+
+namespace __llvm_libc {
+
+int bcmp(const void *lhs, const void *rhs, size_t count);
+void bzero(void *ptr, size_t count);
+int memcmp(const void *lhs, const void *rhs, size_t count);
+void *memcpy(void *__restrict, const void *__restrict, size_t);
+void *memmove(void *dst, const void *src, size_t count);
+void *memset(void *ptr, int value, size_t count);
+
+} // namespace __llvm_libc
+
+namespace {
+
+// Integration tests cannot use the SCUDO standalone allocator as SCUDO pulls
+// various other parts of the libc. Since SCUDO development does not use
+// LLVM libc build rules, it is very hard to keep track or pull all that SCUDO
+// requires. Hence, as a work around for this problem, we use a simple allocator
+// which just hands out continuous blocks from a statically allocated chunk of
+// memory.
+static uint8_t memory[16384];
+static uint8_t *ptr = memory;
+
+} // anonymous namespace
+
+extern "C" {
+
+// Hermetic tests rely on the following memory functions. This is because the
+// compiler code generation can emit calls to them. We want to map the external
+// entrypoint to the internal implementation of the function used for testing.
+// This is done manually as not all targets support aliases.
+
+int bcmp(const void *lhs, const void *rhs, size_t count) {
+  return __llvm_libc::bcmp(lhs, rhs, count);
+}
+void bzero(void *ptr, size_t count) { __llvm_libc::bzero(ptr, count); }
+int memcmp(const void *lhs, const void *rhs, size_t count) {
+  return __llvm_libc::memcmp(lhs, rhs, count);
+}
+void *memcpy(void *__restrict dst, const void *__restrict src, size_t count) {
+  return __llvm_libc::memcpy(dst, src, count);
+}
+void *memmove(void *dst, const void *src, size_t count) {
+  return __llvm_libc::memmove(dst, src, count);
+}
+void *memset(void *ptr, int value, size_t count) {
+  return __llvm_libc::memset(ptr, value, count);
+}
+
+void *malloc(size_t s) {
+  void *mem = ptr;
+  ptr += s;
+  return mem;
+}
+
+void free(void *) {}
+
+void *realloc(void *ptr, size_t s) {
+  free(ptr);
+  return malloc(s);
+}
+
+// The unit test framework uses pure virtual functions. Since hermetic tests
+// cannot depend C++ runtime libraries, implement dummy functions to support
+// the virtual function runtime.
+void __cxa_pure_virtual() {
+  // A pure virtual being called is an error so we just trap.
+  __builtin_trap();
+}
+
+// Integration tests are linked with -nostdlib. BFD linker expects
+// __dso_handle when -nostdlib is used.
+void *__dso_handle = nullptr;
+
+} // extern "C"
+
+void operator delete(void *) {
+  // The libc runtime should not use the global delete operator. Hence,
+  // we just trap here to catch any such accidental usages.
+  __builtin_trap();
+}

libc/test/UnitTest/LibcDeathTestExecutors.cpp

+//===-- Implementation of libc death test executors -----------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "LibcTest.h"
+
+#include "test/UnitTest/ExecuteFunction.h"
+#include "test/UnitTest/TestLogger.h"
+
+#include <cassert>
+
+namespace __llvm_libc {
+namespace testing {
+
+bool Test::testProcessKilled(testutils::FunctionCaller *Func, int Signal,
+                             const char *LHSStr, const char *RHSStr,
+                             const char *File, unsigned long Line) {
+  testutils::ProcessStatus Result = testutils::invoke_in_subprocess(Func, 500);
+
+  if (const char *error = Result.get_error()) {
+    Ctx->markFail();
+    tlog << File << ":" << Line << ": FAILURE\n" << error << '\n';
+    return false;
+  }
+
+  if (Result.timed_out()) {
+    Ctx->markFail();
+    tlog << File << ":" << Line << ": FAILURE\n"
+         << "Process timed out after " << 500 << " milliseconds.\n";
+    return false;
+  }
+
+  if (Result.exited_normally()) {
+    Ctx->markFail();
+    tlog << File << ":" << Line << ": FAILURE\n"
+         << "Expected " << LHSStr
+         << " to be killed by a signal\nBut it exited normally!\n";
+    return false;
+  }
+
+  int KilledBy = Result.get_fatal_signal();
+  assert(KilledBy != 0 && "Not killed by any signal");
+  if (Signal == -1 || KilledBy == Signal)
+    return true;
+
+  using testutils::signal_as_string;
+  Ctx->markFail();
+  tlog << File << ":" << Line << ": FAILURE\n"
+       << "              Expected: " << LHSStr << '\n'
+       << "To be killed by signal: " << Signal << '\n'
+       << "              Which is: " << signal_as_string(Signal) << '\n'
+       << "  But it was killed by: " << KilledBy << '\n'
+       << "              Which is: " << signal_as_string(KilledBy) << '\n';
+  return false;
+}
+
+bool Test::testProcessExits(testutils::FunctionCaller *Func, int ExitCode,
+                            const char *LHSStr, const char *RHSStr,
+                            const char *File, unsigned long Line) {
+  testutils::ProcessStatus Result = testutils::invoke_in_subprocess(Func, 500);
+
+  if (const char *error = Result.get_error()) {
+    Ctx->markFail();
+    tlog << File << ":" << Line << ": FAILURE\n" << error << '\n';
+    return false;
+  }
+
+  if (Result.timed_out()) {
+    Ctx->markFail();
+    tlog << File << ":" << Line << ": FAILURE\n"
+         << "Process timed out after " << 500 << " milliseconds.\n";
+    return false;
+  }
+
+  if (!Result.exited_normally()) {
+    Ctx->markFail();
+    tlog << File << ":" << Line << ": FAILURE\n"
+         << "Expected " << LHSStr << '\n'
+         << "to exit with exit code " << ExitCode << '\n'
+         << "But it exited abnormally!\n";
+    return false;
+  }
+
+  int ActualExit = Result.get_exit_code();
+  if (ActualExit == ExitCode)
+    return true;
+
+  Ctx->markFail();
+  tlog << File << ":" << Line << ": FAILURE\n"
+       << "Expected exit code of: " << LHSStr << '\n'
+       << "             Which is: " << ActualExit << '\n'
+       << "       To be equal to: " << RHSStr << '\n'
+       << "             Which is: " << ExitCode << '\n';
+  return false;
+}
+
+} // namespace testing
+} // namespace __llvm_libc

libc/test/UnitTest/LibcTest.h

 #include "src/__support/CPP/string_view.h"
 #include "src/__support/CPP/type_traits.h"
 #include "test/UnitTest/ExecuteFunction.h"
 #include "test/UnitTest/TestLogger.h"
 
 namespace __llvm_libc {
 namespace testing {
 
-class RunContext;
-
 // Only the following conditions are supported. Notice that we do not have
 // a TRUE or FALSE condition. That is because, C library funtions do not
 // return boolean values, but use integral return values to indicate true or
 // false conditions. Hence, it is more appropriate to use the other comparison
 // conditions for such cases.
 enum TestCondition {
   Cond_None,
   Cond_EQ,
   Cond_NE,
   Cond_LT,
   Cond_LE,
   Cond_GT,
   Cond_GE,
 };
 
 namespace internal {
 
+class RunContext {
+public:
+  enum RunResult { Result_Pass = 1, Result_Fail = 2 };
+
+  RunResult status() const { return Status; }
+
+  void markFail() { Status = Result_Fail; }
+
+private:
+  RunResult Status = Result_Pass;
+};
+
 template <typename ValType>
 bool test(RunContext *Ctx, TestCondition Cond, ValType LHS, ValType RHS,
           const char *LHSStr, const char *RHSStr, const char *File,
           unsigned long Line);
 
 } // namespace internal
 
 struct MatcherBase {
   virtual ~MatcherBase() {}
   virtual void explainError() { tlog << "unknown error\n"; }
   // Override and return true to skip `explainError` step.
   virtual bool is_silent() const { return false; }
 };
 
 template <typename T> struct Matcher : public MatcherBase {
   bool match(T &t);
 };
 
 // NOTE: One should not create instances and call methods on them directly. One
 // should use the macros TEST or TEST_F to write test cases.
 class Test {
 private:
   Test *Next = nullptr;
-  RunContext *Ctx = nullptr;
+  internal::RunContext *Ctx = nullptr;
 
-  void setContext(RunContext *C) { Ctx = C; }
+  void setContext(internal::RunContext *C) { Ctx = C; }
 
 public:
   virtual ~Test() {}
   virtual void SetUp() {}
   virtual void TearDown() {}
 
   static int runTests(const char *);
 

 private:
   virtual void Run() = 0;
   virtual const char *getName() const = 0;
 
   static Test *Start;
   static Test *End;
 };
 
+extern int argc;
+extern char **argv;
+extern char **envp;
+
 namespace internal {
 
 constexpr bool same_prefix(char const *lhs, char const *rhs, int const len) {
   for (int i = 0; (*lhs || *rhs) && (i < len); ++lhs, ++rhs, ++i)
     if (*lhs != *rhs)
       return false;
   return true;
 }

libc/test/UnitTest/LibcTest.cpp

 //===-- Implementation of the base class for libc unittests----------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "LibcTest.h"
 
 #include "src/__support/CPP/string.h"
 #include "src/__support/CPP/string_view.h"
 #include "src/__support/UInt128.h"
-#include "test/UnitTest/ExecuteFunction.h"
 #include "test/UnitTest/TestLogger.h"
-#include <cassert>
 
 namespace __llvm_libc {
 namespace testing {
 
-// This need not be a class as all it has is a single read-write state variable.
-// But, we make it class as then its implementation can be hidden from the
-// header file.
-class RunContext {
-public:
-  enum RunResult { Result_Pass = 1, Result_Fail = 2 };
-
-  RunResult status() const { return Status; }
-
-  void markFail() { Status = Result_Fail; }
-
-private:
-  RunResult Status = Result_Pass;
-};
-
 namespace internal {
 
 // When the value is UInt128 or __uint128_t, show its hexadecimal digits.
 // We cannot just use a UInt128 specialization as that resolves to only
 // one type, UInt<128> or __uint128_t. We want both overloads as we want to
 // be able to unittest UInt<128> on platforms where UInt128 resolves to
 // UInt128.
 template <typename T>
bool test(RunContext *Ctx, TestCondition Cond, ValType LHS, ValType RHS,
   }
 }
 
 } // namespace internal
 
 Test *Test::Start = nullptr;
 Test *Test::End = nullptr;
 
+int argc = 0;
+char **argv = nullptr;
+char **envp = nullptr;
+
+using internal::RunContext;
+
 void Test::addTest(Test *T) {
   if (End == nullptr) {
     Start = T;
     End = T;
     return;
   }
 
   End->Next = T;
bool Test::testMatch(bool MatchResult, MatcherBase &Matcher, const char *LHSStr,
   if (!Matcher.is_silent()) {
     tlog << File << ":" << Line << ": FAILURE\n"
          << "Failed to match " << LHSStr << " against " << RHSStr << ".\n";
     Matcher.explainError();
   }
   return false;
 }
 
-#ifdef ENABLE_SUBPROCESS_TESTS
-
-bool Test::testProcessKilled(testutils::FunctionCaller *Func, int Signal,
-                             const char *LHSStr, const char *RHSStr,
-                             const char *File, unsigned long Line) {
-  testutils::ProcessStatus Result = testutils::invoke_in_subprocess(Func, 500);
-
-  if (const char *error = Result.get_error()) {
-    Ctx->markFail();
-    tlog << File << ":" << Line << ": FAILURE\n" << error << '\n';
-    return false;
-  }
-
-  if (Result.timed_out()) {
-    Ctx->markFail();
-    tlog << File << ":" << Line << ": FAILURE\n"
-         << "Process timed out after " << 500 << " milliseconds.\n";
-    return false;
-  }
-
-  if (Result.exited_normally()) {
-    Ctx->markFail();
-    tlog << File << ":" << Line << ": FAILURE\n"
-         << "Expected " << LHSStr
-         << " to be killed by a signal\nBut it exited normally!\n";
-    return false;
-  }
-
-  int KilledBy = Result.get_fatal_signal();
-  assert(KilledBy != 0 && "Not killed by any signal");
-  if (Signal == -1 || KilledBy == Signal)
-    return true;
-
-  using testutils::signal_as_string;
-  Ctx->markFail();
-  tlog << File << ":" << Line << ": FAILURE\n"
-       << "              Expected: " << LHSStr << '\n'
-       << "To be killed by signal: " << Signal << '\n'
-       << "              Which is: " << signal_as_string(Signal) << '\n'
-       << "  But it was killed by: " << KilledBy << '\n'
-       << "              Which is: " << signal_as_string(KilledBy) << '\n';
-  return false;
-}
-
-bool Test::testProcessExits(testutils::FunctionCaller *Func, int ExitCode,
-                            const char *LHSStr, const char *RHSStr,
-                            const char *File, unsigned long Line) {
-  testutils::ProcessStatus Result = testutils::invoke_in_subprocess(Func, 500);
-
-  if (const char *error = Result.get_error()) {
-    Ctx->markFail();
-    tlog << File << ":" << Line << ": FAILURE\n" << error << '\n';
-    return false;
-  }
-
-  if (Result.timed_out()) {
-    Ctx->markFail();
-    tlog << File << ":" << Line << ": FAILURE\n"
-         << "Process timed out after " << 500 << " milliseconds.\n";
-    return false;
-  }
-
-  if (!Result.exited_normally()) {
-    Ctx->markFail();
-    tlog << File << ":" << Line << ": FAILURE\n"
-         << "Expected " << LHSStr << '\n'
-         << "to exit with exit code " << ExitCode << '\n'
-         << "But it exited abnormally!\n";
-    return false;
-  }
-
-  int ActualExit = Result.get_exit_code();
-  if (ActualExit == ExitCode)
-    return true;
-
-  Ctx->markFail();
-  tlog << File << ":" << Line << ": FAILURE\n"
-       << "Expected exit code of: " << LHSStr << '\n'
-       << "             Which is: " << ActualExit << '\n'
-       << "       To be equal to: " << RHSStr << '\n'
-       << "             Which is: " << ExitCode << '\n';
-  return false;
-}
-
-#endif // ENABLE_SUBPROCESS_TESTS
 } // namespace testing
 } // namespace __llvm_libc

libc/test/UnitTest/LibcTestMain.cpp

 //===-- Main function for implementation of base class for libc unittests -===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "LibcTest.h"
 
 static const char *getTestFilter(int argc, char *argv[]) {
   return argc > 1 ? argv[1] : nullptr;
 }
 
-int main(int argc, char *argv[]) {
+extern "C" int main(int argc, char **argv, char **envp) {
+  __llvm_libc::testing::argc = argc;
+  __llvm_libc::testing::argv = argv;
+  __llvm_libc::testing::envp = envp;
+
   const char *TestFilter = getTestFilter(argc, argv);
   return __llvm_libc::testing::Test::runTests(TestFilter);
 }

libc/test/src/ctype/CMakeLists.txt

-add_custom_target(libc_ctype_unittests)
+add_custom_target(libc-ctype-tests)
 
-add_libc_unittest(
+add_libc_test(
   isalnum_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isalnum_test.cpp
   DEPENDS
     libc.src.ctype.isalnum
 )
 
-add_libc_unittest(
+add_libc_test(
   isalpha_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isalpha_test.cpp
   DEPENDS
     libc.src.ctype.isalpha
 )
 
-add_libc_unittest(
+add_libc_test(
   isascii_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isascii_test.cpp
   DEPENDS
     libc.src.ctype.isascii
 )
 
-add_libc_unittest(
+add_libc_test(
   isblank_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isblank_test.cpp
   DEPENDS
     libc.src.ctype.isblank
 )
 
-add_libc_unittest(
+add_libc_test(
   iscntrl_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     iscntrl_test.cpp
   DEPENDS
     libc.src.ctype.iscntrl
 )
 
-add_libc_unittest(
+add_libc_test(
   isdigit_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isdigit_test.cpp
   DEPENDS
     libc.src.ctype.isdigit
 )
 
-add_libc_unittest(
+add_libc_test(
   isgraph_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isgraph_test.cpp
   DEPENDS
     libc.src.ctype.isgraph
 )
 
-add_libc_unittest(
+add_libc_test(
   islower_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     islower_test.cpp
   DEPENDS
     libc.src.ctype.islower
 )
 
-add_libc_unittest(
+add_libc_test(
   isprint_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isprint_test.cpp
   DEPENDS
     libc.src.ctype.isprint
 )
 
-add_libc_unittest(
+add_libc_test(
   ispunct_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     ispunct_test.cpp
   DEPENDS
     libc.src.ctype.ispunct
 )
 
-add_libc_unittest(
+add_libc_test(
   isspace_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isspace_test.cpp
   DEPENDS
     libc.src.ctype.isspace
 )
 
-add_libc_unittest(
+add_libc_test(
   isupper_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isupper_test.cpp
   DEPENDS
     libc.src.ctype.isupper
 )
 
-add_libc_unittest(
+add_libc_test(
   isxdigit_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     isxdigit_test.cpp
   DEPENDS
     libc.src.ctype.isxdigit
 )
 
-add_libc_unittest(
+add_libc_test(
   toascii_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     toascii_test.cpp
   DEPENDS
     libc.src.ctype.toascii
 )
 
-add_libc_unittest(
+add_libc_test(
   tolower_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     tolower_test.cpp
   DEPENDS
     libc.src.ctype.tolower
 )
 
-add_libc_unittest(
+add_libc_test(
   toupper_test
   SUITE
-    libc_ctype_unittests
+    libc-ctype-tests
   SRCS
     toupper_test.cpp
   DEPENDS
     libc.src.ctype.toupper
 )