[libcxxabi,libunwind] support running tests in standalone mode

[gargaroff]

Summary

Remove check for standalone and shared library mode in libcxxabi to
allow including tests in said mode. This check prevented running the
tests in standalone mode with static libraries, which is the case for
baremetal targets.

Fix check-unwind target trying to use a non-existent llvm-lit executable
in standalone mode. Copy the HandleOutOfTreeLLVM logic from libcxxabi to
libunwind in order to make the tests work in standalone mode.

[gargaroff]

We require this patch for our downstream baremetal target to be able to run the test suites of both libraries on our target.

I'm not sure whether the check in libcxxabi was simply outdated or if we will require some type of check there.

As for the libunwind part: I mostly copy-pasted the HandleOutOfTreeLLVM.cmake from libcxxabi and adapted it to libunwind in places where it was necessary. I compared the previous logic with the new one and although I did find some differences, mostly related to llvm-lit, most of them did not seem to be actually required.

Please note however that I was only able to test our downstream use-case. I hope this doesn't break anything. Please let me know if you spot anything and how we could fix that.

[abidh]

In D86540#2236228, @gargaroff wrote:

We require this patch for our downstream baremetal target to be able to run the test suites of both libraries on our target.

I'm not sure whether the check in libcxxabi was simply outdated or if we will require some type of check there.

I also maintain a downstream baremetal target and have a similar local change to side step this check. Although I did not remove it entirely as I was not sure what it does and instead used NOT LIBCXXABI_BAREMETAL. Anyways, It will be nice to have this fixed upstream.

[gargaroff]

Ping

[ldionne]

libunwind/cmake/Modules/HandleOutOfTreeLLVM.cmake

1 ↗

(On Diff #287659)

Instead of copy-pasting this, it should use the one from libcxx.

libunwind, libcxxabi and libcxx are all required in the tree at all times. A Standalone build refers to where the root of the CMake invocation is, not whether libunwind is actually checked out on its own. Because of this, it should be possible to reuse HandleOutOfTreeLLVM.cmake from libcxx instead of duplicating it.

libunwind/test/CMakeLists.txt

36

Where is this defined?

[gargaroff]

Thanks for the feedback! I'm currently busy with a release, but I'll get back to this afterwards. Don't expect anything for the next two or three weeks.

[gargaroff]

Rebase.
Reuse HandleOutOfTreeLLVM.cmake from libcxx in libunwind.
Define LIBUNWIND_TEST_PARAMS

@ldionne I only made libunwind reuse libcxx's HandleOutOfTreeLLVM.cmake. Libcxxabi is still using a libcxxabi specific one. I'm planning on making libcxxabi also reuse the libcxx one but first wanted to make sure that the approach for libunwind is looking good.

[ldionne]

The approach looks good, with comments.

libcxx/cmake/Modules/HandleOutOfTreeLLVM.cmake

29

I think it might be better to instead provide the LLVM_LIT_OUTPUT_DIR as a "parameter" of this module. Basically, set it before including it in various projects. WDYT?

libunwind/CMakeLists.txt

27

There's already a LIBUNWIND_LIBCXX_PATH defined somewhere in libunwind -- I think we should merge the two variables into one.

[gargaroff]

Reuse LIBUNWIND_LIBCXX_PATH.
Unify HandleOutOfTreeLLVM handling in libcxxabi and libunwind by reusing the one from libcxx.
Define {lib}_STANDALONE_BUILD outside of HandleOutOfTreeLLVM.
Define LLVM_LIT_OUTPUT_DIR outside of HandleOutOfTreeLLVM.

libcxx/cmake/Modules/HandleOutOfTreeLLVM.cmake

29

Yeah sounds good. In that case it would also make sense to do the same for {lib}_STANDALONE_BUILD. That way we can completely remove the package name check from here.

[gargaroff]

libcxx/CMakeLists.txt

33 ↗

(On Diff #293366)

Technically only LIBCXX_BINARY_DIR is required up here, but I wanted to keep those three together

[gargaroff]

Ping

[gargaroff]

Ping

[ldionne]

I love this. Some comments but looks almost ready to go.

libcxxabi/CMakeLists.txt

35

This check isn't necessary -- we already have a similar one at the very top of the file.

libunwind/CMakeLists.txt

33

Can you add this at the very top of the file instead?

if (NOT IS_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}/../libcxx")
  message(FATAL_ERROR "libunwind requires being built in a monorepo layout with libcxx available")
endif()

[ldionne]

Actually, please feel free to ship this with the changes I requested.

[gargaroff]

Address final comments.

I'll land this once the build bots give me the green light. Thanks for the review Louis!

[gargaroff]

...or not. Seems like the pre-merge checks are down at the moment, so I'll just land it right away. If anything breaks, we can always revert it anyway.

libcxx/cmake/Modules/HandleOutOfTreeLLVM.cmake

 elseif (EXISTS "${LLVM_MAIN_SRC_DIR}/cmake/modules")
   list(APPEND CMAKE_MODULE_PATH "${LLVM_MAIN_SRC_DIR}/cmake/modules")
 else()
   message(FATAL_ERROR "Neither ${LLVM_CMAKE_PATH} nor ${LLVM_MAIN_SRC_DIR}/cmake/modules found. "
                       "This is not a supported configuration.")
 endif()
 
 message(STATUS "Configuring for standalone build.")
+
+# In a standalone build, we don't have llvm to automatically generate the
+# llvm-lit script for us.  So we need to provide an explicit directory that
+# the configurator should write the script into.
+if (PACKAGE_NAME STREQUAL "libcxx")
   set(LIBCXX_STANDALONE_BUILD 1)
+  set(LLVM_LIT_OUTPUT_DIR "${libcxx_BINARY_DIR}/bin")
+elseif (PACKAGE_NAME STREQUAL "libunwind")
+  set(LIBUNWIND_STANDALONE_BUILD 1)
+  set(LLVM_LIT_OUTPUT_DIR "${libunwind_BINARY_DIR}/bin")
+else ()
+  message(WARNING "HandleOutOfTreeLLVM.cmake included from unknown package")
+endif ()
 
 # Add LLVM Functions --------------------------------------------------------
 if (WIN32)
   set(LLVM_ON_UNIX 0)
   set(LLVM_ON_WIN32 1)
 else()
   set(LLVM_ON_UNIX 1)
   set(LLVM_ON_WIN32 0)
 endif()
 
 include(AddLLVM OPTIONAL)
 
 # LLVM Options --------------------------------------------------------------
 if (NOT DEFINED LLVM_INCLUDE_TESTS)
   set(LLVM_INCLUDE_TESTS ON)
 endif()
 if (NOT DEFINED LLVM_INCLUDE_DOCS)
   set(LLVM_INCLUDE_DOCS ON)
 endif()
 if (NOT DEFINED LLVM_ENABLE_SPHINX)
   set(LLVM_ENABLE_SPHINX OFF)
 endif()
 
-# In a standalone build, we don't have llvm to automatically generate the
-# llvm-lit script for us.  So we need to provide an explicit directory that
-# the configurator should write the script into.
-set(LLVM_LIT_OUTPUT_DIR "${libcxx_BINARY_DIR}/bin")
-
 if (LLVM_INCLUDE_TESTS)
   # Required LIT Configuration ------------------------------------------------
   # Define the default arguments to use with 'lit', and an option for the user
   # to override.
   set(LLVM_DEFAULT_EXTERNAL_LIT "${LLVM_MAIN_SRC_DIR}/utils/lit/lit.py")
   set(LIT_ARGS_DEFAULT "-sv --show-xfail --show-unsupported")
   if (MSVC OR XCODE)
     set(LIT_ARGS_DEFAULT "${LIT_ARGS_DEFAULT} --no-progress-bar")

libcxxabi/CMakeLists.txt

if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR OR LIBCXXABI_STANDALONE_BUILD)
   set(PACKAGE_NAME libcxxabi)
   set(PACKAGE_VERSION 11.0.0git)
   set(PACKAGE_STRING "${PACKAGE_NAME} ${PACKAGE_VERSION}")
   set(PACKAGE_BUGREPORT "[e-mail]")
 
   # Find the LLVM sources and simulate LLVM CMake options.
   include(HandleOutOfTreeLLVM)
 endif()
 
 # Require out of source build.
 include(MacroEnsureOutOfSourceBuild)
 MACRO_ENSURE_OUT_OF_SOURCE_BUILD(
  "${PROJECT_NAME} requires an out of source build. Please create a separate
  build directory and run 'cmake /path/to/${PROJECT_NAME} [options]' there."
  )
 
 #===============================================================================
if (LIBCXXABI_USE_LLVM_UNWINDER OR LLVM_NATIVE_ARCH MATCHES ARM)
   endif()
 endif()
 
 # Add source code. This also contains all of the logic for deciding linker flags
 # soname, etc...
 add_subdirectory(src)
 
 if (LIBCXXABI_INCLUDE_TESTS)
-  if (LIBCXXABI_STANDALONE_BUILD AND NOT LIBCXXABI_ENABLE_SHARED)
-    # We can't reasonably test the system C++ library with a static
-    # libc++abi.  We either need to be able to replace libc++abi at
-    # run time (with a shared libc++abi), or we need to be able to
-    # replace the C++ runtime (with a non- standalone build).
-    message(WARNING "The libc++abi tests aren't valid when libc++abi "
-                    "is built standalone (i.e. outside of "
-                    "llvm/projects/libcxxabi ) and is built without "
-                    "a shared library.  Either build a shared "
-                    "library, build libc++abi at the same time as "
-                    "you build libc++, or do without testing.  No "
-                    "check target will be available!")
-  else()
-    add_subdirectory(test)
-    add_subdirectory(fuzz)
-  endif()
-endif()

libunwind/CMakeLists.txt

set(CMAKE_MODULE_PATH
   "${CMAKE_CURRENT_SOURCE_DIR}/cmake"
   "${CMAKE_CURRENT_SOURCE_DIR}/cmake/Modules"
   ${CMAKE_MODULE_PATH}
   )
 
 if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR OR LIBUNWIND_STANDALONE_BUILD)
   project(libunwind LANGUAGES C CXX ASM)
 
-  # Rely on llvm-config.
-  set(CONFIG_OUTPUT)
-  if(NOT LLVM_CONFIG_PATH)
-    find_program(LLVM_CONFIG_PATH "llvm-config")
-  endif()
-  if (DEFINED LLVM_PATH)
-    set(LLVM_INCLUDE_DIR ${LLVM_INCLUDE_DIR} CACHE PATH "Path to llvm/include")
-    set(LLVM_PATH ${LLVM_PATH} CACHE PATH "Path to LLVM source tree")
-    set(LLVM_MAIN_SRC_DIR ${LLVM_PATH})
-    set(LLVM_CMAKE_PATH "${LLVM_PATH}/cmake/modules")
-  elseif(LLVM_CONFIG_PATH)
-    message(STATUS "Found LLVM_CONFIG_PATH as ${LLVM_CONFIG_PATH}")
-    set(CONFIG_COMMAND ${LLVM_CONFIG_PATH} "--includedir" "--prefix" "--src-root")
-    execute_process(COMMAND ${CONFIG_COMMAND}
-                    RESULT_VARIABLE HAD_ERROR
-                    OUTPUT_VARIABLE CONFIG_OUTPUT)
-    if (NOT HAD_ERROR)
-      string(REGEX REPLACE "[ \t]*[\r\n]+[ \t]*" ";"
-             CONFIG_OUTPUT ${CONFIG_OUTPUT})
-    else()
-      string(REPLACE ";" " " CONFIG_COMMAND_STR "${CONFIG_COMMAND}")
-      message(STATUS "${CONFIG_COMMAND_STR}")
-      message(FATAL_ERROR "llvm-config failed with status ${HAD_ERROR}")
-    endif()
-
-    list(GET CONFIG_OUTPUT 0 INCLUDE_DIR)
-    list(GET CONFIG_OUTPUT 1 LLVM_OBJ_ROOT)
-    list(GET CONFIG_OUTPUT 2 MAIN_SRC_DIR)
-
-    set(LLVM_INCLUDE_DIR ${INCLUDE_DIR} CACHE PATH "Path to llvm/include")
-    set(LLVM_BINARY_DIR ${LLVM_OBJ_ROOT} CACHE PATH "Path to LLVM build tree")
-    set(LLVM_MAIN_SRC_DIR ${MAIN_SRC_DIR} CACHE PATH "Path to LLVM source tree")
-    set(LLVM_LIT_PATH "${LLVM_PATH}/utils/lit/lit.py")
-
-    # --cmakedir is supported since llvm r291218 (4.0 release)
-    execute_process(
-      COMMAND ${LLVM_CONFIG_PATH} --cmakedir
-      RESULT_VARIABLE HAD_ERROR
-      OUTPUT_VARIABLE CONFIG_OUTPUT
-      ERROR_QUIET)
-    if(NOT HAD_ERROR)
-      string(STRIP "${CONFIG_OUTPUT}" LLVM_CMAKE_PATH_FROM_LLVM_CONFIG)
-      file(TO_CMAKE_PATH "${LLVM_CMAKE_PATH_FROM_LLVM_CONFIG}" LLVM_CMAKE_PATH)
-    else()
-      file(TO_CMAKE_PATH "${LLVM_BINARY_DIR}" LLVM_BINARY_DIR_CMAKE_STYLE)
-      set(LLVM_CMAKE_PATH "${LLVM_BINARY_DIR_CMAKE_STYLE}/lib${LLVM_LIBDIR_SUFFIX}/cmake/llvm")
-    endif()
-  else()
-    message(WARNING "UNSUPPORTED LIBUNWIND CONFIGURATION DETECTED: "
-                    "llvm-config not found and LLVM_MAIN_SRC_DIR not defined. "
-                    "Reconfigure with -DLLVM_CONFIG=path/to/llvm-config "
-                    "or -DLLVM_PATH=path/to/llvm-source-root.")
-  endif()
-
-  if (EXISTS ${LLVM_CMAKE_PATH})
-    # Enable warnings, otherwise -w gets added to the cflags by HandleLLVMOptions.
-    set(LLVM_ENABLE_WARNINGS ON)
-    list(APPEND CMAKE_MODULE_PATH "${LLVM_CMAKE_PATH}")
-    include("${LLVM_CMAKE_PATH}/AddLLVM.cmake")
-    include("${LLVM_CMAKE_PATH}/HandleLLVMOptions.cmake")
-  else()
-    message(WARNING "Not found: ${LLVM_CMAKE_PATH}")
-  endif()
-
   set(PACKAGE_NAME libunwind)
   set(PACKAGE_VERSION 12.0.0git)
   set(PACKAGE_STRING "${PACKAGE_NAME} ${PACKAGE_VERSION}")
   set(PACKAGE_BUGREPORT "[e-mail]")
 
-  if (EXISTS ${LLVM_MAIN_SRC_DIR}/utils/lit/lit.py)
-    set(LLVM_LIT ${LLVM_MAIN_SRC_DIR}/utils/lit/lit.py)
-  else()
-    # Seek installed Lit.
-    find_program(LLVM_LIT "lit.py" ${LLVM_MAIN_SRC_DIR}/utils/lit
-                 DOC "Path to lit.py")
-  endif()
-
-  if (LLVM_LIT)
-    # Define the default arguments to use with 'lit', and an option for the user
-    # to override.
-    set(LIT_ARGS_DEFAULT "-sv")
-    if (MSVC OR XCODE)
-      set(LIT_ARGS_DEFAULT "${LIT_ARGS_DEFAULT} --no-progress-bar")
-    endif()
-    set(LLVM_LIT_ARGS "${LIT_ARGS_DEFAULT}" CACHE STRING "Default options for lit")
-
-    # On Win32 hosts, provide an option to specify the path to the GnuWin32 tools.
-    if (WIN32 AND NOT CYGWIN)
-      set(LLVM_LIT_TOOLS_DIR "" CACHE PATH "Path to GnuWin32 tools")
-    endif()
+  # Add the CMake module path of libcxx so we can reuse HandleOutOfTreeLLVM.cmake
+  set(LIBCXX_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../libcxx)
+  set(LIBCXX_CMAKE_PATH "${LIBCXX_PATH}/cmake/Modules")
+
+  if (EXISTS "${LIBCXX_CMAKE_PATH}")
+    list(APPEND CMAKE_MODULE_PATH "${LIBCXX_CMAKE_PATH}")
   else()
-    set(LLVM_INCLUDE_TESTS OFF)
+    message(FATAL_ERROR "${LIBCXX_CMAKE_PATH} not found. This is not a supported configuration.")
   endif()
 
-  set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX})
-  set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX})
+  include(HandleOutOfTreeLLVM)
 else()
   set(LLVM_LIT "${CMAKE_SOURCE_DIR}/utils/lit/lit.py")
 endif()
 
 #===============================================================================
 # Setup CMake Options
 #===============================================================================
 include(CMakeDependentOption)

 set(LIBUNWIND_GCC_TOOLCHAIN "" CACHE PATH "GCC toolchain for cross compiling.")
 set(LIBUNWIND_SYSROOT "" CACHE PATH "Sysroot for cross compiling.")
 set(LIBUNWIND_TEST_LINKER_FLAGS "" CACHE STRING
     "Additional linker flags for test programs.")
 set(LIBUNWIND_TEST_COMPILER_FLAGS "" CACHE STRING
     "Additional compiler flags for test programs.")
 set(LIBUNWIND_TEST_CONFIG "${CMAKE_CURRENT_SOURCE_DIR}/test/lit.site.cfg.in" CACHE STRING
     "The Lit testing configuration to use when running the tests.")
+set(LIBUNWIND_TEST_PARAMS "" CACHE STRING
+    "A list of parameters to run the Lit test suite with.")
 
 if (NOT LIBUNWIND_ENABLE_SHARED AND NOT LIBUNWIND_ENABLE_STATIC)
   message(FATAL_ERROR "libunwind must be built as either a shared or static library.")
 endif()
 
 # Check that we can build with 32 bits if requested.
 if (CMAKE_SIZEOF_VOID_P EQUAL 8 AND NOT WIN32)
   if (LIBUNWIND_BUILD_32_BITS AND NOT LLVM_BUILD_32_BITS) # Don't duplicate the output from LLVM

libunwind/test/CMakeLists.txt

 set(AUTO_GEN_COMMENT "## Autogenerated by libunwind configuration.\n# Do not edit!")
 configure_lit_site_cfg(
   "${LIBUNWIND_TEST_CONFIG}"
   ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg
   MAIN_CONFIG "${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py")
 
 add_lit_testsuite(check-unwind "Running libunwind tests"
   ${CMAKE_CURRENT_BINARY_DIR}
-  DEPENDS unwind ${LIBUNWIND_TEST_DEPS})
+  DEPENDS unwind ${LIBUNWIND_TEST_DEPS}
+  PARAMS "${LIBUNWIND_TEST_PARAMS}")