[libcxx] Add a CI configuration for standalone building in llvm-project/runtimes

[mstorsjo]

Summary

Generate the llvm-lit script unless invoked from llvm/runtimes.

[mstorsjo]

Ok, so this works for the standalone build, but breaks the full llvm/runtimes build. I’ll look into it a bit later then.

[mstorsjo]

Try to fix the runtimes build

[mstorsjo]

Fix the list separator for when inoking ExternalProject_Add. Surprisingly, the previous form worked with CMake 3.20.1 but not with 3.21.1.

[phosek]

runtimes/CMakeLists.txt

77–82

Can we land this change as a separate commit?

92–102

Rather than removing this altogether, could we either check if the compiler supports these flags or if the compiler is Clang?

92–102

-nostdinc++ should be support by GCC.

114–117

Can we land this change as a separate commit?

[mstorsjo]

runtimes/CMakeLists.txt

77–82

Sure

92–102

I don't remove it altogether, I move setting it to llvm/runtimes/CMakeLists.txt (where it's only used by the currently-built clang, so it's fine to use it unconditionally).

But keeping them here with detection also sounds fine to me. FWIW they're pretty much duplicate and unnecessary right now though, as at least libunwind, libcxxabi and libcxx all try to detect and add these flags themselves.

114–117

Sure, can do.

[mstorsjo]

Rebased after landing other changes that were split out.

The only refactoring left is moving setting LLVM_RUNTIMES_BUILD to llvm-project/runtimes, including llvm-lit if that isn't set, and setting a default value for LLVM_INCLUDE_TESTS.

[mstorsjo]

@phosek - What do you think of the rest of the changes left here in this patch now - do they seem sensible? Should I split them out to one to three more individual changes for the runtimes build (moving setting LLVM_RUNTIMES_BUILD to llvm/runtimes, including llvm-lit when necessary, making sure LLVM_INCLUDE_TESTS is on by default)?

[ldionne]

That is fine with me.

I want to standardize around exactly two types of supported builds:

1. Using <root>/runtimes as a CMake root.
2. Using <root>/llvm as a CMake root and mentioning LLVM_ENABLE_RUNTIMES=..... That is effectively bootstrapping Clang and building the runtimes with the just-built compiler.

In particular, I want to remove the ability to use <root>/llvm as a CMake root and say LLVM_ENABLE_PROJECTS=libcxx,libcxxabi,libunwind as a way of building, since that has a bunch of problems.

Are we on the same page? If so, we'll also want to improve on the terrible terminology we have today. (2) is currently called a "runtimes build", and (1) has no name. I would suggest we call (1) a "build of the runtimes", or just a "normal build", and that we rename (2) to a "bootstrapping build" or something like that.

[mstorsjo]

In D109593#3013819, @ldionne wrote:

That is fine with me.

I want to standardize around exactly two types of supported builds:

1. Using <root>/runtimes as a CMake root.
2. Using <root>/llvm as a CMake root and mentioning LLVM_ENABLE_RUNTIMES=..... That is effectively bootstrapping Clang and building the runtimes with the just-built compiler.

In particular, I want to remove the ability to use <root>/llvm as a CMake root and say LLVM_ENABLE_PROJECTS=libcxx,libcxxabi,libunwind as a way of building, since that has a bunch of problems.

Are we on the same page? If so, we'll also want to improve on the terrible terminology we have today. (2) is currently called a "runtimes build", and (1) has no name. I would suggest we call (1) a "build of the runtimes", or just a "normal build", and that we rename (2) to a "bootstrapping build" or something like that.

That sounds like a good general direction. Distributors of llvm and the runtimes are going to need a decent sized transition period though.

For people using LLVM_ENABLE_PROJECTS, I would believe it should be straightforward for them to migrate. The bigger disruption is going to be when/if we stop allowing building the runtime projects entirely on their own by pointing cmake directly at e.g. libcxx. Judging from last weeks LLVM Distributors Conference, I would say that's the way a majority of distributors build it today, and transitioning that into e.g. 1. above will take some effort in the more complex cases. And some of them build this way with standalone source dirs without the surrounding monorepo, e.g. cc @tstellar.

[phosek]

In D109593#3012195, @mstorsjo wrote:

@phosek - What do you think of the rest of the changes left here in this patch now - do they seem sensible? Should I split them out to one to three more individual changes for the runtimes build (moving setting LLVM_RUNTIMES_BUILD to llvm/runtimes, including llvm-lit when necessary, making sure LLVM_INCLUDE_TESTS is on by default)?

The increased use of LLVM_RUNTIMES_BUILD concerns me a bit. That variable was always meant to be a short term hack to work around the issue with the compiler-rt build, see https://github.com/llvm/llvm-project/blob/47f79c6057764e0c83016269ae2359f8c5c8d135/runtimes/CMakeLists.txt#L134. I should have probably used a more ominous name like DONT_USE_THIS_COMPILER_RT_BUILD_NEEDS_FIXING to make it more clear.

runtimes/CMakeLists.txt

190–192

Could avoid the llvm-lit setup in llvm/runtimes and always do it here unconditionally?

[phosek]

In D109593#3014343, @mstorsjo wrote:

In D109593#3013819, @ldionne wrote:

That is fine with me.

I want to standardize around exactly two types of supported builds:

1. Using <root>/runtimes as a CMake root.
2. Using <root>/llvm as a CMake root and mentioning LLVM_ENABLE_RUNTIMES=..... That is effectively bootstrapping Clang and building the runtimes with the just-built compiler.

In particular, I want to remove the ability to use <root>/llvm as a CMake root and say LLVM_ENABLE_PROJECTS=libcxx,libcxxabi,libunwind as a way of building, since that has a bunch of problems.

Are we on the same page? If so, we'll also want to improve on the terrible terminology we have today. (2) is currently called a "runtimes build", and (1) has no name. I would suggest we call (1) a "build of the runtimes", or just a "normal build", and that we rename (2) to a "bootstrapping build" or something like that.

That sounds like a good general direction. Distributors of llvm and the runtimes are going to need a decent sized transition period though.

For people using LLVM_ENABLE_PROJECTS, I would believe it should be straightforward for them to migrate. The bigger disruption is going to be when/if we stop allowing building the runtime projects entirely on their own by pointing cmake directly at e.g. libcxx. Judging from last weeks LLVM Distributors Conference, I would say that's the way a majority of distributors build it today, and transitioning that into e.g. 1. above will take some effort in the more complex cases. And some of them build this way with standalone source dirs without the surrounding monorepo, e.g. cc @tstellar.

+1 to everything said above.

Another thing I'd point out is that we may want to rethink the use of LLVM_ENABLE_RUNTIMES and <root>/llvm/runtimes. In the bootstrapping build, we want to build Clang and then use to build runtimes, but LLVM isn't aware of Clang, Clang depends on LLVM. To give you a concrete example, if you used -DLLVM_ENABLE_PROJECTS=llvm -DLLVM_ENABLE_RUNTIMES=all, it's not clear what should happen since there's no Clang being built.

I think that in the long run, it'd be better if the bootstrapping build had moved either into Clang or even to the very top-level of the monorepo (that is <root>/CMakeLists.txt) since in practice you'll also want a linker, and possibly Flang and other tools.

[mstorsjo]

runtimes/CMakeLists.txt

190–192

I'm not actually entirely sure where/how llvm-lit is set up in that case - I think it might be built as part of the main llvm build, and then just implicitly assumed to exist there. If I leave this out, I get this:

ninja: Entering directory `/home/martin/code/llvm-project/build/new-standalone'
[0/1] cd /home/martin/code/llvm-project/build/new-standalone/libcxx/test && /usr/bin/python3.8 /home/martin/code/llvm-project/build/new-standalone/bin/llvm-lit /home/martin/code/llvm-project/build/new-standalone/libcxx/test
/usr/bin/python3.8: can't open file '/home/martin/code/llvm-project/build/new-standalone/bin/llvm-lit': [Errno 2] No such file or directory

If we include the llvm-lit subdir when we're called from llvm/runtimes, the nested external cmake build would override/clobber files that the toplevel cmake build produces too. Not sure if it's practically benign or if it would give nasty surprises down the line at some point.

Or should llvm/runtimes pass a more specific HAVE_LLVM_LIT=TRUE, or should we check whether the intended llvm-lit tool actually exists in the expected spot, and if not, include this? (Are there potential race conditions between the toplevel build and the cmake configuration for the runtimes then?)

[ldionne]

I'm glad we're all on the same page.

In D109593#3014343, @mstorsjo wrote:

That sounds like a good general direction. Distributors of llvm and the runtimes are going to need a decent sized transition period though.

Agreed, we can give them like 2 LLVM releases (roughly 1 year).

The bigger disruption is going to be when/if we stop allowing building the runtime projects entirely on their own by pointing cmake directly at e.g. libcxx. Judging from last weeks LLVM Distributors Conference, I would say that's the way a majority of distributors build it today, and transitioning that into e.g. 1. above will take some effort in the more complex cases. And some of them build this way with standalone source dirs without the surrounding monorepo, e.g. cc @tstellar.

Indeed, and that sort of build (standalone builds) are something I've been trying to eradicate for a while. That type of build makes everything a lot more complicated for us. I think moving to a "normal build" using <root>/runtimes as the CMake root should be pretty easy -- they'll just end up with a single CMake invocation instead of two or more.

In D109593#3014375, @phosek wrote:

Another thing I'd point out is that we may want to rethink the use of LLVM_ENABLE_RUNTIMES and <root>/llvm/runtimes. In the bootstrapping build, we want to build Clang and then use to build runtimes, but LLVM isn't aware of Clang, Clang depends on LLVM. To give you a concrete example, if you used -DLLVM_ENABLE_PROJECTS=llvm -DLLVM_ENABLE_RUNTIMES=all, it's not clear what should happen since there's no Clang being built.

I think that in the long run, it'd be better if the bootstrapping build had moved either into Clang or even to the very top-level of the monorepo (that is <root>/CMakeLists.txt) since in practice you'll also want a linker, and possibly Flang and other tools.

Yeah, I agree. How about this:

1. cmake -S <root>/llvm -DLLVM_ENABLE_PROJECTS=libcxx doesn't work anymore.
2. cmake -S <root> -DLLVM_ENABLE_PROJECTS=clang -DLLVM_BOOTSTRAP_RUNTIMES=libcxx performs a bootstrapping build (we can iterate on the naming, today this is -DLLVM_ENABLE_RUNTIMES=libcxx). If you don't add LLVM_ENABLE_PROJECTS=clang, the bootstrapping build can't work.
3. cmake -S <root>/runtimes -DLLVM_ENABLE_RUNTIMES=libcxx performs a normal build of libc++ with whatever system compiler you have.
4. All the other ways to build libc++ & al don't work anymore.

I think we can get there in the next year.

[phosek]

In D109593#3015946, @ldionne wrote:

I'm glad we're all on the same page.

In D109593#3014343, @mstorsjo wrote:

That sounds like a good general direction. Distributors of llvm and the runtimes are going to need a decent sized transition period though.

Agreed, we can give them like 2 LLVM releases (roughly 1 year).

The bigger disruption is going to be when/if we stop allowing building the runtime projects entirely on their own by pointing cmake directly at e.g. libcxx. Judging from last weeks LLVM Distributors Conference, I would say that's the way a majority of distributors build it today, and transitioning that into e.g. 1. above will take some effort in the more complex cases. And some of them build this way with standalone source dirs without the surrounding monorepo, e.g. cc @tstellar.

Indeed, and that sort of build (standalone builds) are something I've been trying to eradicate for a while. That type of build makes everything a lot more complicated for us. I think moving to a "normal build" using <root>/runtimes as the CMake root should be pretty easy -- they'll just end up with a single CMake invocation instead of two or more.

In D109593#3014375, @phosek wrote:

Another thing I'd point out is that we may want to rethink the use of LLVM_ENABLE_RUNTIMES and <root>/llvm/runtimes. In the bootstrapping build, we want to build Clang and then use to build runtimes, but LLVM isn't aware of Clang, Clang depends on LLVM. To give you a concrete example, if you used -DLLVM_ENABLE_PROJECTS=llvm -DLLVM_ENABLE_RUNTIMES=all, it's not clear what should happen since there's no Clang being built.

I think that in the long run, it'd be better if the bootstrapping build had moved either into Clang or even to the very top-level of the monorepo (that is <root>/CMakeLists.txt) since in practice you'll also want a linker, and possibly Flang and other tools.

Yeah, I agree. How about this:

1. cmake -S <root>/llvm -DLLVM_ENABLE_PROJECTS=libcxx doesn't work anymore.
2. cmake -S <root> -DLLVM_ENABLE_PROJECTS=clang -DLLVM_BOOTSTRAP_RUNTIMES=libcxx performs a bootstrapping build (we can iterate on the naming, today this is -DLLVM_ENABLE_RUNTIMES=libcxx). If you don't add LLVM_ENABLE_PROJECTS=clang, the bootstrapping build can't work.
3. cmake -S <root>/runtimes -DLLVM_ENABLE_RUNTIMES=libcxx performs a normal build of libc++ with whatever system compiler you have.
4. All the other ways to build libc++ & al don't work anymore.

I think we can get there in the next year.

Agreed, that's what I had in mind. It'd enable lot of simplification and cleanup throughout the build.

[mstorsjo]

runtimes/CMakeLists.txt

190–192

@phosek - What do you think about this? We can’t get rid of llvm-lit from llvm/runtimes as it’s not set up there but in the main llvm cmake.

[ldionne]

This looks reasonable to me. Unless @phosek objects, I'd like to get this landed so I can start transitioning things to this build ASAP.

[phosek]

runtimes/CMakeLists.txt

190–192

Having a more specific variable would be preferable. What I'm not sure about is where the lit depends comes from even in the bootstrapping build. We explicitly list other test dependencies here: https://github.com/llvm/llvm-project/blob/623f93ed1c99904a2b092c617bb392010e7a0184/llvm/runtimes/CMakeLists.txt#L461 but lit is not in that list. It's possible that this dependency comes from some other edge though. I want to make sure that this dependency is also expressed correctly, rather than it working thus far only by accident (which is a possibility).

[mstorsjo]

Pass HAVE_LLVM_LIT from llvm/runtimes, don't use LLVM_RUNTIMES_BUILD.

[mstorsjo]

runtimes/CMakeLists.txt

190–192

There's no build-time dependency for lit, the bin/llvm-lit script gets generated at configure time by configure_file(llvm-lit.in $out) - so whenever ninja is running, it should already exist. With that in mind, I guess it would be tolerable to try to peek here whether the supposed output llvm-lit executable script exists or not as well? The updated patch passes an explicit HAVE_LLVM_LIT though.

[phosek]

LGTM

[mstorsjo]

Rebased, rerun CI

[mstorsjo]

Retry CI after https://github.com/google/llvm-premerge-checks/pull/356#issuecomment-930272916

libcxx/utils/ci/buildkite-pipeline.yml

agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
+  - label: "New standalone runtimes build"
+    command: "libcxx/utils/ci/run-buildbot new-standalone"
+    artifact_paths:
+      - "**/test-results.xml"
+    agents:
+      queue: "libcxx-builders"
+      os: "linux"
+    retry:
+      automatic:
+        - exit_status: -1  # Agent was lost
+          limit: 2
+    timeout_in_minutes: 120
+
   - label: "Runtimes build"
     command: "libcxx/utils/ci/run-buildbot runtimes-build"
     artifact_paths:
       - "**/test-results.xml"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:

libcxx/utils/ci/run-buildbot

${CMAKE} \
           -GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
           -DCMAKE_BUILD_TYPE=RelWithDebInfo \
           -DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
           -DLLVM_ENABLE_PROJECTS="libcxx;libcxxabi;libunwind" \
           -DLIBCXX_TEST_CONFIG="${MONOREPO_ROOT}/libcxx/test/configs/libcxx-trunk-shared.cfg.in"
 
     check-cxx-cxxabi
 ;;
+new-standalone)
+
+    clean
+
+    echo "--- Generating CMake"
+    ${CMAKE} \
+          -S "${MONOREPO_ROOT}/runtimes" \
+          -B "${BUILD_DIR}" \
+          -GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
+          -DCMAKE_BUILD_TYPE=RelWithDebInfo \
+          -DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
+          -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" \
+          -DLIBCXX_TEST_CONFIG="${MONOREPO_ROOT}/libcxx/test/configs/libcxx-trunk-shared.cfg.in"
+
+    check-cxx-cxxabi
+;;
 runtimes-build)
     clean
 
     echo "--- Generating CMake"
     # TODO: We currently enable modules and assertions in the runtimes build
     #       because that provides coverage for some specific Clang failures
     #       we've been seeing recently, however it would be better to instead
     #       run all CI configurations against a Clang that has assertions enabled.

llvm/runtimes/CMakeLists.txt

 # TODO: This file assumes the Clang toolchain so it'd be better if it lived in
 # Clang, except there already is clang/runtime directory which contains
 # similar although simpler functionality. We should figure out how to merge
 # the two files.
 
 # TODO: Selecting runtimes should be always performed inside the runtimes
 # build, see runtimes/CMakeLists.txt, except that we currently check whether
 # compiler-rt is being built to determine whether to first build builtins
 # or not so we need that information in this file as well.
 set(LLVM_ALL_RUNTIMES "compiler-rt;libc;libcxx;libcxxabi;libunwind;openmp")
 set(LLVM_ENABLE_RUNTIMES "" CACHE STRING
   "Semicolon-separated list of runtimes to build (${LLVM_ALL_RUNTIMES}), or \"all\".")
 if(LLVM_ENABLE_RUNTIMES STREQUAL "all" )
   set(LLVM_ENABLE_RUNTIMES ${LLVM_ALL_RUNTIMES})
 endif()
+set(COMMON_CMAKE_ARGS "-DLLVM_RUNTIMES_BUILD=ON")
 foreach(proj ${LLVM_ENABLE_RUNTIMES})
   set(proj_dir "${CMAKE_CURRENT_SOURCE_DIR}/../../${proj}")
   if(IS_DIRECTORY ${proj_dir} AND EXISTS ${proj_dir}/CMakeLists.txt)
     list(APPEND runtimes ${proj_dir})
   else()
     message(FATAL_ERROR "LLVM_ENABLE_RUNTIMES requests ${proj} but directory not found: ${proj_dir}")
   endif()
   string(TOUPPER "${proj}" canon_name)
llvm_ExternalProject_Add(builtins
                            ${compiler_rt_path}/lib/builtins
                            DEPENDS ${ARG_DEPENDS}
                            CMAKE_ARGS -DLLVM_LIBRARY_OUTPUT_INTDIR=${LLVM_LIBRARY_DIR}
                                       -DLLVM_RUNTIME_OUTPUT_INTDIR=${LLVM_TOOLS_BINARY_DIR}
                                       -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_TRIPLE}
                                       -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=${LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default}
                                       -DCMAKE_C_COMPILER_WORKS=ON
                                       -DCMAKE_ASM_COMPILER_WORKS=ON
+                                      ${COMMON_CMAKE_ARGS}
                                       ${BUILTINS_CMAKE_ARGS}
                            PASSTHROUGH_PREFIXES COMPILER_RT
                            USE_TOOLCHAIN
                            TARGET_TRIPLE ${TARGET_TRIPLE}
                            ${EXTRA_ARGS})
 endfunction()
 
 function(builtin_register_target compiler_rt_path target)
llvm_ExternalProject_Add(builtins-${target}
                            DEPENDS ${ARG_DEPENDS}
                            CMAKE_ARGS -DLLVM_LIBRARY_OUTPUT_INTDIR=${LLVM_LIBRARY_DIR}
                                       -DLLVM_RUNTIME_OUTPUT_INTDIR=${LLVM_TOOLS_BINARY_DIR}
                                       -DLLVM_DEFAULT_TARGET_TRIPLE=${target}
                                       -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON
                                       -DCMAKE_C_COMPILER_WORKS=ON
                                       -DCMAKE_ASM_COMPILER_WORKS=ON
                                       -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON
+                                      ${COMMON_CMAKE_ARGS}
                                       ${${target}_extra_args}
                            USE_TOOLCHAIN
                            TARGET_TRIPLE ${target}
                            ${EXTRA_ARGS})
 endfunction()
 
 # If compiler-rt is present we need to build the builtin libraries first. This
 # is required because the other runtimes need the builtin libraries present
llvm_ExternalProject_Add(runtimes
                                       -DLLVM_INCLUDE_TESTS=${LLVM_INCLUDE_TESTS}
                                       -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_TRIPLE}
                                       -DLLVM_ENABLE_PROJECTS_USED=${LLVM_ENABLE_PROJECTS_USED}
                                       -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=${LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default}
                                       -DLLVM_BUILD_TOOLS=${LLVM_BUILD_TOOLS}
                                       -DCMAKE_C_COMPILER_WORKS=ON
                                       -DCMAKE_CXX_COMPILER_WORKS=ON
                                       -DCMAKE_ASM_COMPILER_WORKS=ON
+                                      ${COMMON_CMAKE_ARGS}
                                       ${RUNTIMES_CMAKE_ARGS}
                            PASSTHROUGH_PREFIXES LLVM_ENABLE_RUNTIMES
                                                 ${ARG_PREFIXES}
                            EXTRA_TARGETS ${extra_targets}
                                          ${test_targets}
                                          ${SUB_COMPONENTS}
                                          ${SUB_CHECK_TARGETS}
                                          ${SUB_INSTALL_TARGETS}
llvm_ExternalProject_Add(runtimes-${name}
                                       -DLLVM_DEFAULT_TARGET_TRIPLE=${target}
                                       -DLLVM_ENABLE_PROJECTS_USED=${LLVM_ENABLE_PROJECTS_USED}
                                       -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON
                                       -DCMAKE_C_COMPILER_WORKS=ON
                                       -DCMAKE_CXX_COMPILER_WORKS=ON
                                       -DCMAKE_ASM_COMPILER_WORKS=ON
                                       -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON
                                       -DLLVM_RUNTIMES_TARGET=${name}
+                                      ${COMMON_CMAKE_ARGS}
                                       ${${name}_extra_args}
                            EXTRA_TARGETS ${${name}_extra_targets}
                                          ${${name}_test_targets}
                            USE_TOOLCHAIN
                            TARGET_TRIPLE ${target}
                            ${EXTRA_ARGS})
 endfunction()
 

runtimes/CMakeLists.txt

if (NOT LLVM_FOUND)
   set(LLVM_LIBRARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/lib)
 endif()
 
 # Setting these variables will allow the sub-build to put their outputs into
 # the library and bin directories of the top-level build.
 set(LLVM_LIBRARY_OUTPUT_INTDIR ${LLVM_LIBRARY_DIR})
 set(LLVM_RUNTIME_OUTPUT_INTDIR ${LLVM_TOOLS_BINARY_DIR})
 
 # This variable makes sure that e.g. llvm-lit is found.
 set(LLVM_MAIN_SRC_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../llvm)
 set(LLVM_CMAKE_PATH ${LLVM_MAIN_SRC_DIR}/cmake/modules)
 
 # This variable is used by individual runtimes to locate LLVM files.
 set(LLVM_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../llvm)
 
 if(APPLE)
   set(LLVM_ENABLE_LIBCXX ON CACHE BOOL "")
 endif()
 
 include(CheckLibraryExists)
 include(CheckCCompilerFlag)
 include(CheckCXXCompilerFlag)
 
 # Disable use of the installed C++ standard library when building runtimes.
 # Check for -nostdlib++ first; if there's no C++ standard library yet,
 # all check_cxx_compiler_flag commands will fail until we add -nostdlib++
 # (or -nodefaultlibs).
 check_c_compiler_flag(-nostdlib++ LLVM_RUNTIMES_SUPPORT_NOSTDLIBXX_FLAG)
 if (LLVM_RUNTIMES_SUPPORT_NOSTDLIBXX_FLAG)
   set(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} -nostdlib++")
 endif()
 check_cxx_compiler_flag(-nostdinc++ LLVM_RUNTIMES_SUPPORT_NOSTDINCXX_FLAG)
 if (LLVM_RUNTIMES_SUPPORT_NOSTDINCXX_FLAG)
   set(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} -nostdinc++")
 endif()
 
 # Avoid checking whether the compiler is working.
 set(LLVM_COMPILER_CHECKED ON)
 
 # Handle common options used by all runtimes.
 include(AddLLVM)
 include(HandleLLVMOptions)
 
 find_package(Python3 REQUIRED COMPONENTS Interpreter)
 
 if (NOT LLVM_DEFAULT_TARGET_TRIPLE)
   include(GetHostTriple)
   get_host_triple(LLVM_DEFAULT_TARGET_TRIPLE)
 endif()
 
+if (NOT LLVM_RUNTIMES_BUILD)
+  option(LLVM_INCLUDE_TESTS "Generate build targets for the runtimes unit tests." ON)
+endif()
+
 # Use libtool instead of ar if you are both on an Apple host, and targeting Apple.
 if(CMAKE_HOST_APPLE AND APPLE)
   include(UseLibtool)
 endif()
 
-# This can be used to detect whether we're in the runtimes build.
-set(LLVM_RUNTIMES_BUILD ON)
-
 foreach(entry ${runtimes})
   get_filename_component(projName ${entry} NAME)
 
   # TODO: Clean this up as part of an interface standardization
   string(REPLACE "-" "_" canon_name ${projName})
   string(TOUPPER ${canon_name} canon_name)
 
   # TODO: compiler-rt has to use standalone build for now. We tried to remove
if(LLVM_INCLUDE_TESTS)
   add_lit_target(check-runtimes
     "Running all regression tests"
     ${RUNTIMES_LIT_TESTSUITES}
     PARAMS ${RUNTIMES_LIT_PARAMS}
     DEPENDS ${RUNTIMES_LIT_DEPENDS}
     ARGS ${RUNTIMES_LIT_EXTRA_ARGS}
     )
   add_custom_target(runtimes-test-depends DEPENDS ${RUNTIMES_LIT_DEPENDS})
+
+  if (NOT LLVM_RUNTIMES_BUILD)
+    # If built by manually invoking cmake on this directory, we don't have
+    # llvm-lit. If invoked via llvm/runtimes, this is already set up at that
+    # level.
+    add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/llvm-lit
+                     ${CMAKE_CURRENT_BINARY_DIR}/llvm-lit)
+  endif()
 endif()
 
 get_property(SUB_COMPONENTS GLOBAL PROPERTY SUB_COMPONENTS)
 if(SUB_COMPONENTS)
   list(REMOVE_DUPLICATES SUB_COMPONENTS)
   foreach(component ${SUB_COMPONENTS})
     if(NOT TARGET ${component})
       message(SEND_ERROR "Missing target for runtime component ${component}!")