test-release.sh: build projects and runtimes lists with semicolons

[dim]

Summary

While doing a test-release.sh run on FreeBSD, I ran into a sed error due
to the introduction of the GNU extension '\s' in commit 500587e23dfd.

Scanning for blanks (spaces and tabs) could be done in a more portable
fashion using the [[:blank:]] character class. But it is easier to avoid
the original problem, which is that the projects and runtime lists have
to be separated by semicolons, and cannot start with a semicolon.

Instead, use the shell's alternate value parameter expansion mechanism,
which makes it easy to append items to lists with separators in between,
and without any leading separator. This also avoids having to run sed on
the end result.

In addition, build any selected runtimes in the second phase, otherwise
the third phase can fail to find several symbols in compiler-rt, if that
has been built. This is because the host's compiler-rt is not guaranteed
to have those symbols.

[hans]

With this, could we get rid of project_list and runtime_list and just use projects and runtimes directly throughout?

[dim]

In D145884#4189341, @hans wrote:

With this, could we get rid of project_list and runtime_list and just use projects and runtimes directly throughout?

Yes, at least for the runtimes part, but the projects part should still be controlled by the if that checks for Phase 3. (Hmm there should be comment above that explaining why, but I digress.)

[dim]

Add another build optimization: in the first phase, we only need to
build a working clang compiler, which is capable of building itself and
the runtimes in the second phase. Therefore, we can skip building any of
the runtimes in the first phase.

[dim]

Rebase against main, and squash.

[tstellar]

llvm/utils/release/test-release.sh

421

Is the phase 2 compiler configured to use the runtimes from phase 2 when building phase 3?

[dim]

llvm/utils/release/test-release.sh

421

Yes, since if you _don't_ build any runtimes in phase 2, you will receive the following error (at least on macOS) while building phase 3:

# Phase 3: Building llvmCore
...
# env CC=/Users/dim/llvm/16.0.0/rc1/Phase2/Release/llvmCore-16.0.0-rc1.install/usr/local/bin/clang CXX=/Users/dim/llvm/16.0.0/rc1/Phase2/Release/llvmCore-16.0.0
-rc1.install/usr/local/bin/clang++ cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DLLVM_ENABLE_PROJECTS=llvm;clang;clang-tools-extra;openmp;lld;lldb;polly;mlir;flang -DLLVM_LIT_ARGS=-j 4 -DLLVM_ENABLE_RUNTIMES=compiler-rt;libcxx;libcxxabi;libunwind -DLLVM_USE_STATIC_ZSTD=ON /Users/dim/llvm/16.0.0/rc1/llvm-project/llvm
...
FAILED: lib/libomp.dylib
: && /Users/dim/llvm/16.0.0/rc1/Phase2/Release/llvmCore-16.0.0-rc1.install/usr/local/bin/clang [... elided long command line to link libomp.dylib ...]
Undefined symbols for architecture arm64:
  "___divdc3", referenced from:
      ___kmpc_atomic_cmplx8_div in kmp_atomic.cpp.o
      ___kmpc_atomic_cmplx4_div_cmplx8 in kmp_atomic.cpp.o
      ___kmpc_atomic_cmplx8_div_cpt in kmp_atomic.cpp.o
      ___kmpc_atomic_cmplx10_div_cpt in kmp_atomic.cpp.o
  "___divsc3", referenced from:
      ___kmpc_atomic_cmplx4_div in kmp_atomic.cpp.o
      ___kmpc_atomic_cmplx4_div_cpt in kmp_atomic.cpp.o
ld: symbol(s) not found for architecture arm64

As far as I'm aware, clang will search for its runtimes relative to its binary, so if they aren't there in ../lib/clang/16, you get an error like above for symbols that are linked in from the builtins library (in this case, complex division).

[dim]

So I've been using this successfully for the last two macOS releases, 16.0.0 and 16.0.1. Is this now OK to commit?

[tstellar]

LGTM.

[amyk]

On Power, I am seeing extra LIT failures after applying this patch that did not happen before:

AddressSanitizer-powerpc64le-linux :: TestCases/Linux/malloc_delete_mismatch.cpp
AddressSanitizer-powerpc64le-linux-dynamic :: TestCases/Linux/malloc_delete_mismatch.cpp
MemorySanitizer-POWERPC64LE :: Linux/sigandorset.cpp
SanitizerCommon-asan-powerpc64le-Linux :: onprint.cpp
SanitizerCommon-tsan-powerpc64le-Linux :: onprint.cpp
UBSan-AddressSanitizer-powerpc64le :: TestCases/Misc/Linux/diag-stacktrace.cpp
UBSan-MemorySanitizer-powerpc64le :: TestCases/Misc/Linux/diag-stacktrace.cpp
UBSan-Standalone-powerpc64le :: TestCases/Misc/Linux/diag-stacktrace.cpp
UBSan-ThreadSanitizer-powerpc64le :: TestCases/Misc/Linux/diag-stacktrace.cpp

As well as object file differences (that appear to occur every time):

# Comparing Phase 2 and Phase 3 files
file Process.cpp.o differs between phase 2 and phase 3
file CRC.cpp.o differs between phase 2 and phase 3
file Compression.cpp.o differs between phase 2 and phase 3
file llvm-config.cpp.o differs between phase 2 and phase 3

It looks like these are all related to not being able to find ZLIB for whatever reason, as I can see in the new test failures similar messages along the lines of:

. . . 
        2: error: failed to decompress '.debug_aranges', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
         3: error: failed to decompress '.debug_info', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
         4: error: failed to decompress '.debug_abbrev', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
         5: error: failed to decompress '.debug_line', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
         6: error: failed to decompress '.debug_str', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
         7: error: failed to decompress '.debug_loc', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
         8: error: failed to decompress '.debug_ranges', LLVM was not built with LLVM_ENABLE_ZLIB or did not find zlib at build time
. . .

I think this makes sense, as I can even see on the Phase 3 of building LLVM, the following message during configuration:

-- Could NOT find ZLIB (missing: ZLIB_LIBRARY) (found version "1.2.11")

Whereas I get a message for Phase 3 saying that the ZLIB library was found (prior to building with this patched version of test-release.sh).

In any case, I just wanted to quickly follow up and report with the testing results that I did with this patch.

[dim]

In D145884#4256208, @amyk wrote:

I think this makes sense, as I can even see on the Phase 3 of building LLVM, the following message during configuration:

-- Could NOT find ZLIB (missing: ZLIB_LIBRARY) (found version "1.2.11")

Whereas I get a message for Phase 3 saying that the ZLIB library was found (prior to building with this patched version of test-release.sh).

Interesting, it will depend on which mechanism is used to find zlib. I am unsure if the standard CMake functionality is used for this, or a hand-written llvm-specific thing. I will see if I can dig a bit into this.

[dim]

In D145884#4256599, @dim wrote:

In D145884#4256208, @amyk wrote:

I think this makes sense, as I can even see on the Phase 3 of building LLVM, the following message during configuration:

-- Could NOT find ZLIB (missing: ZLIB_LIBRARY) (found version "1.2.11")

Whereas I get a message for Phase 3 saying that the ZLIB library was found (prior to building with this patched version of test-release.sh).

Interesting, it will depend on which mechanism is used to find zlib. I am unsure if the standard CMake functionality is used for this, or a hand-written llvm-specific thing. I will see if I can dig a bit into this.

I was able to reproduce this on Ubuntu Linux, but I'm not sure if it has to do with this particular review. If you run the cmake commands with --debug-find (which shows a lot of additional information), you see for example with the stage 2 builtins:

/usr/bin/cmake --debug-find -DCMAKE_C_COMPILER=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/clang -DCMAKE_CXX_COMPILER=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/clang++ -DCMAKE_ASM_COMPILER=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/clang -DCMAKE_AR=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-ar -DCMAKE_RANLIB=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-ranlib -DCMAKE_NM=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-nm -DCMAKE_OBJDUMP=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-objdump -DCMAKE_OBJCOPY=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-objcopy -DCMAKE_STRIP=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-strip -DCMAKE_READELF=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-readelf -DCMAKE_C_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_CXX_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_ASM_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_INSTALL_PREFIX=/usr/local -DLLVM_BINARY_DIR=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj -DLLVM_CONFIG_PATH=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/bin/llvm-config -DLLVM_ENABLE_WERROR=OFF -DLLVM_HOST_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_HAVE_LINK_VERSION_SCRIPT=1 -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=OFF -DLLVM_USE_RELATIVE_PATHS_IN_FILES=OFF -DLLVM_LIT_ARGS=-j 12 -v -DLLVM_SOURCE_PREFIX= -DPACKAGE_VERSION=16.0.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_MAKE_PROGRAM=/bin/ninja -DCMAKE_C_COMPILER_LAUNCHER= -DCMAKE_CXX_COMPILER_LAUNCHER= -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DLLVM_LIBRARY_OUTPUT_INTDIR=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./lib -DLLVM_RUNTIME_OUTPUT_INTDIR=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON -DCMAKE_C_COMPILER_WORKS=ON -DCMAKE_ASM_COMPILER_WORKS=ON -DHAVE_LLVM_LIT=ON -GNinja /home/dim/llvm/16.0.1/final/llvm-project/llvm/runtimes/../../compiler-rt/lib/builtins
...
CMake Debug Log at /usr/share/cmake-3.22/Modules/FindZLIB.cmake:86 (find_library):
  find_library called with the following settings:

    VAR: ZLIB_LIBRARY_RELEASE
    NAMES: "z"
           "zlib"
           "zdll"
           "zlib1"
           "zlibstatic"
    Documentation: Path to a library.
    Framework
      Only Search Frameworks: 0
      Search Frameworks Last: 0
      Search Frameworks First: 0
    AppBundle
      Only Search AppBundle: 0
      Search AppBundle Last: 0
      Search AppBundle First: 0
    CMAKE_FIND_USE_CMAKE_PATH: 1
    CMAKE_FIND_USE_CMAKE_ENVIRONMENT_PATH: 1
    CMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH: 1
    CMAKE_FIND_USE_CMAKE_SYSTEM_PATH: 1

  find_library considered the following locations:

    /usr/sbin/lib/(lib)z(\.so|\.a)
    /usr/sbin/lib/(lib)zlib(\.so|\.a)
    /usr/sbin/lib/(lib)zdll(\.so|\.a)
    /usr/sbin/lib/(lib)zlib1(\.so|\.a)
    /usr/sbin/lib/(lib)zlibstatic(\.so|\.a)
    /usr/sbin/(lib)z(\.so|\.a)
    /usr/sbin/(lib)zlib(\.so|\.a)
    /usr/sbin/(lib)zdll(\.so|\.a)
    /usr/sbin/(lib)zlib1(\.so|\.a)
    /usr/sbin/(lib)zlibstatic(\.so|\.a)
    /usr/bin/lib/(lib)z(\.so|\.a)
    /usr/bin/lib/(lib)zlib(\.so|\.a)
    /usr/bin/lib/(lib)zdll(\.so|\.a)
    /usr/bin/lib/(lib)zlib1(\.so|\.a)
    /usr/bin/lib/(lib)zlibstatic(\.so|\.a)
    /usr/bin/(lib)z(\.so|\.a)
    /usr/bin/(lib)zlib(\.so|\.a)
    /usr/bin/(lib)zdll(\.so|\.a)
    /usr/bin/(lib)zlib1(\.so|\.a)
    /usr/bin/(lib)zlibstatic(\.so|\.a)
    /usr/local/lib/lib/(lib)z(\.so|\.a)
    /usr/local/lib/lib/(lib)zlib(\.so|\.a)
    /usr/local/lib/lib/(lib)zdll(\.so|\.a)
    /usr/local/lib/lib/(lib)zlib1(\.so|\.a)
    /usr/local/lib/lib/(lib)zlibstatic(\.so|\.a)
    /usr/local/lib/(lib)z(\.so|\.a)
    /usr/local/lib/(lib)zlib(\.so|\.a)
    /usr/local/lib/(lib)zdll(\.so|\.a)
    /usr/local/lib/(lib)zlib1(\.so|\.a)
    /usr/local/lib/(lib)zlibstatic(\.so|\.a)
    /usr/local/lib/(lib)z(\.so|\.a)
    /usr/local/lib/(lib)zlib(\.so|\.a)
    /usr/local/lib/(lib)zdll(\.so|\.a)
    /usr/local/lib/(lib)zlib1(\.so|\.a)
    /usr/local/lib/(lib)zlibstatic(\.so|\.a)
    /usr/local/(lib)z(\.so|\.a)
    /usr/local/(lib)zlib(\.so|\.a)
    /usr/local/(lib)zdll(\.so|\.a)
    /usr/local/(lib)zlib1(\.so|\.a)
    /usr/local/(lib)zlibstatic(\.so|\.a)
    /usr/lib/lib/(lib)z(\.so|\.a)
    /usr/lib/lib/(lib)zlib(\.so|\.a)
    /usr/lib/lib/(lib)zdll(\.so|\.a)
    /usr/lib/lib/(lib)zlib1(\.so|\.a)
    /usr/lib/lib/(lib)zlibstatic(\.so|\.a)
    /usr/lib/(lib)z(\.so|\.a)
    /usr/lib/(lib)zlib(\.so|\.a)
    /usr/lib/(lib)zdll(\.so|\.a)
    /usr/lib/(lib)zlib1(\.so|\.a)
    /usr/lib/(lib)zlibstatic(\.so|\.a)
    /usr/lib/(lib)z(\.so|\.a)
    /usr/lib/(lib)zlib(\.so|\.a)
    /usr/lib/(lib)zdll(\.so|\.a)
    /usr/lib/(lib)zlib1(\.so|\.a)
    /usr/lib/(lib)zlibstatic(\.so|\.a)
    /usr/(lib)z(\.so|\.a)
    /usr/(lib)zlib(\.so|\.a)
    /usr/(lib)zdll(\.so|\.a)
    /usr/(lib)zlib1(\.so|\.a)
    /usr/(lib)zlibstatic(\.so|\.a)
    /lib/lib/(lib)z(\.so|\.a)
    /lib/lib/(lib)zlib(\.so|\.a)
    /lib/lib/(lib)zdll(\.so|\.a)
    /lib/lib/(lib)zlib1(\.so|\.a)
    /lib/lib/(lib)zlibstatic(\.so|\.a)
    /lib/(lib)z(\.so|\.a)
    /lib/(lib)zlib(\.so|\.a)
    /lib/(lib)zdll(\.so|\.a)
    /lib/(lib)zlib1(\.so|\.a)
    /lib/(lib)zlibstatic(\.so|\.a)
    /usr/X11R6/lib/lib/(lib)z(\.so|\.a)
    /usr/X11R6/lib/lib/(lib)zlib(\.so|\.a)
    /usr/X11R6/lib/lib/(lib)zdll(\.so|\.a)
    /usr/X11R6/lib/lib/(lib)zlib1(\.so|\.a)
    /usr/X11R6/lib/lib/(lib)zlibstatic(\.so|\.a)
    /usr/X11R6/lib/(lib)z(\.so|\.a)
    /usr/X11R6/lib/(lib)zlib(\.so|\.a)
    /usr/X11R6/lib/(lib)zdll(\.so|\.a)
    /usr/X11R6/lib/(lib)zlib1(\.so|\.a)
    /usr/X11R6/lib/(lib)zlibstatic(\.so|\.a)
    /usr/X11R6/lib/(lib)z(\.so|\.a)
    /usr/X11R6/lib/(lib)zlib(\.so|\.a)
    /usr/X11R6/lib/(lib)zdll(\.so|\.a)
    /usr/X11R6/lib/(lib)zlib1(\.so|\.a)
    /usr/X11R6/lib/(lib)zlibstatic(\.so|\.a)
    /usr/X11R6/(lib)z(\.so|\.a)
    /usr/X11R6/(lib)zlib(\.so|\.a)
    /usr/X11R6/(lib)zdll(\.so|\.a)
    /usr/X11R6/(lib)zlib1(\.so|\.a)
    /usr/X11R6/(lib)zlibstatic(\.so|\.a)
    /usr/pkg/lib/lib/(lib)z(\.so|\.a)
    /usr/pkg/lib/lib/(lib)zlib(\.so|\.a)
    /usr/pkg/lib/lib/(lib)zdll(\.so|\.a)
    /usr/pkg/lib/lib/(lib)zlib1(\.so|\.a)
    /usr/pkg/lib/lib/(lib)zlibstatic(\.so|\.a)
    /usr/pkg/lib/(lib)z(\.so|\.a)
    /usr/pkg/lib/(lib)zlib(\.so|\.a)
    /usr/pkg/lib/(lib)zdll(\.so|\.a)
    /usr/pkg/lib/(lib)zlib1(\.so|\.a)
    /usr/pkg/lib/(lib)zlibstatic(\.so|\.a)
    /usr/pkg/lib/(lib)z(\.so|\.a)
    /usr/pkg/lib/(lib)zlib(\.so|\.a)
    /usr/pkg/lib/(lib)zdll(\.so|\.a)
    /usr/pkg/lib/(lib)zlib1(\.so|\.a)
    /usr/pkg/lib/(lib)zlibstatic(\.so|\.a)
    /usr/pkg/(lib)z(\.so|\.a)
    /usr/pkg/(lib)zlib(\.so|\.a)
    /usr/pkg/(lib)zdll(\.so|\.a)
    /usr/pkg/(lib)zlib1(\.so|\.a)
    /usr/pkg/(lib)zlibstatic(\.so|\.a)
    /opt/lib/lib/(lib)z(\.so|\.a)
    /opt/lib/lib/(lib)zlib(\.so|\.a)
    /opt/lib/lib/(lib)zdll(\.so|\.a)
    /opt/lib/lib/(lib)zlib1(\.so|\.a)
    /opt/lib/lib/(lib)zlibstatic(\.so|\.a)
    /opt/lib/(lib)z(\.so|\.a)
    /opt/lib/(lib)zlib(\.so|\.a)
    /opt/lib/(lib)zdll(\.so|\.a)
    /opt/lib/(lib)zlib1(\.so|\.a)
    /opt/lib/(lib)zlibstatic(\.so|\.a)
    /opt/lib/(lib)z(\.so|\.a)
    /opt/lib/(lib)zlib(\.so|\.a)
    /opt/lib/(lib)zdll(\.so|\.a)
    /opt/lib/(lib)zlib1(\.so|\.a)
    /opt/lib/(lib)zlibstatic(\.so|\.a)
    /opt/(lib)z(\.so|\.a)
    /opt/(lib)zlib(\.so|\.a)
    /opt/(lib)zdll(\.so|\.a)
    /opt/(lib)zlib1(\.so|\.a)
    /opt/(lib)zlibstatic(\.so|\.a)
    /usr/lib/X11/lib/(lib)z(\.so|\.a)
    /usr/lib/X11/lib/(lib)zlib(\.so|\.a)
    /usr/lib/X11/lib/(lib)zdll(\.so|\.a)
    /usr/lib/X11/lib/(lib)zlib1(\.so|\.a)
    /usr/lib/X11/lib/(lib)zlibstatic(\.so|\.a)
    /usr/lib/X11/(lib)z(\.so|\.a)
    /usr/lib/X11/(lib)zlib(\.so|\.a)
    /usr/lib/X11/(lib)zdll(\.so|\.a)
    /usr/lib/X11/(lib)zlib1(\.so|\.a)
    /usr/lib/X11/(lib)zlibstatic(\.so|\.a)
    /registry/lib/(lib)z(\.so|\.a)
    /registry/lib/(lib)zlib(\.so|\.a)
    /registry/lib/(lib)zdll(\.so|\.a)
    /registry/lib/(lib)zlib1(\.so|\.a)
    /registry/lib/(lib)zlibstatic(\.so|\.a)
    /registry/(lib)z(\.so|\.a)
    /registry/(lib)zlib(\.so|\.a)
    /registry/(lib)zdll(\.so|\.a)
    /registry/(lib)zlib1(\.so|\.a)
    /registry/(lib)zlibstatic(\.so|\.a)
    /zlib/lib/(lib)z(\.so|\.a)
    /zlib/lib/(lib)zlib(\.so|\.a)
    /zlib/lib/(lib)zdll(\.so|\.a)
    /zlib/lib/(lib)zlib1(\.so|\.a)
    /zlib/lib/(lib)zlibstatic(\.so|\.a)
    /zlib/(lib)z(\.so|\.a)
    /zlib/(lib)zlib(\.so|\.a)
    /zlib/(lib)zdll(\.so|\.a)
    /zlib/(lib)zlib1(\.so|\.a)
    /zlib/(lib)zlibstatic(\.so|\.a)

  The item was not found.

I.e. CMake searches a *lot* of locations, except the correct one, which is /usr/lib/x86_64-linux-gnu on Ubuntu!

For some unknown reason, this does work correctly for the cmake command that is used to configure the stage 2 runtimes target:

/usr/bin/cmake --debug-find -DCMAKE_C_COMPILER=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/clang -DCMAKE_CXX_COMPILER=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/clang++ -DCMAKE_ASM_COMPILER=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/clang -DCMAKE_AR=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-ar -DCMAKE_RANLIB=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-ranlib -DCMAKE_NM=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-nm -DCMAKE_OBJDUMP=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-objdump -DCMAKE_OBJCOPY=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-objcopy -DCMAKE_STRIP=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-strip -DCMAKE_READELF=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/./bin/llvm-readelf -DCMAKE_C_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_CXX_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_ASM_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_INSTALL_PREFIX=/usr/local -DLLVM_BINARY_DIR=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj -DLLVM_CONFIG_PATH=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.obj/bin/llvm-config -DLLVM_ENABLE_WERROR=OFF -DLLVM_HOST_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_HAVE_LINK_VERSION_SCRIPT=1 -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=OFF -DLLVM_USE_RELATIVE_PATHS_IN_FILES=OFF -DLLVM_LIT_ARGS=-j 12 -v -DLLVM_SOURCE_PREFIX= -DPACKAGE_VERSION=16.0.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_MAKE_PROGRAM=/bin/ninja -DCMAKE_C_COMPILER_LAUNCHER= -DCMAKE_CXX_COMPILER_LAUNCHER= -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCOMPILER_RT_BUILD_BUILTINS=Off -DLLVM_INCLUDE_TESTS=ON -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_ENABLE_PROJECTS_USED=ON -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON -DLLVM_BUILD_TOOLS=ON -DCMAKE_C_COMPILER_WORKS=ON -DCMAKE_CXX_COMPILER_WORKS=ON -DCMAKE_ASM_COMPILER_WORKS=ON -DHAVE_LLVM_LIT=ON -DLLVM_ENABLE_RUNTIMES=compiler-rt;libcxx;libcxxabi;libunwind -GNinja /home/dim/llvm/16.0.1/final/llvm-project/llvm/runtimes/../../runtimes
...
CMake Debug Log at /usr/share/cmake-3.22/Modules/FindZLIB.cmake:86 (find_library):
  find_library called with the following settings:

    VAR: ZLIB_LIBRARY_RELEASE
    NAMES: "z"
           "zlib"
           "zdll"
           "zlib1"
           "zlibstatic"
    Documentation: Path to a library.
    Framework
      Only Search Frameworks: 0
      Search Frameworks Last: 0
      Search Frameworks First: 0
    AppBundle
      Only Search AppBundle: 0
      Search AppBundle Last: 0
      Search AppBundle First: 0
    CMAKE_FIND_USE_CMAKE_PATH: 1
    CMAKE_FIND_USE_CMAKE_ENVIRONMENT_PATH: 1
    CMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH: 1
    CMAKE_FIND_USE_CMAKE_SYSTEM_PATH: 1

  find_library considered the following locations:

    /usr/sbin/lib/(lib)z(\.so|\.a)
    /usr/sbin/lib/(lib)zlib(\.so|\.a)
    /usr/sbin/lib/(lib)zdll(\.so|\.a)
    /usr/sbin/lib/(lib)zlib1(\.so|\.a)
    /usr/sbin/lib/(lib)zlibstatic(\.so|\.a)
    /usr/sbin/(lib)z(\.so|\.a)
    /usr/sbin/(lib)zlib(\.so|\.a)
    /usr/sbin/(lib)zdll(\.so|\.a)
    /usr/sbin/(lib)zlib1(\.so|\.a)
    /usr/sbin/(lib)zlibstatic(\.so|\.a)
    /usr/bin/lib/(lib)z(\.so|\.a)
    /usr/bin/lib/(lib)zlib(\.so|\.a)
    /usr/bin/lib/(lib)zdll(\.so|\.a)
    /usr/bin/lib/(lib)zlib1(\.so|\.a)
    /usr/bin/lib/(lib)zlibstatic(\.so|\.a)
    /usr/bin/(lib)z(\.so|\.a)
    /usr/bin/(lib)zlib(\.so|\.a)
    /usr/bin/(lib)zdll(\.so|\.a)
    /usr/bin/(lib)zlib1(\.so|\.a)
    /usr/bin/(lib)zlibstatic(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/lib/(lib)z(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/lib/(lib)zlib(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/lib/(lib)zdll(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/lib/(lib)zlib1(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/lib/(lib)zlibstatic(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/(lib)z(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/(lib)zlib(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/(lib)zdll(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/(lib)zlib1(\.so|\.a)
    /usr/local/lib/x86_64-linux-gnu/(lib)zlibstatic(\.so|\.a)
    /usr/local/lib/lib/(lib)z(\.so|\.a)
    /usr/local/lib/lib/(lib)zlib(\.so|\.a)
    /usr/local/lib/lib/(lib)zdll(\.so|\.a)
    /usr/local/lib/lib/(lib)zlib1(\.so|\.a)
    /usr/local/lib/lib/(lib)zlibstatic(\.so|\.a)
    /usr/local/lib/(lib)z(\.so|\.a)
    /usr/local/lib/(lib)zlib(\.so|\.a)
    /usr/local/lib/(lib)zdll(\.so|\.a)
    /usr/local/lib/(lib)zlib1(\.so|\.a)
    /usr/local/lib/(lib)zlibstatic(\.so|\.a)
    /usr/local/lib/(lib)z(\.so|\.a)
    /usr/local/lib/(lib)zlib(\.so|\.a)
    /usr/local/lib/(lib)zdll(\.so|\.a)
    /usr/local/lib/(lib)zlib1(\.so|\.a)
    /usr/local/lib/(lib)zlibstatic(\.so|\.a)
    /usr/local/(lib)z(\.so|\.a)
    /usr/local/(lib)zlib(\.so|\.a)
    /usr/local/(lib)zdll(\.so|\.a)
    /usr/local/(lib)zlib1(\.so|\.a)
    /usr/local/(lib)zlibstatic(\.so|\.a)
    /usr/lib/x86_64-linux-gnu/lib/(lib)z(\.so|\.a)
    /usr/lib/x86_64-linux-gnu/lib/(lib)zlib(\.so|\.a)
    /usr/lib/x86_64-linux-gnu/lib/(lib)zdll(\.so|\.a)
    /usr/lib/x86_64-linux-gnu/lib/(lib)zlib1(\.so|\.a)
    /usr/lib/x86_64-linux-gnu/lib/(lib)zlibstatic(\.so|\.a)

  The item was found at

    /usr/lib/x86_64-linux-gnu/libz.so

So there it *does* know that it should look under /usr/libx86_64-linux-gnu... I think this is some weird multilib quirk, but I am still unsure.

[dim]

So this seems to have been caused by @mstorsjo 's rG8368e4d54c459fe173d76277f17c632478e91add, which added:

set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)

to compiler-rt/lib/builtins/CMakeLists.txt. Because this line is there, the checks in CMAKE_DETERMINE_COMPILER_ABI don't run the linker, and this CMake functionality needs to see linker output lines to determine the actual ABI, in this case x86_64-linux-gnu.

E.g. when configuring the runtimes, I eventually see in the cmake debug output:

...
CMakeCCompiler.cmake(56):  if(CMAKE_C_LIBRARY_ARCHITECTURE )
CMakeCCompiler.cmake(57):  set(CMAKE_LIBRARY_ARCHITECTURE x86_64-linux-gnu )
CMakeCCompiler.cmake(60):  set(CMAKE_C_CL_SHOWINCLUDES_PREFIX  )
...

whereas with the builtins, this variable stays empty:

...
CMakeCCompiler.cmake(56):  if(CMAKE_C_LIBRARY_ARCHITECTURE )
CMakeCCompiler.cmake(60):  set(CMAKE_C_CL_SHOWINCLUDES_PREFIX  )
...

Because CMAKE_LIBRARY_ARCHITECTURE isn't set, the find_library() calls for e.g. zlib (but also other libraries) fail to search /usr/lib/x86_64-linux-gnu (or other ABI specific library paths).

In short, I think what @amyk saw is orthogonal to this review, and I would therefore like to commit it. :)

[dim]

FWIW, a build off a stock llvmorg-16.0.1 tag's test-release.sh only builds the runtimes for Phase 3, so you will only see the "could not find ZLIB" once:

# Phase 3: Building llvmCore
...
# env CC=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.install/usr/local/bin/clang CXX=/home/dim/llvm/16.0.1/final/Phase2/Release/llvmCore-16.0.1-final.install/usr/local/bin/clang++ cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_E
NABLE_ASSERTIONS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DLLVM_ENABLE_PROJECTS=llvm;clang;clang-tools-extra;openmp;bolt;lld;lldb;polly;mlir;flang -DLLVM_LIT_ARGS=-j 12 -v -DLLVM_ENABLE_RUNTIMES=compiler-rt;libcxx;libcxxabi;libunwind /home/dim/llvm/1
6.0.1/final/llvm-project/llvm
...
[9365/9507] cd /home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/runtimes/builtins-bins && /usr/bin/cmake -DCMAKE_C_COMPILER=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/clang -DCMAKE_CXX_COMPILER=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/clang++ -DCMAKE_ASM_COMPILER=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/clang -DCMAKE_LINKER=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/ld.lld -DCMAKE_AR=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/llvm-ar -DCMAKE_RANLIB=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/llvm-ranlib -DCMAKE_NM=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/llvm-nm -DCMAKE_OBJDUMP=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/llvm-objdump -DCMAKE_OBJCOPY=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/llvm-objcopy -DCMAKE_STRIP=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/llvm-strip -DCMAKE_READELF=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/llvm-readelf -DCMAKE_C_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_CXX_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_ASM_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_INSTALL_PREFIX=/usr/local -DLLVM_BINARY_DIR=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj -DLLVM_CONFIG_PATH=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/bin/llvm-config -DLLVM_ENABLE_WERROR=OFF -DLLVM_HOST_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_HAVE_LINK_VERSION_SCRIPT=1 -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=OFF -DLLVM_USE_RELATIVE_PATHS_IN_FILES=OFF "-DLLVM_LIT_ARGS=-j 12 -v" -DLLVM_SOURCE_PREFIX= -DPACKAGE_VERSION=16.0.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_MAKE_PROGRAM=/bin/ninja -DCMAKE_C_COMPILER_LAUNCHER= -DCMAKE_CXX_COMPILER_LAUNCHER= -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DLLVM_LIBRARY_OUTPUT_INTDIR=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./lib -DLLVM_RUNTIME_OUTPUT_INTDIR=/home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON -DCMAKE_C_COMPILER_WORKS=ON -DCMAKE_ASM_COMPILER_WORKS=ON -DHAVE_LLVM_LIT=ON -GNinja /home/dim/llvm/16.0.1/final/llvm-project/llvm/runtimes/../../compiler-rt/lib/builtins && /usr/bin/cmake -E touch /home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/runtimes/builtins-stamps//builtins-configure
-- The C compiler identification is Clang 16.0.1
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: /home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/clang
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/dim/llvm/16.0.1/final/Phase3/Release/llvmCore-16.0.1-final.obj/./bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for unwind.h
-- Looking for unwind.h - found
-- Looking for rpc/xdr.h
-- Looking for rpc/xdr.h - not found
CMake Warning at /home/dim/llvm/16.0.1/final/llvm-project/compiler-rt/cmake/Modules/CompilerRTUtils.cmake:281 (message):
  LLVM_CONFIG_PATH is deprecated, please use LLVM_CMAKE_DIR instead
Call Stack (most recent call first):
  CMakeLists.txt:36 (load_llvm_config)


-- Could NOT find Terminfo (missing: Terminfo_LIBRARIES Terminfo_LINKABLE)
-- Could NOT find ZLIB (missing: ZLIB_LIBRARY) (found version "1.2.11")
-- Could NOT find zstd (missing: zstd_LIBRARY)
-- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found version "2.9.13")

Note that it does find libxml2, because it uses a different mechanism (via pkg-config I believe).

[mstorsjo]

In D145884#4262759, @dim wrote:

So this seems to have been caused by @mstorsjo 's rG8368e4d54c459fe173d76277f17c632478e91add, which added:

set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)

to compiler-rt/lib/builtins/CMakeLists.txt. Because this line is there, the checks in CMAKE_DETERMINE_COMPILER_ABI don't run the linker, and this CMake functionality needs to see linker output lines to determine the actual ABI, in this case x86_64-linux-gnu.

CC @phosek - since that change does seem to be causing issues, should we perhaps revert it? (This would require me to revert https://github.com/mstorsjo/llvm-mingw/commit/056cb22fb537e392d525b869f92699441704a78b downstream, to start passing -DCMAKE_C_COMPILER_WORKS=1 when building the builtins again - but that shouldn't be any problem.)

In D145884#4262759, @dim wrote:

Because CMAKE_LIBRARY_ARCHITECTURE isn't set, the find_library() calls for e.g. zlib (but also other libraries) fail to search /usr/lib/x86_64-linux-gnu (or other ABI specific library paths).

I've seen this issue wrt CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY before, but I'm wondering how it ends up here in this case. Since we're only setting CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY while configuring specifically compiler-rt/lib/builtins, I presume there's nothing that looks for zlib within that? Or does the runtimes build somehow reuse caches for later stages, so that the builtins stage which didn't detect that part of the architecture, ends up breaking later stages?

[dim]

In D145884#4264777, @mstorsjo wrote:

...

In D145884#4262759, @dim wrote:

Because CMAKE_LIBRARY_ARCHITECTURE isn't set, the find_library() calls for e.g. zlib (but also other libraries) fail to search /usr/lib/x86_64-linux-gnu (or other ABI specific library paths).

I've seen this issue wrt CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY before, but I'm wondering how it ends up here in this case. Since we're only setting CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY while configuring specifically compiler-rt/lib/builtins, I presume there's nothing that looks for zlib within that? Or does the runtimes build somehow reuse caches for later stages, so that the builtins stage which didn't detect that part of the architecture, ends up breaking later stages?

compiler-rt/lib/builtins/CMakeLists.txt includes compiler-rt/cmake/Modules/CompilerRTUtils.cmake, then it calls load_llvm_config() (a helper function from CompilertRTUtils). load_llvm_config() figures out the path to llvm/cmake, and as far I understand, loads all kinds of settings from there, including paths to llvm headers, libraries, etc. As part of this, it clearly also descends into the "finding required libraries" parts, as it searches for terminfo, zlib, libxml2, etc.

However, when building the runtimes, it starts from the top-level runtimes/CMakeLists.txt, so it doesn't seem to follow this logic.

[mstorsjo]

In D145884#4269286, @dim wrote:

compiler-rt/lib/builtins/CMakeLists.txt includes compiler-rt/cmake/Modules/CompilerRTUtils.cmake, then it calls load_llvm_config() (a helper function from CompilertRTUtils). load_llvm_config() figures out the path to llvm/cmake, and as far I understand, loads all kinds of settings from there, including paths to llvm headers, libraries, etc. As part of this, it clearly also descends into the "finding required libraries" parts, as it searches for terminfo, zlib, libxml2, etc.

However, when building the runtimes, it starts from the top-level runtimes/CMakeLists.txt, so it doesn't seem to follow this logic.

Right, but when the build of compiler-rt/lib/builtins failed to detect zlib, where does that detection result end up used? I presume that the result of that detection ends up in a cmake cache, but I would presume that that cmake cache is specific for the builtins cmake invocation, not influencing other steps (where CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY shouldn't have any effect).

[dim]

In D145884#4269404, @mstorsjo wrote:

In D145884#4269286, @dim wrote:

compiler-rt/lib/builtins/CMakeLists.txt includes compiler-rt/cmake/Modules/CompilerRTUtils.cmake, then it calls load_llvm_config() (a helper function from CompilertRTUtils). load_llvm_config() figures out the path to llvm/cmake, and as far I understand, loads all kinds of settings from there, including paths to llvm headers, libraries, etc. As part of this, it clearly also descends into the "finding required libraries" parts, as it searches for terminfo, zlib, libxml2, etc.

However, when building the runtimes, it starts from the top-level runtimes/CMakeLists.txt, so it doesn't seem to follow this logic.

Right, but when the build of compiler-rt/lib/builtins failed to detect zlib, where does that detection result end up used? I presume that the result of that detection ends up in a cmake cache, but I would presume that that cmake cache is specific for the builtins cmake invocation, not influencing other steps (where CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY shouldn't have any effect).

Well, the builtins and runtimes CMake stuff is run as a separate step from the 'main' build, so I assume the detection results end up in separate CMakeCache files. That said, I am unsure whether any of the builtins or runtimes even care about, or use zlib. In case they do, detecting it properly is important, otherwise the "could not find" message should either be silenced or avoided.

[mstorsjo]

In D145884#4283446, @dim wrote:

In D145884#4269404, @mstorsjo wrote:

In D145884#4269286, @dim wrote:

compiler-rt/lib/builtins/CMakeLists.txt includes compiler-rt/cmake/Modules/CompilerRTUtils.cmake, then it calls load_llvm_config() (a helper function from CompilertRTUtils). load_llvm_config() figures out the path to llvm/cmake, and as far I understand, loads all kinds of settings from there, including paths to llvm headers, libraries, etc. As part of this, it clearly also descends into the "finding required libraries" parts, as it searches for terminfo, zlib, libxml2, etc.

However, when building the runtimes, it starts from the top-level runtimes/CMakeLists.txt, so it doesn't seem to follow this logic.

Right, but when the build of compiler-rt/lib/builtins failed to detect zlib, where does that detection result end up used? I presume that the result of that detection ends up in a cmake cache, but I would presume that that cmake cache is specific for the builtins cmake invocation, not influencing other steps (where CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY shouldn't have any effect).

Well, the builtins and runtimes CMake stuff is run as a separate step from the 'main' build, so I assume the detection results end up in separate CMakeCache files. That said, I am unsure whether any of the builtins or runtimes even care about, or use zlib. In case they do, detecting it properly is important, otherwise the "could not find" message should either be silenced or avoided.

Oh, ok, so the issue is only on the level of printing a surprising failed detection, but no actual noted missing/differing behaviour?

[dim]

In D145884#4303276, @mstorsjo wrote:

In D145884#4283446, @dim wrote:

In D145884#4269404, @mstorsjo wrote:

In D145884#4269286, @dim wrote:

compiler-rt/lib/builtins/CMakeLists.txt includes compiler-rt/cmake/Modules/CompilerRTUtils.cmake, then it calls load_llvm_config() (a helper function from CompilertRTUtils). load_llvm_config() figures out the path to llvm/cmake, and as far I understand, loads all kinds of settings from there, including paths to llvm headers, libraries, etc. As part of this, it clearly also descends into the "finding required libraries" parts, as it searches for terminfo, zlib, libxml2, etc.

However, when building the runtimes, it starts from the top-level runtimes/CMakeLists.txt, so it doesn't seem to follow this logic.

Right, but when the build of compiler-rt/lib/builtins failed to detect zlib, where does that detection result end up used? I presume that the result of that detection ends up in a cmake cache, but I would presume that that cmake cache is specific for the builtins cmake invocation, not influencing other steps (where CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY shouldn't have any effect).

Well, the builtins and runtimes CMake stuff is run as a separate step from the 'main' build, so I assume the detection results end up in separate CMakeCache files. That said, I am unsure whether any of the builtins or runtimes even care about, or use zlib. In case they do, detecting it properly is important, otherwise the "could not find" message should either be silenced or avoided.

Oh, ok, so the issue is only on the level of printing a surprising failed detection, but no actual noted missing/differing behaviour?

Some cursory searching in compiler-rt finds that at least compiler-rt/test/fuzzer/CompressedTest.cpp tries to use zlib, to "consume a Zlib-compressed input". I guess that one test wouldn't work then.

I couldn't find any mention of zlib in the other runtimes, e.g. libcxx, libcxxabi and libunwind.

[mstorsjo]

In D145884#4303549, @dim wrote:

In D145884#4303276, @mstorsjo wrote:

In D145884#4283446, @dim wrote:

Well, the builtins and runtimes CMake stuff is run as a separate step from the 'main' build, so I assume the detection results end up in separate CMakeCache files. That said, I am unsure whether any of the builtins or runtimes even care about, or use zlib. In case they do, detecting it properly is important, otherwise the "could not find" message should either be silenced or avoided.

Oh, ok, so the issue is only on the level of printing a surprising failed detection, but no actual noted missing/differing behaviour?

Some cursory searching in compiler-rt finds that at least compiler-rt/test/fuzzer/CompressedTest.cpp tries to use zlib, to "consume a Zlib-compressed input". I guess that one test wouldn't work then.

Right, but the build that uses CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY is for compiler-rt/lib/builtins only - that won't be set for a build that targets all of compiler-rt, so the fuzzer parts should't be affected, I think?

[dim]

In D145884#4304136, @mstorsjo wrote:

In D145884#4303549, @dim wrote:

In D145884#4303276, @mstorsjo wrote:

In D145884#4283446, @dim wrote:

Well, the builtins and runtimes CMake stuff is run as a separate step from the 'main' build, so I assume the detection results end up in separate CMakeCache files. That said, I am unsure whether any of the builtins or runtimes even care about, or use zlib. In case they do, detecting it properly is important, otherwise the "could not find" message should either be silenced or avoided.

Oh, ok, so the issue is only on the level of printing a surprising failed detection, but no actual noted missing/differing behaviour?

Some cursory searching in compiler-rt finds that at least compiler-rt/test/fuzzer/CompressedTest.cpp tries to use zlib, to "consume a Zlib-compressed input". I guess that one test wouldn't work then.

Right, but the build that uses CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY is for compiler-rt/lib/builtins only - that won't be set for a build that targets all of compiler-rt, so the fuzzer parts should't be affected, I think?

Indeed, it looks like the error won't matter for either the builtins or the runtimes. So it is probably not necessary to revert rG8368e4d54c459fe173d76277f17c632478e91add, as I assume that would cause other trouble.

[mstorsjo]

In D145884#4305349, @dim wrote:

In D145884#4304136, @mstorsjo wrote:

Right, but the build that uses CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY is for compiler-rt/lib/builtins only - that won't be set for a build that targets all of compiler-rt, so the fuzzer parts should't be affected, I think?

Indeed, it looks like the error won't matter for either the builtins or the runtimes.

Ok, great, then we're on the same page - thanks for taking the time to wade through the reasoning!

So it is probably not necessary to revert rG8368e4d54c459fe173d76277f17c632478e91add, as I assume that would cause other trouble.

Maybe; the main trouble would be that I would need to reintroduce -DCMAKE_C_COMPILER_WORKS=TRUE -DCMAKE_CXX_COMPILER_WORKS=TRUE in my setup where I build compiler-rt builtins, but I think that's fine.

For context: I started out wanting to clean this up so that people setting up a toolchain from scratch wouldn't need to set those flags. This worked for compiler-rt/lib/builtins (but with some warts as you've noticed), but turned out to not work for libunwind/libcxxabi/libcxx (which need to do a bit of link testing, while setting CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY makes all link testing no-ops). So when we're not progressing towards the goal of not needing the *_COMPILER_WORKS flags anyway, it might be good to get rid of this inconsistency in compiler-rt/lib/builtins too.

I'll see if I send patch for that later.

llvm/utils/release/test-release.sh

 if [ -z "$NumJobs" ]; then
     NumJobs=`grep -c processor /proc/cpuinfo 2> /dev/null || true`
 fi
 if [ -z "$NumJobs" ]; then
     NumJobs=3
 fi
 
 # Projects list
-projects="llvm clang"
+projects="llvm;clang"
 if [ $do_clang_tools = "yes" ]; then
-  projects="$projects clang-tools-extra"
+  projects="${projects:+$projects;}clang-tools-extra"
 fi
 runtimes=""
 if [ $do_rt = "yes" ]; then
-  runtimes="$runtimes compiler-rt"
+  runtimes="${runtimes:+$runtimes;}compiler-rt"
 fi
 if [ $do_libs = "yes" ]; then
-  runtimes="$runtimes libcxx"
+  runtimes="${runtimes:+$runtimes;}libcxx"
   if [ $do_libcxxabi = "yes" ]; then
-    runtimes="$runtimes libcxxabi"
+    runtimes="${runtimes:+$runtimes;}libcxxabi"
   fi
   if [ $do_libunwind = "yes" ]; then
-    runtimes="$runtimes libunwind"
+    runtimes="${runtimes:+$runtimes;}libunwind"
   fi
 fi
 if [ $do_openmp = "yes" ]; then
-  projects="$projects openmp"
+  projects="${projects:+$projects;}openmp"
 fi
 if [ $do_bolt = "yes" ]; then
-  projects="$projects bolt"
+  projects="${projects:+$projects;}bolt"
 fi
 if [ $do_lld = "yes" ]; then
-  projects="$projects lld"
+  projects="${projects:+$projects;}lld"
 fi
 if [ $do_lldb = "yes" ]; then
-  projects="$projects lldb"
+  projects="${projects:+$projects;}lldb"
 fi
 if [ $do_polly = "yes" ]; then
-  projects="$projects polly"
+  projects="${projects:+$projects;}polly"
 fi
 if [ $do_mlir = "yes" ]; then
-  projects="$projects mlir"
+  projects="${projects:+$projects;}mlir"
 fi
 if [ $do_flang = "yes" ]; then
-  projects="$projects flang"
+  projects="${projects:+$projects;}flang"
 fi
 
 # Go to the build directory (may be different from CWD)
 BuildDir=$BuildDir/$RC
 mkdir -p $BuildDir
 cd $BuildDir
 
 # Location of log files.
case $Flavor in
             ;;
         * )
             echo "# Invalid flavor '$Flavor'"
             echo ""
             return
             ;;
     esac
 
-    if [ "$Phase" -eq "3" ]; then
-      project_list=${projects// /;}
-      # Leading spaces will result in ";<runtime name>". This causes a CMake
-      # error because the empty string before the first ';' is treated as an
-      # unknown runtime name.
-      runtimes=$(echo $runtimes | sed -e 's/^\s*//')
-      runtime_list=${runtimes// /;}
-    else
+    # During the first two phases, there is no need to build any of the projects
+    # except clang, since these phases are only meant to produce a bootstrapped
+    # clang compiler, capable of building the third phase.
+    if [ "$Phase" -lt "3" ]; then
       project_list="clang"
+    else
+      project_list="$projects"
+    fi
+    # During the first phase, there is no need to build any of the runtimes,
+    # since this phase is only meant to get a clang compiler, capable of
+    # building itself and any selected runtimes in the second phase.
+    if [ "$Phase" -lt "2" ]; then
       runtime_list=""
+    else
+      runtime_list="$runtimes"
     fi
+
     echo "# Using C compiler: $c_compiler"
     echo "# Using C++ compiler: $cxx_compiler"
 
     cd $ObjDir
     echo "# Configuring llvm $Release-$RC $Flavor"
 
     echo "#" env CC="$c_compiler" CXX="$cxx_compiler" \
         cmake -G "$generator" \