[libc++][ci] Use ccache in the jobs that build Clang

[ldionne]

Summary

This is an attempt to reduce the time taken by the Bootstrapping
build job and the Clang CI job that builds the compiler from scratch.

[Mordante]

When the cache is preserved between runs we might want to do the same for libc++.

LGTM when the CI is green.

libcxx/utils/ci/buildkite-pipeline-clang.yml

27–28

This gives statistics of the cache usage. I think that will at least initially be useful. Maybe we want to tweak cache parameters, for example the size.

[philnik]

In D150908#4356524, @Mordante wrote:

When the cache is preserved between runs we might want to do the same for libc++.

I don't think that makes a lot of sense. The libc++ build takes only a few seconds and (unfortunately) relies on large parts of our headers, which change with pretty much every patch, making it probably not very profitable. Also, while it's not a big problem, it does add another point of failure to the build. It's most likely worth the small risk for clang builds, but I't not convinced it is for libc++ builds.

[Mordante]

In D150908#4356632, @philnik wrote:

In D150908#4356524, @Mordante wrote:

When the cache is preserved between runs we might want to do the same for libc++.

I don't think that makes a lot of sense. The libc++ build takes only a few seconds and (unfortunately) relies on large parts of our headers, which change with pretty much every patch, making it probably not very profitable. Also, while it's not a big problem, it does add another point of failure to the build. It's most likely worth the small risk for clang builds, but I't not convinced it is for libc++ builds.

I don't recall the last time I had issues with ccache. But as said we "might want to" consider it so we need to measure. I've no idea how much time the runners spend on building libc++. This is the same for a lot of runners since we always build it with C++20. Also not all headers are always touched.

[ldionne]

Poke CI now that images should contain ccache.

[Mordante]

libcxx/utils/ci/buildkite-pipeline-clang.yml

26

I've fixed this in https://reviews.llvm.org/rG4de9936fe0e31ceb817db1cdfc5dd4af2d44e01e
I've created D151780 to validate whether this solves the build issue (https://buildkite.com/llvm-project/libcxx-ci/builds/25187)

[ldionne]

libcxx/utils/ci/buildkite-pipeline-clang.yml

26

Ah, thanks for the catch. Yeah this is an obvious mistake but I forgot to trigger the Clang CI to test this patch.

libcxx/utils/ci/buildkite-pipeline-clang.yml

env:
     LLVM_HEAD_VERSION: "17"
 steps:
   - label: "Building clang"
     commands:
       - "mkdir install"
       # We use Release here to avoid including debug information. Otherwise, the clang binary is very large, which
       # is problematic because we need to upload the artifacts for other jobs to use. This may seem like nothing,
       # but with the number of jobs we run daily, this can result in thousands of GB of network I/O.
-      - "cmake -S llvm -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=install -DLLVM_ENABLE_PROJECTS=\"clang;compiler-rt\""
+      - "cmake -S llvm -B build -G Ninja -DCMAKE_CXX_COMPILER_LAUNCHER="ccache" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=install -DLLVM_ENABLE_PROJECTS=\"clang;compiler-rt\""
       - "ninja -C build install-clang install-clang-resource-headers"
+      - "ccache -s"
       - "tar -cJvf install.tar.xz install/"
       - "buildkite-agent artifact upload --debug install.tar.xz"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
     agents:
       queue: "libcxx-builders"
       os: "linux"

libcxx/utils/ci/run-buildbot

 bootstrapping-build)
     clean
 
     echo "--- Generating CMake"
     ${CMAKE} \
           -S "${MONOREPO_ROOT}/llvm" \
           -B "${BUILD_DIR}" \
           -GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
+          -DCMAKE_CXX_COMPILER_LAUNCHER="ccache" \
           -DCMAKE_BUILD_TYPE=RelWithDebInfo \
           -DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
           -DLLVM_ENABLE_PROJECTS="clang" \
           -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" \
           -DLLVM_RUNTIME_TARGETS="$(${CXX} --print-target-triple)" \
           -DLLVM_TARGETS_TO_BUILD="host" \
           -DRUNTIMES_BUILD_ALLOW_DARWIN=ON \
           -DLLVM_ENABLE_ASSERTIONS=ON