[libc++][CI] Increases the Clang version used.

[Mordante]

Summary

Changes the CI to use the Clang 16 nightly builds instead of Clang 14.
(The libc++15 branch was accidentally build using Clang 14 instead of
Clang 15; hence the skipping of a number.)

Also adds a Clang 15 build to the test matrix.

Based on the private discussion with @ldionne we decided to move
the configuration parameters from the run-buildbot script to the
CI configuration buildkite-pipeline.yml. Other hard-coded values
from the Dockerfile should be move to the CI configuration too. That
will be done in another commit.

C++17 will use Clang-15 since D131479 causes a test to fail.

[Mordante]

Retry CI with new Docker image.

[Mordante]

Fixes CI.

[Mordante]

Attempts to fix the CI.

[Mordante]

Another attempt to fix the CI.

[ldionne]

As discussed during live review, instead I think we should define the compiler in use inside buildkite-pipeline.yml via env (https://buildkite.com/docs/pipelines/command-step#command-step-attributes). This would allow dropping various versioned jobs from the run-buildbot script (e.g. generic-clang-14 would become a call to generic-cxx2b with env set to CXX=clang++-14).

[ldionne]

I think this means we should also somehow hardcode what version of e.g. clang-tidy is used in the .yml file. Basically, the Dockerfile installs a bunch of stuff but makes no decision as to which version of a given tool is used. Then, the .yml file picks the default versions it wants to use, and the run-buildbot script is as version-agnostic as possible.

[Mordante]

Tests the alternative approach as discussed during the live review.
(When this works the other jobs needs to be adjusted too.)

[Mordante]

Test CI with a global version.

[Mordante]

More testing.

[Mordante]

Convert all builders to the new style.

[Mordante]

Test.

[Mordante]

Try to use Clang 16, this might fail if the updated Docker image hasn't propagated to the CI runners.

[Mordante]

Rebased to verify the modular build failures are in ToT too.

[Mordante]

Allow all builds for testing.

[Mordante]

Test modular build with a newer Clang 15.

[Mordante]

Test with new Docker image and rebased.

[ldionne]

There's a few comments, but this LGTM. Thanks a lot, this is a realllllly nice improvement. This will decouple updating the compilers for CI from updating the Docker image on runners, which has been a source of pain in the past.

Just so that we're on the same page, this will also expose us to potential breaks in the nightly Clang build when we update the Docker image. However, I think that's acceptable since we can do basic testing before updating the Docker image.

libcxx/utils/ci/buildkite-pipeline.yml

174 ↗

(On Diff #455143)

I think these jobs were forgotten.

215 ↗

(On Diff #455143)

Is this supported? That would be nice.

356 ↗

(On Diff #455143)

I think this one should be using clang-head too. It just means that we will build Clang from scratch using clang-head, and then we'll build libc++ using the just-built clang-fromscratch.

It's mostly for consistency with other CI jobs, I guess, because in practice we should be able to build Clang from scratch using almost any system compiler.

[Mordante]

Addresses review comments.

[Mordante]

libcxx/utils/ci/buildkite-pipeline.yml

215 ↗

(On Diff #455143)

Interesting suggestion! It works
https://buildkite.com/llvm-project/libcxx-ci/builds/13149

libcxx/utils/ci/run-buildbot

 #!/usr/bin/env bash
 #===----------------------------------------------------------------------===##
 #
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 #
 #===----------------------------------------------------------------------===##
 
 set -ex
 set -o pipefail
 unset LANG
 unset LC_ALL
 unset LC_COLLATE
+export CC=clang-15
+export CXX=clang++-15
 
 PROGNAME="$(basename "${0}")"
 
 function usage() {
 cat <<EOF
 Usage:
 ${PROGNAME} [options] <BUILDER>