[libc++][CI] Uses LLVM 17 in Docker.

[Mordante]

Summary

Updates the LLVM versions used in the Dockerfile. It also removes
obsolete symlinks. This doesn't update the Buildkite jobs, they need to
use the new Docker image before they can be updated.

[philnik]

LGTM assuming the Docker container builds.

libcxx/utils/ci/Dockerfile

79–80

Maybe also remove this?

[ldionne]

That's great, thanks!

[ldionne]

Is LLVM_HEAD_VERSION=17 going to work yet?

[jloser]

In D143007#4103560, @ldionne wrote:

Is LLVM_HEAD_VERSION=17 going to work yet?

I don't think so based on looking at the code:

declare -A LLVM_VERSION_PATTERNS
LLVM_VERSION_PATTERNS[9]="-9"
LLVM_VERSION_PATTERNS[10]="-10"
LLVM_VERSION_PATTERNS[11]="-11"
LLVM_VERSION_PATTERNS[12]="-12"
LLVM_VERSION_PATTERNS[13]="-13"
LLVM_VERSION_PATTERNS[14]="-14"
LLVM_VERSION_PATTERNS[15]="-15"
LLVM_VERSION_PATTERNS[16]="-16"
LLVM_VERSION_PATTERNS[17]=""

if [ ! ${LLVM_VERSION_PATTERNS[$LLVM_VERSION]+_} ]; then
    echo "This script does not support LLVM version $LLVM_VERSION"
    exit 3
fi

since LLVM_VERSION_PATTERNS[17] is an empty string.

[Mordante]

Thanks for the reviews!

In D143007#4103951, @jloser wrote:

In D143007#4103560, @ldionne wrote:

Is LLVM_HEAD_VERSION=17 going to work yet?

I don't think so based on looking at the code:

declare -A LLVM_VERSION_PATTERNS
LLVM_VERSION_PATTERNS[9]="-9"
LLVM_VERSION_PATTERNS[10]="-10"
LLVM_VERSION_PATTERNS[11]="-11"
LLVM_VERSION_PATTERNS[12]="-12"
LLVM_VERSION_PATTERNS[13]="-13"
LLVM_VERSION_PATTERNS[14]="-14"
LLVM_VERSION_PATTERNS[15]="-15"
LLVM_VERSION_PATTERNS[16]="-16"
LLVM_VERSION_PATTERNS[17]=""

if [ ! ${LLVM_VERSION_PATTERNS[$LLVM_VERSION]+_} ]; then
    echo "This script does not support LLVM version $LLVM_VERSION"
    exit 3
fi

since LLVM_VERSION_PATTERNS[17] is an empty string.

AFAIK That part of the code is to make sure lines like

deb http://apt.llvm.org/bullseye/ llvm-toolchain-bullseye-15 main
deb http://apt.llvm.org/bullseye/ llvm-toolchain-bullseye main

work properly in sources.list files in Debian/Ubuntu. Note the the ToT version has no version suffix.

I have tested the Docker file locally with several Clang versions, including 17, before I created this review.

There is one fix queued to be backported to the LLVM-16 after that has landed I'll land this and upload a new Docker image.
(Without that fix the LLVM-16 branch modular build will fail.)

libcxx/utils/ci/Dockerfile

79–80

I prefer to do that separately; I'm not convinced this will work for the LLVM-16 release branch.

[jloser]

In D143007#4104804, @Mordante wrote:

Thanks for the reviews!

In D143007#4103951, @jloser wrote:

In D143007#4103560, @ldionne wrote:

Is LLVM_HEAD_VERSION=17 going to work yet?

I don't think so based on looking at the code:

declare -A LLVM_VERSION_PATTERNS
LLVM_VERSION_PATTERNS[9]="-9"
LLVM_VERSION_PATTERNS[10]="-10"
LLVM_VERSION_PATTERNS[11]="-11"
LLVM_VERSION_PATTERNS[12]="-12"
LLVM_VERSION_PATTERNS[13]="-13"
LLVM_VERSION_PATTERNS[14]="-14"
LLVM_VERSION_PATTERNS[15]="-15"
LLVM_VERSION_PATTERNS[16]="-16"
LLVM_VERSION_PATTERNS[17]=""

if [ ! ${LLVM_VERSION_PATTERNS[$LLVM_VERSION]+_} ]; then
    echo "This script does not support LLVM version $LLVM_VERSION"
    exit 3
fi

since LLVM_VERSION_PATTERNS[17] is an empty string.

AFAIK That part of the code is to make sure lines like

deb http://apt.llvm.org/bullseye/ llvm-toolchain-bullseye-15 main
deb http://apt.llvm.org/bullseye/ llvm-toolchain-bullseye main

work properly in sources.list files in Debian/Ubuntu. Note the the ToT version has no version suffix.

I have tested the Docker file locally with several Clang versions, including 17, before I created this review.

There is one fix queued to be backported to the LLVM-16 after that has landed I'll land this and upload a new Docker image.
(Without that fix the LLVM-16 branch modular build will fail.)

Ah, I see — you're right after closer inspection of the llvm.sh script. This looks fine then — sorry about that!

[Mordante]

In D143007#4104934, @jloser wrote:

Ah, I see — you're right after closer inspection of the llvm.sh script. This looks fine then — sorry about that!

No problem.

libcxx/utils/ci/Dockerfile

 
 # Install Clang <latest>, <latest-1> and ToT, which are the ones we support.
 # We also install <latest-2> because we need to support the "latest-1" of the
 # current LLVM release branch, which is effectively the <latest-2> of the
 # tip-of-trunk LLVM. For example, after branching LLVM 14 but before branching
 # LLVM 15, we still need to have Clang 12 in this Docker image because the LLVM
 # 14 release branch CI uses it. The tip-of-trunk CI will never use Clang 12,
 # though.
-ENV LLVM_HEAD_VERSION=16
+ENV LLVM_HEAD_VERSION=17
 RUN apt-get update && apt-get install -y lsb-release wget software-properties-common
 RUN wget https://apt.llvm.org/llvm.sh -O /tmp/llvm.sh
 RUN bash /tmp/llvm.sh $(($LLVM_HEAD_VERSION - 3)) # for CI transitions
 RUN bash /tmp/llvm.sh $(($LLVM_HEAD_VERSION - 2)) # previous release
 RUN bash /tmp/llvm.sh $(($LLVM_HEAD_VERSION - 1)) # latest release
 RUN bash /tmp/llvm.sh $LLVM_HEAD_VERSION          # current ToT
 
-# Make the latest version of Clang the "default" compiler on the system
-# TODO: After branching for LLVM 16, all jobs will be using an
-#       explicitly-versioned version of Clang instead, so we can get rid of
-#       these symlinks.
-RUN ln -fs /usr/bin/clang++-14 /usr/bin/c++ && [ -e $(readlink /usr/bin/c++) ]
-RUN ln -fs /usr/bin/clang-14 /usr/bin/cc && [ -e $(readlink /usr/bin/cc) ]
-
 # Install clang-format; always use the lastest stable branch.
-# TODO LLVM 16 remove hard-coded version 14 and the symlinks
 RUN apt-get update && apt-get install -y clang-format-$(($LLVM_HEAD_VERSION - 2)) clang-format-$(($LLVM_HEAD_VERSION - 1))
-RUN ln -s /usr/bin/clang-format-14 /usr/bin/clang-format && [ -e $(readlink /usr/bin/clang-format) ]
-RUN ln -s /usr/bin/git-clang-format-14 /usr/bin/git-clang-format && [ -e $(readlink /usr/bin/git-clang-format) ]
 
 # Install clang-tidy
-# TODO LLVM 16 remove hard-coded version 14 and the symlink
-RUN apt-get update && apt-get install -y clang-tidy-$(($LLVM_HEAD_VERSION - 1)) clang-tidy-$LLVM_HEAD_VERSION clang-tidy-14
-RUN ln -s /usr/bin/clang-tidy-14 /usr/bin/clang-tidy && [ -e $(readlink /usr/bin/clang-tidy) ]
+RUN apt-get update && apt-get install -y clang-tidy-$(($LLVM_HEAD_VERSION - 2)) clang-tidy-$((LLVM_HEAD_VERSION - 1))
 
 # Install llvm-dev and libclang-dev to compile custom clang-tidy checks
 RUN apt-get update && apt-get install -y llvm-$(($LLVM_HEAD_VERSION - 1))-dev     llvm-$(($LLVM_HEAD_VERSION - 2))-dev \
                                      libclang-$(($LLVM_HEAD_VERSION - 1))-dev libclang-$(($LLVM_HEAD_VERSION - 2))-dev
 
 # TODO LLVM16 Don't install llvm-16-dev explicitly
 RUN apt-get update && apt-get install -y llvm-16-dev libclang-16-dev
 
 # Install the most recent GCC, like clang install the previous version as a transition.
 ENV GCC_LATEST_VERSION=12
 RUN apt-get update && apt install -y gcc-$((GCC_LATEST_VERSION - 1)) g++-$((GCC_LATEST_VERSION - 1))
 RUN apt-get update && apt install -y gcc-$GCC_LATEST_VERSION g++-$GCC_LATEST_VERSION
 
 # Install a recent CMake
 RUN wget https://github.com/Kitware/CMake/releases/download/v3.21.1/cmake-3.21.1-linux-x86_64.sh -O /tmp/install-cmake.sh