[libc++] This is a dummy Phabricator review to test pre-commit testing

[ldionne]

Trigger build

[ldionne]

Don't run CI for real to speed things up.

[ldionne]

Pretend that one job is failing to see what happens

[ldionne]

Turn on Phabricator reporting

[ldionne]

Turn back on full testing. Let's see how that goes.

[ldionne]

Added exponential backoff retries in phabricator-report

[ldionne]

Don't run the benchmarks for now, that's really too long.

[ldionne]

Don't run the benchmarks, it's too long.

[ldionne]

Adjust how we're determining failures.

[ldionne]

Try running the buildkite agent as non-root.

[ldionne]

Run as non-root in a way that actually works.

[ldionne]

Fix which GCC is used

[ldionne]

Rebase onto master to get some fixes in

[ldionne]

Rebase onto master with GCC fixes

[ldionne]

Also handle non-phab review commits

[ldionne]

Play around with triggering

[ldionne]

Try fixing trigger

[ldionne]

Correct pipeline name

[ldionne]

Improve message

[ldionne]

Rebase onto master

[ldionne]

Rebase onto master

libcxx/utils/ci/Dockerfile

 #
 # To start a shell in the Docker image, use:
 #   $ docker run -it --volume "$(git rev-parse --show-toplevel):/llvm" --workdir "/llvm" $(docker build -q .) bash
 #
 # This will fire up the Docker container and mount the root of the monorepo
 # as /llvm in the container. Be careful, the state in /llvm is shared between
 # the container and the host machine.
 #
+# Finally, a pre-built version of this image is available on DockerHub as
+# ldionne/libcxx-builders. To use the pre-built version of the image, use
+#
+#   $ docker pull ldionne/libcxx-builder
+#   $ docker run -it <options> ldionne/libcxx-builder
+#
+# To update the image, rebuild it and push it to ldionne/libcxx-builder (which
+# will obviously only work if you have permission to do so).
+#
+#   $ docker build -t ldionne/libcxx-builder .
+#   $ docker push ldionne/libcxx-builder
+#
 
 FROM ubuntu:bionic
 
 RUN apt-get update
 RUN apt-get install -y bash curl
 
 # Install the most recently released LLVM
 RUN apt-get install -y lsb-release wget software-properties-common

 # Install a recent CMake
 RUN wget https://github.com/Kitware/CMake/releases/download/v3.18.2/cmake-3.18.2-Linux-x86_64.sh -O /tmp/install-cmake.sh
 RUN bash /tmp/install-cmake.sh --prefix=/usr --exclude-subdir --skip-license
 RUN rm /tmp/install-cmake.sh
 
 # Install other tools used by the build or the test suite
 RUN apt-get install -y ninja-build python3 sphinx-doc
 
-# Install the Buildkite agent and dependencies
-RUN bash -c "$(curl -sL https://raw.githubusercontent.com/buildkite/agent/master/install.sh)"
-RUN apt-get install -y git
-ENV PATH="${PATH}:/root/.buildkite-agent/bin"
-
 # Install the Phabricator Python module to allow uploading results to Phabricator
 RUN apt-get install -y python3-pip
 RUN pip3 install phabricator
 
+# Change the user to a non-root user, since some of the libc++ tests
+# (e.g. filesystem) require running as non-root. Also setup passwordless sudo.
+RUN apt-get install -y sudo
+RUN echo "ALL ALL = (ALL) NOPASSWD: ALL" >> /etc/sudoers
+RUN useradd -u 8877 libcxx-builder
+USER libcxx-builder
+
+# Install the Buildkite agent and dependencies. This must be done as non-root
+# for the Buildkite agent to be installed in a path where we can find it.
+RUN sudo bash -c "$(curl -sL https://raw.githubusercontent.com/buildkite/agent/master/install.sh)"
+RUN sudo apt-get install -y git
+ENV PATH="${PATH}:/home/libcxx-builder/.buildkite-agent/bin"
+
 # By default, start the Buildkite agent (this requires a token).
 CMD buildkite-agent start --tags "queue=libcxx-builders"

libcxx/utils/ci/buildkite-pipeline.yml

 #
 # This file should never contain logic -- all the logic must be offloaded
 # into scripts. This is critical to being able to reproduce CI issues outside
 # of the CI environment, which is important for debugging.
 #
 
 steps:
   - label: "C++03"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx03"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx03 | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "C++11"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx11"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx11 | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "C++14"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx14"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx14 | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "C++17"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx17"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx17 | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "C++20"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx2a"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-cxx2a | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "-fno-exceptions"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-noexceptions"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-noexceptions | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "32 bits"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-32bit"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-32bit | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "GCC/C++20"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-gcc"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-gcc | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "ASAN"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-asan"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-asan | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "TSAN"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-tsan"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-tsan | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "UBSAN"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-ubsan"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-ubsan | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "With LLVM's libunwind"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-with_llvm_unwinder"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-with_llvm_unwinder | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"
 
   - label: "Single-threaded"
-    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-singlethreaded"
+    command: "set -o pipefail && libcxx/utils/ci/run-buildbot.sh x86_64-ubuntu-singlethreaded | libcxx/utils/ci/phabricator-report"
     agents:
       queue: "libcxx-builders"

libcxx/utils/ci/phabricator-report

 #!/usr/bin/env python3
 #===----------------------------------------------------------------------===##
 #
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 #
 #===----------------------------------------------------------------------===##
 
 import argparse
 import io
 import os
 import phabricator
 import re
+import socket
 import subprocess
 import sys
 import time
 
 LLVM_REVIEWS_API = "https://reviews.llvm.org/api/"
 
+def exponentialBackoffRetry(f, exception, maxAttempts=3):
+    """Tries calling a function, but retry with exponential backoff if the
+       function fails with the specified exception.
+    """
+    waitTime = 1
+    attempts = 0
+    while True:
+        try:
+            f()
+            break
+        except exception as e:
+            attempts += 1
+            if attempts == maxAttempts:
+                raise e
+            else:
+                time.sleep(waitTime)
+                waitTime *= 2
+
+def buildPassed(log):
+    """
+    Tries to guess whether a build has passed or not based on the logs
+    produced by it.
+
+    This is really hacky -- it would be better to use the status of the
+    script that runs the tests, however that script is being piped into
+    this script, so we can't know its exit status. What we do here is
+    basically look for abnormal CMake or Lit output, but that is tightly
+    coupled to the specific CI we're running.
+    """
+    # Lit reporting failures
+    matches = re.findall(r"^\s*Failed\s*:\s*(\d+)$", log, flags=re.MULTILINE)
+    if matches and any(int(match) > 0 for match in matches):
+        return False
+
+    # Error while running CMake
+    if 'CMake Error' in log or 'Configuring incomplete, errors occurred!' in log:
+        return False
+
+    # Ninja failed to build some target
+    if 'FAILED:' in log:
+        return False
+
+    return True
+
 def main(argv):
     parser = argparse.ArgumentParser(
         description="""
 This script gathers information about a Buildkite build and updates the
 Phabricator review associated to the HEAD commit with those results.
 
 The intended usage of this script is to pipe the output of a command defined
 in a Buildkite pipeline into it. The script will echo everything to stdout,
while True:
         sys.stdout.write(line)
         sys.stdout.flush() # flush every line to avoid buffering
         log.write(line)
     end = time.time()
 
     # Then, extract information from the environment and post-process the logs.
     log.seek(0)
     log = log.read()
-    result = 'fail' if 'FAILED:' in log else 'pass'
+    result = 'pass' if buildPassed(log) else 'fail'
     resultObject = {
         'name': '{BUILDKITE_LABEL} ({BUILDKITE_BUILD_URL}#{BUILDKITE_JOB_ID})'.format(**os.environ),
         'result': result,
         'duration': end - start,
         'details': log
     }
 
     commitMessage = subprocess.check_output(['git', 'log', '--format=%B' , '-n', '1']).decode()
     phabricatorID = re.search(r'^Phabricator-ID:\s+(.+)$', commitMessage, flags=re.MULTILINE)
     if not phabricatorID:
         raise RuntimeError('Could not find the Phabricator ID in the commit message. '
                            'The commit message was:\n{}'.format(commitMessage))
     else:
         phabricatorID = phabricatorID.group(1)
 
     token = os.environ['CONDUIT_TOKEN']
     phab = phabricator.Phabricator(token=token, host=LLVM_REVIEWS_API)
-    phab.harbormaster.sendmessage(buildTargetPHID=phabricatorID, type=result, unit=[resultObject])
+    exponentialBackoffRetry(
+        lambda: phab.harbormaster.sendmessage(buildTargetPHID=phabricatorID, type=result, unit=[resultObject]),
+        exception=socket.timeout
+    )
 
 if __name__ == '__main__':
     main(sys.argv[1:])

libcxx/utils/ci/run-buildbot.sh

 ninja -C "${BUILD_DIR}" check-cxx-deps cxxabi
 
 echo "+++ Running the libc++ tests"
 ninja -C "${BUILD_DIR}" check-cxx
 
 echo "+++ Running the libc++abi tests"
 ninja -C "${BUILD_DIR}" check-cxxabi
 
-echo "+++ Running the libc++ benchmarks"
-ninja -C "${BUILD_DIR}" check-cxx-benchmarks
+# echo "+++ Running the libc++ benchmarks"
+# ninja -C "${BUILD_DIR}" check-cxx-benchmarks