This is an archive of the discontinued LLVM Phabricator instance.

Improved InformativeMailNotifier.
AbandonedPublic

Authored by ZijunZhao on Sep 9 2021, 6:27 PM.

Details

Summary

Listen for the main branch.

[buildbot] Fix worker for ThinLTO whole program devirtualization

This fixes a syntax error in the extra_configure_args provided to
getClangWithLTOBuildFactory, we shouldn't be surrounding the options
with double quotes.

Once that is fixed, I realized from testing the failing command locally
that it will still fail, because the clang-specific extra_configure_args
are also used by the initial stage1 build with the system compiler,
which is gcc. We only want these for the last stage, where LTO is
performed. Added a new parameter to getClangWithLTOBuildFactory,
extra_configure_args_lto_stage, to pass the options only to that LTO
stage.

Is there a way to test my change to ClangLTOBuilder.py?

jenkins: Do not resume the pipelines if the master restarts

It should help keep them from waiting forever to resume.

[MLGO] Corrected cmake variable

Added DoxygenDocsBuilder to build and publish doxygen documentation. This would build the doxygen documentation for all the LLVM projects which have it, and publish to proper places in llvm.org.

Added getHtmlDocsBuildFactory to build and publish LNT documentation.

This is to build and publish LNT Sphinx documentation.

Added bots to build and publish doxygen docs and LNT sphinx docs.

Fixed bad identation in ClangLTOBuilder.

Renamed HtmlSphinxDocsBuilder to HtmlDocsBuilder.

Use clang-stage2-Rthinlto host-compilers to for LLDB jobs

This should prevent that we have no regression information from LLDB because
the latest stage1 Clang we use as the host-compiler can't compile LLVM itself.

added .pylintrc

So that it can check the python code in the repo
without errors on missing imports.

mlir-nvidia: set -DMLIR_BINDINGS_PYTHON_ENABLED=ON

[jenkins] Fix exceptions raised by urllib.request

[jenkins] Fix typo in the host-compiler path used by lldb-cmake-matrix

Start Docker-izing sanitizer-windows

This defines a new docker image for sanitizer-windows based on the
existing docker image. I chose to dockerize sanitizer-windows first,
because it cycles quickly and doesn't do a two-stage build.

My plan is to have both the existing sanitizer-windows bot and the new
one run concurrently, but the new one will connect to the silent,
staging buildmaster (lab.llvm.org:9994). Once we are ready to cut over,
I will turn off the old sanitizer-windows VM and update this port in the
docker image and redeploy.

I am chosing to stick with the buildbot annotation script approach,
because it allows for much faster iteration: I can adjust paths for the
VS installation for example simply by committing to zorg. However, both
the live and staging bot will use the same script, so the same script
must work for both sanitizer-windows bots at the same time!

Attempt to fix Python 3 issues in annotated scripts

Put back missing strip in annotated script

Fix next Python 3 Unicode issue, the gift that will keep giving for the next 20 years

Try to fix Windows stage 2 build issue

[buildbot] Force lld for ThinLTO whole program devirtualization worker

Force use of lld, so that the configure step uses it. Otherwise we get
an error on the 3rd stage configure which will use the lld-only
--lto-whole-program-devirtualization option.

See: http://lab.llvm.org:8014/#/builders/137/builds/1921/steps/17/logs/stdio

Install Numpy in the MLIR Docker image, this is required for the python bindings

Also use pip to install pybind11 to ensure the most recent version is
available.

clang-ve-ninja: Build and use the LLVM dylib

jenkins/jobs: Checking in Clang jobs pipelines

Update branch names from master -> main

Fix Windows-on-Arm buildbots

... by disabling parts that don't build yet.

compiler-rt\lib\sanitizer_common\sanitizer_unwind_win.cpp assumes WIN64 is
x86_64, so, before that's fixed, disable everything that triggers its build.

Add a libc aarch64 worker and a debug builder running on it.

ClangBuilder: Enable fortran in test-suite

Automatically enable running the fortran parts of the test-suite if
flang is part of the checkout. In order to do this, we need to pass two
CMake defines when configuring the test-suite: one telling it to enable
fortran and one pointing it at flang. The latter picks the most recently
built flang (i.e. it chooses the one from stage 2 if there is one).

At the moment, this only affects clang-cmake-aarch64-full.

Deprecate build cache servers for aarch64 and aarch32 from Buildbot

  • Remove two builders
    • clang-aarch64-linux-build-cache
    • clang-armv7-linux-build-cache

      And underlying servers:
    • packet-linux-aarch64-slave-1
    • packet-linux-armv7-slave-1
  • These servers are not being used in a while, and the build processes provided by them overlap with other builders.

[libc] Make clang and clang++ as the explicit C and C++ compilers.

Explicitly use python3 for lit sandbox

Without this, virtualenv defaults to python2, which causes problems on
the ARM/AArch64 bots which no longer have python2 installed.

Removed lld-x86_64-freebsd builder.

Add worker sie-linux-builder and builder llvm-clang-x86_64-sie-ubuntu-fast.

jenkins: Update branch names for the repos from "master" to "main"

[lldb] Run check-lldb even if check-debuginfo failed

[lldb][monorepo] Run check-lldb even if check-debuginfo failed

Add Python binary path to CMake arguments for the clang-ppc64le-linux builder

LLVM now requires Python 3.6 which is not available in the official Ubuntu 16.04
repositories. This patch adds the path to a manually built Python 3.6 to the
CMake arguments of the clang-ppc64le-linux builder.

[zorg] Add Clang ppc64le cross-targeting buildbot on AIX

Support branches with the ForceSchedulers.

Update extraRecipients for the Windows LLDB box

[zorg] Remove redundant '\' in AIX buildbot config

[lldb] Don't run the debug info tests for the sanitized or matrix job

rdar://74356937

[jenkins] Fix TabError: inconsistent use of tabs and spaces in indentation

Support compatibility with buildbot 2.10.

This removes the deprecation warning and removes the
non-existing method addFactoryArguments (it was not doing
anything anyway for quite some time) which was breaking
the master.

Add flang out-of-tree new driver buildbot.

NFC. Some getSingleBranchSchedulers code cleaning.

Updated tags and removed redundant configure args for llvm-clang-x86_64-expensive-checks-win.

Updated requirements.

Fix android sanitizer tests for multiple devices

jenkins: More updates for "master" to "main" branch

Some builders.py cleaning.

NFC. Better names, more logs.

NFC. Changed getLntSchedulers() to follow the code style of the rest of the file.

And missing import log.

Added builders to build the release/* branches.

One GitHubStatusPush reporter is enough for our needs.

Have the buildbot email me when there is a failure on the sie-linux-builder builder.

Add flang debug new driver buildbot.

Add flang release buildbot.

Add flang release with assertions buildbot.

[zorg] Add buildbot for libcxx build on AIX

deployed new mlir-nvidia image

[clangd-ubuntu-clang] increased RAM

This fixes the out of memory issues when building with TSAN enabled.

fixed typo in comment

[clangd-ubuntu-clang] improved startup error logging

[zorg] Rename cross-targeting buildbot on AIX

Fixed misspelled builder name for e-mail notifications.

More master to main updates

jenkins/jobs: Add timeout to the lnt-ctmark-* jobs

This is so that these don't end up stuck and running forever in case of
any issues.

[sanitizer] Set default rtlib to libgcc

The default --rtlib for Android is about to change from libgcc to
compiler-rt (D96404), which will require a more substantial change to
this script to build the builtins and libunwind. For now, keep using
libgcc.

[sanitizer] Update x86 ABILIST filtering for x86_64 devices

An x86_64 device (e.g. Android emulator or Cuttlefish VM) might have an
ro.product.cpu.abilist like:

x86_64,x86,arm64-v8a,armeabi-v7a,armeabi
x86_64,arm64-v8a,x86,armeabi-v7a,armeabi

When the first arch starts with x86, filter out the remaining archs
that don't start with x86.

[sanitizer] Pass -e dotbytes=10m to wget

When wget writes to a log file, it defaults to 1KiB per dot, which
produces ~20000 lines of log output for the NDK. This "-e dotbytes"
syntax has existed since 1999 in wget.

[sanitizer,android] Don't check devices on error

If we already have a build error we should not check
if tests were executed. If device went offlind for some reasons
we usually show exception as it's infrastructure issue.

Howeve if we already see some compilation error we
don't want it hidden from commiters by exception.

[sanitizers] Fix checkout branch name

[sanitizers] Fix checkout branch name

[zorg] Use boolean flag to determine if building a unified standalone runtimes

For AIX libcxx build, we use ../llvm/runtimes as build source and with
-DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" option, since we are using
the new runtimes build laided out by the "A vision for building the runtimes"
RFC and D93408.

Enable mlir integration tests on the mlir-nvidia bot.

This is in preparation of landing https://reviews.llvm.org/D97463, which moves the cuda-runner tests to the integration test dir.

Fixed mistype in LibcxxAndAbiBuilder.py introduced by 1d2328ff69a8503911e85140b0786429204a2113.

Add builder and worker for XCore as default target.

Don't run testsuite on Linaro quick bots.

[VE] hpce-aurora2 worker replaces nec-arrproto41

Add check-flang option to some of the Flang builders.

Add Flang builder for latest gcc.

Add Flang builder for latest llvm and libc++.

[M68k] Add new worker debian-akiko-m68k for Linux 32-bit M68k

This adds a new builder called debian-akiko-m68k which will
build LLVM and Clang on Debian unstable on m68k.

Author: glaubitz

[libc++] Remove libc++ ARM builders

In accordance with the announcement on libcxx-dev and llvm-dev [1],
we are removing the libc++ specific buildbots in favor of running
those configurations in the pre-commit CI on Buildkite. This commit
removes the ARM-based libc++ build bots.

[1]: http://lists.llvm.org/pipermail/libcxx-dev/2020-November/001040.html

[libc builder] Temporarily disable integration tests and loader tests.

They will be enabled back on a new bot which will do a full libc build.

Added openmp doxygen; increased the doxygent step timeout, as it could take a long time without anything in TTY.

[XCore] Build fewer components. Run 4 parallel jobs.

Do not override LLVM_LIT_ARGS: use -j from the factory.

Fix flang out-of-tree with new driver builder.

[jenkins] Temporarily disable compiler-rt tests until couple of tests are fixed

Revert "[jenkins] Temporarily disable compiler-rt tests until couple of tests are fixed"

This reverts commit 33a9f3c59121b3ad1aad2e5e3c1e7653ac4f8ad1.

Add a mailer config for MLIR testing bots to Google build monitoring mailing-list

Added support for a new src_to_build_dir argument to LLVMBuildFactory to explicitly specify a directory to start a build for.

Added support for a new src_to_build_dir argument to UnifiedTreeBuilder to explicitly specify a directory to start a build for.

Use more efficient linker for flang debug builder.

Use lld linker for flang debug configuration.

Add Clang dependency for new Flang driver in out-of-tree builder.

When building out-of-tree we need to use CLANG_DIR to find
installed Clang components.

Respect jobs property for LIT tests in UnifiedTreeBuilder.

Remove 'release' and 'aarch64' tags from Flang builders.

[libc] Add a full build worker and two builders to run on it.

[libc] Enable full build builders.

[libc builder] Enable linting on full build builders.

[lldb] Move reproducers under the matrix bot

[lldb] Remove lldb-cmake-reproducers

The reproducer were successfully integrated in the matrix bot.

http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-matrix/

Update python deps for buildbot-mlir-nvidia.

Remove Cmake compiler arguments.

This way cmake will pick /usr/local/bin/cc or c++, which we
point to the appropriate compiler.

[lldb] Mark cmake build as unstable when one or more tests fail

Revert "[lldb] Mark cmake build as unstable when one or more tests fail"

This reverts commit 557b1768a04da0d7977cb861fc4c57198d468a78.

[lldb] Remove LLDB_CAPTURE_REPRODUCER export

[VE] use local cache

There is a need to allow SanitizerBuilder to accept extra args so cmake args can be passed in, in turn allowing us to specify -j with cmake.

By default all threads available on the machines are being used by the Buildbot. This patch limits the use of threads to 256.

[zorg] Adjust libcxx buildbot config on AIX

Enable libcxx project only

Enable build_standalone in LibcxxAndAbiBuilder

Unconditionally pre-define CMAKE_ARGS in sanitizer bot.

Fixes the following bug created by
110a39811261ab69894fe77a69813f08a8c1f1a9:

buildbot_cmake.sh: line 38: CMAKE_ARGS: unbound variable

Revert "Respect jobs property for LIT tests in UnifiedTreeBuilder."

This shall be properly handled in LitTestCommand instead.
Similar to how the jobs are handled for NinjaCommand, for example.

This reverts commit 13c3c22377e39033ff843659a8b49be278a20086.

Forgot to declare CMAKE_ARGS ahead of time to prevent unboud variable errors; I noticed a ccache: command not found error in the process and this fixes that too by checking the utility is available before running it.

Revert "Forgot to declare CMAKE_ARGS ahead of time to prevent unboud variable errors;"

This reverts commit 423a80226a9fdc789dc2ef18c4cefdfdf53d5b76.

[PowerPC] Allow the PowerPC RHEL buildbot to use cmake with test-suite

We would like to update the PowerPC RHEL buildbot (clang-ppc64le-rhel) to use
cmake instead of LNT, so that we can streamline our process whenever we update
CMakeList.txt.

[ve] fix cmake cache param

[Zorg] Add space between script and args

Add a space so that the strings are not concatenated incorrectly.

[Zorg] Change make to ninja instead for Clang RHEL buildbot

Changing clang builder's cmake_test_suite to use ninja instead
as the only buildbot that uses it builds and runs with ninja.

Deployed new mlir-nvidia

Revert "[Zorg] Change make to ninja instead for Clang RHEL buildbot"

This reverts commit f785f4216432aa0d249ee3ad1f86515caf4c7b10.

Revert "[PowerPC] Allow the PowerPC RHEL buildbot to use cmake with test-suite"

This reverts commit 4b8974b63dcc1c5eab28a6ec33328b30e63e396b.

Remove 3 Flang workers/builders

The following Flang workers/builders are removed:

  • flang-aarch64-ubuntu
  • flang-aarch64-ubuntu-clang
  • flang-aarch64-ubuntu-gcc10

The corresponding configurations are covered by other, recently added,
Flang workers/builders from Linaro:

  • flang-aarch64-latest-gcc
  • flang-aarch64-ubuntu-latest-clang
  • flang-aarch64-ubuntu-sharedlibs
  • flang-aarch64-ubuntu-release
  • flang-aarch64-ubuntu-rel-assert
  • flang-aarch64-ubuntu-dylib
  • flang-aarch64-ubuntu-debug

When compared to the workers/builders deleted here, the new Flang
workers from Linaro provide much better:

  • granularity in terms of configurations
  • build times

[zorg] Support other relative scripts in AnnotatedBuilder

AnnotatedBuilder should be able to launch both python and bash
scripts. It should launch with python for scripts that specify
the script_interpreter as python, or allow the user to specify
another interpreter. Otherwise it will execute an empty
script_interpreter script directly. Also, move the --jobs option
to be the default of extra_args, otherwise allow the user to
override extra_args.

[zorg] Add buildbot for HIP

Build HIP tests from llvm-test-suite using AMDGPU.
Will be extended to build external repo.

[sanitizer] Add scudo tests on QEMU

[sanitizer] Improve naming of bot steps

[sanitizer] Fix typo in bot script

[sanitizer] Enabled QEMU bot failures

[jenkins] Add groovy script for the llvm-cov job

[sanitizer] Cleanup build dirs hierarchy

[sanitizer] Run scudo on qemu-armhf

Adjust jobs for aix-ppc64-ppc64le worker

since the machine now has additional resources.

[zorg] Add buildbot for Synopsys ARC

Create AWS worker for LLVM Experimental Target : ARC

[zorg] Build and test project depends on project enabled in LibcxxAndAbiBuilder

Build and test project depends on project enabled in LibcxxAndAbiBuilder;
Let all steps respect env passed in;

Fixed bug introduced by D100497 - local variable jobs_flag referenced before assignment.

[zorg] Fix AnnotatedBuilder.py extra_args bug

Do not assign WithProperties("--jobs=%(jobs:-)s") to
extra_args, because extra_args_with_props will perform
WithProperties(...) on all of the args in extra_args.

Bumped urllib3 from 1.26.3 to 1.26.4.

Added enable_runtimes and enable_projects to LLVMBuildFactory.

Added support for LLVM_ENABLE_RUNTIMES to UnifiedTreeBuilder.

Documentation builds do not support LLVM_ENABLE_RUNTIMES.

Added support for LLVM_ENABLE_RUNTIMES to ClangLTOBuilder.

Changed LibcxxAndAbiBuilder to use LLVMBuildFactory enable_projects and enable_runtimes.

Revert "Support compatibility with buildbot 2.10."

Temporarily revert commit 3f44862c69dd1766b51ea408d2cba6ec5c76c30a, as we are still with 2.8.4 in the production.

Will be re-applied after migration to 2.10 or later.

Added "auto" for LLVMBuildFactory enable_runtimes arg; changed default to not use enable_runtimes.

[Zorg][OpenMP] Add CUDA offloading worker.

This worker tests OpenMP offloading for the x86_64 and NVIDIA GPU. In
addition to check-openmp, it runs the SOLLVE Validation & Verification Suite
via the LLVM test-suite External builder. The builder is configured to
only warn if the SOLLVE suite fails, as it also tests features that
have not been implemented in Clang yet.

CUDA is intentionally not installed in a default location (/opt/cuda) to
resemble setups often found in computing clusters with multiple versions
of CUDA to choose from.

[PowerPC] Change RHEL buildbot to unifiedtreebuilder

The patch switches the RHEL buildbot to utilise more jobs when building, as
well as UnifiedTreeBuilder (ie, with cmake and Ninja).

Differential: https://reviews.llvm.org/D101199

Remove -DFLANG_BUILD_NEW_DRIVER=ON from builders. NFCI

FLANG_BUILD_NEW_DRIVER defaults to ON now, so there's no need to add it
anymore.

jenkins/tasks: virtualenv: Use the Python3 interpreter

[sanitizer] Remove disabled line

[sanitizer] Fix COMPILER_RT_DEBUG assignment

Added missing comma in clang-ppc64le-rhel builder configuration.

[sanitizer][NFC] Reuse cmake setup

[sanitizer] Fix asan builder name

[sanitizer] Remove unused GN reference

[sanitizer] Add dedicated QEMU builder

[scudo] Test on mips and ppc QEMU

[sanitizer] Fix "set -e in subshell" in QEMU checkout

[scudo] Remove HWCAP2_MTE workaround

It's not needed after https://patches.linaro.org/patch/433102/

[scudo] Fix "set -e in subsell" issue

[scudo] Combine cmake/ninja steps

[scudo] Print executed tests

[scudo] Install cross compiled libs into STAGE1

We need this for LIT future lit tests.

Update configuration for Arm/Aarch64 cross Windows to Linux toolchan builders.

Those builders:

  • llvm-clang-win-x-armv7l
  • llvm-clang-win-x-aarch64

Run only 1 build at a time for heavily loaded workers.

Fixed undefined runtime_projects in OpenMPBuilder.

[sanitizer] Prepare for D102543

Mips requires special handling

[sanitizer] Remove QEMU from sanitizer-buildbot3

[libfuzzer] Print running tests

[libfuzzer] Remove unnecessary cmake flags

Linking job limit is unnecessary with LLD.
LIBFUZZER_ENABLE_TESTS does not exist any more.

[sanitizer] Run QEMU cmakes in parallel

[sanitizer] Remove nested buildbot_qemu call

Now it has own builder.

[PowerPC] Removed -DLLVM_ENABLE_PROJECTS=flang from PowerPC flang buildbot

With FLANG_NEW_DRIVER default to on now, it is necessary to remove this
option for the bot to be able to build again.

[clangd] Update gRPC dependency for remote index tests

Context https://github.com/clangd/clangd/pull/783

Limit the number of parallel link jobs for the full LTO builder. With the larger number of link jobs the build time grows significantly, and with the default value it randomly runs out of memory.

Limit the number of parallel link jobs for the full LTO rerlease builder.

[MLIR] Update MLIR build config to reflect cmake variable renames

Update CMake variables to reflect the migration for a consistent naming:
https://reviews.llvm.org/D102976

Additional discussion at:
https://llvm.discourse.group/t/mlir-cmake-enable-variable-naming-convention/3520

[zorg] Remove flang-aarch64-ubuntu-out-of-tree-new-driver

The new driver is on by default now, so this builder does the same thing
as flang-aarch64-ubuntu-out-of-tree.

Removed abandoned lldb-sphinx-docs builder.

[sanitizer] Switch to patched QEMU

[sanitizer] Fix QEMU checkout

[sanitizer] Test HWASan + LAM via QEMU.

Uses LAM-enabled full system QEMU emulation to test HWASan's LAM
support.

Depends on:
https://github.com/google/sanitizers/pull/1408
https://github.com/google/sanitizers/pull/1410

[buildbot-clangd-ubuntu-clang] deployed new docker image

This is just the deployment for the container updated in
cf3881f300f973e6dcc12d2c25dff626290642aa

[MLIR] Enable tensorcore integration tests

[VE] disable cmake caches until resolved

Enable retry on failed git operations

This should address issues like
fatal: unable to access 'https://github.com/llvm/llvm-project.git/': Could not resolve host: github.com
in
http://lab.llvm.org:8011/#/builders/131/builds/7

llvmbisect: Add support for monorepo artifacts

This is now compatible with both svn and git/monorepo artifacts, and
also works across them.

[libc] Add a "libc-fuzzer" step in full build mode.

[zorg] Cleanup clang-cmake-{armv7,aarch64}-quick bots

Run "ninja check" for armv7 bot and don't pretend we run
LNT testsuite for the aarch64 bot.

[zorg] Build all targets on Linaro-maintained bots

... not just AArch64 and ARM. Make an exception for the quick
bots, which build a single target.

[zorg] Add AArch64 SVE vector-length-specific (VLS) bots

We add 1-stage bot that tests SVE support by building/running testsuite,
and 2-stage bot that additionally builds 2nd stage compiler with SVE.

[zorg] Rename Linaro's AArch64 and ARM bots to use consistent names

This will simplify Linaro's buildbot maintenance scripts. We also
remove now-unused AArch64 and ARM libcxx workers, since these have
been migrated to buildkite a few months ago.

[zorg] Remove settings that match defaults from Linaro bots

Remove from arguments of ClangBuilder.getClangCMakeBuildFactory

  • checkout_compiler_rt=True
  • checkout_lld=True
  • test=True
  • useTwoStage=False
  • testStage1=True

All these are defaults anyway.

[zorg] Rename Linaro-maintained 2-stage bots

... to have consistent -2stage suffix.

Update compiler on hexagon-build-0x machines

Add flang-x86_64-windows builder.

Add a buildbot to compile flang using msvc. Due to msvc doing some things differently than other compilers, the msvc flang build breaks regularly. This builder will notify committers if their change causes the build to break.

However, the tests never completed successfully on Windows, partially because many tests require a UNIX shell (see D89368). To not constantly fail, tests failures only trigger a warning, until all tests are fixed as well.

Currently running here: http://meinersbur.de:8011/#/builders/146

[M68k] Add new worker suse-gary-m68k-cross for Linux 32-bit M68k

This adds a new builder called clang-m68k-linux-cross which will
cross-build LLVM and Clang on openSUSE Leap.

[libc++] Remove build bots that are handled on BuildKite now

Those builders have been failing for months because they use a version
of Clang that we don't actively support anymore. Furthermore, all those
configurations (single-threaded, no exceptions, with LLVM's unwinder)
are being tested on BuildKite.

At this point, I think those builders are only adding noise and using
resources that could be repurposed to other tasks, so I think it is
reasonable to remove them.

[lldb] Actually let the matrix bot test something

Since https://reviews.llvm.org/D69401 landed the bot has been setting unused
CMake variables for specifying test compilers which meant this bot hasn't
been testing anything since 2019. Reenabling the tests so that we can start
fixing the failing tests.

[NFC] Reformat sanitizer workers

Reassign sanitizer builders/workers

Less builds for sanitizer-buildbot{2,8}

[sanitizer] Simplify clobbering

[sanitizer] Cleanup after build

[sanitizer] Fix compiler_rt_build cleanup

[sanitizer] Add QEMU cleanup

[NFC][sanitizer] Add newline at the end

Disable GPU in polly-arm-linux and aosp-O3-polly-before-vectorizer-unprofitable

[AIX][PowerPC] Reduce number of parallel link jobs for AIX crosstargetting buildbot

Reduce the parallel link jobs so that we reduce machine hangs and timeouts.

[sanitizers] Remove unset variables

It looks nothing else uses them.

[zorg] Reduce # targets built on Linaro flang bots

The flang bots are only running check-flang and are not really
exercising the backend in any way. At the moment most of them
are taking about 1h to run, and we should be able to reduce that
a bit by disabling targets other than the host (AArch64) on them.

[zorg] Hotfix for PPC64le MLIR build issue

The public builder for MLIR on Power currently has an
issue choosing the correct libc. This patch provides
a environment variable to guide CMake to the correct
one.

Move QEMU LAM sanitizer images to env-defined folder.

Also add some nice debug info as to how to build the images yourself.
This also makes the bot selector define the build dir and the QEMU image
dir, rather than these being hardcoded on the script side (which non-bot
users will want to execute to repro the bot).

[jenkins] Update script to use cross project lit test suite

To reflect changes announced in
https://lists.llvm.org/pipermail/llvm-dev/2021-January/148048.html,
this patch updates all occurrences of debuginfo-tests to
cross-project-debuginfo-tests.

Add chown to buildbot_qemu local repro instructions.

[sanitizer] Fix mips QEMU build

[scudo] Disable mips 32bit tests

It times out for unknown reasons.

[sanitizer] Don't tie builders with particular workers

This simplifies management of VMs. We don't need to care which
are online. If some is gone we will just see less frequent builds.

[sanitizer] Missing image is infra issues

This avoids spamming blame list

[sanitizer] Set EXE_LINKER_FLAGS same as SHARED_LINKER_FLAGS

Add cross-project-tests to SIE Linux builder.

Enable LLD tests together with the expensive checks on the expensive check builders.

nclude LLD project to build and run the tests together with LLVM expensive checks on all
appropriate builders:

llvm-clang-x86_64-expensive-checks-ubuntu
llvm-clang-x86_64-expensive-checks-win
llvm-clang-x86_64-expensive-checks-debian
llvm-clang-x86_64-expensive-checks-ubuntu-release
llvm-clang-x86_64-expensive-checks-win-release
llvm-clang-x86_64-expensive-checks-debian-release

The LLD test use the expensive check data structures and it would be good to detect
the failures when they configured with -DLLVM_ENABLE_EXPENSIVE_CHECKS=ON (ref: D105071)

Revert "[sanitizer] Don't tie builders with particular workers"

This reverts commit d37259ec73a4341700e981214b9032631adfdda0.
With some changes.

[zorg] Add HIP builder script

Simple script for HIP builder to build llvm-project incrementally,
then build and execute HIP tests from llvm-test-suite.

[ORC-RT] Disable ORC Runtime on ccache builders.

The ORC runtime contains assembly files that don't seem to build correctly
under ccache (e.g. https://lab.llvm.org/buildbot/#/builders/109/builds/18980).
This commit disables the ORC runtime on those builders, which seems to be the
same fix that was adopted for XRay and other compiler-rt libraries that use
assembly.

Remove aarch64-sve-vector-bits-min compile flag from SVE 2 stage bot

Our build is failing:
https://lab.llvm.org/buildbot/#/builders/176/builds/205

Due to a vectorisation issue being investigated as
https://bugs.llvm.org/show_bug.cgi?id=51182.

Remove the bits-min flag until this bug is fixed so we at least
get coverage of the 2nd stage build with SVE codegen.

jenkins: cmake_builder and clang_builder: Ignore error in the test stage

This allows us to proceed to generating the test result report.

jenkins: Stop uploading the compiler build artifacts to cloud storage

The storage space has been decommissioned.

https://lists.llvm.org/pipermail/llvm-dev/2021-July/151623.html

jenkins: Fix the file path pattern for the test results XML files

Include all the available test results files in the build directory.

Add missing closing quote to SVE 2 stage cmake options

Missing from 24c07902d058abcee3bcf908676bacaae1f3d448.

[Zorg] Use ccache instead incremental build for openmp-offload-cuda-runtime.

Building with LLVM_ENABLE_RUNTIMES=openmp has the disadvantage that if clang changes, the runtime that was built with the previous clang is not rebuilt. That is, if the runtime is miscompiling due to a change in clang, this will only be detected in the next clean build.

For instance, commit rG1100e4aafea233bc8bbc307c5758a7d287ad3bae caused the libomptarget device runtime to miscompile, but the openmp-offload-cuda-runtime builder shows it as green. Tests only started failing with the next clean build. In production, this would have blamed the wrong commit.

In contrast, the openmp-offload-cuda-project builder started failing with the expected commit.

Instead of building incrementally, use ccache to avoid this problem.

Revert "Remove aarch64-sve-vector-bits-min compile flag from SVE 2 stage bot"

This reverts commit 24c07902d058abcee3bcf908676bacaae1f3d448.

The issue is still present but given that it's not sending out any
new notifications I'm going to leave it as is for now.

It actively building is somewhat useful for those working on
the issue to know what the current status is.

Update config for aosp-O3-polly-before-vectorizer-unprofitable

The bot has moved to a new host with a newer AOSP codebase. Update the
buildbot config to match.

Change-Id: I474f4503078acdf4109ce755228379ae81a39be7

[sanitizer] Run two builds at the time

[NFC][sanitizer] Remove misleading comment

[sanitizer] Move cpplint to the end of the build

[sanitizers] Don't run cpplint

The script was removed.

[LLDB] Set lit arg -j4 for Arm/AArch64 Linux buildbots

We have started seeing some sporadic test failures on LLDB Arm/AArch64
Linux buildbots. This patch reduces no of parallel tests to 4 so
to provide some bandwidth for the timing critical and load dependent
test cases.

removing windows deployment

It's been unmaintained for a while and it's unclear
if we need a 32bit setup at all

Add a libc x86_64 windows worker and a debug builder running on it

Add buildbot for OpenMP on AMDGPU

Build OpenMP for AMDGPU target

Added jobs param for hip-vega20-0 worker, as getOpenMPCMakeBuildFactory cannot handle optional params.

Reverted 8ff202c1648d4dba1803aa0cade81790a8382c6c. Added jobs param for the right worker - omp-vega20-0, as getOpenMPCMakeBuildFactory cannot handle optional params.

Update openmp-offload-amdgpu builders

Disable SOLLVE testing for time being.
Clean build for openmp-offload-amdgpu-runtime.
Remove lld from depends_on_projects.
Remove LLD cmake options.

Correct openmp-offload-builders lld, openmp

Remove obsolete libc++ release builder

We should be using BuildKite instead. That build bot is broken now
anyways because it is using an unsupported compiler.

[HIP] use bootstrap to build libc++

Recent libc++ changes request clang-11 or gcc-11 to build.

Switch to a bootstrap build step to use built clang to build libc++.

Add Windows equivalent to libc-linux.py

Create an annotated Windows builder for libc buildbot.

openmp-offload-amdgpu-runtime: Disable testsuite for runtime. set ccace for runtime

disable llvm-test in openmp-offload-amdgpu-runtime
add ccache options for C and CXX

[libc] Get and set vcvars before running the steps.

Also added the CMake flag to pick up MPFR.

[libc | windows] Use the items method to iterate over dict items.

Enable flang examples on a few builders

flang-aarch64-rel-assert (which uses static libs) and
flang-aarch64-sharedlibs (which should be relevant for testing the
plugin support).

[sanitizer] Fix TMPDIR for older Android versions

[sanitizer] Move QEMU image check

[sanitizer] Move QEMU LAM steps

[sanitizer] Move more QEMU LAM steps

[sanitizer] Fix configure_hwasan_lam

[sanitizer] Name "configure hwasan" step

[sanitizer] Revert my Recent QEMU LAM changes

This reverts commit 2b6b16cbc17f5dadc621c9e96082df768be792d6.
This reverts commit b37ed8bbbce7eea797d5f684803f30d7b8532b75.
This reverts commit 75c452375a04c133c82407ecf139934efb4dd23c.
This reverts commit 71d17a80d274ffd4db8f15521d564a18c8e5c1ed.
This reverts commit e257cd977be51b29c9c3c650de4fe300a99c68f2.

[sanitizer] Delay LAM Prerequisites check

Image setup may be finalizing in background

Replace new-pass-manager-fast builder with llvm-dylib-all

Test a LLVM build with the dynamic library enabled. Remove the
clang-x86_64-debian-new-pass-manager-fast builder since the new pass
manager has been the default since Feb 21.

[sanitizer] Update Android bot for D108494

[sanitizer] Fix the previous fix for D108494

openmp-offload-amdgpu-project - disable test suite, since no SOLLVE is run

add libc scudo test to build config

move compiler-rt add before join

disable scudo integration if asan is enabled

Updated configuration for Arm/Aarch64 cross Windows to Linux toolchan release builders.

Find libc++ in the per-target runtime directory

Fixes build after "[CMake] Enable LLVM_ENABLE_PER_TARGET_RUNTIME_DIR by
default on Linux.

Use -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF

On sanitizer bots, revert to all runtime libraries in a common dir for
now.

Add openmp as dependent project , so as to trigger buildbots on openmp commits

Add openmp to dependents for openmp-runtime

add check-tsan-shared

add check-tsan-shared to compiler-rt

zorg/jenkins/jobs/jobs/lldb-cmake-matrix