[PATCH 5/6] [zorg] Add linaro-armv8-01-arm-* slaves

[maxim-kuvyrkov]

Summary

We are going to migrate most of AArch32 builders from armv7
TK1 hardware to armv8 boxes. Since the architecture is slightly
different (armv7 vs armv8) we shall take care to not connect both
armv7 and armv8 for the same builder simultaneously.
We plan to allocate freed up TK1 boards to performance and full bots.

[rovka]

buildbot/osuosl/master/config/builders.py

226 ↗

(On Diff #129781)

If we're not going to connect both of these at the same time, then why don't we remove the TK1 from the slave list and only commit this when the armv8 slaves are ready to take over? Then we can safely disconnect the TK1s any time afterwards.

235 ↗

(On Diff #129781)

Should this still be cortex-a15, or should we be using a different combination of flags?

[maxim-kuvyrkov]

buildbot/osuosl/master/config/builders.py

226 ↗

(On Diff #129781)

We are not going to connect these at the same time *for the same builder*. I plan to de-commission TK1s one-by-one, one builder at a time. I.e.,

1. connect linaro-armv8-01-arm-lnt to silent master, linaro-tk1-03 is connected to main master;
2. confirm that linaro-armv8-01-arm-lnt works OK on silent master;
3. connect linaro-armv8-01-arm-lnt to main master, connect linaro-tk1-03 to silent master;
4. confirm that linaro-armv8-01-arm-lnt works OK on main master;
5. disconnect linaro-tk1-03 from silent master and repurpose hardware for something else.

235 ↗

(On Diff #129781)

I think it should remain cortex-a15 for now. The question is which codegen path should be explicitly exercised? Since cortex-a15 is a popular core, imo, we should leave this as is.

Once migration is finished and we can increase number of configurations that we can test, we should add builders for other popular cores -- as long as they are compatible with hardware underneath.

[rovka]

Ok, LGTM.

zorg/trunk/buildbot/osuosl/master/config/builders.py

return [
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mthumb'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"],
                       submitURL='http://lnt.llvm.org/submitRun',
                       testerName='LNT-Thumb2v7-A15-O3')},
 
         # Cortex-A15 LNT test-suite in test-only mode
         {'name' : "clang-native-arm-lnt",
-         'slavenames':["linaro-tk1-03"],
+         'slavenames':["linaro-tk1-03", "linaro-armv8-01-arm-lnt"],
          'builddir':"clang-native-arm-lnt",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       test=False,
                       useTwoStage=False,
                       runTestSuite=True,
                       testsuite_flags=['--cppflags', '-mcpu=cortex-a15 -marm',
                                        '--threads=4', '--build-threads=4'],
                       extra_cmake_args=["-DCMAKE_C_COMPILER_LAUNCHER='ccache'",
                                         "-DCMAKE_CXX_COMPILER_LAUNCHER='ccache'",
                                         "-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -marm'",
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -marm'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## Cortex-A15 check-all self-host NEON with CMake builder
         {'name': "clang-cmake-armv7-a15-selfhost-neon",
-         'slavenames':["linaro-tk1-04"],
+         'slavenames':["linaro-tk1-04", "linaro-armv8-01-arm-selfhost-neon"],
          'builddir':"clang-cmake-armv7-a15-selfhost-neon",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       useTwoStage=True,
                       testStage1=False,
                       extra_cmake_args=["-DCMAKE_C_COMPILER_LAUNCHER='ccache'",
                                         "-DCMAKE_CXX_COMPILER_LAUNCHER='ccache'",
                                         "-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -marm'",
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -marm'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
                                         "-DLLVM_LIT_ARGS='-sv -j2'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## Cortex-A15 check-all with CMake builder
         {'name': "clang-cmake-armv7-a15",
-         'slavenames':["linaro-tk1-06"],
+         'slavenames':["linaro-tk1-06", "linaro-armv8-01-arm-quick"],
          'builddir':"clang-cmake-armv7-a15",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       extra_cmake_args=["-DCMAKE_C_COMPILER_LAUNCHER='ccache'",
                                         "-DCMAKE_CXX_COMPILER_LAUNCHER='ccache'",
                                         "-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
                                         "-DLLVM_LIT_ARGS='-sv -j4'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## Cortex-A15 check-all self-host with CMake builder
         {'name': "clang-cmake-armv7-a15-selfhost",
-         'slavenames':["linaro-tk1-07"],
+         'slavenames':["linaro-tk1-07", "linaro-armv8-01-arm-selfhost"],
          'builddir':"clang-cmake-armv7-a15-selfhost",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       useTwoStage=True,
                       testStage1=False,
                       extra_cmake_args=["-DCMAKE_C_COMPILER_LAUNCHER='ccache'",
def _get_sanitizer_builders():
           {'name': "sanitizer-windows",
            'slavenames' :["sanitizer-windows"],
            'builddir': "sanitizer-windows",
            'factory': SanitizerBuilderWindows.getSanitizerWindowsBuildFactory(
                         vs='%VS140COMNTOOLS%')},
 
           ## Cortex-A15 check-all full (compiler-rt) with CMake builder; Needs x86 for ASAN tests
           {'name': "clang-cmake-armv7-a15-full",
-           'slavenames':["linaro-tk1-08", "linaro-tk1-09"],
+           'slavenames':["linaro-tk1-08", "linaro-tk1-09", "linaro-armv8-01-arm-full"],
            'builddir':"clang-cmake-armv7-a15-full",
            'factory' : ClangBuilder.getClangCMakeBuildFactory(
                         clean=False,
                         checkout_lld=False,
                         extra_cmake_args=["-DCMAKE_C_COMPILER_LAUNCHER='ccache'",
                                           "-DCMAKE_CXX_COMPILER_LAUNCHER='ccache'",
                                           "-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                           "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                           "-DCOMPILER_RT_TEST_COMPILER_CFLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                           "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
                                           "-DLLVM_LIT_ARGS='-sv -j4'",
                                           "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
           ## Cortex-A15 Thumb2 check-all full (compiler-rt) with CMake builder; Needs x86 for ASAN tests
           {'name': "clang-cmake-thumbv7-a15-full-sh",
-           'slavenames':["linaro-tk1-05"],
+           'slavenames':["linaro-tk1-05", "linaro-armv8-01-arm-full-selfhost"],
            'builddir':"clang-cmake-thumbv7-a15-full-sh",
            'factory' : ClangBuilder.getClangCMakeBuildFactory(
                         clean=False,
                         checkout_lld=False,
                         useTwoStage=True,
                         testStage1=False,
                         extra_cmake_args=["-DCMAKE_C_COMPILER_LAUNCHER='ccache'",
                                           "-DCMAKE_CXX_COMPILER_LAUNCHER='ccache'",
return [
          'builddir' : 'libcxx-libcxxabi-x86_64-linux-ubuntu-gcc-tot-latest-std',
          'factory': LibcxxAndAbiBuilder.getLibcxxAndAbiBuilder(
             env={'PATH': '/usr/local/bin:/usr/bin:/bin',
                  'CC': '/opt/gcc-tot/bin/gcc', 'CXX': '/opt/gcc-tot/bin/g++'}),
         'category': 'libcxx'},
 
         # Cortex-A15 LibC++ and LibC++abi tests (require Clang+RT)
         {'name': 'libcxx-libcxxabi-libunwind-arm-linux',
-         'slavenames': ['linaro-tk1-01'],
+         'slavenames': ['linaro-tk1-01', 'linaro-armv8-01-arm-libcxx'],
          'builddir': 'libcxx-libcxxabi-libunwind-arm-linux',
          'category': 'libcxx',
          'factory': LibcxxAndAbiBuilder.getLibcxxAndAbiBuilder(
             cmake_extra_opts={'CMAKE_C_COMPILER_LAUNCHER': 'ccache',
                               'CMAKE_CXX_COMPILER_LAUNCHER': 'ccache',
                               'CMAKE_C_COMPILER': 'clang',
                               'CMAKE_CXX_COMPILER': 'clang++',
                               'LIBCXXABI_USE_LLVM_UNWINDER': 'ON',
                               'CMAKE_C_FLAGS': '-mcpu=cortex-a15 -marm',
                               'CMAKE_CXX_FLAGS': '-mcpu=cortex-a15 -marm',
                               'LLVM_PARALLEL_LINK_JOBS': '2'})},
 
         {'name': 'libcxx-libcxxabi-libunwind-arm-linux-noexceptions',
-         'slavenames': ['linaro-tk1-01'],
+         'slavenames': ['linaro-tk1-01', 'linaro-armv8-01-arm-libcxx-noeh'],
          'builddir': 'libcxx-libcxxabi-libunwind-arm-linux-noexceptions',
          'category': 'libcxx',
          'factory': LibcxxAndAbiBuilder.getLibcxxAndAbiBuilder(
             cmake_extra_opts={'CMAKE_C_COMPILER_LAUNCHER': 'ccache',
                               'CMAKE_CXX_COMPILER_LAUNCHER': 'ccache',
                               'CMAKE_C_COMPILER': 'clang',
                               'CMAKE_CXX_COMPILER': 'clang++',
                               'LIBCXXABI_USE_LLVM_UNWINDER': 'ON',

zorg/trunk/buildbot/osuosl/master/config/slaves.py

return [
         create_slave("linaro-tk1-02", properties={'jobs' : 4}, max_builds=1),
         create_slave("linaro-tk1-03", properties={'jobs' : 4}, max_builds=1),
         create_slave("linaro-tk1-04", properties={'jobs' : 4}, max_builds=1),
         create_slave("linaro-tk1-05", properties={'jobs' : 4}, max_builds=1),
         create_slave("linaro-tk1-06", properties={'jobs' : 4}, max_builds=1),
         create_slave("linaro-tk1-07", properties={'jobs' : 4}, max_builds=1),
         create_slave("linaro-tk1-08", properties={'jobs' : 4}, max_builds=1),
         create_slave("linaro-tk1-09", properties={'jobs' : 4}, max_builds=1),
+        create_slave("linaro-armv8-01-arm-lnt", properties={'jobs' : 64}, max_builds=1),
+        create_slave("linaro-armv8-01-arm-selfhost", properties={'jobs' : 64}, max_builds=1),
+        create_slave("linaro-armv8-01-arm-selfhost-neon", properties={'jobs' : 64}, max_builds=1),
+        create_slave("linaro-armv8-01-arm-quick", properties={'jobs' : 64}, max_builds=1),
+        create_slave("linaro-armv8-01-arm-full", properties={'jobs' : 64}, max_builds=1),
+        create_slave("linaro-armv8-01-arm-full-selfhost", properties={'jobs' : 64}, max_builds=1),
+        create_slave("linaro-armv8-01-arm-libcxx", properties={'jobs' : 64}, max_builds=1),
+        create_slave("linaro-armv8-01-arm-libcxx-noeh", properties={'jobs' : 64}, max_builds=1),
 
         # AArch64 Linaro slaves
         create_slave("linaro-apm-01", properties={'jobs' : 8}, max_builds=1),
         create_slave("linaro-apm-02", properties={'jobs' : 8}, max_builds=1),
         create_slave("linaro-apm-03", properties={'jobs' : 8}, max_builds=1),
         create_slave("linaro-apm-04", properties={'jobs' : 8}, max_builds=1),
         create_slave("linaro-apm-05", properties={'jobs' : 8}, max_builds=1),
         create_slave("linaro-apm-06", properties={'jobs' : 8}, max_builds=1),