[zorg] Cleanup unnecessary options for ARM and AArch64 bots

[maxim-kuvyrkov]

Summary

"jobs=" settings are inheritted from slave configuration.

"-DCMAKE_C_FLAGS=" and "-DCMAKE_CXX_FLAGS=" for single-stage builds
are passed only to system compiler and don't affect behavior or results.
For two-stage bots (useTwoStage=True) these flags are preserved.

"-DLLVM_LIT_ARGS='-sv -j2'" flags are extraneous, testsuite runs fine
with default parallelism. We preserve "-DLLVM_PARALLEL_LINK_JOBS=2" for
linaro-tk1-* bots, which have only 2GB of RAM.

[rengolin]

buildbot/osuosl/master/config/builders.py

248

You want to keep the -sv as that should very important information about the tests that failed.

287

You're not changing it for LNT, better to do that for both.

I'd also try mcup=native instead...

326

@rovka is this still necessary? Isn't that the default for O0?

Also, mcpu changes. All or nothing.

998

Are the others being set correctly? They weren't when I did this.

1290

why disabling marm/mthumb here?

[rovka]

buildbot/osuosl/master/config/builders.py

326

Nope, we should only need -O0 now.

[maxim-kuvyrkov]

buildbot/osuosl/master/config/builders.py

248

"-sv" are default LLVM_LIT_ARGS set in CMakeLists.txt.

287

You're not changing it for LNT, better to do that for both.

What do you mean? There is no aarch64 LNT bot [yet].

I'd also try mcup=native instead...

Yeah, I plan to switch bots (or add extra variants of bots) for -mcpu=native. That's gonna be a future change.

326

Will fix.

998

I don't follow:
"others" == ???
"They" == ???
Please elaborate :-)

1290

This is a single-stage build, so CMAKE_C_FLAGS and CMAKE_CXX_FLAGS are passed only to host compiler. AFAICT, they do not affect results of libcxx testsuites.

[rengolin]

All of the thumb/arm changes I think shouldn't be removed. The main thing is that we can know that stage 1 clang can be compiled as arm/thumb and it runs on all the tests without failure.

buildbot/osuosl/master/config/builders.py

248

Ah,must be a new thing, then. :)

287

I meant testsuite (using LNT), if you look at the line above you'll see what I mean. :)

998

Well, you're deleting C_FLAGS & CXX_FLAGS which didn't use to be set from COMPILER_RT_FLAGS. Unless they are, this will make Clang not compile with ARM/Thumb + VFP, which is important to make sure Clang works on both Arm and Thumb when running the tests. The VFP test is a bit bogus because our boards have NEON...

1290

The host compiler will compile libcxx with arm/thumb and therefore generate different library code, no? Same for test code.

[maxim-kuvyrkov]

buildbot/osuosl/master/config/builders.py

287

My logic is as follows, let me know if there are mistakes in it:
<cut>
The intention of this bot is to run testsuite on a code path for a popular CPU -- cortex-a57. To achieve this we do a single-stage build of llvm with system compiler (we don't care what tuning system compiler uses as long as ISA is compatible with current CPU), and then build testsuite with stage1 llvm. We care about exercising -mcpu=cortex-a57 code path in llvm, so we add these flags to testsuite.
</cut>

If the above is correct then this change is NFC, which is what I intended it to be.

Now, regarding setting -mcpu=, -mthumb/-marm and other architecture options.
It appears that a cleaner solution would be setting LLVM_TARGET_TRIPLE and/or other configuration options at cmake stage. Make LLVM generate cortex-a57 (or cortex-a15, ...) code by default and stop bothering about flags. WDYT?

998

Same logic (and, same assumptions) about CMAKE_C_FLAGS / CMAKE_CXX_FLAGS apply here.

1290

Yes and no. Library and test code will be different bit-wise, but (unless there are "#ifdef arm / #ifdef thumb" !) should be semantically equivalent.
If there are indeed different code paths for ARM / Thumb modes or other architectural features, then, "yes", we should differentiate here.

[rengolin]

buildbot/osuosl/master/config/builders.py

287

Right, this is the test-suite, your assumptions are correct.

1290

I don't know, but I wouldn't assume so. I think we can leave that change for a different patch, after asking the libc++ folks.

[maxim-kuvyrkov]

I'll amend libcxx and global-isel builders per comments in the final commit. Changes for all other bots seem to be OK.

buildbot/osuosl/master/config/builders.py

1290

OK, sounds good.

buildbot/osuosl/master/config/builders.py

return [
                        extra_cmake_args=['-DLLVM_ENABLE_ASSERTIONS=ON',
                                          '-DLLVM_USE_INTEL_JITEVENTS=TRUE'])},
 
         # Cortex-A15 LNT test-suite in Benchmark mode
         {'name' : "clang-native-arm-lnt-perf",
          'slavenames':["linaro-tk1-02"],
          'builddir':"clang-native-arm-lnt-perf",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
-                      jobs=4,
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       test=False,
                       useTwoStage=False,
                       runTestSuite=True,
                       testsuite_flags=['--cppflags', '-O3 -mcpu=cortex-a15 -mthumb',
                                 '--threads=1', '--build-threads=4',
                                 '--use-perf=all',
                                 '--benchmarking-only', '--exec-multisample=3',
                                 '--exclude-stat-from-submission=compile'],
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mthumb'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mthumb'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM'",
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"],
                       submitURL='http://lnt.llvm.org/submitRun',
                       testerName='LNT-Thumb2v7-A15-O3')},
 
         # Cortex-A15 LNT test-suite in test-only mode
         {'name' : "clang-native-arm-lnt",
          'slavenames':["linaro-tk1-03", "linaro-armv8-01-arm-lnt"],
          'builddir':"clang-native-arm-lnt",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       test=False,
                       useTwoStage=False,
                       runTestSuite=True,
                       testsuite_flags=['--cppflags', '-mcpu=cortex-a15 -marm',
                                        '--threads=4', '--build-threads=4'],
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -marm'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -marm'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM'",
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## Cortex-A15 check-all self-host NEON with CMake builder
         {'name': "clang-cmake-armv7-a15-selfhost-neon",
          'slavenames':["linaro-tk1-04", "linaro-armv8-01-arm-selfhost-neon"],
          'builddir':"clang-cmake-armv7-a15-selfhost-neon",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       useTwoStage=True,
                       testStage1=False,
                       extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -marm'",
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -marm'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                        "-DLLVM_LIT_ARGS='-sv -j2'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## Cortex-A15 check-all with CMake builder
         {'name': "clang-cmake-armv7-a15",
          'slavenames':["linaro-tk1-06", "linaro-armv8-01-arm-quick"],
          'builddir':"clang-cmake-armv7-a15",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                        "-DLLVM_LIT_ARGS='-sv -j4'",
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## Cortex-A15 check-all self-host with CMake builder
         {'name': "clang-cmake-armv7-a15-selfhost",
          'slavenames':["linaro-tk1-07", "linaro-armv8-01-arm-selfhost"],
          'builddir':"clang-cmake-armv7-a15-selfhost",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       useTwoStage=True,
                       testStage1=False,
                       extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                        "-DLLVM_LIT_ARGS='-sv -j4'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## AArch64 Clang+LLVM check-all + test-suite
         {'name': "clang-cmake-aarch64-quick",
          'slavenames':["linaro-apm-01", "linaro-armv8-01-aarch64-quick"],
          'builddir':"clang-cmake-aarch64-quick",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       test=True,
                       useTwoStage=False,
                       runTestSuite=True,
                       testsuite_flags=['--cppflags', '-mcpu=cortex-a57',
                                        '--threads=8', '--build-threads=8'],
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a57'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a57'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'"],
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'"],
                )},
 
         ## AArch64 Self-hosting Clang+LLVM check-all + LLD + test-suite
         ## TODO: Remove the X86 back-end after fixing the 90 bad tests
         ## TODO: Add Compiler-RT after fixing all the failures
         ## TODO: Fix the three remaining test-suite failures
         {'name': "clang-cmake-aarch64-lld",
          'slavenames':["linaro-apm-04", "linaro-armv8-01-aarch64-lld"],
          'builddir':"clang-cmake-aarch64-lld",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=True,
                       test=True,
                       useTwoStage=True,
                       runTestSuite=True,
-                      testsuite_flags=['--cppflags',
-                                       '-mcpu=cortex-a57 -fuse-ld=lld',
+                      testsuite_flags=['--cppflags', '-mcpu=cortex-a57 -fuse-ld=lld',
                                        '--threads=8', '--build-threads=8'],
                       extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a57'",
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a57'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64;X86'",
                                         "-DLLVM_ENABLE_LLD=True"],
                ),
          'category'   : 'lld'},
 
         ## AArch64 Clang+LLVM run test-suite with GlobalISel enabled
         {'name': "clang-cmake-aarch64-global-isel",
          'slavenames':["linaro-apm-06", "linaro-armv8-01-aarch64-global-isel"],
          'builddir':"clang-cmake-aarch64-global-isel",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       test=True,
                       useTwoStage=False,
                       runTestSuite=True,
-                      testsuite_flags=['--cppflags',
-                                       '-mcpu=cortex-a57 -O0 -mllvm -global-isel -mllvm -global-isel-abort=0',
+                      testsuite_flags=['--cppflags', '-mcpu=cortex-a57 -O0 -mllvm -global-isel -mllvm -global-isel-abort=0',
                                        '--threads=8', '--build-threads=8'],
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a57'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a57'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'"],
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'"],
                )},
 
         {'name': 'clang-x86-windows-msvc2015',
          'slavenames': ['windows-gcebot2'],
          'builddir': 'clang-x86-windows-msvc2015',
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                         clean=False,
                         vs='%VS140COMNTOOLS%',
def _get_sanitizer_builders():
 
           ## Cortex-A15 check-all full (compiler-rt) with CMake builder; Needs x86 for ASAN tests
           {'name': "clang-cmake-armv7-a15-full",
            'slavenames':["linaro-tk1-08", "linaro-tk1-09", "linaro-armv8-01-arm-full"],
            'builddir':"clang-cmake-armv7-a15-full",
            'factory' : ClangBuilder.getClangCMakeBuildFactory(
                         clean=False,
                         checkout_lld=False,
-                        extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
-                                          "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
-                                          "-DCOMPILER_RT_TEST_COMPILER_CFLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
+                        extra_cmake_args=["-DCOMPILER_RT_TEST_COMPILER_CFLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                           "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                          "-DLLVM_LIT_ARGS='-sv -j4'",
                                           "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
           ## Cortex-A15 Thumb2 check-all full (compiler-rt) with CMake builder; Needs x86 for ASAN tests
           {'name': "clang-cmake-thumbv7-a15-full-sh",
            'slavenames':["linaro-tk1-05", "linaro-armv8-01-arm-full-selfhost"],
            'builddir':"clang-cmake-thumbv7-a15-full-sh",
            'factory' : ClangBuilder.getClangCMakeBuildFactory(
                         clean=False,
                         checkout_lld=False,
                         useTwoStage=True,
                         testStage1=False,
                         extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mthumb'",
                                           "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mthumb'",
                                           "-DCOMPILER_RT_TEST_COMPILER_CFLAGS='-mcpu=cortex-a15 -mthumb'",
                                           "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                          "-DLLVM_LIT_ARGS='-sv -j2'",
                                           "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         # AArch64 Clang+LLVM+RT check-all + test-suite + self-hosting
         {'name': "clang-cmake-aarch64-full",
          'slavenames':["linaro-apm-02", "linaro-apm-05", "linaro-armv8-01-aarch64-full"],
          'builddir':"clang-cmake-aarch64-full",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
return [
         'category': 'libcxx'},
 
         # Cortex-A15 LibC++ and LibC++abi tests (require Clang+RT)
         {'name': 'libcxx-libcxxabi-libunwind-arm-linux',
          'slavenames': ['linaro-tk1-01', 'linaro-armv8-01-arm-libcxx'],
          'builddir': 'libcxx-libcxxabi-libunwind-arm-linux',
          'category': 'libcxx',
          'factory': LibcxxAndAbiBuilder.getLibcxxAndAbiBuilder(
             cmake_extra_opts={'LIBCXXABI_USE_LLVM_UNWINDER': 'ON',
-                              'CMAKE_C_FLAGS': '-mcpu=cortex-a15 -marm',
-                              'CMAKE_CXX_FLAGS': '-mcpu=cortex-a15 -marm',
                               'LLVM_PARALLEL_LINK_JOBS': '2'})},
 
         {'name': 'libcxx-libcxxabi-libunwind-arm-linux-noexceptions',
          'slavenames': ['linaro-tk1-01', 'linaro-armv8-01-arm-libcxx-noeh'],
          'builddir': 'libcxx-libcxxabi-libunwind-arm-linux-noexceptions',
          'category': 'libcxx',
          'factory': LibcxxAndAbiBuilder.getLibcxxAndAbiBuilder(
             cmake_extra_opts={'LIBCXXABI_USE_LLVM_UNWINDER': 'ON',
                               'LIBCXX_ENABLE_EXCEPTIONS': 'OFF',
                               'LIBCXXABI_ENABLE_EXCEPTIONS': 'OFF',
-                              'CMAKE_C_FLAGS': '-mcpu=cortex-a15 -mthumb',
-                              'CMAKE_CXX_FLAGS': '-mcpu=cortex-a15 -mthumb',
                               'LLVM_PARALLEL_LINK_JOBS': '2'})},
 
         # AArch64 LibC++ and LibC++abi tests (require Clang+RT)
         {'name': 'libcxx-libcxxabi-libunwind-aarch64-linux',
          'slavenames': ['linaro-apm-03', 'linaro-armv8-01-aarch64-libcxx'],
          'builddir': 'libcxx-libcxxabi-libunwind-aarch64-linux',
          'category': 'libcxx',
          'factory': LibcxxAndAbiBuilder.getLibcxxAndAbiBuilder(
             cmake_extra_opts={'LIBCXXABI_USE_LLVM_UNWINDER': 'ON',
-                              'CMAKE_C_FLAGS': '-mcpu=cortex-a57',
-                              'CMAKE_CXX_FLAGS': '-mcpu=cortex-a57',
                               'LLVM_PARALLEL_LINK_JOBS': '4'})},
 
         {'name': 'libcxx-libcxxabi-libunwind-aarch64-linux-noexceptions',
          'slavenames': ['linaro-apm-03', 'linaro-armv8-01-aarch64-libcxx-noeh'],
          'builddir': 'libcxx-libcxxabi-libunwind-aarch64-linux-noexceptions',
          'category': 'libcxx',
          'factory': LibcxxAndAbiBuilder.getLibcxxAndAbiBuilder(
             cmake_extra_opts={'LIBCXXABI_USE_LLVM_UNWINDER': 'ON',
                               'LIBCXX_ENABLE_EXCEPTIONS': 'OFF',
                               'LIBCXXABI_ENABLE_EXCEPTIONS': 'OFF',
-                              'CMAKE_C_FLAGS': '-mcpu=cortex-a57',
-                              'CMAKE_CXX_FLAGS': '-mcpu=cortex-a57',
                               'LLVM_PARALLEL_LINK_JOBS': '4'})},
 
     ]
 
 # Experimental and stopped builders
 def _get_on_demand_builders():
     return [
         ]