[zorg] Cleanup unnecessary options for ARM and AArch64 bots

[maxim-kuvyrkov]

Summary

"jobs=" settings are inheritted from slave configuration.

"-DCMAKE_C_FLAGS=" and "-DCMAKE_CXX_FLAGS=" for single-stage builds
are passed only to system compiler and don't affect behavior or results.
For two-stage bots (useTwoStage=True) these flags are preserved.

"-DLLVM_LIT_ARGS='-sv -j2'" flags are extraneous, testsuite runs fine
with default parallelism. We preserve "-DLLVM_PARALLEL_LINK_JOBS=2" for
linaro-tk1-* bots, which have only 2GB of RAM.

[rengolin]

buildbot/osuosl/master/config/builders.py

248 ↗

(On Diff #133579)

You want to keep the -sv as that should very important information about the tests that failed.

287 ↗

(On Diff #133579)

You're not changing it for LNT, better to do that for both.

I'd also try mcup=native instead...

326 ↗

(On Diff #133579)

@rovka is this still necessary? Isn't that the default for O0?

Also, mcpu changes. All or nothing.

998 ↗

(On Diff #133579)

Are the others being set correctly? They weren't when I did this.

1290 ↗

(On Diff #133579)

why disabling marm/mthumb here?

[rovka]

buildbot/osuosl/master/config/builders.py

326 ↗

(On Diff #133579)

Nope, we should only need -O0 now.

[maxim-kuvyrkov]

buildbot/osuosl/master/config/builders.py

248 ↗

(On Diff #133579)

"-sv" are default LLVM_LIT_ARGS set in CMakeLists.txt.

287 ↗

(On Diff #133579)

You're not changing it for LNT, better to do that for both.

What do you mean? There is no aarch64 LNT bot [yet].

I'd also try mcup=native instead...

Yeah, I plan to switch bots (or add extra variants of bots) for -mcpu=native. That's gonna be a future change.

326 ↗

(On Diff #133579)

Will fix.

998 ↗

(On Diff #133579)

I don't follow:
"others" == ???
"They" == ???
Please elaborate :-)

1290 ↗

(On Diff #133579)

This is a single-stage build, so CMAKE_C_FLAGS and CMAKE_CXX_FLAGS are passed only to host compiler. AFAICT, they do not affect results of libcxx testsuites.

[rengolin]

All of the thumb/arm changes I think shouldn't be removed. The main thing is that we can know that stage 1 clang can be compiled as arm/thumb and it runs on all the tests without failure.

buildbot/osuosl/master/config/builders.py

248 ↗

(On Diff #133579)

Ah,must be a new thing, then. :)

287 ↗

(On Diff #133579)

I meant testsuite (using LNT), if you look at the line above you'll see what I mean. :)

998 ↗

(On Diff #133579)

Well, you're deleting C_FLAGS & CXX_FLAGS which didn't use to be set from COMPILER_RT_FLAGS. Unless they are, this will make Clang not compile with ARM/Thumb + VFP, which is important to make sure Clang works on both Arm and Thumb when running the tests. The VFP test is a bit bogus because our boards have NEON...

1290 ↗

(On Diff #133579)

The host compiler will compile libcxx with arm/thumb and therefore generate different library code, no? Same for test code.

[maxim-kuvyrkov]

buildbot/osuosl/master/config/builders.py

287 ↗

(On Diff #133579)

My logic is as follows, let me know if there are mistakes in it:
<cut>
The intention of this bot is to run testsuite on a code path for a popular CPU -- cortex-a57. To achieve this we do a single-stage build of llvm with system compiler (we don't care what tuning system compiler uses as long as ISA is compatible with current CPU), and then build testsuite with stage1 llvm. We care about exercising -mcpu=cortex-a57 code path in llvm, so we add these flags to testsuite.
</cut>

If the above is correct then this change is NFC, which is what I intended it to be.

Now, regarding setting -mcpu=, -mthumb/-marm and other architecture options.
It appears that a cleaner solution would be setting LLVM_TARGET_TRIPLE and/or other configuration options at cmake stage. Make LLVM generate cortex-a57 (or cortex-a15, ...) code by default and stop bothering about flags. WDYT?

998 ↗

(On Diff #133579)

Same logic (and, same assumptions) about CMAKE_C_FLAGS / CMAKE_CXX_FLAGS apply here.

1290 ↗

(On Diff #133579)

Yes and no. Library and test code will be different bit-wise, but (unless there are "#ifdef arm / #ifdef thumb" !) should be semantically equivalent.
If there are indeed different code paths for ARM / Thumb modes or other architectural features, then, "yes", we should differentiate here.

[rengolin]

buildbot/osuosl/master/config/builders.py

287 ↗

(On Diff #133579)

Right, this is the test-suite, your assumptions are correct.

1290 ↗

(On Diff #133579)

I don't know, but I wouldn't assume so. I think we can leave that change for a different patch, after asking the libc++ folks.

[maxim-kuvyrkov]

I'll amend libcxx and global-isel builders per comments in the final commit. Changes for all other bots seem to be OK.

buildbot/osuosl/master/config/builders.py

1290 ↗

(On Diff #133579)

OK, sounds good.

zorg/trunk/buildbot/osuosl/master/config/builders.py

return [
                        extra_cmake_args=['-DLLVM_ENABLE_ASSERTIONS=ON',
                                          '-DLLVM_USE_INTEL_JITEVENTS=TRUE'])},
 
         # Cortex-A15 LNT test-suite in Benchmark mode
         {'name' : "clang-native-arm-lnt-perf",
          'slavenames':["linaro-tk1-02"],
          'builddir':"clang-native-arm-lnt-perf",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
-                      jobs=4,
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       test=False,
                       useTwoStage=False,
                       runTestSuite=True,
                       testsuite_flags=['--cppflags', '-O3 -mcpu=cortex-a15 -mthumb',
                                 '--threads=1', '--build-threads=4',
                                 '--use-perf=all',
                                 '--benchmarking-only', '--exec-multisample=3',
                                 '--exclude-stat-from-submission=compile'],
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mthumb'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mthumb'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM'",
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"],
                       submitURL='http://lnt.llvm.org/submitRun',
                       testerName='LNT-Thumb2v7-A15-O3')},
 
         # Cortex-A15 LNT test-suite in test-only mode
         {'name' : "clang-native-arm-lnt",
          'slavenames':["linaro-tk1-03", "linaro-armv8-01-arm-lnt"],
          'builddir':"clang-native-arm-lnt",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       test=False,
                       useTwoStage=False,
                       runTestSuite=True,
                       testsuite_flags=['--cppflags', '-mcpu=cortex-a15 -marm',
                                        '--threads=4', '--build-threads=4'],
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -marm'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -marm'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM'",
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## Cortex-A15 check-all self-host NEON with CMake builder
         {'name': "clang-cmake-armv7-a15-selfhost-neon",
          'slavenames':["linaro-tk1-04", "linaro-armv8-01-arm-selfhost-neon"],
          'builddir':"clang-cmake-armv7-a15-selfhost-neon",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       useTwoStage=True,
                       testStage1=False,
                       extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -marm'",
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -marm'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                        "-DLLVM_LIT_ARGS='-sv -j2'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## Cortex-A15 check-all with CMake builder
         {'name': "clang-cmake-armv7-a15",
          'slavenames':["linaro-tk1-06", "linaro-armv8-01-arm-quick"],
          'builddir':"clang-cmake-armv7-a15",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                        "-DLLVM_LIT_ARGS='-sv -j4'",
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## Cortex-A15 check-all self-host with CMake builder
         {'name': "clang-cmake-armv7-a15-selfhost",
          'slavenames':["linaro-tk1-07", "linaro-armv8-01-arm-selfhost"],
          'builddir':"clang-cmake-armv7-a15-selfhost",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       useTwoStage=True,
                       testStage1=False,
                       extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                        "-DLLVM_LIT_ARGS='-sv -j4'",
                                         "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         ## AArch64 Clang+LLVM check-all + test-suite
         {'name': "clang-cmake-aarch64-quick",
          'slavenames':["linaro-apm-01", "linaro-armv8-01-aarch64-quick"],
          'builddir':"clang-cmake-aarch64-quick",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       test=True,
                       useTwoStage=False,
                       runTestSuite=True,
                       testsuite_flags=['--cppflags', '-mcpu=cortex-a57',
                                        '--threads=8', '--build-threads=8'],
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a57'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a57'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'"],
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'"],
                )},
 
         ## AArch64 Self-hosting Clang+LLVM check-all + LLD + test-suite
         ## TODO: Remove the X86 back-end after fixing the 90 bad tests
         ## TODO: Add Compiler-RT after fixing all the failures
         ## TODO: Fix the three remaining test-suite failures
         {'name': "clang-cmake-aarch64-lld",
          'slavenames':["linaro-apm-04", "linaro-armv8-01-aarch64-lld"],
          'builddir':"clang-cmake-aarch64-lld",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=True,
                       test=True,
                       useTwoStage=True,
                       runTestSuite=True,
-                      testsuite_flags=['--cppflags',
-                                       '-mcpu=cortex-a57 -fuse-ld=lld',
+                      testsuite_flags=['--cppflags', '-mcpu=cortex-a57 -fuse-ld=lld',
                                        '--threads=8', '--build-threads=8'],
                       extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a57'",
                                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a57'",
                                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64;X86'",
                                         "-DLLVM_ENABLE_LLD=True"],
                ),
          'category'   : 'lld'},
 
-        ## AArch64 Clang+LLVM run test-suite with GlobalISel enabled
+        ## AArch64 Clang+LLVM run test-suite at -O0 (GlobalISel is now default).
         {'name': "clang-cmake-aarch64-global-isel",
          'slavenames':["linaro-apm-06", "linaro-armv8-01-aarch64-global-isel"],
          'builddir':"clang-cmake-aarch64-global-isel",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,
                       checkout_compiler_rt=False,
                       checkout_lld=False,
                       test=True,
                       useTwoStage=False,
                       runTestSuite=True,
-                      testsuite_flags=['--cppflags',
-                                       '-mcpu=cortex-a57 -O0 -mllvm -global-isel -mllvm -global-isel-abort=0',
+                      testsuite_flags=['--cppflags', '-O0',
                                        '--threads=8', '--build-threads=8'],
-                      extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a57'",
-                                        "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a57'",
-                                        "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'"],
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'"],
                )},
 
         {'name': 'clang-x86-windows-msvc2015',
          'slavenames': ['windows-gcebot2'],
          'builddir': 'clang-x86-windows-msvc2015',
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                         clean=False,
                         vs='%VS140COMNTOOLS%',
def _get_sanitizer_builders():
 
           ## Cortex-A15 check-all full (compiler-rt) with CMake builder; Needs x86 for ASAN tests
           {'name': "clang-cmake-armv7-a15-full",
            'slavenames':["linaro-tk1-08", "linaro-tk1-09", "linaro-armv8-01-arm-full"],
            'builddir':"clang-cmake-armv7-a15-full",
            'factory' : ClangBuilder.getClangCMakeBuildFactory(
                         clean=False,
                         checkout_lld=False,
-                        extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
-                                          "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
-                                          "-DCOMPILER_RT_TEST_COMPILER_CFLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
+                        extra_cmake_args=["-DCOMPILER_RT_TEST_COMPILER_CFLAGS='-mcpu=cortex-a15 -mfpu=vfpv3 -marm'",
                                           "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                          "-DLLVM_LIT_ARGS='-sv -j4'",
                                           "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
           ## Cortex-A15 Thumb2 check-all full (compiler-rt) with CMake builder; Needs x86 for ASAN tests
           {'name': "clang-cmake-thumbv7-a15-full-sh",
            'slavenames':["linaro-tk1-05", "linaro-armv8-01-arm-full-selfhost"],
            'builddir':"clang-cmake-thumbv7-a15-full-sh",
            'factory' : ClangBuilder.getClangCMakeBuildFactory(
                         clean=False,
                         checkout_lld=False,
                         useTwoStage=True,
                         testStage1=False,
                         extra_cmake_args=["-DCMAKE_C_FLAGS='-mcpu=cortex-a15 -mthumb'",
                                           "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a15 -mthumb'",
                                           "-DCOMPILER_RT_TEST_COMPILER_CFLAGS='-mcpu=cortex-a15 -mthumb'",
                                           "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
-                                          "-DLLVM_LIT_ARGS='-sv -j2'",
                                           "-DLLVM_PARALLEL_LINK_JOBS=2"])},
 
         # AArch64 Clang+LLVM+RT check-all + test-suite + self-hosting
         {'name': "clang-cmake-aarch64-full",
          'slavenames':["linaro-apm-02", "linaro-apm-05", "linaro-armv8-01-aarch64-full"],
          'builddir':"clang-cmake-aarch64-full",
          'factory' : ClangBuilder.getClangCMakeBuildFactory(
                       clean=False,