[zorg] Reorganize Polly builders.

[Meinersbur]

Summary

Instead of a single build configuration, add 6 builders of the configurations (static, shared, shlib) x (LINK_POLLY_INTO_TOOLS=ON/OFF) for the 2 gce buildsbots. The test-suite build is extended to include external tests (SPEC benchmarks, povray) and moved to 2 other workers that should be able to compile it faster. Also, the verbosity is reduced (seems to have caused problem when the buildbot master receives too much data) and -polly-process-unprofitable is used for greater test coverage.

Build configurations already running here:
http://meinersbur.de:8011/#/builders/83
http://meinersbur.de:8011/#/builders/145
http://meinersbur.de:8011/#/builders/149
http://meinersbur.de:8011/#/builders/148
http://meinersbur.de:8011/#/builders/147
http://meinersbur.de:8011/#/builders/146
http://meinersbur.de:8011/#/builders/76

[Meinersbur]

@gkistanova ping

[gkistanova]

Thanks for the patch, Michael!

zorg/buildbot/builders/PollyBuilder.py

16

Wrong indentation?

17

Instead of adding a Boolean argument and then handling it to set a cmake parameter, why not let user expliciltly set the
-DPOLLY_ENABLE_GPGPU_CODEGEN in the existing
extraCmakeArgs argument, and add the default if not set explicitly?

86–89

Missing comma and new line between env and logfiles?

163

This effectively prohibits using interpolation with testsuiteComplileArgs.

You ether do interpolation here, for example by calling WithProperties on test suite arguments, or you need to build the flags differently.

[Meinersbur]

• Address review
• Sort builders by -fPIC

[Meinersbur]

zorg/buildbot/builders/PollyBuilder.py

16

Was using tab indentation here. I should use spaces consistently.

17

I moved the POLLY_ENABLE_GPGPU_CODEGEN configuration parameters to the builders, does not introduce additional complexity. Some of the build seem to have set POLLY_ENABLE_GPGPU_CODEGEN already in order to override it.

163

Removing extraTestsuiteCompileArgs. I think I had a use case of it, but it is not used in this patch.

[gkistanova]

Thanks, Michael!

LGTM.

If you want, you can set -DPOLLY_ENABLE_GPGPU_CODEGEN=OFF by default in PollyBuilder by using CmakeCommand.ApplyDefaultOptions.

buildbot/osuosl/master/config/builders.py

'factory' : PollyBuilder.getPollyBuildFactory(
                     env={'LD_LIBRARY_PATH': '/local/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-16.04/lib'},
                     extraCmakeArgs=[
                         "-G", "Ninja",
                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
                         "-DLLVM_DEFAULT_TARGET_TRIPLE=arm-linux-gnueabi",
                         "-DLLVM_TARGET_ARCH=arm-linux-gnueabi",
                         "-DLLVM_ENABLE_ASSERTIONS=True",
                         "-DLLVM_ENABLE_LIBCXX:BOOL=ON",
-                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=OFF",  # Auto-enabled in getPollyBuildFactory
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=OFF",
                         "-DCMAKE_C_COMPILER:FILEPATH=/local/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/clang",
                         "-DCMAKE_CXX_COMPILER:FILEPATH=/local/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/clang++"])},
 
     {'name' : "polly-x86_64-linux",
     'tags'  : ["polly"],
-    'workernames' : ["polly-x86_64-fdcserver", "polly-x86_64-gce1"],
+    'workernames' : ["polly-x86_64-gce1"],
     'builddir': "polly-x86_64-linux",
     'factory' : PollyBuilder.getPollyBuildFactory(
                     clean=False,
                     install=False,
                     make='ninja',
                     extraCmakeArgs=[
                         "-G", "Ninja",
+                        "-DCMAKE_C_COMPILER_LAUNCHER=ccache",
+                        "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache",
+                        "-DBUILD_SHARED_LIBS=ON",
+                        "-DLLVM_ENABLE_ASSERTIONS=True",
+                        "-DLLVM_TARGETS_TO_BUILD='X86;NVPTX'",
+                        "-DCLANG_ENABLE_ARCMT=OFF",
+                        "-DCLANG_ENABLE_STATIC_ANALYZER=OFF",
+                        "-DCLANG_ENABLE_OBJC_REWRITER=OFF",
+                        "-DLLVM_ENABLE_LLD=ON",
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=ON"
+                        ])},
+
+    {'name' : "polly-x86_64-linux-plugin",
+    'tags'  : ["polly"],
+    'workernames' : ["polly-x86_64-gce1"],
+    'builddir': "polly-x86_64-linux",
+    'factory' : PollyBuilder.getPollyBuildFactory(
+                    clean=False,
+                    install=False,
+                    make='ninja',
+                    extraCmakeArgs=[
+                        "-G", "Ninja",
+                        "-DCMAKE_C_COMPILER_LAUNCHER=ccache",
+                        "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache",
+                        "-DLLVM_ENABLE_ASSERTIONS=True",
+                        "-DLLVM_TARGETS_TO_BUILD='X86;NVPTX'",
+                        "-DCLANG_ENABLE_ARCMT=OFF",
+                        "-DCLANG_ENABLE_STATIC_ANALYZER=OFF",
+                        "-DCLANG_ENABLE_OBJC_REWRITER=OFF",
+                        "-DLLVM_ENABLE_LLD=ON",
+                        "-DLLVM_POLLY_LINK_INTO_TOOLS=OFF",
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=OFF"  # Not all required symbols available in opt executable
+                        ])},
+
+    {'name' : "polly-x86_64-linux-noassert",
+    'tags'  : ["polly"],
+    'workernames' : ["polly-x86_64-gce1"],
+    'builddir': "polly-x86_64-linux",
+    'factory' : PollyBuilder.getPollyBuildFactory(
+                    clean=False,
+                    install=False,
+                    make='ninja',
+                    extraCmakeArgs=[
+                        "-G", "Ninja",
+                        "-DCMAKE_C_COMPILER_LAUNCHER=ccache",
+                        "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache",
+                        "-DLLVM_ENABLE_ASSERTIONS=False",
+                        "-DLLVM_TARGETS_TO_BUILD='X86;NVPTX'",
+                        "-DCLANG_ENABLE_ARCMT=OFF",
+                        "-DCLANG_ENABLE_STATIC_ANALYZER=OFF",
+                        "-DCLANG_ENABLE_OBJC_REWRITER=OFF",
+                        "-DLLVM_ENABLE_LLD=ON",
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=ON"
+                        ])},
+
+    {'name' : "polly-x86_64-linux-shared",
+    'tags'  : ["polly"],
+    'workernames' : ["polly-x86_64-gce2"],
+    'builddir': "polly-x86_64-linux",
+    'factory' : PollyBuilder.getPollyBuildFactory(
+                    clean=False,
+                    install=False,
+                    make='ninja',
+                    extraCmakeArgs=[
+                        "-G", "Ninja",
+                        "-DCMAKE_C_COMPILER_LAUNCHER=ccache",
+                        "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache",
                         "-DLLVM_ENABLE_ASSERTIONS=True",
                         "-DLLVM_TARGETS_TO_BUILD='X86;NVPTX'",
                         "-DCLANG_ENABLE_ARCMT=OFF",
                         "-DCLANG_ENABLE_STATIC_ANALYZER=OFF",
+                        "-DCLANG_ENABLE_OBJC_REWRITER=OFF",
+                        "-DLLVM_ENABLE_LLD=ON",
+                        "-DBUILD_SHARED_LIBS=ON",
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=ON"
+                        ])},
+
+    {'name' : "polly-x86_64-linux-shared-plugin",
+    'tags'  : ["polly"],
+    'workernames' : ["polly-x86_64-gce2"],
+    'builddir': "polly-x86_64-linux",
+    'factory' : PollyBuilder.getPollyBuildFactory(
+                    clean=False,
+                    install=False,
+                    make='ninja',
+                    extraCmakeArgs=[
+                        "-G", "Ninja",
+                        "-DCMAKE_C_COMPILER_LAUNCHER=ccache",
+                        "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache",
+                        "-DLLVM_ENABLE_ASSERTIONS=True",
+                        "-DLLVM_TARGETS_TO_BUILD='X86;NVPTX'",
+                        "-DCLANG_ENABLE_ARCMT=OFF",
+                        "-DCLANG_ENABLE_STATIC_ANALYZER=OFF",
+                        "-DCLANG_ENABLE_OBJC_REWRITER=OFF",
+                        "-DLLVM_ENABLE_LLD=ON",
+                        "-DBUILD_SHARED_LIBS=ON",
+                        "-DLLVM_POLLY_LINK_INTO_TOOLS=OFF",
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=ON"
+                        ])},
+
+    {'name' : "polly-x86_64-linux-shlib",
+    'tags'  : ["polly"],
+    'workernames' : ["polly-x86_64-gce2"],
+    'builddir': "polly-x86_64-linux",
+    'factory' : PollyBuilder.getPollyBuildFactory(
+                    clean=False,
+                    install=False,
+                    make='ninja',
+                    extraCmakeArgs=[
+                        "-G", "Ninja",
+                        "-DCMAKE_C_COMPILER_LAUNCHER=ccache",
+                        "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache",
+                        "-DLLVM_ENABLE_ASSERTIONS=True",
+                        "-DLLVM_TARGETS_TO_BUILD='X86;NVPTX'",
+                        "-DCLANG_ENABLE_ARCMT=OFF",
+                        "-DCLANG_ENABLE_STATIC_ANALYZER=OFF",
+                        "-DCLANG_ENABLE_OBJC_REWRITER=OFF",
+                        "-DLLVM_ENABLE_LLD=ON",
+                        "-DLLVM_BUILD_LLVM_DYLIB=ON",
+                        "-DLLVM_LINK_LLVM_DYLIB=ON",
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=ON"
+                        ])},
+
+    {'name' : "polly-x86_64-linux-shlib-plugin",
+    'tags'  : ["polly"],
+    'workernames' : ["polly-x86_64-gce2"],
+    'builddir': "polly-x86_64-linux",
+    'factory' : PollyBuilder.getPollyBuildFactory(
+                    clean=False,
+                    install=False,
+                    make='ninja',
+                    extraCmakeArgs=[
+                        "-G", "Ninja",
+                        "-DCMAKE_C_COMPILER_LAUNCHER=ccache",
+                        "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache",
+                        "-DLLVM_ENABLE_ASSERTIONS=True",
+                        "-DLLVM_TARGETS_TO_BUILD='X86;NVPTX'",
+                        "-DCLANG_ENABLE_ARCMT=OFF",
+                        "-DCLANG_ENABLE_STATIC_ANALYZER=OFF",
+                        "-DCLANG_ENABLE_OBJC_REWRITER=OFF",
+                        "-DLLVM_ENABLE_LLD=ON",
+                        "-DLLVM_BUILD_LLVM_DYLIB=ON",
+                        "-DLLVM_LINK_LLVM_DYLIB=ON",
+                        "-DLLVM_POLLY_LINK_INTO_TOOLS=OFF",
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=ON"
                         ])},
 
     {'name' : "polly-x86_64-linux-test-suite",
     'tags'  : ["polly"],
-    'workernames' : ["polly-x86_64-fdcserver", "polly-x86_64-gce2"],
+    'workernames' : ["polly-x86_64-fdcserver", "minipc-1050ti-linux"],
     'builddir': "polly-x86_64-linux-test-suite",
     'factory' : PollyBuilder.getPollyBuildFactory(
                     clean=False,
                     install=False,
                     make='ninja',
                     extraCmakeArgs=[
                         "-G", "Ninja",
+                        "-DCMAKE_C_COMPILER_LAUNCHER=ccache",
+                        "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache",
                         "-DLLVM_ENABLE_ASSERTIONS=True",
                         "-DLLVM_TARGETS_TO_BUILD='X86;NVPTX'",
                         "-DCLANG_ENABLE_ARCMT=OFF",
                         "-DCLANG_ENABLE_STATIC_ANALYZER=OFF",
+                        "-DCLANG_ENABLE_OBJC_REWRITER=OFF"
                         ],
                     testsuite=True,
-                    extraTestsuiteCmakeArgs=["-G", "Ninja"]
+                    extraTestsuiteCmakeArgs=[
+                        "-G", "Ninja",
+                        "-DTEST_SUITE_COLLECT_COMPILE_TIME=OFF",
+                        "-DTEST_SUITE_COLLECT_STATS=OFF",
+                        "-DTEST_SUITE_COLLECT_CODE_SIZE=OFF",
+                        WithProperties("-DTEST_SUITE_EXTERNALS_DIR=%(builddir)s/../../test-suite-externals"),
+                      ]
                     )},
 
 # AOSP builders.
 
     {'name' : "aosp-O3-polly-before-vectorizer-unprofitable",
     'tags'  : ["polly", "aosp"],
     'workernames' : ["hexagon-build-03"],
     'builddir': "aosp",
     'factory' : AOSPBuilder.getAOSPBuildFactory(
                     device="angler",
                     extra_cmake_args=[
                         "-G", "Ninja",
                         "-DLLVM_TARGETS_TO_BUILD='ARM;AArch64'",
                         "-DLLVM_DEFAULT_TARGET_TRIPLE=arm-linux-androideabi",
                         "-DLLVM_TARGET_ARCH=arm-linux-androideabi",
                         "-DLLVM_ENABLE_ASSERTIONS=True",
                         "-DLLVM_ENABLE_LIBCXX:BOOL=ON",
-                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=OFF",  # Auto-enabled in getPollyBuildFactory (used indirectly)
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=OFF",
                         "-DCMAKE_C_COMPILER:FILEPATH=/local/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/clang",
                         "-DCMAKE_CXX_COMPILER:FILEPATH=/local/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/clang++"],
                     timeout=240,
                     target_clang=None,
                     target_flags="-Wno-error -O3 -mllvm -polly -mllvm -polly-position=before-vectorizer -mllvm -polly-process-unprofitable -fcommon",
                     jobs=32,
                     extra_make_args=None,
                     env={'LD_LIBRARY_PATH': '/local/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-16.04/lib'},
'factory' : PollyBuilder.getPollyBuildFactory(
                     jobs=16,
                     checkAll=True,
                     env={'LD_LIBRARY_PATH': '/local/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-16.04/lib'},
                     extraCmakeArgs=[
                         "-G", "Ninja",
                         "-DLLVM_REVERSE_ITERATION:BOOL=ON",
                         "-DLLVM_ENABLE_ASSERTIONS=True",
                         "-DLLVM_ENABLE_LIBCXX:BOOL=ON",
+                        "-DPOLLY_ENABLE_GPGPU_CODEGEN=ON",
                         "-DCMAKE_C_COMPILER:FILEPATH=/local/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/clang",
                         "-DCMAKE_CXX_COMPILER:FILEPATH=/local/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/clang++"])},
 
 # LLDB builders.
 
     {'name' : "lldb-x86_64-debian",
     'tags'  : ["lldb"],
     'workernames' : ["lldb-x86_64-debian"],

zorg/buildbot/builders/PollyBuilder.py

 from buildbot.process.properties import WithProperties
 from buildbot.steps.shell import ShellCommand, WarningCountingShellCommand
 from buildbot.plugins import steps
 from zorg.buildbot.commands.LitTestCommand import LitTestCommand
 from zorg.buildbot.process.factory import LLVMBuildFactory
 
 def getPollyBuildFactory(
     clean=False,
     install=False,
     make='make',
     jobs=None,
     checkAll=False,
     env=None,
     extraCmakeArgs=None,
     testsuite=False,extraTestsuiteCmakeArgs=None,
     **kwargs):
 
     if extraCmakeArgs is None:
         extraCmakeArgs=[]
     if extraTestsuiteCmakeArgs is None:
         extraTestsuiteCmakeArgs = []
 
     llvm_srcdir = "llvm.src"
     llvm_objdir = "llvm.obj"
     llvm_instdir = "llvm.inst"
def getPollyBuildFactory(
     # Prepare environmental variables. Set here all env we want everywhere.
     merged_env = {
                    'TERM' : 'dumb'     # Make sure Clang doesn't use color escape sequences.
                  }
     if env:
         merged_env.update(env)  # Overwrite pre-set items with the given ones, so user can set anything.
 
     depends_on_projects = ['llvm','clang','polly']
-    if testsuite:
-        # XRay tests in test-suite require compiler-rt
-        depends_on_projects += ['compiler-rt']
 
     cleanBuildRequestedByProperty = lambda step: step.build.getProperty("clean", False)
     cleanBuildRequested = lambda step: clean or step.build.getProperty("clean", default=step.build.getProperty("clean_obj"))
 
     f = LLVMBuildFactory(
             depends_on_projects=depends_on_projects,
             llvm_srcdir=llvm_srcdir,
             obj_dir=llvm_objdir,
f.addStep(steps.RemoveDirectory(name='clean-build-dir',
                            dir=llvm_objdir,
                            warnOnFailure=True,
                            doStepIf=cleanBuildRequested))
 
     # Create configuration files with cmake
     cmakeCommand = ["cmake", "../%s/llvm" % llvm_srcdir,
                     "-DCMAKE_COLOR_MAKEFILE=OFF",
                     "-DPOLLY_TEST_DISABLE_BAR=ON",
-                    "-DPOLLY_ENABLE_GPGPU_CODEGEN=ON",
                     "-DCMAKE_BUILD_TYPE=Release",
                     "-DLLVM_POLLY_LINK_INTO_TOOLS=ON",
                     "-DLLVM_ENABLE_PROJECTS=%s" % ";".join(f.depends_on_projects),
                    ] + cmake_install + extraCmakeArgs
     f.addStep(ShellCommand(name="cmake-configure",
                            command=cmakeCommand,
                            haltOnFailure=False,
                            description=["cmake configure"],
                            workdir=llvm_objdir,
-                           env=merged_env))
+                           env=merged_env,
+                           logfiles={
+                                'CMakeCache.txt' : llvm_objdir + '/CMakeCache.txt',
+                           }))
 
     # Build
     f.addStep(WarningCountingShellCommand(name="build",
                            command=build_cmd,
                            haltOnFailure=True,
                            description=["build"],
                            workdir=llvm_objdir,
                            env=merged_env))
if testsuite:
             src_dir=testsuite_srcdir,
             alwaysUseLatest=True)
 
         f.addStep(steps.RemoveDirectory(name='test-suite_clean-build-dir',
                            dir=testsuite_builddir,
                            haltOnFailure=False,
                            warnOnFailure=True))
 
-        # -Wno-unused-command-line-argument is needed because linking will not uses the "-mllvm -polly" argument.
+        # -Wno-unused-command-line-argument is needed because linking will not use the "-mllvm -polly" argument.
         f.addStep(ShellCommand(name='test-suite_cmake-configure',
                            description=["Test-Suite: cmake"],
                            command=["cmake", '-B', testsuite_builddir, '-S', testsuite_srcdir,
                                     "-DCMAKE_BUILD_TYPE=Release",
-                                    "-DTEST_SUITE_COLLECT_STATS=ON",
                                     "-DTEST_SUITE_EXTRA_C_FLAGS=-Wno-unused-command-line-argument -mllvm -polly",
-                                    "-DTEST_SUITE_EXTRA_CXX_FLAGS=-mllvm -polly",
+                                    "-DTEST_SUITE_EXTRA_CXX_FLAGS=-Wno-unused-command-line-argument -mllvm -polly",
                                     "-DTEST_SUITE_LIT_FLAGS=-vv;-o;report.json",
                                     WithProperties("-DCMAKE_C_COMPILER=" + clangexe),
                                     WithProperties("-DCMAKE_CXX_COMPILER=" + clangxxexe),
                                     WithProperties("-DTEST_SUITE_LLVM_SIZE=" + sizeexe),
                                     WithProperties("-DTEST_SUITE_LIT=" + litexe),
                                 ] + extraTestsuiteCmakeArgs,
                            haltOnFailure=True,
                            workdir='.',
-                           env=merged_env))
+                           env=merged_env,
+                           logfiles={
+                                'CMakeCache.txt' : testsuite_builddir + '/CMakeCache.txt',
+                            }))
 
         f.addStep(WarningCountingShellCommand(name='test-suite_build',
                            description=["Test-Suite: build"],
                            # Continue building; programs that don't compile will fail with NOEXE.
                            command=[make, 'all', '-k0'] + jobs_cmd,
                            haltOnFailure=False,
                            flunkOnFailure=True,
                            workdir=testsuite_builddir,
                            env=merged_env))
 
         f.addStep(LitTestCommand(name='test-suite_run',
                             description=['Test-Suite: run'],
-                            command=[WithProperties(litexe), '-vv', '-o', 'report.json', '.'],
+                            command=[WithProperties(litexe), '-vv', '-s', '-o', 'report.json', '.'],
                             haltOnFailure=True,
                             workdir=testsuite_builddir,
                             logfiles={
                                 'test.log'   : 'test.log',
                                 'report.json': 'report.json'
                                  },
                             env=merged_env))
 
     return f