[Zorg][PowerPC] Implement a TestSuiteBuilder

[Conanap]

Summary

This patch implements TestSuiteBuilder, which utilises
UnifiedTreeBuilder to build with ninja, and then checks
out llvm-test-suite, builds it and runs it.

The patch also changes the PPC RHEL bot to this new builder.

Patch has been tested on a local server.

[gkistanova]

Thanks, Albion!

The patch looks good. There are few things, though.

1. Could you remove unrelated changes to the .gitignore file, please? Please feel free to make a separate patch if you want this change in.

2. jobs is a worker property, not a build configuration. We have it historically in old builders, but there is no need to keep it in new. Think of a case when different computers are used as workers for the same builder, one is stronger and could handle more parallel jobs, and the other one needs less. NinjaCommand handles the property; you can just remove the jobs arguments for your usage.

+ some inline comment.

zorg/buildbot/builders/TestSuiteBuilder.py

2

There is no need to rename the getCmakeWithNinjaBuildFactory as there is no name collision here.

[Conanap]

Removed jobs option, removed .gitignore changes, removed import rename.

[Conanap]

@gkistanova this should hopefully have addressed all your concerns. Please let me know if there's anything else, thanks!

[gkistanova]

Perfect! Thanks for making the changes!
Let me stage this for you for a day or two in case you will want to make some changes after seeing it in action.

[Conanap]

@gkistanova Thanks! I left it on staging for a day and am pretty happy with where it's at. Let me know if there's anything else you want me to change before approval and commit. Thanks!

[Conanap]

gentle ping @gkistanova

[gkistanova]

LGTM with a nit pick.
Please remove steps from the import and feel free to commit.

zorg/buildbot/builders/TestSuiteBuilder.py

4

You do not use steps, so you can remove it from the import.

[Conanap]

I've removed the steps import in the actual commit. Thanks!

buildbot/osuosl/master/config/builders.py

'factory' : ClangBuilder.getClangCMakeBuildFactory(
                     extra_cmake_args=[
                         "-DLLVM_ENABLE_ASSERTIONS=ON",
                         "-DPYTHON_EXECUTABLE:FILEPATH=/usr/local/bin/python3"])},
 
     {'name' : "clang-ppc64le-rhel",
     'tags'  : ["clang", "ppc", "ppc64le"],
     'workernames' : ["ppc64le-clang-rhel-test"],
     'builddir': "clang-ppc64le-rhel",
-    'factory' : UnifiedTreeBuilder.getCmakeWithNinjaBuildFactory(
+    'factory' : TestSuiteBuilder.getTestSuiteBuildFactory(
                     depends_on_projects=["llvm", "clang", "clang-tools-extra", "lld", "compiler-rt"],
                     checks=["check"],
                     enable_runtimes="auto",
                     extra_configure_args=[
                         "-DLLVM_ENABLE_ASSERTIONS=On", "-DCMAKE_C_COMPILER=clang",
                         "-DCMAKE_CXX_COMPILER=clang++",
                         "-DCLANG_DEFAULT_LINKER=lld",
                         "-DCMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN:PATH=/opt/rh/devtoolset-7/root/usr",

zorg/buildbot/builders/TestSuiteBuilder.py

+from zorg.buildbot.builders.UnifiedTreeBuilder import getCmakeWithNinjaBuildFactory
+
+from buildbot.plugins import util
+
+from zorg.buildbot.commands.CmakeCommand import CmakeCommand
+from zorg.buildbot.commands.NinjaCommand import NinjaCommand
+from zorg.buildbot.commands.LitTestCommand import LitTestCommand
+
+# This builder is uses UnifiedTreeBuilders and adds running
+# llvm-test-suite with cmake and ninja step.
+
+def addTestSuiteStep(
+            f,
+            compiler_dir = None,
+            env = None,
+            lit_args = [],
+            **kwargs):
+
+    cc = util.Interpolate('-DCMAKE_C_COMPILER=' + '%(prop:builddir)s/'+compiler_dir+'/bin/clang')
+    cxx = util.Interpolate('-DCMAKE_CXX_COMPILER=' + '%(prop:builddir)s/'+compiler_dir+'/bin/clang++')
+    lit = util.Interpolate('%(prop:builddir)s/' + compiler_dir + '/bin/llvm-lit')
+    test_suite_src_dir = util.Interpolate('%(prop:builddir)s/' + 'test/test-suite')
+    test_suite_workdir='test/sandbox/test-suite'
+    cmake_lit_arg = util.Interpolate('-DTEST_SUITE_LIT:FILEPATH=' + '%(prop:builddir)s/' + compiler_dir + '/bin/llvm-lit')
+    # used for cmake building test-suite step
+    options = [cc, cxx, cmake_lit_arg]
+
+    f.addGetSourcecodeForProject(
+        project='test-suite',
+        src_dir=test_suite_src_dir,
+        alwaysUseLatest=True)
+
+    f.addStep(CmakeCommand(name='cmake Test Suite',
+                           haltOnFailure=True,
+                           description='Running cmake on Test Suite dir',
+                           workdir=test_suite_workdir,
+                           options=options,
+                           path=test_suite_src_dir,
+                           generator='Ninja'))
+
+    f.addStep(NinjaCommand(name='ninja Test Suite',
+                           description='Running Ninja on Test Suite dir',
+                           haltOnFailure=True,
+                           workdir=test_suite_workdir))
+
+    f.addStep(LitTestCommand(name='Run Test Suite with lit',
+                             haltOnFailure=True,
+                             description='Running test suite tests',
+                             workdir=test_suite_workdir,
+                             command=[lit] + lit_args + ['.'],
+                             env=env,
+                             **kwargs))
+
+    return f
+
+def getTestSuiteBuildFactory(
+           depends_on_projects = None,
+           enable_runtimes = None,
+           targets = None,
+           llvm_srcdir = None,
+           obj_dir = None,
+           checks = None,
+           install_dir = None,
+           clean = False,
+           extra_configure_args = None,
+           env = None,
+           **kwargs):
+
+    # handle the -DCMAKE args for lit
+    lit_args = list()
+    if any("DLLVM_LIT_ARGS" in arg for arg in extra_configure_args):
+        arg = [arg for arg in extra_configure_args
+                if "DLLVM_LIT_ARGS" in arg][0]
+        lit_args = arg.split("=")[1]
+        lit_args = lit_args.split(" ")
+
+    f = getCmakeWithNinjaBuildFactory(
+            depends_on_projects = depends_on_projects,
+            enable_runtimes = enable_runtimes,
+            targets = targets,
+            llvm_srcdir = llvm_srcdir,
+            obj_dir = obj_dir,
+            checks = checks,
+            install_dir = install_dir,
+            clean = clean,
+            extra_configure_args = extra_configure_args,
+            env = env,
+            **kwargs)
+
+
+    addTestSuiteStep(f,
+           compiler_dir=f.obj_dir,
+           env=env,
+           lit_args=lit_args,
+           **kwargs)
+
+    return f