[Zorg] Change make to ninja instead for Clang RHEL buildbot

[Conanap]

Summary

Changing it to use ninja instead as it builds and runs with ninja.

[Conanap]

Using ninja instead

[stefanp]

zorg/buildbot/builders/ClangBuilder.py

498

It looks like you are modifying a step in ClangCMakeBuildFactory to use ninja instead of make. My worry is that this factory is used in a number of bots (see config/builders.py) not only PowerPC bots. It this going to cause problems on other platforms? Other bots?

[Conanap]

zorg/buildbot/builders/ClangBuilder.py

498

Hey Stefan, this part of the patch (cmake_test_suite) was added just for the RHEL buildbot; you can see the other fabricator patch here (committed on Monday): https://reviews.llvm.org/D99097

[saghir]

This looks good to me, approving to unblock the failing bot.

[stefanp]

Thank you for the explanation!
LGTM.

[gkistanova]

This patch has broken zorg. I reverted the both this one and D99097 as they seems closely related, and D100431 fixes D99097.

[gkistanova]

Sorry for reverting your patches, but I do not see a quick and easy way to fix the problems it has introduced.

Besides, with your last change to use ninja to run tests I don't think you need all these changes after all. It looks like you can use UnifiedTreeBuilder as is and it will do what you want. If not, we can discuss what is missing,

Though, for the sake of completeness of this review, please see my comments inline.

zorg/buildbot/builders/ClangBuilder.py

493

You cannot concatenate a string with WithProperties.
Trying to do this would give you a builtins.TypeError: must be str, not WithProperties error.

494

The same is here. No string and WithProperties concatenations.

556

lnt_setup is not defined if cmake_test_suite is True.

[Conanap]

This change is no longer needed

zorg/buildbot/builders/ClangBuilder.py

if runTestSuite:
 
         # LNT Command line (don't pass -jN. Users need to pass both --threads
         # and --build-threads in nt_flags/test_suite_flags to get the same effect)
         use_runtest_testsuite = len(nt_flags) == 0
         # with LNT, the workdir needs to be test/sandbox,
         # but the CMakeList.txt and Makefiles are in %(builddir)s/test/test-suite.
         test_suite_work_dir = 'test/sandbox'
         if cmake_test_suite:
-            cmake_test_suite_cmd = ['cmake', '-G', 'Unix', 'Makefiles',
-                                    '--cc', cc,
-                                    '--cxx', cxx]
-            test_suite_cmd = ['make', '-k']
-            test_suite_work_dir = test_suite_dir
+            cmake_test_suite_cmd = ['cmake', '-G', 'Ninja',
+                                    '-DCMAKE_C_COMPILER=' + cc,
+                                    '-DCMAKE_CXX_COMPILER=' + cxx,
+                                    '-DTEST_SUITE_LIT:FILEPATH=' + lit,
+                                    test_suite_dir]
+            test_suite_build_cmd = ['ninja', '-k', '0']
+            test_suite_cmd = ['ninja', 'check']
         elif not use_runtest_testsuite:
             test_suite_cmd = [python, lnt, 'runtest', 'nt',
                               '--no-timestamp',
                               '--sandbox', sandbox,
                               '--test-suite', test_suite_dir,
                               '--cc', cc,
                               '--cxx', cxx]
             # Append any option provided by the user
if runTestSuite:
                                env=env))
         f.addStep(ShellCommand(name='recreate sandbox',
                                command=['virtualenv', '--python=python3', 'sandbox'],
                                haltOnFailure=True,
                                description='recreating sandbox',
                                workdir='test',
                                env=env))
         f.addStep(ShellCommand(name='setup lit',
                                command=[python, lnt_setup, 'develop'],
                                haltOnFailure=True,
                                description='setting up LNT in sandbox',
                                workdir='test/sandbox',
                                env=env))
         if cmake_test_suite:
             f.addStep(ShellCommand(
                                name='cmake test-suite',
                                command=cmake_test_suite_cmd,
                                haltOnFailure=True,
                                description=['running cmake for test suite'],
                                workdir=test_suite_work_dir,
                                env=test_suite_env))
+            f.addStep(ShellCommand(
+                               name='Ninja test-suite',
+                               command=test_suite_build_cmd,
+                               haltOnFailure=True,
+                               description=['Using ninja to build test suite'],
+                               workdir=test_suite_work_dir,
+                               env=test_suite_env))
         f.addStep(LitTestCommand(
                                name='test-suite',
                                command=test_suite_cmd,
                                haltOnFailure=True,
                                description=['running the test suite'],
                                workdir=test_suite_work_dir,
                                logfiles={'configure.log'   : 'build/configure.log',
                                          'build-tools.log' : 'build/build-tools.log',
                                          'test.log'        : 'build/test.log',
                                          'report.json'     : 'build/report.json'},
                                env=test_suite_env))
 
     return f