Changing number max number of build for ppc64le-flang-mlir-rhel-test

[kamaub]

Summary

The ppc64le-flang-mlir-rhel-test worker runs two builders,
ppc64le-mlir-rhel-clang and ppc64le-flang-rhel-clang, but has its
max_builds=1 which leads to an uncesscary bottleneck.

[gkistanova]

Hello Kamau,

ppc64le-flang-rhel-clang seems using a lot of resources. It takes up to 15 minutes to build when it uses the computer exclusively.

What is the expected build time when 2 ppc64le-flang-rhel-clang builds will be running simultaneously on the same computer?

[kamaub]

In D123060#3428541, @gkistanova wrote:

Hello Kamau,

ppc64le-flang-rhel-clang seems using a lot of resources. It takes up to 15 minutes to build when it uses the computer exclusively.

What is the expected build time when 2 ppc64le-flang-rhel-clang builds will be running simultaneously on the same computer?

Hello Galina,
Thank you for your review, and i'm really sorry for the late reply, I wanted to test this change on a local version of master first
(it only took about 1 hour to spin up and test) and some other work kept pulling me away.

What is the expected build time when 2 ppc64le-flang-rhel-clang builds will be running simultaneously on the same computer?

The motivation here is to have the worker ppc64le-flang-mlir-rhel-test run both its builders, ppc64le-mlir-rhel-clang
and ppc64le-flang-rhel-clang, at the same time. I did not initially setup the bot but I noticed that the build times were on average
5 minutes but the time between builds was 15 minutes and then I also noticed that neither of the two builders ran at the same time
so that why I wanted to let the worker run two builds so they could run at the same time and not have the builders unnecessarily
wait for each other to finish.

I increased the max_builds to two and saw that both bots could ran at the same time as oppose to before when they ran one after
the other. A similar thing is done with the sanitizer-buildbot2 worker which runs the sanitizer-x86_64-linux-fast and
sanitizer-x86_64-linux builders at the same time.

ppc64le-flang-rhel-clang seems using a lot of resources. It takes up to 15 minutes to build when it uses the computer exclusively.

This is actually the smallest and least intensive of the workers we run on the machine and I think that the intension was that
ppc64le-mlir-rhel-clang and ppc64le-flang-rhel-clang run at the same time and I also don't anticipate a slowing down of
the other bots by enabling this worker to run both builders based on the machine capabilities.
I could see a case where there were no build request for ppc64le-mlir-rhel-clang for example and so two builder request
for ppc64le-flang-rhel-clang get started so then a build request that comes in later for ppc64le-mlir-rhel-clang gets delayed
because the workers has already started the two for flang.

Is this possible? In that case maybe I should just make a worker for each builder.

[gkistanova]

Depending on the build requests queues any combination of 2 concurrent builds is possible. Yes, it is possible to have 2 ppc64le-flang-rhel-clang builds running and ppc64le-mlir-rhel-clang builds waiting.

[kamaub]

Creating separate workers for the flang and mlir ppc64le builders to remove any
dependency on build request order between the two bots as was pointed out in a
previous review.

[kamaub]

In D123060#3447349, @gkistanova wrote:

Depending on the build requests queues any combination of 2 concurrent builds is possible. Yes, it is possible to have 2 ppc64le-flang-rhel-clang builds running and ppc64le-mlir-rhel-clang builds waiting.

Thank you for clarifying, I've added a new worker and changed the old one's name to prevent this build request conflict, if this change is acceptable could you please add the two new workers to server's local.cfg with the same password as ppc64le-flang-mlir-rhel-test.
Both the staging and main of course, when this lands I will move these two bots to staging just to observe and make sure it goes smoothly.

[gkistanova]

LGTM.
Please feel free to connect the new workers to the staging.

buildbot/osuosl/master/config/builders.py

'factory' : UnifiedTreeBuilder.getCmakeWithNinjaWithMSVCBuildFactory(
                         "-DMLIR_ENABLE_BINDINGS_PYTHON=ON",
                         "-DLLVM_ENABLE_WERROR=ON",
                         "-DLLVM_TARGETS_TO_BUILD='host;NVPTX;AMDGPU'",
                     ])},
 
     {'name' : 'ppc64le-mlir-rhel-clang',
     'tags'  : ["mlir", "ppc", "ppc64le"],
     'collapseRequests' : False,
-    'workernames' : ['ppc64le-flang-mlir-rhel-test'],
+    'workernames' : ['ppc64le-mlir-rhel-test'],
     'builddir': 'ppc64le-mlir-rhel-clang-build',
     'factory' : UnifiedTreeBuilder.getCmakeWithNinjaBuildFactory(
                     clean=True,
                     depends_on_projects=['llvm', 'mlir'],
                     targets = ['check-mlir-build-only'],
                     checks = ['check-mlir'],
                     extra_configure_args=[
                         '-DLLVM_TARGETS_TO_BUILD=PowerPC',
'factory' : UnifiedTreeBuilder.getCmakeWithNinjaBuildFactory(
                         "-DCMAKE_CXX_COMPILER=g++",
                         "-DLLVM_INSTALL_UTILS=ON",
                         "-DCMAKE_CXX_STANDARD=17",
                     ])},
 
     {'name' : 'ppc64le-flang-rhel-clang',
     'tags'  : ["flang", "ppc", "ppc64le"],
     'collapseRequests' : False,
-    'workernames' : ['ppc64le-flang-mlir-rhel-test'],
+    'workernames' : ['ppc64le-flang-rhel-test'],
     'builddir': 'ppc64le-flang-rhel-clang-build',
     'factory' : UnifiedTreeBuilder.getCmakeWithNinjaBuildFactory(
                     clean=True,
                     depends_on_projects=['llvm', 'mlir', 'clang', 'flang'],
                     checks=['check-flang'],
                     extra_configure_args=[
                         '-DLLVM_TARGETS_TO_BUILD=PowerPC',
                         '-DLLVM_INSTALL_UTILS=ON',

buildbot/osuosl/master/config/workers.py

return [
         create_worker("ppc64be-sanitizer", properties={'jobs': 16}, max_builds=1),
 
         # POWER 8 PowerPC little endian (powerpc64le)
         create_worker("ppc64le-clang-lnt-test", properties={'jobs': 8}, max_builds=1),
         create_worker("ppc64le-clang-multistage-test", properties={'jobs': 8}, max_builds=1),
         create_worker("ppc64le-sanitizer", properties={'jobs': 4}, max_builds=1),
         create_worker("ppc64le-lld-multistage-test", max_builds=1),
         create_worker("ppc64le-clang-rhel-test", properties={'jobs': 192}, max_builds=1),
-        create_worker("ppc64le-flang-mlir-rhel-test", max_builds=1),
+        create_worker("ppc64le-flang-rhel-test", max_builds=1),
+        create_worker("ppc64le-mlir-rhel-test", max_builds=1),
 
         # SPARC 64-bit big endian (sparc64)
         create_worker("debian-stadler-sparc64", properties={'jobs': 4}, max_builds=1),
 
         # Ubuntu x86-64, Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz
         create_worker("hexagon-build-02", properties={'jobs': 12, 'loadaverage': 32},
             max_builds=1),