[zorg] correct amdgpu openmp builders

[david-salinas]

Summary

Correct errors with amdgpu openmp builders

[ronlieb]

@JonChesterfield @Meinersbur

we are trying to get the builder running and seeing the following error.
so we are removing the depends_on_projects clause as an experiment.

Traceback (most recent call last):

• <exception caught here> --- File "/home/stagemaster/buildbot/master/buildbot/process/buildstep.py", line 555, in startStep yield defer.gatherResults(dl)

twisted.internet.defer.FirstError: FirstError[#30, [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.defer.FirstError'>: FirstError[#2, [Failure instance: Traceback: <class 'KeyError'>: 'jobs'
/usr/local/lib/python3.6/dist-packages/twisted/internet/defer.py:151:maybeDeferred
/home/stagemaster/buildbot/master/buildbot/process/properties.py:961:getRenderingFor
/home/stagemaster/buildbot/master/buildbot/process/properties.py:961:<listcomp>
/home/stagemaster/buildbot/master/buildbot/process/properties.py:224:render

• <exception caught here> ---

/usr/local/lib/python3.6/dist-packages/twisted/internet/defer.py:151:maybeDeferred
/home/stagemaster/buildbot/master/buildbot/process/properties.py:463:getRenderingFor
/home/stagemaster/buildbot/master/buildbot/process/properties.py:418:getitem
/home/stagemaster/buildbot/master/buildbot/process/properties.py:145:getitem
]]
]]

[Meinersbur]

From the error message this may fix the problem:

diff --git a/buildbot/osuosl/master/config/workers.py b/buildbot/osuosl/master/config/workers.py
index 18d78b28..63bf8a87 100644
--- a/buildbot/osuosl/master/config/workers.py
+++ b/buildbot/osuosl/master/config/workers.py
@@ -258,5 +258,5 @@ def get_all():
         create_worker("arc-worker", properties={'jobs': 16}, max_builds=1),

         # OpenMP on AMDGPU, Ubuntu 18.04.5, Intel(R) Xeon(R) Gold 5218 @ 2.30GHz with 64GB Memory, 1 Vega20 GPU with 16GB Memory
-        create_worker("omp-vega20-0", max_builds=1),
+        create_worker("omp-vega20-0", properties={'jobs': 6}, max_builds=1),
         ]

Did you consider setting up your own buildbot-master, possibly running on the same machine, to test configurations?

[david-salinas]

@Meinersbur Thanks. I'm going to try your suggested changes to workers.py. Yeah, I had started to look into trying to setup a buildbot-master server locally, but I ran into connection timeouts trying to access the buildbot site today - may just be my local connection though.

[david-salinas]

Patch 2. Change to workers.py

[ronlieb]

@Meinersbur ok with you also ?

[ronlieb]

Galina fixed out workers issue, thank you.
David, please abandon this patch.
i will push another to change the settings for : "openmp-offload-amdgpu-project", and : "openmp-offload-amdgpu-runtime",

[david-salinas]

Abandoning in favour of https://reviews.llvm.org/D107694

buildbot/osuosl/master/config/builders.py

 # OpenMP AMDGPU Builders
     {'name' : "openmp-offload-amdgpu-project",
     'tags'  : ["openmp"],
     'workernames' : ["omp-vega20-0"],
     'builddir': "openmp-offload-amdgpu-project",
     'factory' : OpenMPBuilder.getOpenMPCMakeBuildFactory(
                         clean=False,
                         enable_runtimes=[],
-                        depends_on_projects=['llvm','clang','openmp','lld'],
                         extraCmakeArgs=[
                             "-DCMAKE_BUILD_TYPE=Release",
                             "-DLLVM_TARGETS_TO_BUILD=X86;AMDGPU",
                             "-DLLVM_ENABLE_ASSERTIONS=ON",
                             "-DLLVM_ENABLE_LLD=ON",
                             "-DCLANG_DEFAULT_LINKER=lld",
                             ],
                         install=True,
                         testsuite=True,
-                        testsuite_sollvevv=True,
+                        testsuite_sollvevv=False,
                         extraTestsuiteCmakeArgs=[
                             "-DTEST_SUITE_SOLLVEVV_OFFLOADING_CFLAGS=-fopenmp-targets=amdgcn-amd-amdhsa;-Xopenmp-target=amdgcn-amd-amdhsa",
                             "-DTEST_SUITE_SOLLVEVV_OFFLOADING_LDLAGS=-fopenmp-targets=amdgcn-amd-amdhsa;-Xopenmp-target=amdgcn-amd-amdhsa",
                         ],
                     )},
 
     {'name' : "openmp-offload-amdgpu-runtime",
     'tags'  : ["openmp"],
     'workernames' : ["omp-vega20-0"],
     'builddir': "openmp-offload-amdgpu-runtime",
     'factory' : OpenMPBuilder.getOpenMPCMakeBuildFactory(
-                        clean=False,
+                        clean=True,
                         enable_runtimes=['openmp'],
-                        depends_on_projects=['llvm','clang','openmp','lld'],
                         extraCmakeArgs=[
                             "-DCMAKE_BUILD_TYPE=Release",
                             "-DLLVM_TARGETS_TO_BUILD=X86;AMDGPU",
                             "-DLLVM_ENABLE_ASSERTIONS=ON",
                             "-DLLVM_ENABLE_LLD=ON",
                             "-DCLANG_DEFAULT_LINKER=lld",
                             ],
                         install=True,
                         testsuite=True,
-                        testsuite_sollvevv=True,
+                        testsuite_sollvevv=False,
                         extraTestsuiteCmakeArgs=[
                             "-DTEST_SUITE_SOLLVEVV_OFFLOADING_CFLAGS=-fopenmp-targets=amdgcn-amd-amdhsa;-Xopenmp-target=amdgcn-amd-amdhsa",
                             "-DTEST_SUITE_SOLLVEVV_OFFLOADING_LDLAGS=-fopenmp-targets=amdgcn-amd-amdhsa;-Xopenmp-target=amdgcn-amd-amdhsa",
                         ],
                     )},
 
 
 # Libc++ builders.

buildbot/osuosl/master/config/workers.py

return [
 
         # XCore target, Ubuntu 20.04 x64 host
         create_worker("xcore-ubuntu20-x64", properties={'jobs': 4}, max_builds=1),
 
         # ARC Worker, CentOS 7.9 x86_64 Intel Xeon Platinum 8000 @ 3.6GHz, 32GB RAM
         create_worker("arc-worker", properties={'jobs': 16}, max_builds=1),
 
         # OpenMP on AMDGPU, Ubuntu 18.04.5, Intel(R) Xeon(R) Gold 5218 @ 2.30GHz with 64GB Memory, 1 Vega20 GPU with 16GB Memory
-        create_worker("omp-vega20-0", max_builds=1),
+        create_worker("omp-vega20-0", properties={'jobs': 6}, max_builds=1),
         ]