[zorg] Add Clang ppc64le cross-targeting buildbot on AIX

[Xiangling_L]

@gkistanova Hi Galina, we are trying to set up a cross-targeting buildbot for AIX platform. And I sent you an email on Wed, Jan 20th, 2021 requesting access for the new build bot.

Can you please confirm that you received it. Thanks.

[hubert.reinterpretcast]

buildbot/osuosl/master/config/builders.py

634

I think having the host platform described more clearly makes the non-native default target more obvious.

655

There's at least one typo to correct. Just use the value from a build for ppc64le Linux.

[Xiangling_L]

Addressed the comments;

[hubert.reinterpretcast]

buildbot/osuosl/master/config/builders.py

637

Please consider changing this to match the name.

651–652

I think you reported "offline" that you needed to add -qaltivec.

[Xiangling_L]

buildbot/osuosl/master/config/builders.py

637

Oops... I missed this.

651–652

Thanks for reminding. I will update this. I was waiting for build to finish. It usually took around 6 hours.

[Xiangling_L]

Update build dir name accordingly;
Ad -qaltivec into cmake config;

[hubert.reinterpretcast]

@Xiangling_L, sorry if this is adding extra respin test cycles. I've done as much review this time around as I can do for the patch as it is currently posted. Thanks.

buildbot/osuosl/master/config/builders.py

635

Minor comment: Usage of the "ppc" tag seems to imply it is a generic tag for anything Power-related. It makes sense to add it here.

643

Not sure if other people have a different opinion about this, but I think being explicit for some settings used by "nearby" entries make sense.

646

Most configurations using involving Ninja appear to use getCmakeWithNinjaBuildFactory. The ones that don't mostly also have make='ninja'.

647

My understanding is that "llvm" is not needed in the LLVM_ENABLE_PROJECTS list. I guess we could keep it, but I don't think we should keep the trailing semicolon.

[daltenty]

buildbot/osuosl/master/config/builders.py

646

Generator options already seem to exist in the ClangBuilder implementation, so I don't think we should be passing them here.

656

The binutils directory here is used for the gold plugin, so we don't need it for AIX.

657

Looks like the jobs property on the worker will set the lit threads, so we don't need that part I think.

buildbot/osuosl/master/config/workers.py

121

Just noting we'll have to boost this when we add the native builder for example (we're still waiting on more machine resources though).

[Xiangling_L]

Addressed the comments;

[hubert.reinterpretcast]

LGTM with some last changes based on @daltenty's comments.

buildbot/osuosl/master/config/builders.py

641

The make='ninja' line is not needed given @daltenty's comment.

646

@daltenty, just confirming you are saying that https://reviews.llvm.org/source/zorg/browse/main/zorg/buildbot/builders/ClangBuilder.py gives us Ninja by default?

[daltenty]

buildbot/osuosl/master/config/builders.py

646

@daltenty, just confirming you are saying that https://reviews.llvm.org/source/zorg/browse/main/zorg/buildbot/builders/ClangBuilder.py gives us Ninja by default?

Certainly seems that way, every generator option is hard coded to Ninja

[Xiangling_L]

Remove make='ninja';

[gkistanova]

Almost there. One more change it will be ready to land.

buildbot/osuosl/master/config/builders.py

649

No need to specify the LLVM_ENABLE_PROJECTS. The build factory will take care of this for you.

[Xiangling_L]

buildbot/osuosl/master/config/builders.py

649

Thanks, I will update the patch accordingly.

[Xiangling_L]

Removed redundant -DLLVM_ENABLE_PROJECTS option;

[gkistanova]

Thanks!

buildbot/osuosl/master/config/builders.py

'factory' : ClangBuilder.getClangCMakeBuildFactory(clean=False,
                         "-DLLVM_ENABLE_ASSERTIONS=On", "-DCMAKE_C_COMPILER=clang",
                         "-DCMAKE_CXX_COMPILER=clang++",
                         "-DCLANG_DEFAULT_LINKER=lld",
                         "-DCMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN:PATH=/opt/rh/devtoolset-7/root/usr",
                         "-DCMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN:PATH=/opt/rh/devtoolset-7/root/usr",
                         "-DLLVM_BINUTILS_INCDIR=/usr/include", "-DBUILD_SHARED_LIBS=ON", "-DLLVM_ENABLE_WERROR=ON",
                         '-DLLVM_LIT_ARGS=-vj 20'])},
 
+    {'name' : "clang-ppc64-aix-ppc64le",
+    'tags'  : ["clang", "aix", "ppc", "ppc64le"],
+    'workernames' : ["aix-1"],
+    'builddir': "clang-ppc64-aix-ppc64le",
+    'factory' : ClangBuilder.getClangCMakeBuildFactory(
+                    jobs=4,
+                    clean=False,
+                    checkout_clang_tools_extra=False,
+                    checkout_compiler_rt=False,
+                    checkout_lld=False,
+                    checkout_libcxx=False,
+                    useTwoStage=False,
+                    runTestSuite=False,
+                    stage1_config='Release',
+                    extra_cmake_args=[
+                        "-DLLVM_ENABLE_ASSERTIONS=On",
+                        "-DCMAKE_C_COMPILER=/opt/IBM/xlC/16.1.0/bin/xlclang",
+                        "-DCMAKE_CXX_COMPILER=/opt/IBM/xlC/16.1.0/bin/xlclang++",
+                        "-DCMAKE_C_FLAGS=-qmaxmem=-1\ -qarch=pwr8\ -qaltivec\ -qalias=noansi",
+                        "-DCMAKE_CXX_FLAGS=-qmaxmem=-1\ -qarch=pwr8\ -qaltivec\ -qalias=noansi",
+                        "-DPython3_EXECUTABLE:FILEPATH=/opt/freeware/bin/python3_64",
+                        "-DLLVM_ENABLE_ZLIB=OFF", "-DLLVM_APPEND_VC_REV=OFF",
+                        "-DLLVM_DEFAULT_TARGET_TRIPLE=powerpc64le-unknown-linux-gnu",
+                        "-DLLVM_ENABLE_WERROR=ON"])},
+
     {'name' : "clang-s390x-linux",
     'tags'  : ["clang"],
     'workernames' : ["systemz-1"],
     'builddir': "clang-s390x-linux",
     'factory' : ClangBuilder.getClangCMakeBuildFactory(
                     jobs=4,
                     clean=False,
                     checkout_lld=False,

buildbot/osuosl/master/config/workers.py

return [
         create_worker("sanitizer-buildbot4", properties={'jobs': 64}, max_builds=2),
         # Ubuntu 14.04 x86_64 6-core z440 workstation
         create_worker("sanitizer-buildbot6", properties={'jobs': 6}, max_builds=1),
         # Debian 7.7 x86_64 GCE instance
         create_worker("sanitizer-buildbot7", properties={'jobs': 64}, max_builds=3),
         # Debian 7.7 x86_64 GCE instance
         create_worker("sanitizer-buildbot8", properties={'jobs': 64}, max_builds=3),
 
+        # POWER 8 PowerPC AIX 7.2
+        create_worker("aix-1", properties={'jobs': 4}, max_builds=1),
+
         # IBM z13 (s390x), Ubuntu 16.04.2
         create_worker("systemz-1", properties={'jobs': 4, 'vcs_protocol': 'https'}, max_builds=4),
 
         # Windows Server 2012 x86_64 16-core GCE instance
         create_worker("sanitizer-windows", properties={'jobs': 16}, max_builds=1),
         # Windows Server 2012 x86_64 32-core GCE instance
         create_worker("windows-gcebot2", properties={'jobs': 32}, max_builds=1),