[zorg] Add Clang ppc64le cross-targeting buildbot on AIX

[Xiangling_L]

@gkistanova Hi Galina, we are trying to set up a cross-targeting buildbot for AIX platform. And I sent you an email on Wed, Jan 20th, 2021 requesting access for the new build bot.

Can you please confirm that you received it. Thanks.

[hubert.reinterpretcast]

buildbot/osuosl/master/config/builders.py

611

I think having the host platform described more clearly makes the non-native default target more obvious.

632

There's at least one typo to correct. Just use the value from a build for ppc64le Linux.

[Xiangling_L]

Addressed the comments;

[hubert.reinterpretcast]

buildbot/osuosl/master/config/builders.py

614

Please consider changing this to match the name.

628–629

I think you reported "offline" that you needed to add -qaltivec.

[Xiangling_L]

buildbot/osuosl/master/config/builders.py

614

Oops... I missed this.

628–629

Thanks for reminding. I will update this. I was waiting for build to finish. It usually took around 6 hours.

[Xiangling_L]

Update build dir name accordingly;
Ad -qaltivec into cmake config;

[hubert.reinterpretcast]

@Xiangling_L, sorry if this is adding extra respin test cycles. I've done as much review this time around as I can do for the patch as it is currently posted. Thanks.

buildbot/osuosl/master/config/builders.py

612

Minor comment: Usage of the "ppc" tag seems to imply it is a generic tag for anything Power-related. It makes sense to add it here.

620

Not sure if other people have a different opinion about this, but I think being explicit for some settings used by "nearby" entries make sense.

623

Most configurations using involving Ninja appear to use getCmakeWithNinjaBuildFactory. The ones that don't mostly also have make='ninja'.

624

My understanding is that "llvm" is not needed in the LLVM_ENABLE_PROJECTS list. I guess we could keep it, but I don't think we should keep the trailing semicolon.

[daltenty]

buildbot/osuosl/master/config/builders.py

623

Generator options already seem to exist in the ClangBuilder implementation, so I don't think we should be passing them here.

633

The binutils directory here is used for the gold plugin, so we don't need it for AIX.

634

Looks like the jobs property on the worker will set the lit threads, so we don't need that part I think.

buildbot/osuosl/master/config/workers.py

127

Just noting we'll have to boost this when we add the native builder for example (we're still waiting on more machine resources though).

[Xiangling_L]

Addressed the comments;

[hubert.reinterpretcast]

LGTM with some last changes based on @daltenty's comments.

buildbot/osuosl/master/config/builders.py

618

The make='ninja' line is not needed given @daltenty's comment.

623

@daltenty, just confirming you are saying that https://reviews.llvm.org/source/zorg/browse/main/zorg/buildbot/builders/ClangBuilder.py gives us Ninja by default?

[daltenty]

buildbot/osuosl/master/config/builders.py

623

@daltenty, just confirming you are saying that https://reviews.llvm.org/source/zorg/browse/main/zorg/buildbot/builders/ClangBuilder.py gives us Ninja by default?

Certainly seems that way, every generator option is hard coded to Ninja

[Xiangling_L]

Remove make='ninja';

[gkistanova]

Almost there. One more change it will be ready to land.

buildbot/osuosl/master/config/builders.py

626

No need to specify the LLVM_ENABLE_PROJECTS. The build factory will take care of this for you.

[Xiangling_L]

buildbot/osuosl/master/config/builders.py

626

Thanks, I will update the patch accordingly.

[Xiangling_L]

Removed redundant -DLLVM_ENABLE_PROJECTS option;

[gkistanova]

Thanks!

buildbot/osuosl/master/config/builders.py

'factory' : ClangBuilder.getClangCMakeBuildFactory(clean=False,
                         "-DLLVM_ENABLE_ASSERTIONS=On", "-DCMAKE_C_COMPILER=clang",
                         "-DCMAKE_CXX_COMPILER=clang++",
                         "-DCLANG_DEFAULT_LINKER=lld",
                         "-DCMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN:PATH=/opt/rh/devtoolset-7/root/usr",
                         "-DCMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN:PATH=/opt/rh/devtoolset-7/root/usr",
                         "-DLLVM_BINUTILS_INCDIR=/usr/include", "-DBUILD_SHARED_LIBS=ON", "-DLLVM_ENABLE_WERROR=ON",
                         '-DLLVM_LIT_ARGS=-vj 20'])},
 
+    {'name' : "clang-aix-linux",
+    'tags'  : ["clang", "aix", "ppc64le"],
+    'workernames' : ["aix-1"],
+    'builddir': "clang-aix-linux",
+    'factory' : ClangBuilder.getClangCMakeBuildFactory(
+                    jobs=4,
+                    clean=False,
+                    checkout_clang_tools_extra=False,
+                    checkout_compiler_rt=False,
+                    checkout_lld=False,
+                    stage1_config='Release',
+                    extra_cmake_args=[
+                        "-G", "Ninja",
+                        "-DLLVM_ENABLE_PROJECTS=clang;llvm;",
+                        "-DLLVM_ENABLE_ASSERTIONS=On",
+                        "-DCMAKE_C_COMPILER=/opt/IBM/xlC/16.1.0/bin/xlclang",
+                        "-DCMAKE_CXX_COMPILER=/opt/IBM/xlC/16.1.0/bin/xlclang++",
+                        "-DCMAKE_C_FLAGS=-qmaxmem=-1\ -qarch=pwr8\ -qalias=noansi",
+                        "-DCMAKE_CXX_FLAGS=-qmaxmem=-1\ -qarch=pwr8\ -qalias=noansi",
+                        "-DPython3_EXECUTABLE:FILEPATH=/opt/freeware/bin/python3_64",
+                        "-DLLVM_ENABLE_ZLIB=OFF", "-DLLVM_APPEND_VC_REV=OFF",
+                        "-DLLVM_DEFAULT_TARGET_TRIPLE=powerpc6le-gnu-linux",
+                        "-DLLVM_BINUTILS_INCDIR=/usr/include", "-DLLVM_ENABLE_WERROR=ON",
+                        "-DLLVM_LIT_ARGS=--threads=4\ -v"])},
+
     {'name' : "clang-s390x-linux",
     'tags'  : ["clang"],
     'workernames' : ["systemz-1"],
     'builddir': "clang-s390x-linux",
     'factory' : ClangBuilder.getClangCMakeBuildFactory(
                     jobs=4,
                     clean=False,
                     checkout_lld=False,

buildbot/osuosl/master/config/workers.py

return [
         create_worker("sanitizer-buildbot4", properties={'jobs': 64}, max_builds=2),
         # Ubuntu 14.04 x86_64 6-core z440 workstation
         create_worker("sanitizer-buildbot6", properties={'jobs': 6}, max_builds=1),
         # Debian 7.7 x86_64 GCE instance
         create_worker("sanitizer-buildbot7", properties={'jobs': 64}, max_builds=3),
         # Debian 7.7 x86_64 GCE instance
         create_worker("sanitizer-buildbot8", properties={'jobs': 64}, max_builds=3),
 
+        # POWER 8 PowerPC AIX 7.2
+        create_worker("aix-1", properties={'jobs': 4}, max_builds=1),
+
         # IBM z13 (s390x), Ubuntu 16.04.2
         create_worker("systemz-1", properties={'jobs': 4, 'vcs_protocol': 'https'}, max_builds=4),
 
         # Windows Server 2012 x86_64 16-core GCE instance
         create_worker("sanitizer-windows", properties={'jobs': 16}, max_builds=1),
         # Windows Server 2012 x86_64 32-core GCE instance
         create_worker("windows-gcebot2", properties={'jobs': 32}, max_builds=1),