Page MenuHomePhabricator

[zorg] Add buildbot for HIP
ClosedPublic

Authored by ashi1 on Apr 5 2021, 11:42 AM.

Details

Summary

Build HIP tests from llvm-test-suite using AMDGPU.
There will be an upcoming patch to add HIP test to llvm-test-suite/External/HIP.

Diff Detail

Event Timeline

ashi1 created this revision.Apr 5 2021, 11:42 AM
ashi1 requested review of this revision.Apr 5 2021, 11:42 AM
ashi1 added inline comments.
buildbot/osuosl/master/config/builders.py
1697

@gkistanova - is it okay to temporarily use an external script, while we are bringing up the build? Or does it need to go into zorg first? Thank you.

Everything else looks Ok.

buildbot/osuosl/master/config/builders.py
1697

This is fine, but please provide here a real path where the script will be placed to when ready. Just have it as a local uncommitted patch in the builder source tree for now.

ashi1 marked an inline comment as done.Apr 7 2021, 11:58 AM

This HIP builder is dependent on two other patches:

  1. hip-build.sh script found at https://reviews.llvm.org/D100060
  2. llvm-test-suite - adding HIP tests found at https://reviews.llvm.org/D99997
buildbot/osuosl/master/config/builders.py
1697

That sounds good. I've opened a patch for that script (although still under-development):
https://reviews.llvm.org/D100060

ashi1 added a reviewer: tra.Apr 9 2021, 11:22 AM
ashi1 marked an inline comment as done.
ashi1 added inline comments.Apr 9 2021, 11:25 AM
buildbot/osuosl/master/config/builders.py
1697

AnnotatedBuilder.py seems to enforce running python on scripts in relative local paths. Should I wrap my bash script with a python script so that it can run the bash script? I noticed cuda also using full path /buildbot/cuda-build outside of zorg. Thanks.

@gkistanova , is it okay to submit this with a full system path temporarily? /buildbot/hip-build.sh? Or do we have to switch to a python script?

No. Cuda builders got that in the code base as a temporary solution while they are working on their annotated script. Exactly the same arguments as yours. And I see that being confusing, and don't think it worth it after all.

To get it working in the production you need to add the support for shell scripts anyway. With that in place you could put your script in the right place and keep working on that while your builder is staged. It seems the right thing to do.

Please feel free to ask if you have questions or will need help with adding the shell script support.

ashi1 added a comment.EditedApr 16 2021, 11:17 AM

No. Cuda builders got that in the code base as a temporary solution while they are working on their annotated script. Exactly the same arguments as yours. And I see that being confusing, and don't think it worth it after all.

To get it working in the production you need to add the support for shell scripts anyway. With that in place you could put your script in the right place and keep working on that while your builder is staged. It seems the right thing to do.

Please feel free to ask if you have questions or will need help with adding the shell script support.

Hi Galina, I've opened a small patch that should execute .py scripts with python, and all other scripts directly. Is there any public buildbot test that ensures the change works, I won't have access to all the various environments using AnnotatedBuilder.py. Thank you.
https://reviews.llvm.org/D100666

ashi1 updated this revision to Diff 339634.EditedApr 22 2021, 8:00 AM

Hi @gkistanova, I've changed the buildbot to use a local relative path (in annotated/hip-build.sh dir).
I will keep it as a local uncommitted script during bring-up. D100666 was submitted, and script_interpreter="python" is disabled for this bash script.

Looks good with a nitpick. Please see my comment inline.

I will keep it as a local uncommitted script during bring-up.

This is fine, but please make sure everything is complete and committed before you will be ready to move the bot to the production.

buildbot/osuosl/master/config/builders.py
1699

How about script_interpreter=None instead?

ashi1 updated this revision to Diff 339759.Apr 22 2021, 12:39 PM
ashi1 marked an inline comment as done.

Revised to @gkistanova 's comments.

This revision is now accepted and ready to land.Apr 22 2021, 9:50 PM
This revision was landed with ongoing or failed builds.Apr 23 2021, 7:11 AM
This revision was automatically updated to reflect the committed changes.