This is an archive of the discontinued LLVM Phabricator instance.

Differential D84956

[zorg] Add Flang and MLIR PowerPC buildbot on Red Hat
ClosedPublic

Authored by zacharyselk on Jul 30 2020, 10:51 AM.

Details

Reviewers
gkistanova
sscalpone
zacharyselk
DavidTruby
Commits
rZORG2e57006f2b77: PowerPC Flang+MLIR buildbot cmake fix
rZORG714b345535c0: Add Flang and MLIR PowerPC buildbot on Red Hat

Diff Detail

Event Timeline

zacharyselk created this revision.Jul 30 2020, 10:51 AM
Herald added a reviewer: DavidTruby. · View Herald TranscriptJul 30 2020, 10:51 AM
Herald added a reviewer: sscalpone. · View Herald Transcript
Herald added subscribers: aaron.ballman, rriddle, shchenz, nemanjai. · View Herald Transcript
zacharyselk requested review of this revision.Jul 30 2020, 10:51 AM
Herald added subscribers: stephenneuendorffer, • wuzish. · View Herald TranscriptJul 30 2020, 10:51 AM
Harbormaster completed remote builds in B66426: Diff 281984.Jul 30 2020, 10:51 AM
zacharyselk updated this revision to Diff 282059.Jul 30 2020, 3:04 PM
  • Removing explicit job count
Harbormaster completed remote builds in B66472: Diff 282059.Jul 30 2020, 3:04 PM
nemanjai added inline comments.Jul 31 2020, 5:06 AM
buildbot/osuosl/master/config/builders.py
1594

Shouldn't this be --threads=256 instead of -j256?

nemanjai added a comment.Jul 31 2020, 5:41 AM

Since we are adding another bot, we need to adjust the number of threads that are used by LIT for all the bots running on that machine. Thanks.

buildbot/osuosl/master/config/builders.py
519

Please add LLVM_LIT_ARGS=-v --threads=256

558

Please add LLVM_LIT_ARGS=-v --threads=256

1069

Please change this from 10 to 256.

DavidTruby resigned from this revision.Jul 31 2020, 5:51 AM
zacharyselk updated this revision to Diff 282216.Jul 31 2020, 7:22 AM
  • Adjusted lit test job numbers
Harbormaster completed remote builds in B66557: Diff 282216.Jul 31 2020, 7:22 AM
gkistanova accepted this revision.Aug 3 2020, 11:48 AM

Looks Ok with one problem.

buildbot/osuosl/master/config/slaves.py
80

Did you miss a comma at the end?

This revision is now accepted and ready to land.Aug 3 2020, 11:48 AM
zacharyselk updated this revision to Diff 282901.Aug 4 2020, 6:42 AM
zacharyselk marked 2 inline comments as done.
  • Added a forgotten comma
Harbormaster completed remote builds in B66928: Diff 282901.Aug 4 2020, 6:42 AM
gkistanova accepted this revision.Aug 7 2020, 9:49 PM

Thanks! Please feel free to commit.

zacharyselk accepted this revision.Aug 7 2020, 9:57 PM

Great! I don't have commit rights, so if someone would be able to commit for me that would be fantastic.

buildbot/osuosl/master/config/slaves.py
80

Yes I did, I am surprised that the Harbormaster did not complain about that. I will amend the diff.

Closed by commit rZORG714b345535c0: Add Flang and MLIR PowerPC buildbot on Red Hat (authored by zacharyselk, committed by gkistanova). · Explain WhyAug 11 2020, 10:50 PM
This revision was automatically updated to reflect the committed changes.
gkistanova added a commit: rZORG714b345535c0: Add Flang and MLIR PowerPC buildbot on Red Hat.
zacharyselk reopened this revision.Aug 12 2020, 11:28 AM
This revision is now accepted and ready to land.Aug 12 2020, 11:28 AM
zacharyselk updated this revision to Diff 285149.Aug 12 2020, 11:28 AM
  • Removing explicit job count
  • Adjusted lit test job numbers
  • Added a forgotten comma
  • Removed quotes that were causing issues
Harbormaster completed remote builds in B68149: Diff 285149.Aug 12 2020, 11:28 AM
Closed by commit rZORG2e57006f2b77: PowerPC Flang+MLIR buildbot cmake fix (authored by zacharyselk). · Explain WhyAug 13 2020, 5:22 PM
This revision was automatically updated to reflect the committed changes.
zacharyselk added a commit: rZORG2e57006f2b77: PowerPC Flang+MLIR buildbot cmake fix.

Revision Contents

PathSize
buildbot/
osuosl/
master/
config/
44 lines
1 line

Diff 281984

buildbot/osuosl/master/config/builders.py

Show First 20 Lines • Show All 510 Lines • ▼ Show 20 Lines return [
{'name' : "clang-ppc64le-linux-multistage", {'name' : "clang-ppc64le-linux-multistage",
'slavenames' :["ppc64le-clang-multistage-test"], 'slavenames' :["ppc64le-clang-multistage-test"],
'builddir' :"clang-ppc64le-multistage", 'builddir' :"clang-ppc64le-multistage",
'factory' : ClangBuilder.getClangCMakeBuildFactory(clean=False, 'factory' : ClangBuilder.getClangCMakeBuildFactory(clean=False,
checkout_lld=False, checkout_lld=False,
useTwoStage=True, useTwoStage=True,
stage1_config='Release', stage1_config='Release',
stage2_config='Release', stage2_config='Release',
extra_cmake_args=['-DLLVM_ENABLE_ASSERTIONS=ON', '-DBUILD_SHARED_LIBS=ON']), extra_cmake_args=['-DLLVM_ENABLE_ASSERTIONS=ON', '-DBUILD_SHARED_LIBS=ON']),
nemanjaiUnsubmitted
Not Done
ReplyInline Actions

Please add LLVM_LIT_ARGS=-v --threads=256

nemanjai: Please add `LLVM_LIT_ARGS=-v --threads=256`
'category' : 'clang'}, 'category' : 'clang'},
{'name': "clang-ppc64be-linux", {'name': "clang-ppc64be-linux",
'slavenames':["ppc64be-clang-test"], 'slavenames':["ppc64be-clang-test"],
'builddir':"clang-ppc64be", 'builddir':"clang-ppc64be",
'factory' : ClangBuilder.getClangCMakeBuildFactory(clean=False, 'factory' : ClangBuilder.getClangCMakeBuildFactory(clean=False,
checkout_lld=False, checkout_lld=False,
useTwoStage=False, useTwoStage=False,
Show All 22 Lines return [
useTwoStage=False, useTwoStage=False,
runTestSuite=True, runTestSuite=True,
stage1_config='Release', stage1_config='Release',
nt_flags=['--threads=16', '--build-threads=16'], nt_flags=['--threads=16', '--build-threads=16'],
extra_cmake_args=["-DLLVM_ENABLE_ASSERTIONS=On", "-DCMAKE_C_COMPILER=clang", extra_cmake_args=["-DLLVM_ENABLE_ASSERTIONS=On", "-DCMAKE_C_COMPILER=clang",
"-DCMAKE_CXX_COMPILER=clang++", "-DCMAKE_CXX_COMPILER=clang++",
"-DCMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN:PATH=/opt/rh/devtoolset-7/root/usr", "-DCMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN:PATH=/opt/rh/devtoolset-7/root/usr",
"-DCMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN:PATH=/opt/rh/devtoolset-7/root/usr", "-DCMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN:PATH=/opt/rh/devtoolset-7/root/usr",
"-DLLVM_BINUTILS_INCDIR=/usr/include", "-DBUILD_SHARED_LIBS=ON", "-DLLVM_ENABLE_WERROR=ON"]), "-DLLVM_BINUTILS_INCDIR=/usr/include", "-DBUILD_SHARED_LIBS=ON", "-DLLVM_ENABLE_WERROR=ON"]),
nemanjaiUnsubmitted
Done
ReplyInline Actions

Please add LLVM_LIT_ARGS=-v --threads=256

nemanjai: Please add `LLVM_LIT_ARGS=-v --threads=256`
'category' : 'clang'}, 'category' : 'clang'},
{'name': "clang-s390x-linux", {'name': "clang-s390x-linux",
'slavenames':["systemz-1"], 'slavenames':["systemz-1"],
'builddir':"clang-s390x-linux", 'builddir':"clang-s390x-linux",
'factory' : ClangBuilder.getClangCMakeBuildFactory(jobs=4, 'factory' : ClangBuilder.getClangCMakeBuildFactory(jobs=4,
clean=False, clean=False,
checkout_lld=False, checkout_lld=False,
▲ Show 20 Lines • Show All 494 Lines • ▼ Show 20 Lines return [
'category' : 'lld'}, 'category' : 'lld'},
{'name' : "ppc64le-lld-multistage-test", {'name' : "ppc64le-lld-multistage-test",
'slavenames' : ["ppc64le-lld-multistage-test"], 'slavenames' : ["ppc64le-lld-multistage-test"],
'builddir' : "ppc64le-lld-multistage-test", 'builddir' : "ppc64le-lld-multistage-test",
'factory': UnifiedTreeBuilder.getCmakeWithNinjaMultistageBuildFactory( 'factory': UnifiedTreeBuilder.getCmakeWithNinjaMultistageBuildFactory(
extra_configure_args=[ extra_configure_args=[
'-DLLVM_ENABLE_ASSERTIONS=ON', '-DLLVM_ENABLE_ASSERTIONS=ON',
'-DLLVM_LIT_ARGS="-svj 10"'], '-DLLVM_LIT_ARGS="-svj 10"'],
nemanjaiUnsubmitted
Done
ReplyInline Actions

Please change this from 10 to 256.

nemanjai: Please change this from 10 to 256.
depends_on_projects=['llvm', 'clang', 'lld']), depends_on_projects=['llvm', 'clang', 'lld']),
'category' : 'lld'}, 'category' : 'lld'},
{'name' : "lld-x86_64-ubuntu-fast", {'name' : "lld-x86_64-ubuntu-fast",
'slavenames' : ["as-builder-4"], 'slavenames' : ["as-builder-4"],
'builddir' : "lld-x86_64-ubuntu-fast", 'builddir' : "lld-x86_64-ubuntu-fast",
'mergeRequests': False, 'mergeRequests': False,
'factory': UnifiedTreeBuilder.getCmakeWithNinjaBuildFactory( 'factory': UnifiedTreeBuilder.getCmakeWithNinjaBuildFactory(
Show All 39 Lines return [
depends_on_projects=['llvm','mlir'], depends_on_projects=['llvm','mlir'],
vs="autodetect", vs="autodetect",
checks=['check-mlir'], checks=['check-mlir'],
extra_configure_args=[ extra_configure_args=[
"-DLLVM_BUILD_EXAMPLES=ON", "-DLLVM_BUILD_EXAMPLES=ON",
"-DLLVM_ENABLE_PROJECTS=mlir", "-DLLVM_ENABLE_PROJECTS=mlir",
"-DLLVM_TARGETS_TO_BUILD='host;NVPTX;AMDGPU'", "-DLLVM_TARGETS_TO_BUILD='host;NVPTX;AMDGPU'",
])}, ])},
{'name': 'ppc64le-mlir-rhel-clang',
'mergeRquests': False,
'slavenames': ['ppc64le-flang+mlir-rhel-test'],
'builddir': 'ppc64le-mlir-rhel-clang-build',
'factory': UnifiedTreeBuilder.getCmakeWithNinjaBuildFactory(
clean=True,
depends_on_projects=['llvm', 'mlir'],
checks=['check-mlir'],
extra_configure_args=[
'-DLLVM_TARGETS_TO_BUILD="PowerPC"',
'-DLLVM_INSTALL_UTILS=ON',
'-DCMAKE_CXX_STANDARD=17',
'-DLLVM_ENABLE_PROJECTS="mlir"',
'-DLLVM_LIT_ARGS=-v -j256'
],
env={
'CC': 'clang',
'CXX': 'clang++',
'LD': 'lld'
}),
'category': 'mlir'},
] ]
# Sanitizer builders. # Sanitizer builders.
def _get_sanitizer_builders(): def _get_sanitizer_builders():
return [ return [
{'name': "sanitizer-x86_64-linux", {'name': "sanitizer-x86_64-linux",
'slavenames' :["sanitizer-buildbot1", "sanitizer-buildbot2"], 'slavenames' :["sanitizer-buildbot1", "sanitizer-buildbot2"],
'builddir': "sanitizer-x86_64-linux", 'builddir': "sanitizer-x86_64-linux",
'factory': SanitizerBuilder.getSanitizerBuildFactory()}, 'factory': SanitizerBuilder.getSanitizerBuildFactory()},
{'name': "sanitizer-x86_64-linux-fast", {'name': "sanitizer-x86_64-linux-fast",
'slavenames' :["sanitizer-buildbot1", "sanitizer-buildbot2"], 'slavenames' :["sanitizer-buildbot1", "sanitizer-buildbot2"],
'builddir': "sanitizer-x86_64-linux-fast", 'builddir': "sanitizer-x86_64-linux-fast",
'factory': SanitizerBuilder.getSanitizerBuildFactory()}, 'factory': SanitizerBuilder.getSanitizerBuildFactory()},
▲ Show 20 Lines • Show All 413 Lines • ▼ Show 20 Lines return [
depends_on_projects=['llvm','mlir','clang','flang'], depends_on_projects=['llvm','mlir','clang','flang'],
extra_configure_args=[ extra_configure_args=[
"-DLLVM_TARGETS_TO_BUILD=X86", "-DLLVM_TARGETS_TO_BUILD=X86",
"-DLLVM_INSTALL_UTILS=ON", "-DLLVM_INSTALL_UTILS=ON",
"-DCMAKE_CXX_STANDARD=17", "-DCMAKE_CXX_STANDARD=17",
], ],
), ),
'category' : 'flang'}, 'category' : 'flang'},
{'name': 'ppc64le-flang-rhel-clang',
'mergeRquests': False,
'slavenames': ['ppc64le-flang+mlir-rhel-test'],
'builddir': 'ppc64le-flang-rhel-clang-build',
'factory': UnifiedTreeBuilder.getCmakeWithNinjaBuildFactory(
clean=True,
depends_on_projects=['llvm', 'mlir', 'clang', 'flang'],
checks=['check-flang'],
extra_configure_args=[
'-DLLVM_TARGETS_TO_BUILD="PowerPC"',
'-DLLVM_INSTALL_UTILS=ON',
'-DCMAKE_CXX_STANDARD=17',
'-DLLVM_ENABLE_PROJECTS="flang"',
'-DLLVM_LIT_ARGS=-v -j256'
nemanjaiUnsubmitted
Not Done
ReplyInline Actions

Shouldn't this be --threads=256 instead of -j256?

nemanjai: Shouldn't this be `--threads=256` instead of `-j256`?
],
env={
'CC': 'clang',
'CXX': 'clang++',
'LD': 'lld'
}),
'category': 'flang'},
] ]
# Experimental and stopped builders # Experimental and stopped builders
def _get_on_demand_builders(): def _get_on_demand_builders():
return [ return [
] ]
def _get_experimental_scheduled_builders(): def _get_experimental_scheduled_builders():
▲ Show 20 Lines • Show All 222 Lines • Show Last 20 Lines

buildbot/osuosl/master/config/slaves.py

Show First 20 Lines • Show All 71 Lines • ▼ Show 20 Lines return [
# POWER 8 PowerPC little endian (powerpc64le) # POWER 8 PowerPC little endian (powerpc64le)
create_slave("ppc64le-clang-test", properties={'jobs': 4}, max_builds=1), create_slave("ppc64le-clang-test", properties={'jobs': 4}, max_builds=1),
create_slave("ppc64le-clang-lnt-test", properties={'jobs': 8}, max_builds=1), create_slave("ppc64le-clang-lnt-test", properties={'jobs': 8}, max_builds=1),
create_slave("ppc64le-clang-multistage-test", properties={'jobs': 8}, max_builds=1), create_slave("ppc64le-clang-multistage-test", properties={'jobs': 8}, max_builds=1),
create_slave("ppc64le-sanitizer", properties={'jobs': 4}, max_builds=1), create_slave("ppc64le-sanitizer", properties={'jobs': 4}, max_builds=1),
create_slave("ppc64le-lld-multistage-test", properties={'jobs': 40}, max_builds=1), create_slave("ppc64le-lld-multistage-test", properties={'jobs': 40}, max_builds=1),
create_slave("ppc64le-clang-rhel-test", properties={'jobs': 4}, max_builds=1), create_slave("ppc64le-clang-rhel-test", properties={'jobs': 4}, max_builds=1),
create_slave("ppc64le-flang+mlir-rhel-test", properties={'jobs': 1536}, max_builds=1)
gkistanovaUnsubmitted
Not Done
ReplyInline Actions

Did you miss a comma at the end?

gkistanova: Did you miss a comma at the end?
zacharyselkAuthorUnsubmitted
Not Done
ReplyInline Actions

Yes I did, I am surprised that the Harbormaster did not complain about that. I will amend the diff.

zacharyselk: Yes I did, I am surprised that the Harbormaster did not complain about that. I will amend the…
# Ubuntu x86-64, Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz # Ubuntu x86-64, Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz
create_slave("hexagon-build-02", properties={'jobs': 12, 'loadaverage': 32}, create_slave("hexagon-build-02", properties={'jobs': 12, 'loadaverage': 32},
max_builds=1), max_builds=1),
# Ubuntu x86-64, Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz # Ubuntu x86-64, Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz
create_slave("hexagon-build-03", properties={'jobs': 16, 'loadaverage':32}, create_slave("hexagon-build-03", properties={'jobs': 16, 'loadaverage':32},
max_builds=1), max_builds=1),
▲ Show 20 Lines • Show All 167 Lines • Show Last 20 Lines