Add a new builder and a new worker for LoongArch.
The worker runs build, check-all and test-suite on Loongson-3C5000L-LL
machine.
Details
Diff Detail
- Repository
- rZORG LLVM Github Zorg
- Build Status
Buildable 199691 Build 303238: arc lint + arc unit
Event Timeline
Move @wangleiat and me to subscribers as we have no authority to review buildbot changes.
Add some guys who help reviewing LoongArch changes to the subscriber list. I think they may be interested in this change. Thanks.
Thanks for the builder contribution, and congratulations! I'm very pleased to be a part of this bring-up effort. Graduating from experimental status Soon™️ :-)
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
814 | It seems single quotes are used for string literals elsewhere. Consistency would be nice. | |
825 | I currently build AArch64 and RISCV too, as they are prominent arches too. Maybe adding Mips would be nice as well, AFAIK Loongson claimed to "maintain the previous MIPS models for 10 years" a few years ago. | |
buildbot/osuosl/master/config/workers.py | ||
67 | mention CLFS and give a link to the CLFS repo, given it's not a widely-known distro spin? |
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
825 | https://llvm.org/docs/HowToAddABuilder.html#best-practices-for-configuring-a-fast-builder says:
I think most targets including X86/AArch64/RISCV/Mips are already coverred by other bots. To make the bot faster, we should turn on as few as targets here. |
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
825 |
Hmm I must be so coffee-deprived while writing that comment that I forgot this. No problem only building x86 and ourselves then. |
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
814 | Okay so the job name and builder name is way too similar then. I'd suggest picking a name for the builder instance, such as loongson-loongarch64-clfs-clang-01, while keeping the job name as-is. See how e.g. Linaro builders all have names like linaro-ARCH-OS-CC-NUMBER. (The architecture name is considered by many to be too loooong, so you may choose to use a shorter one like loongson-loong64-xxx instead, the important part is to clearly convey the information that this is a LoongArch builder contributed by Loongson running CLFS & Clang.) |
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
814 | Thanks. I will change the worker name to "loongson-loongarch64-clfs-clang-01" according to your suggestion. The builder name will be changed to "clang-loongarch64-linux", thus it is not too similar to the worker name, and also keeps consistency with other builders. |
hi, @gkistanova , I sent the buildbot-worker-access-name and buildbot-worker-access-password to your email (gkistanova@gmail.com) on November 25, 2022. And updated the buildbot-worker-access-name to “loongson-loongarch64-clfs-clang-01” on November 28, 2022. I would like to confirm whether you have received these two emails. Thanks!
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
825 | On the Arm32 bots, we really only build Arm, not even x86. Of all targets, x86 is perhaps the one with the most coverage, so you really don't need to build it on other targets. | |
827 | Avoid adding temporal state on the commit. Whenever you fix those issues you won't come back here and update (or at least, shouldn't). These files are rarely changed even when the bots themselves change. The only reason you'll have to change this again is when LoongArch moves out of experimental, or if you want to build more stuff on the same build. |
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
827 | No, I mean just remove the comments because what compiler you use and why is a matter for the bot maintainer only. If you need a special compiler, go for it, don't need to explain how you got there. :) You should create issues (either llvm or your local bugtracker) to fix those problems, and before we move out of experimental, you need to be able to build it with any GCC/clang that we officially support (because other bots will start building your backend and running your tests), but until then, take your time, fix your bugs, etc. Changes to this file are rare and require additional screening from other bot owners, Galina, as well as a build master restart (by Galina), so you shouldn't add comments about internal infrastructure. |
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
825 | So we can define: | |
827 | Sorry, I’m a little confused. The default gcc13 in’/usr/bin’ on the bot has the bug mentioned here https://github.com/llvm/llvm-project/issues/59227. But if it was fixed in future, we may choose to use it but not ‘/home/loongson/gcc/gcc-install/bin/gcc'. This would cause a change to this file again. So why not remove these 2 lines this time and make the make the custom gcc as default? '-DCMAKE_C_COMPILER=/home/loongson/gcc/gcc-install/bin/gcc', And I think the bug of gcc13 is not LoongArch specific because we met the same issue on X86. What’s more, we find the specific gcc commit causing introducing this bug. This commit seems is not arch related. PS: The reason why no bots report such issue is that no bots use latest gcc13 and run check-clang in release build. |
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
825 | If that works, sure. | |
827 | Whatever works on your system. I normally just install clang from packages and use that to build, but any compiler you have that works should be fine. If GCC 13 doesn't work then this is a bug, as the current requirements state GCC >= 7. But that's not for this patch. Here you just build however you can. |
buildbot/osuosl/master/config/builders.py | ||
---|---|---|
825 |
I just confirmed this works both for cross and native builds. Will save so much time each run... | |
827 | IMO, I don't think hard-coding the absolute path is a good idea either, due to the involved process hinted by @rengolin; as the builder is a CLFS installation which is already hugely non-standard by itself, I'd say just swap out the gcc and g++ commands and symlink the "fixed" version into place before the situation is resolved. (CLFS for LoongArch is AFAIK basically maintained by one person, with dubious automation and reproducibility, if at all. I remember the CLFS for LoongArch maintainer said he has a huge script automating everything, but it's never shared on the project repo at least not yet.) And, as the nonstandard toolchain is a problem in need of fixing, I'd suggest following the advice of making a public issue, so people don't forget to revisit later. |
Hi @gkistanova, this change has been accepted and could you please take a look? We hope the bot for LoongArch could be setted up as soon as possiple so that we can request to move LoongArch out of experimental before LLVM16 release branch is created. Thanks.
Galina may be on extended holiday.
When maintaining bots at Linaro, we had a local Buildmaster where we could test our changes and new bots. If you have one, you could expose that for now, so that you can share build failures in case of breakages from others, until Galina comes back.
Thanks Renato.
If self-maintained buildbot can meet the requirement for a back-end to be promoted to official, we would have a try.
It's a bit of a grey area, but the key requirement is:
"The build target check-all must pass with the new target built, and where applicable, the test-suite must also pass without errors, in at least one configuration (publicly demonstrated, for example, via buildbots)."
It doesn't specify where.
The tests must be "demonstrated on a public buildbot", which can be tied to your own Buildmaster, or in Github Actions, Jenkins, Buildkite, DevOps, etc.
The main reason to get this bot in the upstream Buildmaster is that users get warned on breakages and provides a single entry point to CI, so that people can find your builders.
We've been trying to spread the cost of CI for many years but it has proven a hard problem for compiler developers to solve. :)
It seems single quotes are used for string literals elsewhere. Consistency would be nice.