We're hitting EAGAIN on the upstream buildbots when constructing threads. Overall buildbot memory usage doesn't seem too high and we've set the number of allowed processes (NPROC) to unlimited. We can try reducing the number of jobs run specifically when running lld tests.
Details
Diff Detail
- Repository
- rZORG LLVM Github Zorg
Event Timeline
I don't think this is going to help, lit tasks are running inside the console Ninja pool so they're already limited to a single job. Instead, we would need to pass -j to lit through LIT_OPTS environment variable, see http://llvm.org/docs/CommandGuide/lit.html#general-options.
zorg/buildbot/builders/FuchsiaBuilder.py | ||
---|---|---|
156 | I've never seen this option used with =, have you checked whether this works? The quotes shouldn't be needed either. |
zorg/buildbot/builders/FuchsiaBuilder.py | ||
---|---|---|
156 | I think = should work. Never used LIT_OPTS before and I just copied from an example I found in some lit tests. ninja check-clang seems to run much slower with this so I would *think* it's working. Removed it regardless since -j32 just seems cleaner. |
Specifying 32 jobs as a hard coded value in the build factory does not look right. You may want to parametrize this somehow.
zorg/buildbot/builders/FuchsiaBuilder.py | ||
---|---|---|
137 | I know that this is how it was, but since you changed this line, could you use the str.format() interface instead, please? We are Python 3.x only now. No need to keep the backward compatibility. | |
156 | Isn’t the job number specific to a particular builder hardware? |
Updated such that the number of jobs is passed as an argument to getFuchsiaToolchainBuildFactory.
zorg/buildbot/builders/FuchsiaBuilder.py | ||
---|---|---|
156 | From what I can tell, the job number can be freely adjusted to whatever value but is only limited by the number of cores on the hardware. My intention was to have all current fuchsia toolchain builders to run at most 32 jobs for specifically the check-lld step. Is there a more proper way to set this other than in the builder factory? |
I know that this is how it was, but since you changed this line, could you use the str.format() interface instead, please?
We are Python 3.x only now. No need to keep the backward compatibility.