Page MenuHomePhabricator

[lit] Raise the default soft process limit when possible
ClosedPublic

Authored by hfinkel on Oct 2 2015, 10:26 AM.

Details

Summary

It is common to have a default soft process limit, at least on some families of Linux distributions, of 1024. This is normally more than enough, but if you have many cores, and you're running tests that create many threads, this can become a problem. My POWER7 development machine has 48 cores, and when running the lld regression tests, which often want to create up to 48 threads, I run into problems. lit, by default, will want to run 48 tests in parallel, and 48*48 < 1024, and so many tests fail like this:

terminate called after throwing an instance of 'std::system_error'

what():  Resource temporarily unavailable

or lit fails like this when launching a test:

OSError: [Errno 11] Resource temporarily unavailable

lit can easily detect this situation and attempt to repair it before launching tests (by raising the soft process limit to something that will allow ncpus^2 threads to be created), and should do so to prevent spurious test failures.

This is the follow-up to this thread: http://lists.llvm.org/pipermail/llvm-dev/2015-October/090942.html

Diff Detail

Event Timeline

hfinkel updated this revision to Diff 36374.Oct 2 2015, 10:26 AM
hfinkel retitled this revision from to [lit] Raise the default soft process limit when possible.
hfinkel updated this object.
hfinkel added reviewers: ruiu, chandlerc, cmatthews.
hfinkel set the repository for this revision to rL LLVM.
hfinkel added a subscriber: llvm-commits.
ruiu edited edge metadata.Oct 2 2015, 10:32 AM
ruiu added a subscriber: ruiu.

Patch LGTM. Did this fix your problem on your machine?

hfinkel updated this revision to Diff 36376.Oct 2 2015, 10:32 AM
hfinkel edited edge metadata.
hfinkel removed rL LLVM as the repository for this revision.

Don't print the note if it did not work.

In D13389#258733, @ruiu wrote:

Patch LGTM. Did this fix your problem on your machine?

Thanks! Yes, this fixes the problem on my machine.

chandlerc accepted this revision.Oct 2 2015, 10:39 AM
chandlerc edited edge metadata.

LGTM, but it looks like your comments are weirdly indented (see below). Submit whenever.

utils/lit/lit/run.py
231–234

The comment is weirdly indented here...

This revision is now accepted and ready to land.Oct 2 2015, 10:39 AM

Thanks.

utils/lit/lit/run.py
231–234

*sigh* tabs; I'll fix prior to submitting.

hfinkel closed this revision.Oct 2 2015, 10:53 AM

r249161.