Fail early, when we discover no tests at all, or filter out all of them.
There is also --allow-empty-runs to disable test to allow workflows
like LIT_FILTER=abc ninja check-all. Apparently check-all invokes
lit multiple times if certain projects are enabled, which would produce
unwanted "empty runs". Specify via LIT_OPTS=--allow-empty-runs.
There are 3 causes for empty runs:
- No tests discovered. This is always an error. Fix test suite config or command line.
- All tests filtered out. This is an error by default, but can be suppressed via --alow-empty-runs. Should prevent accidentally passing empty runs, but allow the workflow above.
- The number of shards is greater than the number of tests. Currently, this is never an error. Personally, I think we should consider making this an error by default; if this happens, you are doing something wrong. I added a warning but did not change the behavior, since this warrants more discussion.