This is an archive of the discontinued LLVM Phabricator instance.

[test] Exit with an error if no tests are run.
ClosedPublic

Authored by rupprecht on Aug 3 2020, 3:46 PM.

Details

Summary

If the test suite is misconfigured when it's run (a bad regexp, wrong test directory, etc.), the test suite may not discover any tests. When this happens, the test runner exits happily because no tests failed:

Ran 0 tests in 0.000s
RESULT: PASSED (0 passes, 0 failures, 0 errors, 0 skipped, 0 expected failures, 0 unexpected successes)

Change this to return an error so the misconfiguration can be more easily detected. Verified that lldb-dotest -p TestDoesNotExist.py successfully fails.

Diff Detail

Event Timeline

rupprecht created this revision.Aug 3 2020, 3:46 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 3 2020, 3:46 PM
rupprecht requested review of this revision.Aug 3 2020, 3:46 PM
JDevlieghere accepted this revision.Aug 3 2020, 4:02 PM

LGTM

lldb/packages/Python/lldbsuite/test/dotest.py
1043

Maybe did not discover any (matching) tests to make it clear that this might be the result of user input?

This revision is now accepted and ready to land.Aug 3 2020, 4:02 PM
MaskRay added a subscriber: MaskRay.Aug 3 2020, 4:04 PM
MaskRay added inline comments.
lldb/packages/Python/lldbsuite/test/dotest.py
1043

sys.stderr.write or print(..., file=sys.stderr) (which appends a newline)

Trailing full stops are not recommended for error messages if the project does not already have the convention http://llvm.org/docs/CodingStandards.html#error-and-warning-messages

Looking around, several messages do not have a full stop.

JDevlieghere added inline comments.Aug 3 2020, 4:08 PM
lldb/packages/Python/lldbsuite/test/dotest.py
1043

Let's use logging.error like the other error messages if we want this to go to stderr.

rupprecht updated this revision to Diff 282759.Aug 3 2020, 4:16 PM
  • Use logging.error
  • Remove trailing stop
  • Edit message to indicate that matching tests weren't found (user error)
rupprecht marked 3 inline comments as done.Aug 3 2020, 4:18 PM

Thanks for the quick review!

I have a few more boring changes coming up, but hopefully some interesting ones after that.

This revision was landed with ongoing or failed builds.Aug 3 2020, 4:32 PM
This revision was automatically updated to reflect the committed changes.