This is an archive of the discontinued LLVM Phabricator instance.

[lldb][tests] Make it possible to expect failure for a whole category
ClosedPublic

Authored by tatyana-krasnukha on Dec 26 2019, 8:35 AM.

Details

Summary

There already are decorators and "--excluded" option to mark test-cases/files
as expected to fail. However, when a new test file is added and it relates
to a feature that a target doesn't support, this requires either adding decorators
to that file or modifying the file provided as "--excluded" option value.

The purpose of this patch is to avoid any modifications in such cases.
E.g. if a target doesn't support "watchpoints" and passes "--xfail-category watchpoint"
to dotest, a testing job will not fail after a new watchpoint-related test file is added.

Diff Detail

Repository
rLLDB LLDB

Event Timeline

Herald added a project: Restricted Project. · View Herald TranscriptDec 26 2019, 8:35 AM

This LGTM but I'm not the right person to sign this off. I guess Pavel and Jonas know this code better.

If I understand correctly this is the equivalent of skip-category but you want to XFAIL the tests instead of skip them? Seems reasonable to me :-)

labath added a comment.Jan 6 2020, 6:18 AM

Seems reasonable, just make the option name consistent.

packages/Python/lldbsuite/test/dotest_args.py
85–88 ↗(On Diff #235349)

Please use a consistent name for the two options.

Increased number of lines of context.

Jonas, yes, it is.

Pavel, names are consistent (if I understand your comment correctly). The "--xfail-category" is singular like the "--skip-category".

This revision is now accepted and ready to land.Jan 6 2020, 10:15 AM
This revision was automatically updated to reflect the committed changes.