This is an archive of the discontinued LLVM Phabricator instance.

[dotest] Consider unexpected passes as failures.
ClosedPublic

Authored by JDevlieghere on Dec 18 2018, 9:47 AM.

Details

Summary

Unexpected successes should be considered failures because they can hide regressions when not addressed. When a test is fixed and not re-enabled, it can easily regress without us noticing.

I couldn't find a good way to make this change other than changing it in the unittest2 framework. I know this is less than optimal but since we have the dependency checked in and the change is pretty fundamental to the framework I think it's not unreasonable.

Diff Detail

Repository
rL LLVM

Event Timeline

JDevlieghere created this revision.Dec 18 2018, 9:47 AM
labath accepted this revision.Dec 20 2018, 11:20 AM

Looks good to me. Modifying this code is not ideal, but this is a pretty small but fundamental change in how unittest works. Changing the treatment of unexpected successes anywhere else would be a hack. It also looks like this is not the first change we've made in this piece of code.

This revision is now accepted and ready to land.Dec 20 2018, 11:20 AM
This revision was automatically updated to reflect the committed changes.