Differential D32178
Delete unstable integration tests Authored by jroelofs on Apr 18 2017, 10:07 AM.
Details
These tests are breaking when tested under the upstream 3.8.1 release + the 10.12 / 16C58 sdk. They use headers from the host SDKs, so they are not stable with respect to adding new language features, such as class @properties. Regression tests should not depend at all on the state of the host system. If these tests are still useful, they ought to live somewhere else (test-suite?).
Diff Detail Event TimelineComment Actions Thanks for working on this! We had some more discussion about these tests yesterday, and we would prefer to keep them actually. We might want to use something like REQUIRES instead, but right now we are still not sure what to check for. Comment Actions I think you'd have to check for a maximum host SDK version, and then bump it upstream every time there's a new one thats known to work with the then-current trunk. It can't be done based on language features because by definition, one cannot know what features are going to be added in the future. All of that being said, we'd still have a situation where the test means different things on different host systems purely based on what's installed on that system (which is not a good thing IMO). Comment Actions Ping. @arphaman I'd like to remove them, absent a concrete plan to fix this from folks with a vested interest in them (i.e. you). Comment Actions Clang's regression test suite is not a sensible home for these tests. We should have a home and a plan for system-specific integration tests, but this is not it. Perhaps this should instead be a part of LNT / the test-suite project? Comment Actions @jroelofs Comment Actions I've managed to put together a solution that adds REQUIRES to the tests that ensure they only work with the latest SDK (10.12). WDYT? Comment Actions I do agree that these tests would be better off in another place, but we currently don't have the bandwidth to facilitate that move. |