This is an archive of the discontinued LLVM Phabricator instance.

test-suite: Change extension used for reference outputs by Makefile-based harness so we can start improving how the CMake-based harness works without breaking the old system or the buildbots
AbandonedPublic

Authored by Abe on Oct 4 2016, 2:47 PM.

Details

Summary

Altered "Makefile.programs" to look for "<foo>.reference_output_for_old_Makefile-based_harness" instead of "<foo>.reference_output" so we can separate the expected outputs on a test-by-test basis, added 670 symlinks so that it should work as-is, i.e. no real change for now, just making room for future improvements.

The idea here is to keep overhead low while allowing us to break symlinks in the future and have different reference outputs for each testing system. This may be important in some cases, e.g. the FP-related test-suite improvements, unless the Makefile-based harness is going to be completely removed from trunk very very soon.

Of note, the only file changed is "Makefile.programs", which has 32 lines of change according to Phabricator.

Diff Detail

Event Timeline

Abe updated this revision to Diff 73557.Oct 4 2016, 2:47 PM
Abe retitled this revision from to test-suite: Change extension used for reference outputs by Makefile-based harness so we can start improving how the CMake-based harness works without breaking the old system or the buildbots.
Abe updated this object.
Abe added reviewers: sebpop, rengolin, scanon, ddunbar.
Abe updated this object.
Abe added a subscriber: cfe-commits.
rengolin edited edge metadata.Oct 5 2016, 2:50 AM

Can you expand on how you plan to add the second set of reference outputs?

We already have multiple reference outputs, for example big endian vs. little endian. I was expecting you to have done a similar thing, which doesn't involve changing every single reference file. :)

One way this could be simpler is to change the Makefile on each affected directories to duplicate the tests & check differently against the reference file. It can be the same reference, but with different thresholds for different fp-contract options.

Abe abandoned this revision.Oct 6 2016, 8:32 AM