This is an archive of the discontinued LLVM Phabricator instance.

[test-suite] Don't allow FP_TOLERANCE and HASH_PROGRAM_OUTPUT to occur toghether.
ClosedPublic

Authored by chatur01 on Sep 30 2015, 5:24 AM.

Details

Summary

LNT's benchmarks produce their output into files which are then compared in
various ways.

The benchmarks output numerical scores, which can then be compared by a custom
program in the test-suite tree which does a floating-point comparison with a
given tolerance (fpcmp).

The test-suite system allows allows users to do a hash of the program output
for a subsequently speedier comparison. This is used for benchmarks that
generate a significant amount of output (currently defined as >100k), The
problem is that some benchmarks that chose this feature also chose a
floating-point comparison of these hashes, which at best is nonsense, and at
worse causes an infinite loop.

The fpcmp program is not resilient when it's input is weird like this. For a
particular case of an observed mafft failure, it caused fpcmp to go into an
infinite loop.

Diff Detail

Event Timeline

chatur01 retitled this revision from to [test-suite] Don't allow FP_TOLERANCE and HASH_PROGRAM_OUTPUT to occur toghether..
chatur01 updated this object.
chatur01 added a subscriber: llvm-commits.
rengolin accepted this revision.Sep 30 2015, 5:53 AM
rengolin added a reviewer: rengolin.
rengolin added a subscriber: rengolin.

I'm surprised this was there all the time... :)

Makes total sense. LGTM. Thanks!

This revision is now accepted and ready to land.Sep 30 2015, 5:53 AM
chatur01 closed this revision.Sep 30 2015, 6:26 AM