Page MenuHomePhabricator

Fix incorrect truncation at the overflow boundary

Authored by pirama on Jun 21 2015, 11:26 AM.



This patch fixes incorrect truncation when the input wider value is
exactly 2^dstBits. For that value, the overflow to infinity is not
correctly handled. The fix is to replace a strict '>' with '>='.

truncdfsf2(340282366900000000000000000000000000000.0) returns infinity
truncdfsf2(340282366920938463463374607431768211456.0) returns 0
__truncdfsf2(400000000000000000000000000000000000000.0) returns infinity

Likewise, truncdfhf2 and truncsfhf2 (and consequently gnu_f2h_ieee)
are discontinuous at 65536.0.

This patch adds tests for all three cases, along with adding a missing
header include to fp_test.h.

Diff Detail


Event Timeline

pirama updated this revision to Diff 28091.Jun 21 2015, 11:26 AM
pirama retitled this revision from to Fix incorrect truncation at the overflow boundary.
pirama updated this object.
pirama edited the test plan for this revision. (Show Details)
pirama added reviewers: joerg, ab, srhines.
pirama added a subscriber: Unknown Object (MLST).
ab added a reviewer: scanon.Jun 22 2015, 3:58 PM

Nice catch! LGTM; Steve?


scanon accepted this revision.Jun 22 2015, 5:41 PM
scanon edited edge metadata.


This revision is now accepted and ready to land.Jun 22 2015, 5:41 PM
This revision was automatically updated to reflect the committed changes.