This is an archive of the discontinued LLVM Phabricator instance.

[APFloat] Fix incorrect fptrunc rouning when from semantics exp lower than minimal exp of target semantics
Needs ReviewPublic

Authored by kpolushin on Jul 29 2020, 3:58 AM.

Details

Summary

Incorrect calculation of the exponentChange in the case when the exponent
of the original semantics is lower than the minimum exponent of the target semantics,
led to an incorrect shift and an incorrect value of the lost fraction.

For example:

fptrunc double 0x10000001 to float

Round to the minimum positive float 0x36A0000000000000 (denorm),
although in the case of rounding to the nearest, this should have reduced to 0.
Since the lost fraction was calculated incorrectly, the function IEEEFloat::roundAwayFromZero,
called from IEEEFloat::normalize returned true (ignoring differences in changed exponent semantics),
which resulted in rounding up.

Diff Detail

Event Timeline

kpolushin created this revision.Jul 29 2020, 3:58 AM
kpolushin requested review of this revision.Jul 29 2020, 3:58 AM
lattner edited reviewers, added: scanon; removed: lattner.Jul 30 2020, 12:27 PM

I'm not sure what specific problem you're trying to solve here, but this change in itself looks incorrect.

The computation done here follows the same logic that would occur later in normalize(). exponentChange is (correctly) set to toSemantics.minExponent - exponent here so that the subsequent exponent += exponentChange will then set exponent to toSemantics.minExponent.