Page MenuHomePhabricator

[Fixed Point Arithmetic] Validation Test for Saturated Subtraction on Signed _Fracts
Needs ReviewPublic

Authored by leonardchan on May 16 2018, 4:03 PM.



This patch includes the logic for subtraction on saturated _Fract types and a test for thm.

  • Overflow occurs when the sign bit or MSB of the LHS is different from the resulting sign, depending on the sign of the RHS. Cap at the max value for the type if the LHS is positive, but subtracting a negative number makes the result negative. Cap at the min value if the LHS is negative, but subtracting a positive number makes the result positive.
  • Also fixed incorrect minimum value for each _Fract type
  • Getters for each fixed point min and max value for a given type
  • Correction when casting from a fixed point to a float. If the fixed point data and sign bits do not take up the whole width of the integer (ie. has padding), we will need to either zero out the padding or extend the sign into the padding if it is negative to ensure the correct floating point value is produced after casting.

Diff Detail

rC Clang