In my database, I have a field with a value of 556.08…
…and when I multiply that by 50, it ends up as 27804.000000000004 in the database instead of 27804.
Has anyone else experienced any weirdness like this?
Hi philnauta
don’t worry, everything is fine !
What it is: Numbers of type double precision have a fraction part 52 bits long. 2^52 = 4’503’599’627’370’496, i.e. you cannot expect to retain more than 16 valid digits in a multiplication.
Why it is: The last digit 4 in your result is what is called the machine epsilon, the round-off error, or truncation error. It is the least difference possible between the maximum mathematical precision in god’s universe, and the limited double precision realized on a bit-based human computer.
How you solve it: In bubble there are several types of rounding, trimming and truncation commands you can use to fine tune the appearance of your results.
Many regards, O.
1 Like
Thanks @ones, interesting explanation. I still don’t understand why 556.08 * 50 has any decimal places in the first place (it should be an integer), but I’ll make use of rounding anyway.
The number 556.08 is not an integer. In most programming languages it is understood that the “pollution” of an integer calculus by a floating point or double precision number automatically treats all the arguments and so the result as floating point or double precision. The information that the result is an integer must come from the programmer, who will typically use rounding or trimming to get rid of undezired digits behind the decimal point.
This topic was automatically closed after 14 days. New replies are no longer allowed.