Why is there an error in the calculation of 366.0/5.0?
3 views (last 30 days)
Show older comments
I input the following code in Matlab and the result obtained has a small error, as shown below:
>> format long
>> 367/5
ans =
73.400000000000006
0 Comments
Answers (2)
Walter Roberson
on 2 May 2024
By default, MATLAB calculates using IEEE 754 Double Precision representation of numbers. IEEE 754 is a binary floating point representation. One thing about binary floating point representations is that 1/10 is not exactly representable in any finite length binary floating point.
This is not a bug; it is an inherent limitation of binary floating point representation.
The mathematical reasons why 1/10 is not exactly representable in binary is the same reason why 1/3 and 1/6, 1/7, 1/9, and 1/13 are not exactly representable in finite decimal. Every finite base representation has numbers that cannot be exactly finitely represented.
2 Comments
James Tursa
on 4 May 2024
Edited: James Tursa
on 4 May 2024
The three closest numbers to 73.4 representable in IEEE double precision are:
format longg
x = [73.4-eps(73.4);73.4;73.4+eps(73.4)];
The binary floating point hex patterns of these three numbers are (they differ by 1 in the trailing bit):
num2hex(x)
The exact decimal conversions of these binary floating point numbers are:
fprintf('%60.50f\n',x)
MATLAB (or more precisely, the CPU floating point processor) picked the closest one for the result. There are no numbers (including 73.4 exactly) inbetween these numbers that are exactly representable in IEEE double precision.
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!