Precision of rounding numbers

I am having trouble with this one.I don't know but MATLAB answer to 1/(0.1*0.00001) is not 1000000 but 999999.999999999880000, 1000000 is displayed but when I am running it with my code it always fall on D>0 so upon investigating using fprintf('%.20f',D) I found out that its answer is not 1M but 999999-ish. I am using this for evaluating discriminant therefor I can't arrived with D==0. I am really on the edge figuring this out. Any recommendation or solution to this one? Thanks a lot!

Answer 1

[VBBV]

 D = 1/(0.1*0.00001) 

D = 1.0000e+06

 fprintf('%.2d',D)  % try using %d format specifier

1.00e+06

As you are using %f it is meant for floatpoint numbers,you must use %d

Answer 2

[Walter Roberson]

 D = 1/(0.1*0.00001)

D = 1.0000e+06

 fprintf('%.999g\n', D)

999999.999999999883584678173065185546875

You can see that the result is not an integer.

This is expected. MATLAB computes using IEEE 754 Binary Floating Point Numbers. Binary Floating Point numbers are unable to exactly represent 1/10 or 1/100000 -- for the same mathematical reason that no finite decimal expansion is able to exactly represent 1/3 or 1/7 .

In other words, it is not a bug in MATLAB: you are encountering a limitation that will happen any time you use numbers with a finite precision, no matter what fixed numeric base you use. If you were to use base 2520, you would be able to exactly represent 1/2, 1/3, 1/4, 1/5, 1/6, 1/7, 1/8, 1/9, 1/10 -- but you would not be able to exactly represent 1/11 .