Excel Calculation is Incorrect

O

Overlanda

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly
 
D

Don S

Put this into a cell and change the format of the cell to number with 16
decimal places...

=1131.97-1123

The answer is 8.9700000000000300

Notice the 3 at the 14th place after the decimal.

Haha what a joke.

I have tried on two Excel 2003 boxes - one with SP1 and one without. Both
calculated it incorrectly
Click to expand...


I'm not a math wiz, but I think it has something to do with binary
(computer) calculation of decimal (base 10) numbers. Use
=round(toberounded) & you'll have fewer "errors".

Don S
 
Top