Unacceptable floating point errors

J

Jeff in GA

Alain,

I'm sure that in your social circles you are regarded as a clever person,
and you evidently hold yourself in the highest regard, so you are perhaps
quite fortunate.

However, my math problem is quite simple, and I'm sorry it has made you so
defensive: .29 - .28 = .010000. But I now realize that I'm not being
realistic, and so that makes me a bad customer, a very very bad customer.

JoeU2004 has done a fine job of explaining the limitations induced by how
Microsoft decided to implement their calculations. (Per your other posting,
I'm pretty sure that IEEE is *not* a subversive organization, but that would
make a great rumor!)

If you knew the sort of math that I do, you would feel quite foolish for
having made your assertions. But, by all means, this forum could use some
levity.

....Jeff
 
D

David Biddulph

We know what the answer should be, and we also know the limitations of a
fixed point binary machine with a number such as 1/10, just as we know the
limitations of a fixed point decimal machine for a number such as 1/3, and
we treat the results accordingly.

You are welcome to use whichever machine and whichever software you prefer.
If you prefer to use something other than Excel, you are welcome to do so.
If you want to use something that works in decimal rather than binary, you
are welcome to do so. If you wish to write some software of your own to
replace Excel, then again you are welcome to do so.
 
A

Alain Dekker

Hi Jeff,

No offense intended, dude!

I think by now you possibly do understand that the digitial representation
of a decimal number (ie. in binary) is limited by memory and the nature of
binary itself. But you're also missing a more important, and more subtle,
concept - the issue of "significant figures". When you practice chemistry,
for example, you discover (much to the surprise of many and the indignation
and refusal to accept of a few) that it is actually not scientifically
accurate to say you're going to pippette 10ml of liquid A and to expect
exactly 10.00000ml (or even 10.0ml). Most people struggle with this, but
you're *actually* saying you're going to pipette (usually) between 5 and 15
ml of liquid. If you really wanted exactly 10.00000ml (to seven significant
figures) then you'd need a super accurate pipette and the hands of a
surgeon.

The bottom line is that 0.01 *IS EXACTLY THE SAME AS* 0.00999999999999995
when represented in 32-bit binary. They're not even slightly different.
Excel, as far as I know, uses double floating point precision ie. 32 bits to
represent both the mantissa and floating point part, and so has got it
spot-on accurate. This is not a joke. Your pocket calculator just does a
rounding job for visual purposes to make it seem like its done the right
calculation.

If you think you can come up with a better system (and I am trying to put
this in a friendly way, please!) you are deluded. I don't think its an
exaggeration to say that tens of billions of man-hours by some of the
smartest people of the last 50 years have come up with what we have today.
Its not perfect but it is very good and you use their system thousands of
times a day without knowing it.

BTW, did you spot the flaw in the mathematical puzzle? Quite a nice little
party trick!

If you still feel like abusing Microsoft Excel or myself after this, please
go ahead. I could do with a laugh.

Regards,
Alain
 
J

Jeff in GA

Alain,

I'm glad I was able to instigate such an inspired thread.

I do not mean this in a derogatory way, but many of you are so accustomed to
operating within the machine that the logic of your matrix is perfectly valid
to you. Worse yet, some of you react with indignation, if not derision, when
a person suggests that .29 - .28 should equal .010000000000

For you "0.01 *IS EXACTLY THE SAME AS* 0.00999999999999995", but even Excel
doesn't think it is the same thing when using conditional statements.
Moreover, the unpredictability of it makes it even more insidious. I.e., the
problem does not occur for .30 - .29 and many other pairs, so it has the
effect of being a random error generator, IMO.

I realize that the very notion of "customer" is silly and old fashioned, but
to the customer the reaction from those inside the MS matrix appears to be:
"you're a fool to expect the mathematical results to be correct, and a moron
if you don't understand the unpredictable intricacies of floating point
mathematics, and that this imprecision has the blessing of the IEEE!"

Though it appears stupid to you, I was simply expecting .29 - .28 to equal
exactly .01, and I was simply astonished when it didn't, and I honestly could
not imagine why it didn't.

In order to deal with really moronic customers like me (there's that
"customer" word again -- I just can't help myself!), MS would be better
served to provide a more conspicuous forewarning of this possible occurrence.
I realize that it does not behoove an arrogant corporation to phrase it
thusly, but the message should be along the lines that: "In certain
instances, a mathematical result is produced which is not perfectly precise.
Unfortunately, this is unavoidable to due to the limitations inherent to any
calculation algorithm that reduces the values to binary equivalents [link to
technical explanation] and we regret any inconvenience that may present. Here
is a how you can anticipate and minimize the consequences of this problem
[link]â€.

My point is that the explanation from MS should respect the fact that many
of us are reasonably expecting consistent precision (e.g., like .29 - .28 =
..01000), and that we are not prepared for unpredictable and sporadic
exceptions. It's like the clock that strikes thirteen, then you wonder if
all the other times were correct.

....Jeff
 
K

Kevin Andreoli

Jeff in GA said:
In order to deal with really moronic customers like me (there's that
"customer" word again -- I just can't help myself!), MS would be better
served to provide a more conspicuous forewarning of this possible
occurrence.


If you feel that way about it Jeff then take it up with Microsoft. There
ain't any of them in here, just a bunch of users like yourself.
 
J

Jeff in GA

That's a good point. I don't know who is MS (if anyone) and who isn't.

I guess I was hoping that MS somehow participates in the process of helping
their customers, or otherwise would read these threads out of concern.

What was I thinking?!?

:
 
A

Alain Dekker

Jeepers creepers Jeff, are you being dense on purpose or did you just not
sleep well last night???

I'm not from MS nor have I ever met anyone from MS. Mostly the people who
reply in these forums are well-meaning users who are freely giving of their
own time. MS have over a billion customers and if you mulitply that by the
number of different products each uses on average (say Windows, Word, Excel,
COM, VB, .NET, etc) they probably have tens of billions of customers.

Imagine if everyone who thought they were right on some dinky issue, like
yourself, had a personal line to MS-central? Since most questions, like
yours, are ignorant and a waste of time, they'd be swamped with garbage.

Lets face facts, most of humanity, and therefore most of MS customers, are
pretty stupid.

MS are not the most customer-friendly company, I'd agree, but as a software
developer they do an excellent job of providing developer tools and
documentation.

If you want to learn about binary represenation, typed data and other basic
computer concepts, dip your toes into Wikipedia or the Microsoft Developer
Network (MSDN). You'll find all your answers and more there and then next
time you can be the one to feel superior and amused when some moron comes
along and says "hey, 0.29 - 0.28 returned 0.00999999999995, what a load of
crap!".

I thought by this time you'd have calmed down. Hey, maybe you're actually
quite sophisticated and you've been yanking our chains for some fun. Who
knows. For a while this has been amusing but now I'm bored.

Bye, bye.
Alain
 
A

Alain Dekker

Good article.

David Biddulph said:
MS's explanation is at
http://support.microsoft.com/kb/78113
(linked from the info to which Berndt pointed you).
--
David Biddulph

Jeff in GA said:
Alain,
...
In order to deal with really moronic customers like me (there's that
"customer" word again -- I just can't help myself!), MS would be better
served to provide a more conspicuous forewarning of this possible
occurrence.
I realize that it does not behoove an arrogant corporation to phrase it
thusly, but the message should be along the lines that: "In certain
instances, a mathematical result is produced which is not perfectly
precise.
Unfortunately, this is unavoidable to due to the limitations inherent to
any
calculation algorithm that reduces the values to binary equivalents [link
to
technical explanation] and we regret any inconvenience that may present.
Here
is a how you can anticipate and minimize the consequences of this problem
[link]". ...
 
C

Chip Pearson

I'm willing to bet that academic-grade math software, which is used in
colleges and many commercial settings, (e.g., Mathematica), can provide

Some software gives you better accuracy than does Excel. Some software
uses a larger data width that can provide more accuracy out farther to
the right. Also, some software can perform some mathematical
operations symbolically rather than numerically.

But even in the real world, rounding to some degree is necessary. No
matter how far out you carry out the calculations, you'll never get a
computationally correct answer to the addition 1/3 + 1/3 + 1/3 = 1.
No "academic grade" math software that doesn't use symbolic
manipulation will give you answer of 1. Take it out to 1000000 decimal
places and you'll still not equal to 1.

Professional computer programmers know the limitations of
computational arithmetic and write code to accommodate those
limitations. For example, when testing equality of floating point
numbers, code is not generally written as

If (X - Y) = 0 Then
' do something

Instead, code is written as

If Abs(X - Y) <= Epsilon Then
' do something

where Epsilon is some value, such as 0.00000001, scaled to the
compiler's representation of floating point numbers. Some programming
languages have this constant built in as a native element of the
language. In other languages you declare it yourself.

Greater accuracy in computed floating point numbers comes at a cost of
performance. It takes more operations to calculate a more accurate
representation of a quantity. At some point, the software must take an
approximation. Whether that approximation is at about 7 places in
Single Precision Floating Points, about at 15 places in Double
Precision Floating Points, or 1000000 places in some hypothetical
software, there will necessarily be some rounding. As long as you
are limited by a finite number of decimal places, rounding is
inevitable.

The designers of the software and programming languages take into
consideration the real world needs of the end users, the applications
built with the code and compiler, and the performance of the hardware
to decide how accurate the representation of the quantity needs to be.
For nearly all purposes, 15 digits of precision is adequate.

In order to make software and data consistent and sharable among
different systems, applications, and platforms, some standardized
format must be adhered to. For most software, that standard is the 8
byte Double Precision Floating Point standard published by IEEE. Is it
perfect? No. Is it the best possible standard? No. But it is what
nearly all software uses. Without some standard, you couldn't share
data between different programs. Would you leave it up to the user to
instruct the software to use 128 bits rather than 64? And then assume
that all users of the same data know to use 128 rather than 64? Can't
happen in the real world.

Software that uses the IEEE standard isn't "defective". The
limitations are known or should be known by the users and developers.
One could make the argument that the documentation is deficient by not
making clear the limitations of the software, but as long as the
standard is followed, the software does what it is designed to do.


Cordially,
Chip Pearson
Microsoft Most Valuable Professional
Excel Product Group, 1998 - 2009
Pearson Software Consulting, LLC
www.cpearson.com
(email on web site)
 
J

JoeU2004

Errata....
Arguably, if 0.29 were represented by adding one more bit (2^-54), it
would be exactly 0.290000000000000,03552713678800500929355621337890625,
about 0.000000000000000035 (18 fractional digits). In that case, 0.29 -
0.28 would be 0.0100000000000000,088817841970012523233890533447265625,
which Excel would display as 0.010...0 to 15 significant digits.

I had decided not to correct the several typos in my posting, most of which
are not material. But I do want to correct a potential misimpression, even
though no one has (yet) made a point of it, just for the record.

First, what I meant to write is: if I increment the least significant bit
by adding 2^-54, it would have the result above, which is about
0.000000000000000035 (18 fractional digits) more than 0.29.

But I do not mean to suggest that Excel is flawed for not doing so in its
conversion.

The issue, again, has to do with binary floating point arithmetic.

The string "0.29" is converted to a number by computing 2/10 + 9/100. That
happens to result in exactly
0.289999999999999,980015985556747182272374629974365234375, which is less
than 0.29.

I had noted previously that the string "0.28" happens to convert, by
computing 2/10 + 8/100, to the exact number
0.280000000000000,0266453525910037569701671600341796875, which is more than
0.28.

The different directions in conversion -- one less, the other more -- is
partly responsible for the failure of the simplistic case of 0.29 - 0.28 not
appearing to be 0.01 when formatted to 16 decimal places (15 significant
digits).

That might raise the speculation that Excel should always convert to a
binary value greater than (if not equal to) the original numeric string.

But for this example, it is not a panacea.

Even though the result of 0.29 - 0.28 (in A3, say) might appear to be 0.01
when formatted to 16 decimal places, the formula =A3=0.01 returns FALSE (!).

The reason is: the exact result of (0.29 + 2^-54) - 0.28 is
0.0100000000000000,088817841970012523233890533447265625. That is different
enough from the internal representation of the constant 0.01 for Excel to
recognize the difference, despite its heuristics to adjust infinitesimal
differences. The exact internal representation of the constant 0.01 is
0.0100000000000000,0020816681711721685132943093776702880859375.

Moreover, one might reasonably question the wisdom of trying to make this
specific example work better.

The reason is: there are 18 valid representations of each of 0.29 and 0.28,
any of which might be the result of an arithmetic expression. The 324
combinations of subtracting 0.28 from 0.29 result in any of 35 values from
about 0.00999999999999907 to about 0.01000000000000095. Only 18
combinations result in what appears to be 0.01 when formatted to 16 decimal
places. But =A3=0.01 returns FALSE for all of those combinations, since all
18 are exactly 0.0100000000000000,088817841970012523233890533447265625.


----- original message -----
 
D

David Biddulph

Yes, you are quite correct, Jeff. If you are starting off with numbers with
a resolution of 0.01 (such as in many currency claculations), and if you
want precision, it would certainly make sense to multiply up and do the
calculation in cents (integers) rather than in dollars (floating point).
 
S

Simon Murphy

Jeff
You can't have a greater degree of accuracy in a result than was present
in the arguments
..30 - .29 does = .01 to 2 significant digits, beyond that the result is
undefined. .01000000000 is just one of many valid ones

..01 is the 2sd representation of any number from .005 to less than .015.

If you have a need for such high precision then you probably need to use
specialist tools rather than a general purpose product like Excel.

cheers
Simon
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top