Excel 2003 display error or just rounding error?

[MBurch]

I am not a math wizard so this might just be a rounding error.
Excel 2003 SP3 (11.8169.8172)
Columns formatted to numeric with 20 decimal places.

sqrt(x)
where x=
2=1.41421356237310000000
18=4.24264068711929000000
72=8.48528137423857000000
200=14.14213562373100000000
450=21.21320343559640000000
882=29.69848480983500000000
1568=39.59797974644670000000
2592=50.91168824543140000000

sqrt((x^2)/2) * (x + 1)
where x=
1=1.41421356237310000000
2=4.24264068711929000000
3=8.48528137423857000000
4=14.14213562373100000000
5=21.21320343559640000000
6=29.69848480983500000000
7=39.59797974644670000000
8=50.91168824543140000000

this is where it is different
a7=6
b7=36 = a7^2
c7=18 = b7/2
f7=sqrt(c7) = 4.24264068711928000000 <- different. if anything should round
up?

sqrt(18) = 4.24264068711928000000

Windows calculator puts the result as 4.242640687119285146405661726291

 

[MBurch]

:

Realized that the line above

18=4.24264068711929000000

was the wrong pasted line, should have been

 

[Bob Buckland ?:-\)]

Hi M.,

You may want to also post in the MS Excel discussion group using the link below.

==========
I am not a math wizard so this might just be a rounding error.
Excel 2003 SP3 (11.8169.8172)
Columns formatted to numeric with 20 decimal places.

sqrt(x)
where x=
2=1.41421356237310000000
18=4.24264068711929000000
72=8.48528137423857000000
200=14.14213562373100000000
450=21.21320343559640000000
882=29.69848480983500000000
1568=39.59797974644670000000
2592=50.91168824543140000000

sqrt((x^2)/2) * (x + 1)
where x=
1=1.41421356237310000000
2=4.24264068711929000000
3=8.48528137423857000000
4=14.14213562373100000000
5=21.21320343559640000000
6=29.69848480983500000000
7=39.59797974644670000000
8=50.91168824543140000000

this is where it is different
a7=6
b7=36 = a7^2
c7=18 = b7/2
f7=sqrt(c7) = 4.24264068711928000000 <- different. if anything should round
up?

sqrt(18) = 4.24264068711928000000

Windows calculator puts the result as 4.242640687119285146405661726291 >>
--
Please let us know if this has helped,

Bob Buckland ?
MS Office System Products MVP

LINKS
A. Specific newsgroup/discussion group mentioned in this message:
news://msnews.microsoft.com/microsoft.public.excel
or via browser:
http://microsoft.com/communities/newsgroups/en-us/?dg=microsoft.public.excel

B. MS Office Community discussion/newsgroups via Web Browser
http://microsoft.com/office/community/en-us/default.mspx
or
Microsoft hosted newsgroups via Outlook Express/newsreader
news://msnews.microsoft.com

 

[Val]

The more operations you do with floating point numbers, the fuzzier they get
at the low end. Double precision floating point values (8 bytes) have about
15 digits of precision.