Currency and Number Fields-maximum decimal pt values

[Roland]

What are the internal Project size limitations to the right of the decimal
place for the Currency and Number type fields? I know you can set the
display size for currency fields to a maximum of 2 characters but the actual
retained database values can extend out farther.

i.e. - 10.32 might actually be stored as 10.3187231 in the database

Thanks!

 

[Jim Aksel]

Number returns Double
Cost type variables return type is "Variant," for all intent is Double

Double precision floating point provides a relative precision of about 16
decimal digits. However, project will always round to 2 decimal places in
the display.
--
If this post was helpful, please consider rating it.

Jim Aksel, MVP

Check out my blog for more information:
http://www.msprojectblog.com

 

[Roland]

Thanks for feedback Jim.

Number returns Double
Cost type variables return type is "Variant," for all intent is Double

Double precision floating point provides a relative precision of about 16
decimal digits. However, project will always round to 2 decimal places in
the display.
--
If this post was helpful, please consider rating it.

Jim Aksel, MVP

Check out my blog for more information:
http://www.msprojectblog.com

 

[Roland]

Do you know why these two values wouldn't be the same? It appears the Base
Cost is rounding up at the 8th position.

BCWS Baseline Cost
36842.6188861355 36842.6188861400

Thanks.

 

[Rob Schneider]

The "why" is that it is a computer doing the arithmetic. If you seek
this sort of precision, you probably need to find another tool.

--rms

www.rmschneider.com

 

[Roland67]

The "why" is because our system has customized EV calculations. Since these
numbers don't match, our evaluation tools have to have more intelligence than
is necessary. Also, we receive unexpected results due to this kind of
rounding. When this happens our auditors jump in our shorts for an answer. If
I answer "it is a computer doing the arithmetic" then that satisfy them.

 

[Rob Schneider]

Gosh.

--rms

www.rmschneider.com

The "why" is because our system has customized EV calculations. Since these
numbers don't match, our evaluation tools have to have more intelligence than
is necessary. Also, we receive unexpected results due to this kind of
rounding. When this happens our auditors jump in our shorts for an answer. If
I answer "it is a computer doing the arithmetic" then that satisfy them.

 

[Roland67]

Bad day. Your response just hit me the wrong way. Sorry about that.