precision in functions

B

Bruce Baker

In a function such as PMT, what is the scale & precision used internally
when calculating the result ?

Thanks
Bruce
 
X

xlbo

As far as I know, excel calculates to 15 sig fig

From excel Help
15 digit precision Excel stores and calculates with 15 significant digits of precision

Excel calculates stored, not displayed, values The displayed, and printed, value depends on how you choose to format and display the stored value

Precision of calculations: When a formula performs calculations, Excel usually uses the values stored in cells referenced by the formula. For example, if two cells each contain the value 10.005 and the cells are formatted to display values in currency format, the value $10.01 is displayed in each cell. If you add the two cells together, the result is $20.01 because Excel adds the stored values 10.005 and 10.005, not the displayed values. You can change the precision of calculations so that Excel uses the displayed value instead of the stored value when it recalculates formulas

HT
Geof
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Excel TVM functions of RATE, NPER, PMT, PV, and FV fail to produceresults that are known about Geome 0
Any way to know the PRECISE indent values? 3
Possible problem with PMT and FV functions 2
Function to establish mortgage bond size. 2
financial function 1
different result with excel and calculator 1
Calculation using IEEE 754 single precision format 4
PMT function and 1st payment partway through a period 3

Top