15 Significant Digits Limitation a Mistake for Spatial Information

J

Jerry W. Lewis

The OP's question was less than specific. Your answer only applies to one of
the possible interpretations.

Hardware limitations would restrict to 8-byte IEEE double precision (DP,
though PC hardware that does not support 10-byte reals has not been made for
nearly 20 years).

However, DP does not restrict to 15 decimal digits. Most 16 digit numbers
can be represented in DP, so many packages can display more than 15 digits.

It takes 17 digits to uniquely determine a DP value, so most packages
(including Excel's VBA, but not Excel) will use 17 digits of the input value
to obtain the best binary representation.

Input and output precision are more tightly linked in Excel (and in VBA)
than in most packages, since in most packages the source code is not altered
on input. In the Excel model input becomes output, so some alteration on
input of the source is unavoidable if more than 15 figures are displayed
(e.g. 9007199254740993 has no DP representation, and so must change on entry
to either 9007199254740992 or 9007199254740994 if Excel were to support 16
figures on input and output. However 15 digit input means that you cannot
easily get full DP input precision, and 15 digit output means that you cannot
easily maintain DP precision in the face of subsequent editing.

Thus 15 digit I/O is a design decision, not a physical constraint. It is
not clear to me that MS adequately thought it through before making the
decisions (particularly since they decided differently for Excel and VBA),
but since the decisions were made, I doubt that MS will reconsider them.

Jerry
 
Top