From: Andrei Chichak (acpmiedm_at_telusplanet.net)
Date: Thu Dec 11 2003 - 09:19:37 PST
At 12:57 PM 12/10/2003 +1000, you wrote:
>I have for many years used the freeware calculator inteliCalc 1.0
>My use has generally been for multiplication and trig.
>
>I do not fix the decimal place and leave the setting on standard
>
>Today when I did a simple subtraction 22.35 - 22.8 the answer returned was
>0.449999999999999
>
>Has anyone else had this problem?
Sure, FORTRAN had this "problem" since back in the '50s. I was told about
it in second year CS in our numerical analysis class.
The basic problem is a base 2 representation of a base 10 number. With a
lot of values you will get repeating sequences of bits to represent the
value. Add in a large helping of finite precision and you get odd
representations.
Some values, like pi, have no repeating sequences, so translating to base 2
will give you a different approximation and the inherent representation
problems.
The result can be made arbitrarily close to the true result by adding
precision or by changing the display routines to avoid these 9999
situations (as done in most languages since they learned from FORTRAN).
The difference between the expected answer and your answer is 1e-10, either
reduce your displayed number of significant digits back to the number that
the original equation allows (if there is a built in method to do this or
else visually)(correct response according to physicists) or get the Newton
Development Group to adjust the floating point code :-)
>It seems that this occurs with many instances where the number being
>subtracted lies between 0.5 and 0.99 below the whole number from which it
>is being subtracted
You will also see big problems when huge numbers are used with values very
close to zero. This is a problem with the way floating point numbers are
represented in IEEE format. There isn't anything that you can do about it
other than being very careful to watch the relative magnitudes of your
operands and avoiding the issues.
>Is it possible the program has become corrupted?
It is more probable that you have found a fault in the program. Programs do
not decay on their own like hardware does. On the other hand there is
typically one bug for every 10 lines of code.
Fire up a copy of NTK and use the debugger to ask your newton what 22.35 -
22.8 is. If it gives the repeating 9s I would be very worried since the
answer is supposed to be negative. On the other hand 22.8 - 22.35 may give
you repeating 9's indicating that the representation is provided by the
Newton ROM routines.
>Gordon
Andrei
-- This is the NewtonTalk list - http://www.newtontalk.net/ for all inquiries List FAQ/Etiquette/Terms: http://www.newtontalk.net/faq.html Official Newton FAQ: http://www.chuma.org/newton/faq/
This archive was generated by hypermail 2.1.5 : Thu Dec 11 2003 - 12:00:00 PST