# Floating Point Accuracy

## FPU Accuracy

It is possible to make FPU's which perform the basic
arithmetic functions (+,-,*,/,sqrt) perfectly, i.e.
the result returned is the most accurate which is possible
in the format used.
Apart from the occasional bug, all (?) FPU's
which use IEEE binary arithmetic
achieve this performance.

Two factors affect the accuracy of the values returned by the basic
trig instructions which compute sin(), cos(), tan(), etc. These are:
- The accuracy of the internal representation
of pi.
- The accuracy of the instructions
for reduced arguments.

To compute the value of a basic trig function (sin(), cos(), etc)
for an argument which is larger
than some small value (typically pi/2) the usual method is to first
subtract an appropriate multiple of pi.
This process is called argument reduction.
To accurately reduce an argument for a trig function,
a very accurate value of pi is needed.
For example, on the Intel 80x86 architecture the fsin
instruction (used to compute sin()) takes a 64 bit precision
argument and returns a result of the same precision.
To accurately compute the sin() of the number which is closest
to pi in this format requires a pi with more than 128 bits
accuracy.

For larger arguments, even greater accuracy is required.
For example the integer

8227740058411162616 = 2618971001542672383.9999999689975244 * pi

is exactly representable as a 64 bit precision number.
The sin() of this number is
-9.73971495111e-08

and requires a pi with about 150 bit accuracy to be able to
get a result with 64 bit accuracy.
Intel 80x86 processors (at least up to the Pentium Pro processors)
have an internal pi which is accurate to about 69 bits.
On these processors, the results of the fsin instruction
therefore has reduced accuracy for arguments which are close
to multiples of pi or are large in magnitude.

To determine to accuracy of the internal pi in Intel compatible
processors, in May 1997 I posted a small test
program to a Linux newsgroup.
The program determines the accuracy of the result of
the fsin instruction for 36 different arguments.

I received about 30 replies.
The results were:

-------------+---------+-----------+-----------+------------------------------
CPU | Nr of | pi | Incorrect |
type | reports | precision | tests | Notes
| | (bits) | |
-------------+---------+-----------+-----------+------------------------------
486 DX2/66 | 5 | 69.40 | 17,27,29 | Intel, Cyrix and AMD
Pentium/PPro | 16 | 69.40 | 21,22,29 | Pentium-66, P133 and PPro-200
K6 | 2 | 69.40 | 21,24,29 | 166, 200
6x86 | 7 | 72.02 | 29 | IBM and Cyrix. P150+ to 200+
Emulator | 1 | 129.48 | 17 | Linux version 1.22
K5 PR133 | 2 | >150 (2) | 22,27 |
-------------+---------+-----------+-----------+------------------------------
Note: in each case where the result was incorrect, the error was one ulp.
(2) The program could not measure the pi precision of a K5, implying a
precision of better than about 150 bits.