Remix.run Logo
How much precision can you squeeze out of a table?(johndcook.com)
32 points by nomemory 4 hours ago | 2 comments
PaulHoule 4 hours ago | parent | next [-]

When I was in high school I thought book this was so much fun

http://182.160.97.198:8080/xmlui/bitstream/handle/123456789/...

which is all about the kind of numerical analysis you would do by hand and introduces a lot of really cool math like the calculus of differences

https://en.wikipedia.org/wiki/Finite_difference

gus_massa 3 hours ago | parent | prev [-]

Nitpicking:

> You’re never going to get error less than 10E−15 since that’s the error in the tabulated values,

If you have like 100 (or 400) values in the table, you can squeeze one more digit.

In the simple case, imagine you have the constant pi, with 15 digits, repeated 100 times. If the rounding was done in a random direction like

  floor(pi * 1E15 + random() - 1/2 ) / 1E15
then you can use statistic to get an average with an error that is like K/sqrt(N) where K is a constant and N is the number of samples. If you were paying attention in you statistic clases, you probably know the value of K, but it's not my case. Anyway, just pick N=100 or N=400 or something like that and you get another digit for "free".

Nobody builds a table for a constant and uses a uniform offset for rounding. Anyway, with some smooth function that has no special values in the points where it's sample, the exact value of the cut decimal is quite "random" and you get a similar randomization.