Remix.run Logo
pclmulqdq 2 days ago

There's an implied one bit, so you actually have a 53 bit significand (and 53-bit precision) given only a 52 bit mantissa.

fiforpg 2 days ago | parent [-]

Right, I did realize after posting that close to numbers of the form

1{hidden bit} + (1-2^-52){mantissa with all ones}

the relative accuracy — corresponding to the absolute accuracy of a single bit in mantissa — is about 2^-53. The hidden bit is easy to forget about...