The only consumer GPUs ten years ago that offered decent FP64 performance were the GTX TITAN series. And they were beasts! It's a shame nothing quite like them exists anymore. But they were the highest of high-end cards, certainly not that common or cheap.

AMD Hawaii GPUs in their professional variant (FirePro), which were cheap, unlike the "datacenter" GPUs of today, and the more recent Radeon VII had much better FP64 performance per $ than GTX Titan.

Moreover, there were claims that the memory errors on GTX Titan were quite frequent. On graphics applications memory errors seldom matter, but if you have to do a computation twice to be certain that there were no memory errors affecting the results, that removes much of the performance advantage of a GPU.