There is a reason all the big supercomputers have started using GPUs in the last decade. They are much more efficient. If you want 32bit parallel performance just buy some consumer GPUs and hook them up. If you need 64bit buy some prosumer GPUs like the RTX 6000 Pro and you are done.

Nobody is really building CPU clusters these days.

Unfortunately even the RTX 6000 Pro has nerfed double-precision throughput at about 2 TFLOPS, 64x slower than single precision. For comparison an EPYC 9755 does ~10 TFLOPS, while drawing less power. An A100 -- if you can find one -- is in the same ballpark.

The best option for DP throughput for hobbyists interested in HPC might be old AMD cards from before they, too, realized that scientific folks would pay up the nose for higher precision.

Well, El Capitan uses AMD CPUs (which have integrated GPU capabilities) and it is right on top of the rankings lately.

Frontier is right behind it with the same arrangement.

Having honest to god dedicated GPUs on their own data bus with their own memory isn't necessarily the fastest way to roll.