This very much depends on the simulation. CFD historically used 64-bit, with extended precision used internal to an extremely small number of poorly conditioned operations, or with very stiff equations; today it's not at all unusual to use single precision or (strategically) mixed precision, depending on the problem. I think the strongest statement you could make about simulation workloads, rather than them "needing" a specific precision, is that they are usually properly analyzed to determine the needed precision, and then run at that precision.

I think the term in this case is marketing.

NVIDIA used to use the term supercomputer more often five or six years ago. It's now fully committed to the term AI factory. I think this is supposed to make you think of their expensive kit as a productive asset building your business rather than an expensive tool for your boffins. Your average business executive is instinctively going to question whether they need a supercomputer but don't yet have the same preconceptions about AI factories.

I don't think that Nvidia single handedly caused the move away from the term supercomputer. But I think they had a hand helping push firmly in that direction.

> PYTORCH_ROCM_ARCH="gfx900;gfx906;gfx908;gfx90a;gfx942;gfx950;\ gfx1030;gfx1100;gfx1101;gfx1102;gfx1103;\ gfx1150;gfx1151;gfx1200;gfx1201"

That kicked off my PTSD to setup ROCm on my 6600XT and 9070XT, not even with different containers. I have to build an image for EACH architecture. That's duping 15GB for each arch on my already terribly small SSD.

That only works if you have all the unit tests and making sure the behaviors between different GPUs are expected, and can tolerate difference in implementation details such as numerical stability and complexity for some functions. AMD and Nvidia are doing vastly different computation for different implementation to the same algorithm for their APIs (can't tell due to NDA, but one algorithm Nvidia did it using broadcast, while AMD is using tile), and that's one of the reason I gave up. And perhaps only for light kernels doing basic BLAS operations and GEMMs. With a standard to follow, that's one of places where it indeed is giving it a good chance, for example on Tensorflow with ROCm or PyTorch. You basically have an insurance of the behavior and a translation layer for ROCm and CUDA by Python.

But anything other than that, nah.

Edit: Maybe I remember what algorithm that is. It is a trick to do coding theory stuff by using NTT (Number Theoretic Transform, the friend of DFT, the Discrete Fourier Transform, but on a ring and hence applicable to finite field) on GPU to do computation over a GF(2^8) field, for my former employment but I went back to using GFNI. Still I have a NDA to adhere to and can't really explain what happens.

Using AI slop in software that runs ML models is like buying a Ferrari with a motorbike engine. Might be good for demos or as a proof of concept, but that can’t be used for serving LLMs at scale