It's mind blowing seeing these multi exaflop single rack systems.

The world's first exaflop supercomputer was Frontier. It was launched only 4 years ago in 2022.

It's not a fair comparison of course. FP4 in Helios barely qualifies as floating point. Frontier was proper fp64, 16 times the bit width and probably 256x as many transistors.

All the same just wow. Much compute.

▲

ACCount37 3 hours ago | parent [-]

Workloads did change over time. Back when we were first approaching practical exascale, the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision.

Now, the dominant compute-hungry workload is AI, where precision takes second place to the independent parameter count. To the point that the capacity of BF16, which were originally designed as a radical optimization for AI workloads, is sometimes considered wasteful now.

AI workloads have some truly peculiar and counterintuitive properties - the kind of things you might expect to see in biology instead of conventional computing. Intrinsic error tolerance, for one. It did necessitate some rethinking and reprioritization, and I'm not quite sure if we converged to the general shape of an "optimal" AI accelerator as of yet.

▲

ux266478 3 hours ago | parent | next [-]

> the dominant workload for a supercomputer was thought to be physics simulations - and they often benefit from high numerical precision.

It still is. Just like how "mainframe" used to be a very general word, and over time gained a very unintuitive definition referring to a very specific type of computer with a specific purpose, "supercomputer" almost invariably means it's a highly bespoke cluster dealing with FP64 workloads. I don't see anyone referring to these AI clusters as supercomputers, for the same reason they aren't referring to the racks as mainframes.

I wonder if there's a term for this kind of semantic narrowing?

	▲	fancyfredbot 2 hours ago \| parent [-]
		I think the term in this case is marketing. NVIDIA used to use the term supercomputer more often five or six years ago. It's now fully committed to the term AI factory. I think this is supposed to make you think of their expensive kit as a productive asset building your business rather than an expensive tool for your boffins. Your average business executive is instinctively going to question whether they need a supercomputer but don't yet have the same preconceptions about AI factories. I don't think that Nvidia single handedly caused the move away from the term supercomputer. But I think they had a hand helping push firmly in that direction.

▲

storus an hour ago | parent | prev [-]

FP64 is not that precise; proper simulations usually need much higher precision. Even ancient Intel could do 80-bit FP.

▲

addaon an hour ago | parent [-]

This very much depends on the simulation. CFD historically used 64-bit, with extended precision used internal to an extremely small number of poorly conditioned operations, or with very stiff equations; today it's not at all unusual to use single precision or (strategically) mixed precision, depending on the problem. I think the strongest statement you could make about simulation workloads, rather than them "needing" a specific precision, is that they are usually properly analyzed to determine the needed precision, and then run at that precision.

	▲	storus 16 minutes ago \| parent [-]
		One could argue that the interesting parts of simulation require higher precision. When you are in stable conditions, that's where your intuition might be sufficient already; once you hit those ugly parts then you start requiring as high precision as you can get to have any correlation with reality. Even a simple solar system simulation through gravity interactions has significant drifts in FP64.