Remix.run Logo
Tepix a day ago

People are buying dual Epyc Zen5 systems to get 24 DDR5-6000 memory channel bandwidth for inferemcing large LLMs on CPU. Clearly there is a demand for very fast memory.

bobim a day ago | parent [-]

Sure, implicit finite elements analysis scales up to two cores per DDR4 channel. Core density just grew up faster than bandwidth and it makes all those high cores cpus a waste for this kinds of workloads.