Someone on HN pointed out the Intel Xeon Max 9480. 64 GiB of HBM2 attached directly so the thing can boot without any RAM sticks if necessary. Cool processor but it couldn't use the memory bandwidth very well apparently.

At least in the initial benchmarks Xeon Max could not reach anything close to the theoretical memory bandwidth.

I have no idea whether that was corrected later with some microcode update or a new mask stepping.

Regardless, now Xeon Max is obsolete.

The Chinese have revived this idea in the custom CPU with an Armv9.3-A ISA that is used in their latest supercomputer, which also uses HBM memory, but which is in a complete other league, by having a 5 times higher memory bandwidth than the theoretical memory bandwidth of Xeon Max, and the ratio is much greater when considering the actual memory bandwidth of Xeon Max, and not the theoretical value.

That memory bandwidth of 8 TB/s per socket has enabled the Chinese supercomputer to exceed the throughput of the US supercomputers, which use GPUs, but even these GPUs have a lower memory bandwidth than the Chinese CPUs.

However, AMD has announced a pair of new datacenter GPUs, one for HPC (Instinct MI430X) and one for AI (Instinct MI455X), which increase many times the memory bandwidth, both over their older models and over the Chinese CPUs. MI455X has a monstrous HBM interface, 24,576 bits wide, for a 23.3 TB/s memory bandwidth.

MI455X is what AMD will sell to Anthropic (in the form of complete Helios racks, with EPYC servers full of GPUs) in the deal discussed in a couple of other HN threads (Reuters title: "AMD to sell Anthropic tens of billions in AI servers, invest up to $5 billion in startup"), i.e. Anthropic will not pay all in cash, but $5B will paid by giving a share to AMD.