Sort of interesting design It has half a terabyte of SRAM on the accelerator with a 128 GB of RAM replacing HBM with SRAM but keeping regular RAM.

▲

2001zhaozhao 4 hours ago | parent [-]

half a gigabyte not terabyte lol

▲

zrm 3 hours ago | parent [-]

Yeah, this thing is unusual. It resembles a consumer GPU but using normal DRAM with more channels, so then you get more RAM but less memory bandwidth than GDDR/HBM, which is the same thing Apple does. And the TDP is pretty attractive for what it is.

But what is that amount of SRAM for? It simultaneously seems like a lot, because SRAM is so expensive, but still not very much, because what can you fit in 0.5GB? Is there a specific workload that needs something that looks like this?

	▲	zamadatix 3 hours ago \| parent [-]
		The Intel D270, Nvidia DGX Spark, and (somewhat) AMD Strix Halo also use the same approach, so it seems to be getting pretty common. SRAM wise it doesn't seem too insane. About the same ratio of memory:SRAM as the 5090 (IIRC). Still more than I would have expected. I assume they found it helps process the current layer faster or something.