The SSDs are necessary because Apple's architecture doesn't allow RAM upgrades. Reminds me of someone who said "640KB ought to be enough for anybody".

▲

wtallis an hour ago | parent | next [-]

To be fair, nobody has upgradable memory in any system that has enough memory bandwidth and compute power to run LLMs with decent performance. It might be interesting to compare against some decade-old x86 server or workstation stuffed full of LRDIMMs to reach 1.5–2TB of RAM, but the bandwidth would be only slightly faster than a desktop today with high-end DDR5: nowhere close to GPU bandwidth. So performance would still suck.

Designing for extreme expandability comes with pretty steep tradeoffs.

	▲	LTL_FTC an hour ago \| parent [-]
		Take a look at AMD’s 12-channel memory servers. The newer Epycs are up to 16-channels now, 1.6TB/s. Pretty great for inference.

▲

kulahan an hour ago | parent | prev | next [-]

Gates never said that, for what it's worth.

	▲	buzzerbetrayed an hour ago \| parent [-]
		> In an April 1985 InfoWorld editorial, James Fawcette wrote that Gates had said something like: “When we set the upper limit of PC-DOS at 640K, we thought nobody would ever need that much memory.” So yes, Bill Gates denies that story. So it just depends on who you believe. But to flat out say he never said it is too confident.

▲

lowbloodsugar an hour ago | parent | prev [-]

You would have the same issue with a GPU. I have 8 slots on my CPU but it cant handle 2.8TB of ram. I cant even buy 2TB for it because even though the mobo is rated for it, nobody ever made those chips (someone will sell me 8x256gb 8Rx4 for $15k but there’s no way the memory controller can handle that). 1TB is $10k and forced it to run slower.