> 3090 would be nice

They would need 3x speedup over the current generation to approach 3090. A100 that has +- the 3090 compute but 80GB VRAM (so fits LLaMA 70B) does prefill at 550tok/s on a single GPU: https://www.reddit.com/r/LocalLLaMA/comments/1ivc6vv/llamacp...

the GB10 is only the same performance as a 3090. gb10 uses way less power.

i'm not sure why anyone would buy a mac studio instead of a gb10 machine for this use case.

	▲	rbanffy 3 hours ago \| parent \| next [-]
		> i'm not sure why anyone would buy a mac studio instead of a gb10 For an AI-only use case, the GB10s make sense, but they are only OK as desktop workstations, and I’m not sure for how long DGX OS will be updated, as dedicated AI machines have somewhat short lives. Apple computers, OTOH, have much longer lives, and desktops live the longest. I retired my Mac Mini a year after the machine was no longer getting OS updates, and it was still going strong.
	▲	villgax 5 hours ago \| parent \| prev [-]
		it's just people looking to do experiments locally on the main machine rather than just get a dedicated spark, which can be used properly as a headless box than a Mac of which you are at the mercy of system shenanigans albiet still bearable compared to windows