| ▲ | mackopes 3 days ago | ||||||||||||||||
Show me another consumer hardware that handles inference and/or training better. How many RTX5090s would you need? | |||||||||||||||||
| ▲ | spogbiper 3 days ago | parent | next [-] | ||||||||||||||||
https://liliputing.com/nvidia-dgx-spark-is-3000-ai-supercomp... looks like there will be several good options "soon"? | |||||||||||||||||
| |||||||||||||||||
| ▲ | NitpickLawyer 3 days ago | parent | prev [-] | ||||||||||||||||
For local inference macs have indeed shined through this whole LLM thing, and came out as the preferred device. They are great, the dev experience is good, speeds are ok-ish (a bit slower w/ the new "thinking" models / agentic use with lots of context, but still manageable). But nvda isn't that far behind, and has already moved to regain some space with their PRO6000 "workstation" GPUs. You get 96GB of VRAM for ~7.5k$, which is more than a comparable RAM mac, but not 30k you previously had to shell for top of the line GPUs. So you get a "prosumer" 5090 with a bit more compute and 3x VRAM, in a computer that can sell for <10k$ and beat any mac at both inference and training, for things that "fit" in that VRAM. Macs still have the advantage for larger models, tho. The new DGX spark should join that market soon(tm). But they allegedly ran into problems on several fronts. We'll have to wait and see. | |||||||||||||||||