| ▲ | adastra22 a day ago |
| Serious question: why not run DGX Spark or Framework Desktop, at 30%-50% lower cost? |
|
| ▲ | haskman 15 hours ago | parent | next [-] |
| A DGX spark is cheaper than a mac mini m5 with half the memory, half the disk, and no cuda, and no Linux. Why not indeed! |
|
| ▲ | jnwatson a day ago | parent | prev | next [-] |
| M5 Ultra has 4-5x the memory bandwidth of both. 1.2 TB/s memory bandwidth opens up good performance on relatively large models. |
| |
| ▲ | vardalab a day ago | parent [-] | | you can get 4xGB10 for <20K so that gets you about the same tg and pp will be probably better. Power consumption though will be something like 200W idle so that's a bummer. And you get VLLM and SGLANG unlike them mac where one has to use oMLX (nice but not the same concurrency or cache performance)
I have 128GB M5 Max laptop and sill prefer to run things on other boxes in basement because its no fun to have the primary device being overloaded. | | |
| ▲ | OliverGuy 18 hours ago | parent [-] | | 4XGB10 is in quite a different price bracket than a single M5 128GB Mac Studio... |
|
|
|
| ▲ | HNDevsSuck a day ago | parent | prev [-] |
| [dead] |
