| ▲ | labcomputer 16 hours ago | |
A $2500 Mac Studio with M5 Max running Qwen3.8-27b or Gemma4:31b in oMLX gets about the same speeds you quoted... and it comes with a warranty and has enough memory not to choke at longer context lengths. | ||
| ▲ | ActorNightly 6 hours ago | parent [-] | |
Qwen3.8-27b can be ran at 100 tok/sec average https://github.com/L-Forster/open-jet/tree/master/megakernel. Gemma4:31b, which is the better model with more complex architecture (and thus can't be compiled into a megakernel), is gonna be way slower in prefill, and ~50% slower than a 3090 even with oMLX. You can't get away from much slower shared ram vs vram, and less parallelism compared to 3090. The other advantage of an eGPU setup, you can simply add/swap performance. Dual 4070s give you 24b of ram and you can still get about 80% of the performance of a single 3090, and actually you get faster prefill due to more parallelism. So in terms of money/usefulness, Mac always loses. | ||