| ▲ | Tepix 5 hours ago | ||||||||||||||||||||||
You can get two Strix Halo PCs with similar specs for that $4000 price. I just hope that prompt preprocessing speeds will continue to improve, because Strix Halo is still quite slow in that regard. Then there is the networking. While Strix Halo systems come with two USB4 40Gbit/s ports, it's difficult to a) connect more than 3 machines with two ports each b) get more than 23GBit/s or so per connection, if you're lucky. Latency will also be in the 0.2ms range, which leaves room for improvement. Something like Apple's RDMA via Thunderbolt would be great to have on Strix Halo… | |||||||||||||||||||||||
| ▲ | coder543 5 hours ago | parent | next [-] | ||||||||||||||||||||||
As you allude, the prompt processing speeds are a killer improvement of the Spark which even 2 Strix Halo boxes would not match. Prompt processing is literally 3x to 4x higher on GPT-OSS-120B once you are a little bit into your context window, and it is similarly much faster for image generation or any other AI task. Plus the Nvidia ecosystem, as others have mentioned. One discussion with benchmarks: https://www.reddit.com/r/LocalLLaMA/comments/1oonomc/comment... If all you care about is token generation with a tiny context window, then they are very close, but that’s basically the only time. I studied this problem extensively before deciding what to buy, and I wish Strix Halo had been the better option. | |||||||||||||||||||||||
| ▲ | Aurornis 5 hours ago | parent | prev [-] | ||||||||||||||||||||||
The primary advantage of the DGX box is that it gives you access to the nVidia ecosystem. You can develop against it almost like a mini version of the big servers you're targeting. It's not really intended to be a great value box for running LLMs at home. Jeff Geerling talks about this in the article. | |||||||||||||||||||||||
| |||||||||||||||||||||||