| ▲ | zamadatix 2 hours ago | |
Make RAM half the price it was before and you'd still have very few people trying to do local AI on it because it's just so damn slow for the task. Having 12 channels of DDR5 in an Epyc system with NVMe storage is still one of the slowest ways I can run LLMs locally. | ||
| ▲ | ASalazarMX 2 hours ago | parent [-] | |
Funny, I'm running local LLMs in a modest iMac M4, they are slow indeed but not utterly slow. A dedicated system should be way faster I'd guess. | ||