"Having your data locally" is definitely the biggest one.
- The general consensus seems to be that the frontier models are kind of (understandably) "benchmaxxed" towards coding benchmarks and have suffered in some other areas like writing, art, etc. Lot of tweaked/specialized models on huggingface
- Getting around the censorship in commercial models
- While local models don't match frontier coding agents, you can run them for multiple 24/7 with a long running goal (The fans in Macs are presumably not built to run 24/7, but this can be mitigated to a considerable extent IME by simply pointing a desktop fan at it, and you can throttle things in software. On the discrete GPU side of things, you can usually get big power savings without much performance impact by throttling the GPU clock; things are usually bottlenecked on memory)
- Even small 27B/35B models are getting darn good, definitely more than powerful enough for classification etc... you don't need 128GB to run them
- Right now, providers like OpenAI are running at huge losses. At some point, they are going to need to turn a profit, at which point they may get significantly more expensive, and "hosting your own" may make more economic sense than it does today -- and be a valuable professional skill, even.
- Even more speculative: perhaps the future for many applications is training and running your own hyper-specialized models. Right now, training your own is nowhere near cost effective on self-hosted hardware vs. renting GPUs
Note: whether any of these use cases are "worth it" is up to the individual. I'm not trying to sell anybody on the idea. I don't personally do much local LLM stuff at the moment, but it's attractive and I've been close to pulling the trigger...
