| ▲ | wez470 an hour ago | ||||||||||||||||
How cost effective had it been? I don't want to rely on 3rd parties either, but my experience is it's super expensive to run locally versus using a provider (both in terms of hardware costs and ongoing electricity usage). | |||||||||||||||||
| ▲ | lrvick an hour ago | parent [-] | ||||||||||||||||
No third party can ever provide the privacy and sovereignty of being able to run locally with abliterated models. I would do it this way even if the providers were much cheaper as I value those properties a -lot-. In practice though, I can produce about 3B tokens/month of Qwen 3.8 Flash Next which is the model I crank almost 24/7. That would cost something like 1k/mo on OpenRouter. My setup costs about $300/mo in electricity, though getting solar soon to drop that to zero. | |||||||||||||||||
| |||||||||||||||||