|
| ▲ | jobs_throwaway an hour ago | parent | next [-] |
| It depends what you mean by locally. I don't foresee running a model on my laptop anytime soon to power a coding agent. Far more likely is an infra team at my company operating an open source model on cloud infrastructure. When they're already paying $1000 / month / dev, it starts to pencil pretty quickly. |
|
| ▲ | simplyluke an hour ago | parent | prev | next [-] |
| > that you can run locally That's doing a lot of work here. The future I see isn't most companies buying hundreds of thousands in hardware to run models, it's them adding a line item to their AWS bill. Inference costs on the larger hosted open source models are dramatically lower than the frontier labs API pricing. |
| |
| ▲ | apocalyptic0n3 an hour ago | parent [-] | | > it's them adding a line item to their AWS bill That's the future Amazon sees too. We just had a week long session with the AWS team and they pushed that to us multiple times. |
|
|
| ▲ | PeterStuer an hour ago | parent | prev | next [-] |
| Many business tasks do not need the latest frontier models. I have a production system running since early GPT-4o. It now runs with GPT-5.2, not for improvements, but because it is cheaper. I could invest in switching to a local model, I tried and it works well enough, but api costs for this task are so low, it barely scratches $30/month. So I am using the local machine for other things and leave the inference on OpenAI, for now. |
|
| ▲ | applfanboysbgon 26 minutes ago | parent | prev | next [-] |
| Opus 4.6 is a February model. Every time this subject comes up it seems like people post intentionally misleading things and move the goalposts. The goalpost we've been bludgeoned with over and over again is that, in particular, Everything Changed in November 2025. That GPT 5.2 and Claude 4.5 were the inflection point. That is actually 6 months ago. And DeepSeek 4 is already there. > run locally You can't run DeepSeek locally on consumer hardware[1], but you can on enterprise hardware, and enterprise spend is the subject of this conversation -- and even if you aren't self-hosting, it doesn't matter, because you can just get your inference from one of the the many companies serving DeepSeek, who trivially undercut the pricing of OpenAI/Anthropic because they didn't have to spend hundreds of billions on training frontier from scratch but instead only invest in supporting inference, which is already profitable. [1] Since this misconception comes up all the time, I'll go ahead and pre-empt it: no, training a 32b parameter model on outputs from DeepSeek and running that locally is not "running DeepSeek", despite the hundreds of stupid articles and Youtube videos making that idiotic claim that they're running it on a 5090. |
| |
| ▲ | simonw 21 minutes ago | parent [-] | | > You can't run DeepSeek locally on consumer hardware Maybe not DeepSeek v4 Pro, but I've run DeepSeek v4 Flash on my 128GB MacBook Pro using antirez's carefully quantized https://github.com/antirez/ds4 and it's impressive. | | |
| ▲ | applfanboysbgon a few seconds ago | parent [-] | | Oh sure, yeah, that's nothing to sneeze at either. I think unqualified "DeepSeek" should generally refer to the full model, though, especially in the context of GPT5.2-grade quality. |
|
|
|
| ▲ | PunchyHamster an hour ago | parent | prev [-] |
| But one will be in few months. And then you have choice of paying say $100k for hardware and pay just power cost (or pay someone to do that for you), or pay way, way more for your team to have access to marginal improvement. And 5% worse model for 10% of the price of the bleeding edge will be worth it for majority of people |
