Remix.run Logo
▲ Aurornis an hour ago

> as a clear sign they are scared shitless of local models,

The number of people with hardware to run a model like that is very small.

It's going to be many years before hardware to run medium size local models is widely available. Running mid-size local models doesn't even register as a factor in pricing decisions right now.

It's such a small part of the market and will be for many years, until RAM and other hardware prices come down.

▲stymaar 17 minutes ago | parent | next [-]

> The number of people with hardware to run a model like that is very small.

True, but the number of people who pay the non-subsidized API price is also very small, and it's all in big companies who would have no issue buying or renting hardware for that purpose.

▲locknitpicker an hour ago | parent | prev [-]

> The number of people with hardware to run a model like that is very small.

That's not the problem anyone is facing, and analysis anyone is doing. At all.

The problem is the huge AI-related budget that companies who dove head first onto AI are facing.

Therefore the problem everyone is faced with boils down to whether they keep paying the likes of OpenAI a de-facto business tax, or they replace that tax with a one-off CapEx invoice followed by a utilities bill.

And you don't need a frontier model to do most things. In fact, the biggest challenge in using local models in agentic coding is the models supporting these agent-based tools.

> It's going to be many years before hardware to run medium size local models is widely available.

Nonsense. Speaking for myself, my employer already introduced a hard cap on AI credits while rolling out hardware support for local models, and even went to the extents of rolling out workstations with 64GB of RAM as a baseline. Consequently, we are using the likes of gemma4 for a fair share of everyday tasks. This has a transformative effect on your workflows because suddenly you don't ration tokens, and agent orchestration doesn't become a cash guzzler. So far the only constraints are tool support and token throughput, but that's not a concern.