Could you explain your mental model of the situation a bit more?

Let's take something like GML 5.2 that is open source - anyone can run it on NVIDIA or AMD chips. People are currently making money by running GLM 5.2 Nvidia and AMD chips and selling inference on aggregators like openRouter.

So the cost to serve is like cost of Chip or financing for Chip + power and rackspace somewhere. We basically know that each year you will be able to server exponentially more inference per $ on Nvidia and AMD hardware as it gets to newer generations, so why would you expect the cost of inference on open models to also increase?

Like let's say not through a Chinese subsidized provider but like BaseTen https://www.baseten.co/pricing/ why would that get more expensive year over year?

> Could you explain your mental model of the situation a bit more?

I'm talking specifically about US-based AI companies / hardware companies and their circular investments and their customer-facing pricing.

> We basically know that each year you will be able to server exponentially more inference per $ on Nvidia and AMD hardware as it gets to newer generations, so why would you expect the cost of inference on open models to also increase?

Because the cost of training continues to increase while getting increasingly harder and harder to make gains. On top of that, there is the delay between investing in the datacenters, staff, running the training, operational costs of customer use, etc and actually recouping profit via customer sales. Your costs go up because their costs go up. Likely even more so now because of the increased risk the US Gov will shut down sales and use of their newest models. That risk will get bundled in with pricing.