Why assume token costs will increase as opposed to become commoditized?

Self-hosting takes either cloud- or on-prem-engineering chops, which are in short supply. In my view, that becomes the primary driver for lowering token costs. Secondary are the alternative model sources like Z.ai and Moonshot, which between open weights and lower prices help to drive commoditization. But there is a political risk with these, for example, Z.al is on the U.S. Department of Commerce Entity List, which blocks American companies from selling advanced technology and goods to the firm.

▲

binary132 5 days ago | parent | next [-]

you seem to be discounting or unaware of the possibility of a) automating a significant chunk of new buildouts, and b) domestic companies seeking a slice of the “sell open model hosting for less than competitors” pie.

I truly believe the cost per token for most people will eventually go approximately to zero even in the absence of subsidy unless there is a dramatic state intervention to curtail free computing and hardware R&D and manufacturing.

▲

tomrod 5 days ago | parent [-]

Fair callout. Strongly discounting (b) and (a) is underspecified for me to respond reasonably.

Your premise of full token commoditization has a major wrinkle though. GPU-centric data centers with never allow tokens to go to zero since you have a 1.5 year technical deprecation and 5 year life on the GPUs. Maybe Cerebras or similar chip builders, after writing model weights to silicon, will improve those returns on capital and enabling the scaling you are envisioning.

From my perch, AI at the edge is severely underserved, and the GTM for the hyperscalers mostly ignore it. So if someone figures out how to scale AI at the edge (nvidia + HF perhaps) then I think your premise is on point.

	▲	visarga 5 days ago \| parent [-]
		Maybe they still want to serve the feee tokens if that gets them good training data.

▲

visarga 5 days ago | parent | prev [-]

On macs you just install a desktop app. What can be simpler? Decent Qwen models already doing some simpler types of agentic work.

	▲	tdeck 5 days ago \| parent [-]
		Sure but if the customer installs the app on their own computer how do companies charge them a recurring subscription fee? Adobe can get away with it but most cannot.

▲

Yizahi 5 days ago | parent | prev | next [-]

That would be even worse for them, since then current customers of these AI-middleman corpos would just start deploying their own self-hosted and private LLM services. In before the complexity argument, there may well be some completely different middlemen corpos, who will assist in such deployments and initial ramp-ups.

▲

altern8 4 days ago | parent | prev | next [-]

Right now they're losing billions/year, it's not sustainable

▲

bigstrat2003 5 days ago | parent | prev | next [-]

Because that's been the playbook of every tech company for a decade or two. It should be our expectation that this stuff will get an expensive rug pull the instant the companies can get away with it.

	▲	DonsDiscountGas 5 days ago \| parent \| next [-]
		> the instant the companies can get away with it LLMs have no lock in. Switching costs are zero. It's really not clear that instant will ever come
	▲	bluefirebrand 5 days ago \| parent \| prev [-]
		Yes, this exactly. People in this thread are discussing it as if it is a question of pure economics. It's not! It's a question of "how much do we think we can get away with charging"

▲

pj_mukh 5 days ago | parent | prev [-]

There seems to be a weird self-fulfilling narrative here:

1) "AI companies will go bust because tokens will cost too much" >> 2) "Don't let them build data centers that will sit empty" >> 3) Demand outstrips supply by a lot >> 4) Token costs too much.

This is no different from the NIMBY's that block all the housing then sit in a huff saying "see! everyone's leaving. good thing we didn't build anything!" when the rents go up.

▲

sensanaty 4 days ago | parent [-]

Talk about a goomba fallacy. The people who are against the data center buildups (which is increasingly becoming a bipartisan issue the world over thanks to the antics of the AI psychos) are not the same people complaining about token prices. I'm sure many of them would even be gleeful if the AI companies went belly up

▲

pj_mukh 4 days ago | parent [-]

Doesn't matter if they're different people or if it's politically popular.

It's how the cycle would go, and in the end the same people will complain about how AI "is only for rich people" (how high token prices will express itself in populist language)

The exact thing happened to housing.

▲

sensanaty 4 days ago | parent [-]

Again, the people who are anti-AI don't give a shit about token prices. They will care that AI is only for rich people, but this won't be due to token prices, but because only the wealthy will be the ones with any kind of job (if the dream of the AI overlords is to come to fruition anyways) or income stream due to displacement caused by AI

	▲	pj_mukh 4 days ago \| parent [-]
		"dream of the AI overlords" I don't care what their dream is, I care about what's happening IRL. Please read the original article again, AI is clearly being used by workers to expand their productivity (and their earning capacity). The main thing that will screw that up is if token prices spike. "AI is only for rich people, but this won't be due to token prices" Statement doesn't make sense. Higher prices means unaffordability. I don't know why that would change for AI tokens. Higher token prices is what will make sure advanced models are only available to capital owners This feels "unintuitive" only because it's not the dominant narrative, but that's not a measure of accuracy. "Everybody believe so" is not a good defense.