I think there is growing confusion in this discussion. I am saying "I see the fixed nature of ASICs creating a barrier to their adoption despite how cheap they are".

Many of the responses seem to say "but there is a market for cheap models".

There is a world where we have cheap models and we don't have ASICs powering them. Things like TPUs and NPUs, which are programmable, are likely to fill that role. They are optimized for inference while also allowing different (and updated) models to run on them.

Given two companies competing on the cheap end. First company goes TPU, second goes ASIC: who wins? My bet is on TPU since they can update their model, even if their hardware is slightly more expensive and slightly slower, since the optionality of new models beats the performance gap. That may not hold forever but given the pace and volatility of the current LLM market, I believe it will hold for some time.

▲

margalabargala 17 hours ago | parent [-]

The reason you are getting "but there is a market for cheap models" as a response, is people saying in a roundabout way "but soon models may not need to be updated".

Models like Kimi 3, GLM 5.2, or even Fable 5 for that matter are reasonable to burn to.ASIC because they are over the threshold of "good enough to be generally useful", something that will continue to be true in the future.

Most people do not need the latest model, they need a sufficient model. If I had Fable 5 on an ASIC, I imagine I would use that and ignore paying API rates for Fable 6.

▲

stillpointlab 17 hours ago | parent [-]

> but soon models may not need to be updated

Which is why my original post mentioned the volatility in models. We aren't just doing research on frontier, there is a huge amount of research on quantization, distillation, etc. that is changing the landscape at the low-end almost as much as it is changing on the frontier.

And it is also why I mention revealed preference. What feels sufficient / "good enough" today is a moving target. This isn't just a question of what you want, it is a question of what is economically viable for the entity that will be designing, manufacturing and marketing this ASIC of which you speak.

▲

margalabargala 16 hours ago | parent [-]

That moving target is different for everyone and their use case, and for me, it's already passed. I remember using Opus 4.5 and thinking "this is good enough to do everything I want it to do properly" and I stand by that. Paying $10k for unlimited Opus 4.5 running at 9k tok/s would IMO allow me to do more, faster, than putting the same money into $200/mo anthropic subscriptions for the latest and greatest.

This is me personally. The calculus is different for other people. But I suspect Kimi 3 is pretty darn close to that tipping point for an awful lot of people.

	▲	daadx 12 hours ago \| parent [-]
		Unless he's talking about personal use then he's a bozo - he won't get to decide. The managers of the firm will. They dont care that its faster unless it translates into the financials. They want lower costs, higher revenues - explain how it fits brudda.