This whole business model of trying to shave off or arbitrage a fraction of the money going to OpenAI and Anthropic just sucks. And it seems precarious. There's no honest way to resell tokens at a profit, and everyone knows it.

▲

thelastbender12 2 days ago | parent | next [-]

Sorry, how is this new pricing anything but honest? They provide an editor you can use to - optimize the context you send to the LLM services - interact with the output that comes out of them

Why does not justify charging a fraction of your spend on the LLM platform? This is pretty much how every service business operates.

	▲	hu3 2 days ago \| parent [-]
		There's now greater incentive for Zed to stuff more content in the prompts to inflate tokens used and thus profit more. Or at least be less zealous. This is not a new concern. And is not unique to Zed.

▲

drakythe 2 days ago | parent | prev | next [-]

For companies where that is their entire business model I absolutely agree. Zed is a solid editor with additional LLM integration features though, so this move would seem to me to just cover their costs + some LLM integration development funds. If their users don't want to use the LLM then no skin off Zed's back unless they've signed some guaranteed usage contract.

▲

Havoc 2 days ago | parent | prev | next [-]

>There's no honest way to resell tokens at a profit, and everyone knows it.

Agree with the sentiment, but I do think there are edge cases.

e.g. I could see a place like openrouter getting away with a tiny fractional markup based on the value they provide in the form of having all providers in one place

	▲	Lalabadie 2 days ago \| parent [-]
		The issue with a model like this (fixed small percentage) is that your biggest clients are the most incentivized to move away. At scale, OpenRouter will instead get you the lower high-volume fees they themselves get from their different providers.

▲

dinvlad 2 days ago | parent | prev [-]

The whole business model even for OAI/Anthropic is unsustainable.. they are already running it at a huge loss atm, and will do for the foreseeable future. The economics simply doesn't work, unfortunately or not

▲

porridgeraisin 2 days ago | parent [-]

OAI/Anthropic/Big AI players today never ever need to become profitable... The same way roads never need to become profitable by themselves. This per token charge is like tolls, it has nothing to do with the costs.

AI, just like cloud was before this, is being treated as "infrastructure". The reason people invest in roads is not to make money from the road itself, but rather recuper all the losses from the new extremely profitable businesses that will come on top of this infrastructure. To the stock ticker, this looks like a boom+bust cycle. But really, it's a bust+boom cycle as far as the investors are concerned. Saudi Arabia don't give a hoot about transformers, but they know if they invest their large amounts of capital in the new infrastructure - huge models in expensive datacenters using cities worth of power, they can invest in the many new hyper profitable businesses on top. Also helps that it locks in demand for oil, I guess :P

A similar example for cloud would be the gajillion cloud SaaS, DevOps, cloud security whatever businesses that only exist today because the whole cloud infra segment ran unprofitable for a long time and created the infrastructure for all this.

These new businesses will not be multi hundred-billion dollar businesses, no. They will all be million dollar businesses. But you'll have a million such businesses. Everyone that's stuffing money into AI today is hoping that there will be a huge product layer [1] on top of this new infrastructure once it's all consolidated at a few major players allowing "infra" costs to drop considerably, which they can milk.

The stock of all these companies going high is just wall st making it easy for people to buy debt from this infrastructure. If they're indisciplined, they will co-mingle this debt with normal people's daily lives debt and sell it as one, and everything gets affected badly. If they're not, then the stock market will complain, but normal folks will be kind of OK.

[1] well, everyone has this same idea, meaning there also a lot of short term investors trying to make gains while openai/nvidia/etc are on their way up, sort of like greater fool investing, but let's ignore them for the purposes of this argument.

[Note] of course, whether cloud/ai/whatever are actually useful infra that deserves to be force-created is up for debate.. many disagree, not me though.

▲

stalfosknight a day ago | parent | next [-]

Wasn't this the thinking behind the massive overbuilding of fiber in the 90s?

	▲	porridgeraisin a day ago \| parent [-]
		Yes, this was the thinking behind the telco boom(or bust depending on who you ask), cloud in the 2010s too, was the same.

▲

osn9363739 2 days ago | parent | prev [-]

I don't even know where to start with this. Comparing private companies to roads? This makes no sense right? These companies, like roads never have to turn a profit?

	▲	porridgeraisin 2 days ago \| parent [-]
		It might sound weird, but yes. The economic perspective is the same. You're making a huge upfront unprofitable investment into something, so that a lot of insanely profitable investments can be made 10 years in the future, that uses the result of todays investment as infrastructure. Whether it's as important as roads, all of that is not relevant from an economic standpoint -- I'm just explaining the rationale behind the investments today. > These companies It's useful to mentally think of it as an implicit "team effort" among all relevant, rich companies. Openai (just an example) themselves are being "allowed" by everyone else in the "team" to not make money, because everyone knows that at the end there will be a gajillion new product companies they can all make their money back on. Openai/stargate is in that sense just a "front" for this massive infrastructure investment. Sama/Microsoft themselves will make money that way, either by building those products (just think of the ai integrations in MS enterprise in 10 years...) or by investing in others building those products. Defense folks have investments into this for the same reason. That's why I said it's similar to roads, no one expects to make money from roads. But on those nice roads amazon will create a 1-day delivery product, charge you 10$ a month for it, and investors will make money from that [1]. [1] yes yes this specific example is bunk and historically wrong, but wanted to drive home the parallel with a small example