Remix.run Logo
bilater 2 hours ago

This kind of scarce thinking is almost always wrong and will lead you to down a sad dark HN loser path. Tokens will get cheaper. It costs OpenAI less money to serve GPT-5.5 than GPT-4. Ppl don't understand how much efficiency gains are being made with algo breakthroughs as well as hardware improvements that counter balance the demand rise. Open source models are 3-6 months behind. The world is your oyster stop worrying about how things will go wrong start thinking about what you can do today so you don;'t end up like the writer.

drzaiusx11 a few seconds ago | parent | next [-]

To be frank, we live in sad dark HN loser times.

blahblaher 2 hours ago | parent | prev | next [-]

"It costs OpenAI less money to serve GPT-5.5 than GPT-4." does it though? do you have the numbers? Or you just making stuff up?

ralusek 37 minutes ago | parent | next [-]

We used to not know, but now because open source models are being hosted and served by people whose only incentive is making profit on directly running inference, we have a ballpark idea.

alex_sf 30 minutes ago | parent [-]

There's no reason to think that the latest frontier models have similar inference costs to open source models.

It would be more surprising if the surrounding architecture hasn't significantly diverged. If it _hasn't_ significantly diverged, then given the performance difference it would imply that the frontier models have significantly greater param counts, which would result in a higher cost.

simianwords an hour ago | parent | prev [-]

GPT-4 (original API):

Input: $30 / 1M tokens

Output: $60 / 1M tokens

GPT-5.5:

Input: $5 / 1M tokens

Output: $30 / 1M tokens

Costs have been reducing by over 5x year over year. Inference cost concern is mostly performative.

https://simianwords.bearblog.dev/conclusive-proofs-that-llm-...

Edit: can't reply but companies aren't selling inference at loss. In the blog post I point to third party hosting of open models like Deepseek which are also going down. They are not VC backed.

I also point to Gemma 31B which you can run on your laptop today that beats most models from 2024.

zamalek an hour ago | parent | next [-]

What they charge people says nothing about what it costs them. Off the top of my head, one confounding factor is trying to win back marketshare from Anthropic.

We will only know the actually situation once Anthropic goes public and we can look at their books.

rafaelero an hour ago | parent [-]

I think it's pretty safe to assume they are not losing money on inference.

basilgohar 39 minutes ago | parent | next [-]

Based on what? They haven't even IPOed.

multjoy 36 minutes ago | parent | prev | next [-]

I think it’s safe to assume that they are bleeding cash.

alex_sf 29 minutes ago | parent | prev | next [-]

It's silicon valley and they are trying to aggressively grow. Your baseline assumption should be the exact opposite.

37 minutes ago | parent | prev [-]
[deleted]
IncRnd 36 minutes ago | parent | prev | next [-]

If you go to https://developers.openai.com/api/docs/pricing, you will see the actual prices, which do not match what you posted:

GPT-4.1 Input: $2.00 / 1M Tokens Output: $8.00 / 1M Tokens

raincole 29 minutes ago | parent [-]

The parent comment is correct. They are talking about GPT-4, which was really expensive by today's standard. After GPT4o came out, GPT-4 was completely forgotten.

Ygg2 an hour ago | parent | prev | next [-]

That's pricing.

Pricing has no correlation with profit. It can be artificially lowered to kill competition, and artificially inflated to maximize profit.

alex_sf an hour ago | parent | prev [-]

The price a company charges, _particularly_ a high growth VC-backed one, is a poor signal for their costs.

That blog post is not very compelling either. Without knowing details of the architecture, comparing the various frontier models to open models doesn’t make sense.

nicce 30 minutes ago | parent | prev | next [-]

This is only true if there is enough competition with equally good SOTA models. Otherwise, the price of the best models will keep increasing until people don't buy them anymore and use humans instead. Regardless of how much it costs to operate in reality. There is a reason why non-profit unnamed company turned to profit company.

aleqs 2 hours ago | parent | prev | next [-]

Very much agree - efficiency improvements are very real both on model and hardware side. The reliance on proprietary OpenAI/anthropic APIs is a problem though, one that will naturally resolve itself in the favour of self-hosted/open models.

raincole an hour ago | parent | prev | next [-]

It's just like saying every dependency is a ticking bomb. In a very strict sense, it's true. But it really doesn't matter for most businesses (and absolutely doesn't matter for early stage startups.)

drzaiusx11 2 minutes ago | parent [-]

Depends on the domain really, along with you and your user's aversion to risk. On average I'd agree your take holds true though.

new_account_100 2 hours ago | parent | prev | next [-]

Where do you work?

094459 an hour ago | parent | prev | next [-]

moores law ftw

runtime_terror an hour ago | parent | prev | next [-]

Lot of "trust me bro" vibes with this post

gjsman-1000 an hour ago | parent | prev | next [-]

> sad dark HN loser path

Assertion assertion assertion wishful thinking assertion.

Show, don't tell. Show us that we're wrong and this isn't a VC black hole. The CEO of Enron as late as September 2001 could've called every critic a sad dark loser with nobody challenging him publicly. Jim Cramer famously yelled anyone pulling their money from Bear Sterns in 2008 was "silly, do not be silly" exactly 8 days before their collapse and a -92% stock drop. In COVID, calling everyone paranoid and sensationalist about some mythical new flu was popular in December 2019 and gone by March 2020. How about Uber, the seeming go-to for how VCs can turn a money hole into a profitable business? The average price increase is now 18% per year and still going up, with an over 60% increase in 5 years. Does anyone still talk about the "sad dark HN loser path" of those who doubted VR in 2018? How's your VR startup doing?

mmcnl 36 minutes ago | parent | prev | next [-]

I don't think so. AI use is still very limited. For OpenAI and Anthropic and the AI boom to match their valuation, AI adoption needs to increase substantially. The current constraint is data centers. Pricing will be heavily influenced by market dynamics. Plenty of things that should be cheap aren't because of scarcity (simple example: RAM).

toasty228 4 minutes ago | parent | prev [-]

Datacenter GPUs pinned to 100% won't make it to their 3rd anniversary, models are getting larger and larger, they get smarter by running longer "reasoning" loops, there is no indication that it'll get better soon.

> Open source models are 3-6 months behind.

On the benchmarks included in their training set yes, not in real life