Seeing that R&D costs are the lion's share, I wonder if we are at a point where the focus can shift to improving the cost of inference.

Unless we are genuinely pushing to find AGI, at which point nothing matters, LLMs in their current form don't replace knowledge workers but are an effective force multiplier. How good is enough?

For instance, I pay about $1-2 a month for DeepSeek. It's not as sophisticated as Claude, but it still doubles my productivity as a SWE.

If Fable comes out and demands 50x the price of DeepSeek in order for Anthropic to make a profit on it, how much more productive would I be compared to my personal experience + DeepSeek? 3x? 50x?

Is it cost effective for a business to hire someone without SWE experience + Fable verses hiring someone with SWE experience and DeepSeek? When does R&D hit diminishing returns?

> I wonder if we are at a point where the focus can shift to improving the cost of inference.

There's always working on improving the cost of inference, but I don't think this is an area of R&D that will slow down. The reason is:

1. A better competitor model risks eating away at how much they can charge for inference (i.e. revenue) 2. Whoever unlocks AGI will unlock even more growth 3. Even when you unlock AGI, you'll want to throw gobs of money at it to improve itself and all sorts of things.

> If Fable comes out and demands 50x the price of DeepSeek in order for Anthropic to make a profit on it, how much more productive would I be compared to my personal experience + DeepSeek? 3x? 50x?

You're pricing it wrong and looking at it wrong. First, the per token price doesn't consider that a smarter model can end up using fewer tokens overall to achieve a result. Secondly, if the difference is between failing to accomplish the task and accomplishing the task, suddenly that 50x can seem like a bargain.

> Is it cost effective for a business to hire someone without SWE experience + Fable verses hiring someone with SWE experience and DeepSeek? When does R&D hit diminishing returns?

At this time, someone without SWE experience + <name AI model> vs someone good with SWE experience and <name another AI model> is a no-brainer. The AI model is an accelerant but the "no SWE experience" will be accelerated into a wall. Now maybe that doesn't matter for prototyping and certain other things, but anything in production the lack of experience will hurt them with things they won't even know about or even know how to look for it (e.g. slow, insecure, etc).

Let's put it this way, how much is 5% productivity bump worth to you?

If you're in the US and you're making 100k a year, that's worth 5k or $416/m. So you can buy two of the most expensive plans on the frontier models.

This focus on cost optimization is insane. Just use the frontier models. Even a marginal bump is worth whatever the hell they're charging, at least for now.

A thousand 90 IQ cannot do what a 145 IQ can do.

Similarly, some bosses might believe that they can hire 100 cheap, unmotivated SWEs to replace Linus Torvalds or Fabrice Bellard and achieve something slightly worse. But in certain areas, it doesn't work like that.

That's interesting on Deepseek. But I think as long as the models are still making noticeable gains with each iteration it's hard to say "good enough."

If a model comes and makes developer + Deepseek even a little more productive, from employers perspective, it'd still make sense to pay a lot of money for that.

Deepseek shines for personal usage because it's possible to use it however you want and whenever you want with no session/weekly limits stress because you use the API and it's priced very reasonably.

For clarity, inference is typically a COGS and therefore hits Gross Margin vs model training which would typically be in OpEx (where R&D lives) and would hit operating margin.

Even if you discount superhuman AI (which I would emphasize that frontier researchers do not discount and expect to see soon) think it’s still hard to have enough confidence that the ground is solid. Someone in 2024 trying to go down this route would have invested a lot of now-pointless effort into prompt engineering.

> Unless we are genuinely pushing to find AGI, at which point nothing matters, LLMs in their current form don't replace knowledge workers but are an effective force multiplier. How good is enough?

There's a non-negligible percentage of the industry who have a pseudo-religious belief in AGI, so I wouldn't be surprised if that was, in fact, the goal.

Who knows, maybe they'll stop once the money dries up.

> Unless we are genuinely pushing to find AGI, at which point nothing matters

I think the third coming out Jesus Christ in closer than AGI. Seriously, I dread how much of Silicon Valley is wrapped in this narrative of AGI and Singularity.

How can all these "rationalists" fail to see that this is what religion looks like: Faith and promises of heaven and hell.