> if I give you $15B, you will probably make a lot more than $15B with it

"probably" is the key word here, this feels like a ponzi scheme to me. What happens when the next model isn't a big enough jump over the last one to repay the investment?

It seems like this already happened with GPT-5. They've hit a wall, so how can they be confident enough to invest ever more money into this?

I mean, this is how semiconductors have worked forever. Every new generation of fab costs ~2x what the previous generation did, and you need to build a new fab ever couple of years. But (if you could keep the order book full for the fab) it would make a lot of money over its lifetime, and you still needed to borrow/raise even more to build the next generation of fab. And if you were wrong about demand .... you got into a really big bust, which is also characteristic of the semiconductor industry.

This was the power of Moore's Law, it gave the semiconductor engineers an argument they could use to convince the money-guys to let them raise the capital to build the next fab- see, it's right here in this chart, it says that if we don't do it our competitors will, because this chart shows that it is inevitable. Moore's Law had more of a financial impact than a technological one.

And now we're down to a point where only TSMC is for sure going through with the next fab (as a rough estimate of cost, think 40 billion dollars)- Samsung and Intel are both hemming and hawing and trying to get others to go in with them, because that is an awful lot of money to get the next frontier node. Is Apple (and Nvidia, AMZ, Google, etc.) willing to pay the costs (in delivery delays, higher costs, etc.) to continue to have a second potential supplier around or just bite the bullet and commit to TSMC being the only company that can build a frontier node?

And even if they can make it to the next node (1.4nm/14A), can they get to the one after that?

The implication for AI models is that they can end up like Intel (or AMD, selling off their fab) if they misstep badly enough on one or two nodes in a row. This was the real threat of Deepseek: if they could get frontier models for an order of magnitude cheaper, then the entire economics of this doesn't work. If they can't keep up, then the economics of it might, so long as people are willing to pay more for the value produced by the new models.

Do they have a function to predict in advance if the next model is going to be profitable?

If not, this seems like a recipe for bankruptcy. You are always investing more than you're making, right up until the day you don't make it back. Whether that's next year or in ten or twenty years. It's basically impossible to do it forever - there simply isn't enough profit to be had in the world if you go forward enough orders of magnitude. How will they know when to hop off the train?

if you're referring to https://youtu.be/GcqQ1ebBqkc?t=1027 he doesn't actually say that each model has been profitable.

He says "You paid $100 million and then it made $200 million of revenue. There's some cost to inference with the model, but let's just assume in this cartoonish cartoon example that even if you add those two up, you're kind of in a good state. So, if every model was a company, the model is actually, in this example is actually profitable. What's going on is that at the same time"

notice those are hypothetical numbers and he just asks you to assume that inference is (sufficiently) profitable.

He doesn't actually say they made money by the EoL of some model.

That can only be true if someone else is subsidizing Anthropic's compute. The calculation is simple: Annualized depreciation costs on the AI buildout (hundreds of billions, possibly a trillion invested) are more that the combined total annualized revenue of the inference industry. A more realistic computation of expenses would show the each model line very deeply in the red.

> According to Dario, each model line has generally been profitable: i.e. $200MM to train a model that makes $1B in profit over its lifetime.

Surely the Anthropic CEO will have no incentive to lie.

Well how much of it is correlation vs causation. Does the next generation of model unlock another 10x usage? Or was Claude 3 "good enough" that it got traction from early adopters and Claude 4 is "good enough" that it's getting a lot of mid/late adopters using it for this generation? Presumably competitors get better and at cheaper prices (Anthropic charges a premium per token currently) as well.

What about inference costs?

My guess would be that it parallels other backend software revolutions.

Initially, first party proprietary solutions are in front.

Then, as the second-party ecosystem matures, they build on highest-performance proprietary solutions.

Then, as second parties monetize, they begin switching to OSS/commodity solutions to lower COGS. And with wider use, these begin to outcompete proprietary solutions on ergonomics and stability (even if not absolute performance).

While Anthropic and OpenAi are incinerating money, why not build on their platforms? As soon as they stop, scales tilt towards an apache/nginx type commoditized backend.

"May 21, 2025

This is an extraordinary moment.

Computers are now seeing, thinking and understanding.

Despite this unprecedented capability, our experience remains shaped by traditional products and interfaces."

I don't even want to learn about them every line is so exhausting

I was expecting a wedding or birth announcement from that picture framing and title.

"We would like to introduce you to the spawn of Johnny Ive and Sam Altman, we're naming him Damien Thorn."

And don't forget the furnace furnace: gas/coal to power all this.

The economics will work out when the district heating is run off the local AI/cash furnace.

Like trying to have a lunch conversation with coworkers about anything other than work

I wouldn't describe it as a singularity point. I don't mean that they'll get models to design better model architectures, or come up with feature improvements for the inference/training host frameworks, etc.

Instead, I mean that these later-generation models will be able to be fine-tuned to do things like e.g. recognizing and discretizing "feature circuits" out of the larger model NN into algorithms, such that humans can then simplify these algorithms (representing the fuzzy / incomplete understanding a model learned of a regular digital-logic algorithm) into regular code; expose this code as primitives/intrinsics the inference kernel has access to (e.g. by having output vectors where every odd position represents a primitive operation to be applied before the next attention pass, and every even position represents a parameter for the preceding operation to take); cut out the original circuits recognized by the discretization model, substituting simple layer passthrough with calls to these operations; continue training from there, to collect new, higher-level circuits that use these operations; extract + burn in + reference those; and so on; and then, after some amount of this, go back and re-train the model from the beginning with all these gained operations already being available from the start, "for effect."

Note that human ingenuity is still required at several places in this loop; you can't make a model do this kind of recursive accelerator derivation to itself without any cross-checking, and still expect to get a good result out the other end. (You could, if you could take the accumulated intuition and experience of an ISA designer that guides them to pick the set of CISC instructions to actually increase FLOPS-per-watt rather than just "pushing food around on the plate" — but long explanations or arguments about ISA design, aren't the type of thing that makes it onto the public Internet; and even if they did, there just aren't enough ISAs that have ever been designed for a brute-force learner like an LLM to actually learn any lessons from such discussions. You'd need a type of agent that can make good inferences from far less training data — which is, for now, a human.)

By all accounts, what's in Google's racks right now (TPU v5e, v6e) is vaguely H100-adjacent, in both raw performance and supported model size.

If Google wants anything better than that? They, too, have to wait for the new hardware to arrive. Chips have a lead time - they may be your own designs, but you can't just wish them into existence.

Well considering it has been possible to produce similar doctored images for decades at this point, I think we can conclude legislation has not been fast enough.

That article is nothing to do with AI, really.

Thats the thing, what’s “illegal” will challenge our whole society when it comes do dynamically generated real interactive avatars that are new humans

When it comes to sexually explicit content in general with adults, all of our laws rely on the human actor existing

FOSTA and SESTA is related to user generated content of humans, for example. They rely on making sure an actual human isnt being exploited and burdening everyone with that enforcement. When everyone can just say “thats AI” nobody’s going to care and platforms will be willing to take that risk of it being true again - or a new hit platform will. That kind of content currently Doesnt exist in large quantities yet, until a video model ungimped can generate it.

Concerns about trafficking only rely on actual humans not entirely new avatars

regarding children there are more restrictions that may already cover this, there is a large market for just adult looking characters though and worries about underage can be tackled independently. or be found entirely futile. not my problem, focus on what you can control. this is whats coming though.

people already dont mind parasocial relationships with generative AI and already pay for that, just add nudity

It's going to be really weird when huge swaths of the internet are illegal to visit outside the USA because you keep running into that kind of AI generate "content".

Why is this illegal btw? I mean whats stopping an AI company from releasing a proper NSFW model? I hope it doesn't happen but I want to know what prevents them from doing it now.

Not my experience. Have you used Veo 3?