> Then why have they been lagging behind OpenAI and Anthropic

Because they're not desperate. Slow and steady wins the race, at this rate all Google has to do is wait for OpenAI and Anthropic to exhaust themselves on aggressive training, then they can casually amble along right past them.

Are you ready to call this race 5 years in? I'm waiting another 15 years and at least one or two new LLM architectures.

> Then why have they been lagging behind OpenAI and Anthropic for most of the last few years, and only briefly been at the frontier?

One possible explanation: because Google is a little bit more frugal and focuses on how to make providing AI models financially feasible - combined with some willingness to burn money so that they don't strongly fall behind on their AI models.

On the other hand, OpenAI and Anthropic at least formerly concentrated on building and providing the best models that they could with concerns about financial feasibility taking a backseat.

Just to be clear: I do have the impression that by now (likely because of pressure from investors) OpenAI and Anthropic take these financial concerns more seriously, but nevertheless Google's vs OpenAI's/Anthropic's "DNAs" concerning on what to focus on differ.

> Then why have they been lagging behind OpenAI and Anthropic for most of the last few years, and only briefly been at the frontier?

Because it's not an existential battle for Google. If OAI or Anthropic disappear from the absolute frontier for ~8 months the news cycle and churn will diminish them to the second rate. Google is processing near 4 quadrillion tokens every month, that's - I'm sure - significantly more than OAI or Anthropic, because Google is interested more so in their flash models and getting these competitive, which they are.

From a business perspective a frontier model does not make much sense anymore if you are not a startup. Neither for Amazon, nor for Google. Their clouds need models that are fast and perform well in their agent frameworks nothing were a frontier model excels at.

Most Google products even use flash lite underneath, so their frontier model is mostly used for distillation.

Alphabet issued a very oversubscribed 100-year bond with 6.1% yield earlier this year to raise capital for datacenter expension.

Meanwhile, Anthropic/OpenAI will struggle to survive the next 24 months on their current trajectory.

Lagging by what metric exactly? Is Toyota lagging behind McLaren? (company vs company)

Could it just be that Demis was checked out of the race and they lacked leadership?

The financials for Anthropic and OpenAI are likely borderline suicidal, google and co are publicly traded. Moreover, all innovations downstream to them dont they? Why not just stay slightly behind, especially given many have stake in those other companies?

True, but have other hyperscalers caught up to Google's AI data centers?: fully liquid cooled, torus networking(?), 100,000+ TPUs interconnected, etc.

Google is already on gen 8 of its TPUs and is certainly already working on the next version or two.

Moore's law stalled ~2015. Unfortunately, no way current models will run on the <100W thermal budget of a cell phone. Printing the weights directly into a chip would help efficiency a lot, but not enough.

They probably said the same thing about social media back in the day.

I'm sure the thinking out there, and hence investment, is all about how to tether the user to the most addictive, network-effected, incredibly deep, server-side, moat-able version of AI possible.

> China will always be generations behind until they crack domestic EUV

Are you sure?

--

China Just Built What TSMC Said Was Impossible

https://www.youtube.com/watch?v=Pk-w279ESHg

--

China Just Built What ASML Feared Most

https://www.youtube.com/watch?v=YiPgSm62fiM

Do you think that they won't?

yeah this would be one interpretation where "winner take all" could still be right. though with the recent accelerating release cadence, doesn't it seem like the head start / lead has been shrinking over time?

Based on historical developments the cost of compute will go down again eventually, decreasing the cost of training AI models of the same quality as today even further. That part is what I would be the most certain about.

https://www.anthropic.com/research/glm-5-3-and-the-spread-of...

It doesn't need to. It can use your cash to pay the people who are. Those in power like it more that way.

You say that because the 'most' existing models have done is hack governments and companies. Can't you think of worse things a model could do; accidentally or by instruction?

If anyone other than NVIDIA has a moat, they for sure never talk about it

The problem is twofold. One, even a monopoly AI provider wouldn't have pricing power against its suppliers. Its suppliers are energy, semiconductors, and real estate. Semiconductors maybe they could get some leverage on but energy and real estate have plenty of other buyers. Two, there's still no evidence of a runaway scenario (ie a small lead turns into a big lead over time) and there's still no evidence that there's some resource that you can deny everyone else that they can't build your product also. You can't hoard energy, compute, memory, data, human talent, or customers.

The net effect is that the most likely scenario is if one big lab fails, they will likely all fail. Their revenues are all correlated.

To go to your dotcom comparison, the winner will be the ones picking through the assets that were written down by orders of magnitude and trying new products with the technology until one sticks to the wall. But I don't know if a dramatic crash is guaranteed either.

I suspect this is going to end up like most services provided e.g. cloud stuff, balkanized between a couple major players and an assortment of DIY or less popular options if you don't like those ecosystems, plus some UX/DX focused wrappers that use the big players under the hood.

I think that would be a pretty satisfactory outcome compared to one hypercompany consuming trillions of dollars of the world economy.

“Divide the world” sounds ominous. Here’s another scenario to consider:

Internet access is not really unlimited, but for many people with fiber at home, it effectively is and we pay a flat rate.

Perhaps by the end of next year, most programmers will stop thinking about metered access for AI? For many people, the cheaper models (about as good as today’s frontier models) will be good enough.

Which might sound good, but the downside is that it will also be easier to build an AI botnet without the users paying for it noticing. Particularly when people are running AI inference on their own hardware.

Or, like airlines, the ones that are left will have great technology but be not so great from a business and financial perspective. To me AI seems like a commodity service.

THe problem with analogies is that they are imperfect.

I would argue those who already rule the world, will continue to do so.

What happens to OAI and Anthropic? No idea, probs go bust. Google just has to offer a half-decent offering in the long run and have a cost-advantage and it'll eventually knock OAI and Anthropic out as firms figure out what combination of models they want to be best for their economics and generating returns. Enterprises trust google over OAI and Anthropic. A clear signal of this was the Apple deal.

Dont forget those sweet returns fellas! CEO's are hired to make the owners wealthier. That is not gone.

> I guess if one of them hits singularity, it could in theory just wipe out all the rest

The story that some AI company might reach singularity and then "everything will be different" is another science-fiction story that executives of AI companies love to tell to justify the staggering amount of necessary investments and cash burn. :-)

Sure, it's happening...but, is IT happening? By that, I mean, we can see that the models are able to iterate at a pace and scale that humans can't match, and that provides gains in model performance and efficiency. But, humans are still needed in the loop, and not just because it's necessary for safety/alignment reasons. I don't think any significant discovery has been made by models on their own, and I don't know that LLMs will ever have the capacity to invent. They can synthesize from known data amazingly well, and since they know everything "known data" is extremely broad. But, the leaps, so far, have all come from humans.

So far, I don't think the models are capable of running away on their own. Of course, it would be playing with fire to not at least consider the risks of such a runaway scenario and build in safeguards against it. But, there is no model that can build a better model on its own, thus far, to the best of my knowledge (which is far more limited than the models, so maybe I should ask them).

Those people have a lot of overlap with the LessWrong crowd. They do not have RSI now and probably never will

The companies whose insane valuation is based on accomplishing thing X say they’re getting closer to accomplishing thing X?

At least they’re led by trustworthy and honest people or we’d need to take their claims with some dose of skepticism.

Is it recursive self improvement if Claude Code writes your pytorch for you?

Why does your outcome make any sense?

Already businesses that have more compute and access to data seem to eat the world around them. If, and ya its and if, we can make something that self learns into RSI it's not looking like any business that came before this.

Europe doesn’t even have cheap electricity

Yes, it may come down to how well they get efficiencies from scale.

People always compare the inflated API prices, but subscription prices of American models are competitive for the intelligence. You get >20x the subscription cost in tokens.

Agree entirely but that's the point, if it's a margin knife-fight with marginal product differentiation/pricing power nobody is going to be making bank.

If they have to recoup training costs then they don't have much choice