Compute has already lost value for me. Six months ago I thought you needed a 1T+ model to be useful coding. Now I am able to get by just fine with a 27b model.

I see two factors converging to cause a collapse of this house of cards:

1. People are realizing that what they need isn't more general intelligence, it's more specialization. A small but well tuned coding model, a small but well tuned customer service model, a small but well tuned document explorer.

2. Specialized hardware - TPUs and NPUs - especially coming out of china. The latest GLM model was trained and runs on Huawei hardware. Nvidia is only worth so much because they are the biggest and best provider of the kind of compute needed to run llms, but the export bans mean china has a lot of incentive to topple that monopoly.

The amount of compute we need to do the things llms do is falling rapidly, the number of people who can provide that compute is rising.

LLMs needing less compute would actually be a good thing for Nvidia due to Jevons paradox. Right now token costs are an impediment to using AI more broadly, and more efficient models would help adoption in cases where AI has proven to be useful, like coding.

https://en.wikipedia.org/wiki/Jevons_paradox

> People are realizing that what they need isn't more general intelligence, it's more specialization. A small but well tuned coding model...

It’s not quite as simple as that. Several studies have shown the opposite: models trained on more diverse knowledge tend to cross-pollinate across domains. So a more generalized model can actually perform better than a specialized one.

That’s why you’re not seeing tons of tiny models (one for Python, one for Pascal, one for Rust, etc).

I think what will keep the industry afloat, all else failing, is the surveillance industry! Nothing like a fat reoccurring cheque from the government to check if little Jimmy is committing thought crime!

That's unless the code produced in the future is much more complex than today's.

I've been thinking about that and that's why Nvidia's prices are surprising to me. Investors should know that better than me so there must be something I don't know

You're not considering video which OpenAI opted out of when they retired Sora.

Generative video requires significantly more computing power and energy than generative text.

OpenAI is fucked, compute is still needed, it's just them that isn't.

They need the right harness and either your help it auto produces in time enough content to further improve.

If you reshuffle your argument, and apply the same facts you get to a similar conclusion but with a drastically different spin.

> it's more specialization

China, constrained by hardware, and talent (not to slight the Chinese, but they are limited to domestic resources - and much of the US effort is very international). They did, what the Chinese do, and optimized the process of production, and drastically lowered the cost of development of their models. Cheeper to build, cheaper to run is just good economics.

Meanwhile in the us, we have open AI doing "experiments" - it looks like the costs around the hugging face hack are going to be about the same as China would spend on building out one of their smaller efforts (several million dollars). (Depending on whos numbers you trust, the fact that I can even make this claim should make you raise an eyebrow).

Go back to the 80s' and "expert systems" - most people will tell you that for their time, they were amazing, and useful. People would have loved to have more of them but they were so cost prohibitive that we all but abandoned them for serious use. The US frontier labs seem to have forgotten this lesson and their calls to "slow down" look like an excuse to "cut the waste so we can move to making money".