Even if NVidia crashed and there ends up being an oversupply of GPUs or TSMC fab slots, this doesn't change that OpenAi and Google still need memory for their own hardware accelerators, so the price of boards wouldn't go down by much

▲

kodoman 3 hours ago | parent [-]

Part of me wonders (maybe naively) if with falling cost of AI inference and improvement of models means there is an oversupply of inference infrastructure. since we may reach a point where no one needs more and everything you could want can be done by the machine and adding more compute for most users is not any kind of benefit (like it was before LLMs there was an obvious ceiling where the majority of people did not need more compute).

I know Jevons paradox is a thing and I could see it going the other way as well but the first possibility seems to be ignored and I am not sure we can. Eventually demand for training might dry up as well if are models become largely very good and only a bit of fine tuning is required for most applications.

▲

mrgordon 2 hours ago | parent | next [-]

I don't see how it helps to think so narrowly about the computation needed solely for training when we are clearly headed to a future where tons of realtime applications (think robots) need to do insane amounts of processing ASAP while doing time-sensitive tasks.

IMO its copium to think that chips which have 10x more demand than supply are suddenly going to be easy to get as "AI is eating the world" like software did over the past decades.

	▲	PeterHolzwarth 2 hours ago \| parent [-]
		You are definitely right, mrgordon - this isn't an infinite scaling, "one extra copy cost essentially nothing," software issue. Video cards are physical. You can't just sell more copies with almost zero marginal cost like software. Each video card is a physical good that is subjected to all those physical laws.

▲

jiggawatts 2 hours ago | parent | prev [-]

Some years ago I came across this site: https://sites.google.com/site/1024cores

The author is a developer working on software that scales to 1,024 cores and greater -- which is within spitting distance now thanks to 256-core AMD EPYC processors. If you count hyperthreads as a "virtual core" then a dual-socket system gets you there!

That blog made me realise that almost all software can't utilise 1,024 cores, not even close! Microsoft SQL Server, as a random example, tops out at 64 cores for any single query.

Similarly, people couldn't utilise a machine that big, not with pre-AI software, except for certain embarrassingly parallel codes such as simulations and 3D graphics rendering.

Now... NOW... there's no upper limit. None. One person(!) can cheerfully utilise millions of cores, billions even, and there's no end in sight.

If you think one conversation thread in "ChatGPT" is all we need, you're not seeing the big picture of what's possible here.

Those math proofs OpenAI produced were created with tens of thousands of agents running in parallel, orchestrated by thousands more agents, and overseen by hundreds of agents, all triggered by someone typing as little as a sentence or two to start that avalanche of compute!

With that kind of AI power at your fingertips, huge, unthinkable projects become a fun exercise for the weekend:

- Translate the entirety of Wikipedia to every human language.

- Rewrite the entire Linux kernel to Rust in one go

- Rewrite all open source C code to Rust.

- Formalise every maths paper ever written.

Etc, etc, ... without end, without bound.