Yeah, that's probably true, but we're also seeing that there's still tons of inefficiencies in how LLMs are being run. Seems like every couple months there's some new technique to squeeze more performance out of less hardware. KV caching improvements, fast attention, speculative decoding, dynamic quantization, quantization aware training, etc.

That said, I recently replaced my five year old self-built PC (with a top-of-the-line desktop CPU, chipset, memory, and GPU of the time) with a new everything-the-best build, and while it's clear we're not keeping up with Moore's Law anymore, it's still 4-5 times faster for compute-intensive stuff, especially parallelizable tasks. We're still getting faster/cheaper. So, the time scale is maybe ten years rather than five.

Really the biggest concerns are not computers getting spectacularly faster, but 'intelligence' algorithms getting orders of magnitude better.

Drop the power requirements 1000 fold, and yea you will be able to make your own SOTA model on the cheap. The problem is the person that has a few exaflops of power will still leave you in the dust in the intelligence explosion that would happen after an event like this.

That has never been true, unfortunately. The 2005 top500 was led by bluegene/L achieving 280 FP64 TFlop/s.

Apple is talking about 17.5 FP16 TFlop/s on the iphone 17 neural engine. So 20 years later we are still nowhere near, not even at reduced precision.

Single clock speed hasn't had much of an upgrade, but the architecture for doing exactly what they are doing? That will improve for at least 5-10 years. There are both huge power gains from Processing in Memory (PIM) chips (70-80% discount in energy), and improvements to engineering to make memory cheaper and cheaper.

In five years I think you will be able to train a frontier modem for much less money than today and the power hungry hardware of today will be cheap second hand due to the power usage.

>but I don't think the rate of improvement of compute/watt will match the previous decades.

Unless we invest heavily in research and find new way to do chips. But I think there's not enough motivation and money to do that.

In fairness I completely agree with 99% of their comment.

I was nitpicking the use of the word “moat”. For it to be a moat, it’d need to be more expensive to traverse than to build.

Instead, the big AI firms are trying to create a monopoly on capital in an area where real costs are dropping 90% year over year.

I think in general stripping away the parts you agree with from the argument works great, because it strips away a whole lot of potential for ending up indirectly arguing over things that aren't in contention, and it often also defuses the rest when it turns out the core of the argument perhaps is much smaller than people are willing to get invested in.

OTOH I have often witnessed people agreeing without realizing it. I‘ve been able to defuse a bunch of arguments by pointing that out.

Yeah, more valuable than the comments I came to read (even if those are interesting too!)

Usually people are taught these techniques at the management courses. If you're at a BigCorp where they push managers through such courses - you can hear a lot of that stuff in your manager's speech if you pay attention to it.

[dead]

Yeah, I think a lot of the "slow down" rumblings we're hearing from OpenAI and Anthropic are really overtures toward regulatory capture; basically, "now that we're in the lead, we need to lock this shit down so nobody else can catch up."

I assume this is an honest question, in which case the answer is funding is not really finite.

They did get a bunch of investment grants from Trump, so your tax money (and power bills) are subsidizing them. They also arranged for ETFs to eliminate consumer protection rules to force everyone’s retirement to buy SpaceX/Anthropic/OpenAI shortly after IPO. That totals $3T in valuation (unless it goes up in first week trading), so your retirement is basically going to be weighing “AI bubble” similar to “MAGA”, and then everything else is rounding error. (The rule changes waive profitability requirements, and shorten the cooldown from IPO to indexing from a year to weeks).

I guess that’s one way to try to make capital finite.

or just "buy" your competition like big tech did

every major tech company literally have deal,ownership,alliance etc

they literally not gonna gobble up entirely to trigger anti-trust case

Yes, but surely AI are going to save us from the bloated stack of modern software.