You're validly critiquing where it is now.

The hype people are excited because they're guessing where it's going.

This is notable because it's a milestone that was not previously possible: a driver that works, from someone who spent ~zero effort learning the hardware or driver programming themselves.

It's not production ready, but neither is the first working version of anything. Do you see any reason that progress will stop abruptly here?

▲

1024core 7 hours ago | parent | next [-]

Not a huge fan of @sama, but he is quoted as saying: this is the worst these models will every be!

Puts all criticism in a new perspective.

▲

slopinthebag 7 hours ago | parent | next [-]

That's like Bill Gates saying XP is the worst Windows will ever be

▲

usef- 7 hours ago | parent | next [-]

Not Windows: Operating systems. We did get more capable operating systems. The point of the quote is "this is the worst the SOTA will ever be".

If Windows XP were fully supported today I still wouldn't use it, personally, despite having respect for it in its era. The core technology of how, eg OS sandboxing, security, memory, driver etc stacks are implemented have vastly improved in newer OSes.

▲

slopinthebag 7 hours ago | parent [-]

You're just moving the goal posts unfortunately. The point is that positive progress is never actually guaranteed.

	▲	usef- 6 hours ago \| parent [-]
		Of course not. But I believe your Windows example was implying fundamental tech got worse. The original "worst" quote is implying SOTA either stays the same (we keep using the same model) or gets better. People have been predicting that progress will halt for many years now, just like the many years of Moore's law. By all indications AI labs are not running short of ideas yet (even judging purely by externally-visible papers being published and model releases this week). We're not even throwing all of what is possible on current hardware technology at the issue (see the recent demonstration chips fabbed specifically for LLMs, rather than general purpose, doing 14k tokens/s). It's true that we may hit a fundamental limit with current architectures, but there's no indication that current architectures are at a limit yet.

▲

k1musab1 7 hours ago | parent | prev [-]

Aged like milk.

▲

cactusplant7374 7 hours ago | parent | prev [-]

That assumes he is all knowing.

▲

democracy 2 hours ago | parent | prev | next [-]

>> Do you see any reason that progress will stop abruptly here?

I do. When someone thinks they are building next generation super software for 20$ a month using AI, they conveniently forget someone else is paying the remaining 19,980$ for them for compute power and electricity.

▲

staplers 7 hours ago | parent | prev | next [-]

People abstract upon new leaps in invention way too early though. Believing these leaps are becoming the standard. Look at cars, airplanes, phones, etc.

After we landed on the moon people were hyped for casual space living within 50 years.

The reality is it often takes much much longer as invention isn't isolated to itself. It requires integration into the real world and all the complexities it meets.

Even moreso, we may have ai models that can do anything perfectly but it will require so much compute that only the richest of the rich are able to use it and it effectively won't exist for most people.

▲

slopinthebag 7 hours ago | parent | prev [-]

> Do you see any reason progress will stop abruptly here?

Yeah, money and energy. And fundamental limitations of LLM's. I mean, I'm obviously guessing as well because I'm not an expert, but it's a view shared by some of the biggest experts in the field ¯\_(ツ)_/¯

I just don't really buy the idea that we're going to have near-infinite linear or exponential progress until we reach AGI. Reality rarely works like that.

▲

selridge 7 hours ago | parent | next [-]

So far the people who bet against scaling laws have all lost money. That does not mean that their luck won’t change, but we should at least admit the winning streak.

▲

slopinthebag 7 hours ago | parent [-]

You mean Moore's law? Which is now dead?

▲

selridge 7 hours ago | parent | next [-]

No I don't mean that. I mean the LLM parameter scaling laws. More importantly, it doesn't matter if I mean that or Moore's law or anything else, because I'm not making a forward looking prediction.

Read what I wrote.

I'm saying is if you bet AGAINST [LLM] scaling laws--meaning you bet that the output would peter out naturally somehow--you've lost 100% so far.

100%

Tomorrow could be your lucky day.

Or not.

▲

slopinthebag 5 hours ago | parent [-]

This weekend I had 100% success at the blackjack table, until I didn't and lost.

I guess we'll see :)

	▲	selridge 4 hours ago \| parent [-]
		You gonna go read up on some 0% success rate strategies on the way? What I’m saying is that we act as though claims about these scaling laws have never been tested. People feel free to just assert that any minute now the train will stop. They have been saying that since the Stochastic parrots. It has not come true yet. Tomorrow could be it. Maybe the day after. But it would then be the first victory.

▲

_zoltan_ 7 hours ago | parent | prev [-]

it's not dead. it's enough to look at GB200/GB300 vs Vera Rubin specs.

▲

azakai 6 hours ago | parent | prev | next [-]

At the very least, computers are still getting faster. Models will get faster and cheaper to run over time, allowing them more time to "think", and we know that helps. Might be slow progress, but it seems inevitable.

I do agree that exponential progress to AGI is speculation.

▲

conception 6 hours ago | parent | prev | next [-]

You think all AI companies will never release a better model days after they all release better models?

That is a position to take.

▲

empthought 7 hours ago | parent | prev [-]

I know some proponents have AGI as their target, but to me it seems to be unrelated to the steadily increasing effectiveness of using LLMs to write computer code.

I think of it as just another leap in human-computer interface for programming, and a welcome one at that.

	▲	nitwit005 6 hours ago \| parent [-]
		If you imagine it just keeps improving, the end point would be some sort of AGI though. Logically, once you have something better at making software than humans, you can ask it to make a better AI than we were able to make.