Remix.run Logo
usef- 6 hours ago

Of course not. But I believe your Windows example was implying fundamental tech got worse.

The original "worst" quote is implying SOTA either stays the same (we keep using the same model) or gets better.

People have been predicting that progress will halt for many years now, just like the many years of Moore's law. By all indications AI labs are not running short of ideas yet (even judging purely by externally-visible papers being published and model releases this week).

We're not even throwing all of what is possible on current hardware technology at the issue (see the recent demonstration chips fabbed specifically for LLMs, rather than general purpose, doing 14k tokens/s). It's true that we may hit a fundamental limit with current architectures, but there's no indication that current architectures are at a limit yet.