Indeed. A mouse that runs through a maze may be right to say that it is constantly hitting a wall, yet it makes constant progress.

An example is citing Mr Sutskever's interview this way:

> in my 2022 “Deep learning is hitting a wall” evaluation of LLMs, which explicitly argued that the Kaplan scaling laws would eventually reach a point of diminishing returns (as Sutskever just did)

which is misleading, since Sutskever said it didn't hit a wall in 2022[0]:

> Up until 2020, from 2012 to 2020, it was the age of research. Now, from 2020 to 2025, it was the age of scaling

The larger point that Mr Marcus makes, though, is that the maze has no exit.

> there are many reasons to doubt that LLMs will ever deliver the rewards that many people expected.

That is something that most scientists disagree with. In fact the ongoing progress on LLMs has already accumulated tremendous utility which may already justify the investment.

[0]: https://garymarcus.substack.com/p/a-trillion-dollars-is-a-te...

Why would the public care what was possible in 2015? They see the results from 2023-2025 and aren't impressed, just like Sutskever.

What exactly are they doing? I've seen a lot of hype but not much real change. It's like a different way to google for answers and some code generation tossed in, but it's not like LLMs are folding my laundry or mowing my lawn. They seem to be good at putting graphic artists out of work mainly because the public abides the miserable slop produced.

Not really.

Any fool could have anticipated the eventual result of transformer architecture if pursued to its maximum viable form.

What is impressive is the massive scale of data collection and compute resources rolled out, and the amount of money pouring into all this.

But 10 years ago, spammers were building simple little bots with markov chains to evade filters because their outputs sounded plausibly human enough. Not hard to see how a more advanced version of that could produce more useful outputs.