You have no evidence that self improvement can work at generalized tasks. Go is a simple game with a clear win condition, but deep strategy and near infinite permutations of how a game plays out. Winning a game of Go is a task well suited to machine learning.

One might say you are engaging in wishful thinking by believing it’ll just continue to work across all domains. The world is much bigger than a Go board.

Anyone who works on stuff, especially novel stuff, quickly exposes the weaknesses of LLMs.

> You have no evidence that self improvement can work at generalized tasks.

I never made that claim. I just said it's way too early to rule it out: there is no logical reason why AIs will (always) need to have a human in the loop, and we don't have enough experience with LLMs to know what their true limits are.

I referenced AlphaGo not because the game of Go is exactly like every other task AI might perform in the future, but because the evolution from AlphaGo (which was trained on human games) to AlphaGo Zero (which was not) shows that at least in certain domains, it's not only possible to take the human out of the loop, it can actually make AI perform better.

I'm not claiming this will definitely be possible in every other domain, but people who state it definitely won't be, are jumping the gun.