It's amusing to read people in the past writing about the prospect of superhuman intelligence. The real problems have turned out to be different. Sycophancy and hallucinations, which are part of being confidently wrong, remains a big problem. Needing square miles of data centers was an issue in 1950s science fiction, and disappeared by the 1980s. Yet now they're being built, with private funding and the prospect of profit. The need for way too much training data indicates something is still wrong with the current approach.

None of that was predicted.

I predicted on this site in 2016 the massive social and economic impacts AGI would have and specifically when RL data loops are not available to anyone but major players:

https://news.ycombinator.com/item?id=12168228

I even wrote up a whole article that specifically called RL loop based development as the future:

https://medium.com/@andrewkemendo/the-ai-revolution-will-be-...

> Reinforcement Learning tasks rely on ridiculous amounts of data. Whereas with traditional software architecture, where you accomplish tasks through explicit task instruction, RL trains for tasks based on millions of tests through a reward system. Most importantly once you have trained it to some minimum level, if you deploy it correctly, then it should continue improving — so long as you bake feedback into the UX. Imagine that instead of telling excel what to do, you and every other user will have a conversation with excel, improving the system incrementally.