>I acknowledge that he has predicted some things in a falsifiable way and turned out correct, but this isn’t one of them. You’re reading hindsight into the text.

I read that blog years ago. Believe me, my opinions are not hindsight.

>Incorrect. He was speculating on what would happen if one gave GPT-2 “near-infinite training data and compute.” It’s a thought-experiment, not a prediction. Near-infinite amounts of anything is a fantasy.

Thought experiments can generate predicitons. His claim was essentially: If you scale data and compute sufficiently, this technology can learn enough of the underlying structure of mathematics to write proofs.

This is meaningful when others around you are saying this is a dead end and that the technology is fundamentally incapable of this regardless of degree of investment and scaling. It shows a much better calibrated sense of the potential of the architecture than those who said otherwise.

If your objection is that "near infinite" makes it insufficiently quantitative to count as a falsifiable forecast, then fine. But at that point we're mostly arguing over what deserves the label "prediction" rather than whether Scott correctly identified an important capability the architecture could develop.

And i'm not trying to say this makes Scott (or the lesswrong crowd) geniuses.