Remix.run Logo
▲ killerstorm a day ago

It's not so much they _turned_ wrong, they were obviously wrong at the time he made statements, or shortly after. He is just extremely biased against transformers, and it's easy to see why - transformers are already invented, and as a scientist he wants to invent something new.

More specifically:

* he said that autoregressive transformer is fundamentally flawed: if it makes one error, all further generated tokens make no sense; that's, obviously, bullshit: there are many ways to implement backtracking, and there are many papers demonstrating backtracking at a time he did that lecture. And, of course, it just doesn't matter - reasoning model can just write "Oh, that's wrong. Let's try a different thing". - it's just not derailed by one mistake.

* he said that GPT-1000 can't possible have understanding of physical world, when GPT-4 already existing at that time could solve a problem he described.

So it's not like he was bad at predicting things, he's intellectually dishonest.

▲redox99 a day ago | parent [-]

Yeah that slide of him "proving" autoregressive is flawed was really the nail in the coffin. I stopped listening to him after that.