| ▲ | killerstorm a day ago | |
It's not so much they _turned_ wrong, they were obviously wrong at the time he made statements, or shortly after. He is just extremely biased against transformers, and it's easy to see why - transformers are already invented, and as a scientist he wants to invent something new. More specifically: * he said that autoregressive transformer is fundamentally flawed: if it makes one error, all further generated tokens make no sense; that's, obviously, bullshit: there are many ways to implement backtracking, and there are many papers demonstrating backtracking at a time he did that lecture. And, of course, it just doesn't matter - reasoning model can just write "Oh, that's wrong. Let's try a different thing". - it's just not derailed by one mistake. * he said that GPT-1000 can't possible have understanding of physical world, when GPT-4 already existing at that time could solve a problem he described. So it's not like he was bad at predicting things, he's intellectually dishonest. | ||
| ▲ | redox99 a day ago | parent [-] | |
Yeah that slide of him "proving" autoregressive is flawed was really the nail in the coffin. I stopped listening to him after that. | ||