My favorite LeCun was when he was talking about LLMs understanding the real world. In the clip he essentially said "If I have a water bottle on this book and I push the book, the water bottle will move as well. A 3 year old can tell you this. But an LLM has no concept of this effect since it's not written explicitly in any books". The clip is then immediately followed by asking this exact question to a number of LLMs and giving exactly the correct answer.

From the interview, I see he still harps on about the limitations of LLMs and harps on about real world AI. I just can't take him seriously because pretty much everything he says is wrong and hypothetical. He still can't point to a single thing that an LLM fails at that require his so called "world models"

He'll just throw some random stuff out there like "we're not at level 5 driving yet" despite having hundreds of millions of miles driven autonomously every day. He just seems to keep holding on to his beliefs (much of them economic) and refuses to adjust his priors.

> "we're not at level 5 driving yet" despite having hundreds of millions of miles driven autonomously every day

Those are two very different things; you know that, right? You could even drive billions of miles, but that doesn't mean we have attained Level 5.

Level 5 definition: https://www.zf.com/products/en/cars/stories/6_levels_of_auto...

We're still far from solving real-world physical interaction, even with the world knowledge of LLMs incorporated into VLAs. It's the reason robotics startups trying to deploy into people's homes are so reliant on teleop; the hardware is sufficient (we know that because the robots can solve tasks when piloted by a human), but the intelligence is still insufficient.

I don't think JEPA is necessarily the solution (although I'm bullish on world models), but I don't understand why people feel so infuriated about LeCun. The reason he is famous today is because he spent years taking a contrarian stance, working on neural networks when they were seen as dead and buried. He was eventually proven right. Nowadays he holds strong views which contradict the LLM zeitgeist --- what's shocking about that?

[dead]