Actually no, it's not interesting at all. Vague dismissal of an outsider is a pretty standard response by insecure academic types. It could have been interesting and/or helpful to the conversation if they went into specifics or explained anything at all. Since none of that's provided, it's "OpenAI insider" vs John Carmack AND Richard Sutton. I know who I would bet on.

It seems that you’ve only read the first part of the message. X sometimes aggressively truncates content with no indication it’s done so. I’m not sure this is complete, but I’ve recovered this much:

> I read through these slides and felt like I was transported back to 2018.

> Having been in this spot years ago, thinking about what John & team are thinking about, I can't help but feel like they will learn the same lesson I did the hard way.

> The lesson: on a fundamental level, solutions to these games are low-dimensional. No matter how hard you hit them with from-scratch training, tiny models will work about as well as big ones. Why? Because there's just not that many bits to learn.

> If there's not that many bits to learn, then researcher input becomes non-negligible.

> "I found a trick that makes score go up!" -- yeah, you just hard-coded 100+ bits of information; a winning solution is probably only like 1000 bits. You see progress, but it's not the AI's.

> In this simplified RL setting, you don't see anything close to general intelligence. The neural networks aren't even that important.

> You won't see _real_ learning until you absorb a ton of bits into the model. The only way I really know to do this is with generative modeling.

> A classic example: why is frame stacking just as good as RNNs? John mentioned this in his slides. Shouldn't a better, more general architecture work better?

> YES, it should! But it doesn't, because these environments don't heavily encourage real intelligence.

It's a OpenAI researcher that's worked on some of their most successful projects, and I think the criticism in his X thread is very clear.

Systems that can learn to play Atari efficiently are exploiting the fact that the solutions to each game are simple to encode (compared to real world problems). Furthermore, you can nudge them towards those solutions using tricks that don't generalize to the real world.

Alex Nichol worked on "Gotta Learn Fast" in 2018 which Carmack mentions in his talk, he also worked on foundational deep learning methods like CLIP, DDPM, GLIDE, etc. Reducing him to a "seething openai insider" seems a bit unfair

Do you have an X account, if you're not logged in you'll only see the first post in the thread.

He did go into specifics and explained his point. Or have you only read his first post?

It’s not vague, did you only see the first tweet or the entire thread?