It seems that you’ve only read the first part of the message. X sometimes aggressively truncates content with no indication it’s done so. I’m not sure this is complete, but I’ve recovered this much:

> I read through these slides and felt like I was transported back to 2018.

> Having been in this spot years ago, thinking about what John & team are thinking about, I can't help but feel like they will learn the same lesson I did the hard way.

> The lesson: on a fundamental level, solutions to these games are low-dimensional. No matter how hard you hit them with from-scratch training, tiny models will work about as well as big ones. Why? Because there's just not that many bits to learn.

> If there's not that many bits to learn, then researcher input becomes non-negligible.

> "I found a trick that makes score go up!" -- yeah, you just hard-coded 100+ bits of information; a winning solution is probably only like 1000 bits. You see progress, but it's not the AI's.

> In this simplified RL setting, you don't see anything close to general intelligence. The neural networks aren't even that important.

> You won't see _real_ learning until you absorb a ton of bits into the model. The only way I really know to do this is with generative modeling.

> A classic example: why is frame stacking just as good as RNNs? John mentioned this in his slides. Shouldn't a better, more general architecture work better?

> YES, it should! But it doesn't, because these environments don't heavily encourage real intelligence.