This is really interesting, and supports my priors that we are best served by seeking architectural improvements to produce efficient and effective "fluid" intelligence.

The models: 1) fail to induce the correct model of the world, even if they get components right. 2) are hobbled by their ability to memorize past worlds, when faced with true novelty. 3) are poor at adapting their models, when new data refutes them.

These are all core elements of deep intelligence, and IMHO the ability of current architectures to massively memorize, without a proper balancing motive for compression and refactoring, leads to these outcomes.