Try to get your LLM of choice to find its way out of a labyrinth that you describe in text form. It's absolutely awful even with the simplest mazes. I'm not sure the problem here is memory, though? I think it has to do with spatial reasoning. I'd be willing to bet every company right now is working on spatial reasoning (at least up to 3D) and as soon as that is working, a huge amount of pieces will fall into place.

Spatial reasoning is weak, but still I frequently see models come up with the right answer in reasoning steps, only to make the wrong move in the following turn because they forget what they just learned. For models with hidden reasoning it's often not even possible to retain the reasoning tokens in context through multiple steps, but even if you could the context windows are big but not big enough to contain all the past reasoning for every step for hundreds of steps. And then even if they were the retrieval from context for abstract concepts (vs verbatim copying) is terrible.

Text is too lossy and inefficient. The models need to be able to internally store and retrieve a more compact, abstract, non-verbal representation of facts and procedures.