A while ago I tried something similar but tried to boil it down to the simplest thing I could come up with. I ended up making a standard maze into a first-person perspective where it unfolds one step at a time, and seeing if a model could solve it without re-entering areas it had already fully explored. They all failed.

Setup: a maze generator generates a square maze and puts the start and end on opposite corners. It doesn't show the full maze to the LLM, just has the LLM explore it one square at a time like a text adventure. It tells the LLM which directions of its current position has walls (relative direction: front, back, left, right). The LLM then chooses between move forward, turn left, turn right. That's pretty much it.

First attempt: Just maintain all the above in a chat, step by step. It'd get lost pretty quickly and start re-exploring already-explored area quite readily. Not very surprising, as we all know they can get lost in long chat threads. The chat model seemed to just go forward or turn right forever (which can work in some mazes), whereas the thinking model did seem to avoid backtracking until it got to T-junctions of a wrong way, where it always seemed to go back and forth forever.

Second attempt: After each step, tell the LLM to "externalize" everything it knew about the maze, and then feed that to a brand new LLM context. The idea was to avoid long chat context problems and see if the LLM could adequately represent its internal state and knowledge such that a "new" LLM could take over. This really didn't end up working any better. The biggest problem was that sometimes it would think that "turn left" would also change the position, and sometimes not. There were other issues too, so I didn't go much further with this approach.

Third attempt: Tell the LLM the premise of the game, and tell it to create a python state machine that stores all the state information it would need to represent its progress through the maze, and then to emit specific keywords when it needed to interact with it (and I added some code that served as a proxy). This also didn't work great. The state machine was close, but one thing it always forgot to do was relate index with direction. So if it's "in cell (5, 5) and facing up", it wouldn't know whether "forward" would be an increase or decrease in the x or y index.

I was also humored by its sycophancy here. I'd ask it

"Would adding a map to the state machine output be useful?"

"Yes, that is a great idea, let's do that!"

It'd do a great job of adding the map, but then I'd ask, "Does a map create more opportunity confusion?"

"Yes, that's an excellent insight, let's remove it!"

"No, really, you're the LLM, you're the one who's going to be using this app. I'm asking you, what do you think?"

"Whatever you want to do, just tell me"

Eventually, as the OP pointed out, these costs do add up pretty quickly. All I was after was "does externalizing the state help solve some of the long chat context problems", and the answer was "no" enough for me.

EDIT: Note that in all cases, they 100% emitted valid commands. And also I never noticed a case where "move forward" was chosen when there was a wall in front of them, nor "turn" when they were in the middle of a corridor.