Very interesting conversation I'm still listening too. One bit I disagreed with is that I still think that an LLM's context is more like a person's sensory memory[1] than their working memory. The way that data falls off the end of the buffer regardless of how much attention it provokes is entirely unlike our own working memory. On the other hand a reasoning model's scratchpad seems to fit the analogy much better.

[1]https://en.wikipedia.org/wiki/Sensory_memory