I love this idea! I agree that LLMs often don't have the knowledge that's actually important (a frustration I have with ChatGPT is that it remembers unimportant things and then over-indexes on them in its responses).

Totally agree. The current memory implementations feel a bit meh. you don’t really control what’s stored or how it’s used. I’ve been thinking of it more as a visual knowledge layer you can inspect + operate on, rather than something implicit inside the model. Curious if you’ve found any tools that handle this well today?