I agree. Compaction sucks, so I made tools that let the LLM selectively delete (and recall if needed) chunks of its context. You might want to try context bonsai if you're routinely hitting the auto-compaction wall.

https://github.com/Vibecodelicious/context-bonsai-agents

▲

tekacs a day ago | parent | next [-]

I've implemented a similar approach – although I'm surprised not to see mention of cache prefix busting in there!

▲

planckscnst a day ago | parent | next [-]

I did calculations on the prefix cache effect on costs of sessions where I used it and found that the removal of tokens from context had a much bigger effect on reducing costs than cache busting had on increasing them. I should re-do that and publish it.

▲

jeremyjh a day ago | parent | prev [-]

Yes the idea is cool but this could really hammer usage, especially just leaving it up to the agent to decide when to do it. I'm not surprised though, considering the github account is named "Vibecodelicious" and became active in December.

Poking around in the repo the whole implementation is an unsupervised LLM fever-dream.

	▲	planckscnst a day ago \| parent [-]
		It depends on what you mean by "unsupervised" - I've been strictly working through the agent, through specs the entire time, but it's been very supervised, I just leave the mistakes in-place and have it work from there. However at this point it can completely maintain itself. When a new version of Claude or OpenCode is released, it updates itself to work on the latest version. It can also add new implementations for harnesses pretty reliably. It's actually pretty fun to watch it at this point. "Make this work on Hermes agent and message me when you're done" and an hour later or so, I can go play with it in Hermes.

▲

adyavanapalli a day ago | parent | prev | next [-]

This is actually brilliant. I've seen other approaches where the context is selectively pruned, but never in a recoverable way.

P.S. Love the "bonsai" name btw!

	▲	8note 19 hours ago \| parent [-]
		ive built a variation that spins the chunks out as if they had been done by a subagent, and so the summary is like what the subagent returns i couldnt figure out a nice enough way to reload the conversation, and didnt want to have to worry about the cache rebuilds so its a bit languishing

▲

znnajdla a day ago | parent | prev [-]

I have a hunch that OpenAIs proprietary compaction endpoint actually does something similar on the backend.

	▲	tekacs a day ago \| parent [-]
		When I've asked Codex agents about things that were in their context window, they've never – to my experience – been able to actually retrieve something from before compaction when using the proprietary compaction endpoint. Instead, they've had to consult their actual transcript. So... at least as of a week ago or so, I don't believe so.