Both clawdbot and pi have improved and expanded functionality a lot during the last 3 weeks, maybe worth another look? What you have described sounds a lot like the experience I’m having.

At this point I really don't want to even imagine the headache of implementing their codebase updates into all the custom scaffolding that my own fork relies on now. I think my plan going forward is to cherry pick features that sound interesting and re-implement on my own using my agent that has proper documentation on my personal configuration. Will check out what's new though.