I'm also trying to build something similar for agent orchestration where one terminal is controlling multiple terminals. I tried using tmux but it's very good at sending the initial text to the tmux sessions, but I've not been able to get an agent to have a proper back and forth controlling multiple tmux sessions. I know we can use send-keys, but reading the session or knowing when that session is complete is kind of up in the air. And then if the main orchestrator terminal has checked all the sessions to see if they're actually working and doing things, the main session kind of stop so I've kind of been thinking about a cron that periodically checks in and nudges it to check the sessions again. Are they still working? Do they need more guidance? Essentially having one terminal control others, but having that back and forth with the terminals has been pretty challenging to achieve. Have you gotten anywhere with this?

I recently built something just like this: https://github.com/sethdeckard/atria

It supports tmux, but you can use it without via embedded terminal. It also has native integration with a few select terminals that expose the right kind of APIs.

It sounds like maybe you haven't seen agent teams mode, which this project is using.

https://code.claude.com/docs/en/agent-teams