All that is completely UI-agnostic and not related to GUI/TUI distinction in absolutely any way. Keypress automation is terrible, if you have to resort to jank like this you'd better use something like AT-SPI (which no TUI implements) or at least DOM manipulation and userscripts on web, although I've never seen a use case for that, sanely designed agents are malleable enough to avoid it.

The point is that I agree with the OP, terminals are fundamentally unfit for applications like agents that are not REPL and need complex interfaces. All attempts to fit them into it I've seen are 99% cargo cult, like the entire TUI zoo that keeps layering the cruft just because people are culturally averse to clean first-principle designs or put pretentious "aesthetics" first. Just look at the amount of misconceptions voiced ITT. This incidental complexity is destined to snowball into web/Electron-like Rube Goldberg machines pretty fast, and I guarantee you will hate it. It's already spinning out of control with agents.