IMO what's being called an 'Agent Development Environment' solves this issue for you. I like Orca but there's a few dozen of them.

Basically they give you a way to view the code the llm has generated/changed easily, annotate that code for the llm, and manage multiple agents and at once.