I was fairly skeptical all the way until a month ago, when I got access to Github Copilot's preview of ChatGPT5 agent, it writes code, tries it out, and fixes any problems. Up until last month, it was letting the LLM write code, then have errors, and spend forever trying to figure out the code, or figuring out how to get the LLM to fix it (usually making it worse each time!).