Surely we all run these agents in isolated environments since months back already, right? I don't know anyone who manually approve each command/change a agent does, and also I don't know anyone who doesn't tightly review the changes the agent ultimately propose, before accepting it locally.

I'm sure people do that, I myself do, but just a couple weeks ago Github issue navigation was broken, you couldn't get past page 1 by the UI, a couple weeks before that github pages domain associations were broken too, just right now I saw on upwork a job ad where they were saying "we use AI to speed our workflow and you'll need to too" and then the list of urgent fixes ranged from: forms on administrators pages not working, values are being misplaced, forms somewhere else not saving on submission, ui bugs with panels not opening, sections on pages X with bugs on... So on and so forth.

I mean, these problems were there before (careless development) and it's probably possible to stabilise at some point with proper guardrails, it could also be that the first version was done with AI at meaningless cost compared to hiring someone, but I am a bit weary of this extending to ever more areas of software production without an accompanying increase in "true" model capabilities.

The more it kinda "works" the less people will be incentivised to review it properly and understand it - there's already people saying the future is agents talking to agents - when I saw minority report it sounded a bit too far-fetched but it might not be that far-fetched after all.