You still need people in the mix that understand the scope, scale and impact of the exploits/bugs found. Just letting agents go wild is how you get slop over time... You can probably get away without them to an extent, but I'd suggest that you're likely to increase the risk of errors and misbehavior in practice over time by not checking agent work.

Even checking human work is often a shortcoming of processes in practice.