Until the agents start touching things that already worked well before and break them.

They do need a competent developer operating them. That has nothing to do with whether nor not specs have value.