I think it also depends on how well defined requirements are. I can have Claude nearly one shot a Terraform provider but it's just taking 2 API contracts and combining them together (the provider contract and the API you're automating)

On the other hand, some things are incredibly product intensive and poorly defined and it seems LLMs will latch on to an arbitrary direction with much lower success rate.

It is also worse in "brown field" where there's lots of tribal knowledge and technical debt. Two systems may both exist that look very similar but oftentimes there's the one on life support that almost no one uses or cares about and the LLM doesn't know it should largely ignore that until it's thoroughly documented (which can be a significant undertaking)