With generated code, the directory structure, interface design and general state management is usually a haphazard mess. Even with the best frontier models. But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt. Subtle things like which error states are "oh shit we need to bail" vs "this isn't a deal breaker." Sometimes it will ask, but more often than not it will just make a decision and it's often the wrong one. If I don't have a fully kitted out test suit and a good type checker to verify the final product against, the the whole looping thing is just useless to me and I'm back to reviewing every line of code it puts out and having to draw on my years of architecture experience to make sure we don't build a giant pile of trash.

Because they are designed to be used by managers who don't know how to answer these questions and don't want to be asked them. Just have the magic answers box pick something.

The generated code is fine at the functional level, the directory structure is usually the standard pattern for the given type of project.

The error types and codes, it will produce to spec.

If you type 'make me that thingy' - yes, it's probably not going to do what you want, but if you give it spec and guidance, it usually will.

The 'interface design' ... not very good though.

They're RLHF'ed to an inch of their lives to be able to one-shot complete tasks, since requiring human input defeats the purpose of being able to replace the labor force.

But once the insanity ends LLMs will be packaged as tools for developers to use to boost their productivity, and we'll consider them as we do IDE's and debuggers and stuff. But we have to get through this hype cycle first.