Good article. We really need more nuance in the discussion of using LLMs for SE. Not all tasks are the same; we shouldn't expect to use the same process across them!

In my experience, the more constrained the LLM is, either by existing patterns in the code base or by prior art in general, the better the LLM is. When I've asked it do architectural or design work, it has been much less successful. That's one dimension on which LLM capabilities vary. There are certainly more!

Absolutely. When you narrow the solution space by adding more constraints, your probability of success increases. At least, that’s what I’ve noticed.

For vague, divergent tasks, the overall experience hasn’t been quite as good.