As I see it there are currently 3 broad classes of LLM usage:

1. enhanced autocomplete -- either line (e.g. JetBrains small language models) or longer predictions;

2. chat -- asking questions related to the task you are doing;

3. tool assisted/agentic -- letting models interact with tools to perform tasks.

I personally find 1 & 2 helpful for my development tasks. For 3, I'm cautious of using it for several reasons:

1. lack of control of what gets sent to the model (e.g. if you have confidential user data, proprietary code and/or data, access tokens/private keys);

2. letting the models run arbitrary commands (modifying the system, deleting files, etc.);

3. not having control over the design process, formatting of the files, etc.

Not having control of the design process is not the right way to use the leading models IMO due to the ROI of how much you get out from controlling the architecture phase.