> Typing things faster doesn't make you get to the goal faster, since finding the overall design is the limiting thing.

interesting point and that matches my experience quite well. LLMs have been horrendous at creating a good design. Even on a micro scale I almost always have them refactor the functions they write

I certainly get a productivity boost at actually doing the implementation.. but the implementation is already there in my head or on paper. It's really hard to know the true improvement

I do find them useful for brainstorming. I can throw a bunch of code and tests at it and ask what edge cases I might want to consider, or anything I've missed. 9/10 of their suggestions I just skip over but often there's a few I integrate

Getting something that works vs creating something that'll do well in the medium-long term is just such a different thing that I'm not sure if they'll be able to improve at the second