I find mostly what I do is think about how to improve the product, what the user wants, and what information the LLM needs and how to get it into the context window.

Sure. Ultimately it's still you who is responsible for the code you ship I assume, no matter how it came about.