Do we even need to get LLMs to create code in the end then? Can we imagine a future where the thing simply does it all, you deploy the LLM like a docker image and have it process all requests directly. Sounds far fetched today, coming to your cluster in five years.

In theory yes, but you would need LLMs that basically have frontier level capabilities but with zero latency, meaning you enter a prompt and as you type the prompt is regenerating instantly like some kind of autocomplete.

I think that would feel like crack for some people who are addicted to using or building stuff with LLMs.