>"Stage 2: “Can we tweak the prompt without deploying?”

Are we playing philosophy here? If you move some part of the code from the repo and into a database, then changing that database is still part of the deployment, but now you just made your versioning have identity crisis. Just put your prompts in your git repo and say no when someone requests an anti-pattern be implemented.

I think the core challenge here is that being able to (in "development") quickly change the prompt or other parameters and re-run the system to see how it changes is really valuable for making a tight iteration loop.

It's annoying/difficult in practice if this is strictly in code. I don't think a database is necessarily the way to go, but it's just a common pattern I see. And I really strongly believe this is more of a need for a "development time override" than the primary way to deploy to production, to be clear.