I think the future will be:

1. Someone prompts 2. Server searches for equivalent prompts, if something similar was asked before, return that response from cache. 3. If prompt is unique enough, return response from LLM and cache new response. 4. If user decides response isn’t specific enough, ask LLM and cache.