Please pardon the pure speculation incoming. Yes, caching the answer doesn't seem useful. Caching the progression, the graph, may be. This is similar to making code changes with ed(1) instead of editing in vi.

The transform script(s) are cached and can be played back or adjusted. Surely for some breadth of question inputs, they map more often to similar answers--but not static answers; instead, evented edits.

It's nearly untenable for a human to keep private edit scripts to generate code changes. The extra steps for custom regex, essentially one-offs for a shared codebase, is inefficient. But maybe not to an LLM.

I don't understand how this fits LLM architecture at all