ZIM or not, I think the “LLM as optional sidecar” part is the right idea.

The durable asset is the knowledge base itself. A local model can be useful on top, but it should stay a layer, not become the dependency.