> ...used some kind of semantic search to show the user a search result from the local database immediately, and then followed that up with a very small LLM writing a text answer. This gives actually-good answers in a very reasonable timeframe

I have tried this very same thing on https://grimoire.bio. Large corpus of biological knowledge with detailed visualizations that are shown immediately upon searching, but adding LLM answer when missing/needed.