Isn't this what Apple tried with Siri? I don't see anyone use it, and adding an LLM to the mix is going to make it less accurate.

They wrote a whole ass paper about SLMs that do specifically this - expert small language models with narrow expertise.

And then went for a massive (but private and secure) datacenter instead.