The problem with that approach is that “how do I handle this query”, and “did I handle that query ok” are both AI-hard problems, and must be tackled near the user, where the large model is not.

It also requires a relatively complex distributed architecture.

I worry about how successful they can be with this approach, in terms of best user experience.