The biggest issue with business users (who don’t know SQL) writing SQL with an LLM is that there is no one to validate that query and now that business user will treat that LLM response as canon to share in meetings, presentations, and with clients. The LLM may have forgotten a filter, used the wrong definition of revenue, or misunderstood the user’s intent by writing a query that answers their question in the most literal way.

That is the crux of the issue. No amount of semantic layers and context will help this until an LLM can read the user’s mind to remove ambiguity in the prompt.

I see most of the benefits of LLMs to be used by analysts who know SQL to work more productivly.

I recently shared a similar rationale for why a custom query language can help users even in the age of LLMs.

https://nchammas.com/writing/custom-query-language-design#wh...

Users can't "own" queries they don't understand. An LLM helping a user avoid annoying syntax errors is very useful; but helping a user write a query they ultimately cannot understand is fraught with the problems you describe.

also, the same metric can be used in different ways and for different purposes across a business, so you might need different validation rules and definitions.

Maybe one separate semantics validation layer could help, but costs 2x or possibly Nx if you need to recover and turn a wrong query into a correct one