If the data is this large then why not ask the llm to build a query

That's more-or-less what RAG is; a tool to surface relevant data so that AI can make informed decisions. Efficient retrieval is important so that AI doesn't need to ingest the whole corpus for each query.

My case:

With a very large dataset of totally unnormalized data, semantic folding is a silver bullet.

latency is often the reason, distilling interactions into significantly better retrieval is the second.