Just yesterday I saw a talk from Spotify where they presented their own fast retrieval mechanism "Parallel Fusion Router" [1]. It's also in the "learned retrieval" paradigm - in their case they use a teacher LLM to train a much smaller router LLM how to retrieve, using an LLM-as-a-judge based reward function.

From my own experience, these learned methods from Apple, Spotify and others, are highly relevant to hyperscale only - when you are dealing with hundreds of millions of items to choose from (or more), and require super high relevance in the top-5. For all other use cases (up to a few million items) I found that just building a BM25 index, and giving the LLM a BM25 "search tool" and casting a "wide net" wins every single time compared to learned methods (though the latter are more publishable in recsys and icml).

[1] https://research.atspotify.com/2025/9/you-say-search-i-say-r...

It seems, that the amount of information that "typically" an person would want to work with, can be handled with pretty much any technique by the compute that most of us have, at a performance level that is satisfying.. It's only when you start to increase the information volume, decrease the compute, or increase the performance expectations, that it is really impactful to start to find methods that really make the most of the deep structures of that information to query it. Complexity of the original information and what accuracy means for your use case also comes into play I suppose.

I've been experimenting with hybrid memory retrieval using SQLite FTS5/BM25 for lexical search and dense embeddings from Nomic Embed Text v1.5 (768D), all-MiniLM-L6-v2 (384D), and Qwen3-Embedding (0.6B), with NumPy cosine similarity, Matryoshka-style dimensional truncation (64D → 768D reranking), and optional graph-based candidate expansion. In one 40-query benchmark, combining retrieval channels improved candidate recall from 30/40 with compact vectors alone to 40/40, although Top-1 accuracy was only 33/40. Another metric I was looking at was number of final candidates found to filter. If its very high, you can get better accuracy, but at the expense of resource use, and it can also lead to worse accuracy if your tying to choose top result efficently.. it makes a needle in a needle stack.

> I found that just building a BM25 index, and giving the LLM a BM25 "search tool" and casting a "wide net" wins every single time compared to learned methods

Funny, I did exactly this for my last RAG project. Works great for a small dataset.

> found that just building a BM25 index, and giving the LLM a BM25 "search tool" and casting a "wide net" wins every single time compared to learned methods

I've been saying this for a while now. Lexical is approximately the same thing as semantic when a frontier LLM is driving the search terms.