I think the whole field of vector databases is mostly just one huge misunderstanding. Most of you are not Google or any other big tech company so so won't have billions of embeddings.

It's crazy how people add bloat and complexity to their stuff just because they want to do medium scale RAG with ca. 2 million embeddings.

Here comes the punchline, you do not need a fancy vector database in this case. I stumbled over https://github.com/sqliteai/sqlite-vector which is a SQLite extension and I wonder why no one else did this before, but it simply implements a highly optimized brute force search over the vectors, so you get sub 100ms queries over millions of vectors with perfect recall. It uses dynamic runtime dispatch that makes use of the available SIMD instructions your CPU has. Turns out this might be all you need. No need for memory a memory hungry search index (like HNSW) or writing a huge index to disk (like DiskANN).

There’s vss as duckdb extension too that builds a hnsw index.

https://github.com/duckdb/duckdb-vss

Since duckdb is already columnar, it goes brrrrr with single digit millisecond vector similarly lookups.

Might be all you need, except an open source licence:

> For production or managed service use, please contact SQLite Cloud, Inc for a commercial license.