Zvec: A lightweight, fast, in-process vector database

wittlesus 15 minutes ago | parent | next [-]

Genuine question for anyone running in-process vector search in production: when do you reach for something like this vs. an external service?

The appeal of in-process is obvious — no network hop, simpler deployment, lower latency on small-to-medium datasets. But I'm curious about the operational story. How do you handle index updates while serving queries? Is there a write lock during re-indexing, or can you do hot swaps?

The mceachen comment about sqlite-vec being brute-force is interesting too. For apps with under ~100K embeddings, does the algorithmic difference even matter practically, or is the simpler deployment story more important than raw QPS?

▲

simonw 5 hours ago | parent | prev | next [-]

Their self-reported benchmarks have them out-performing pinecone by 7x in queries-per-second: https://zvec.org/en/docs/benchmarks/

I'd love to see those results independently verified, and I'd also love a good explanation of how they're getting such great performance.

	▲	ashvardanian 4 hours ago \| parent \| next [-]
		8K QPS is probably quite trivial on their setup and a 10M dataset. I rarely use comparably small instances & datasets in my benchmarks, but on 100M-1B datasets on a larger dual-socket server, 100K QPS was easily achievable in 2023: https://www.unum.cloud/blog/2023-11-07-scaling-vector-search... ;) Typically, the recipe is to keep the hot parts of the data structure in SRAM in CPU caches and a lot of SIMD. At the time of those measurements, USearch used ~100 custom kernels for different data types, similarity metrics, and hardware platforms. The upcoming release of the underlying SimSIMD micro-kernels project will push this number beyond 1000. So we should be able to squeeze a lot more performance later this year.
	▲	panzi 4 hours ago \| parent \| prev [-]
		PGVectorScale claims even more. Also want to see someone verify that.

▲

clemlesne 7 hours ago | parent | prev | next [-]

Did someone compared with uSearch (https://github.com/unum-cloud/USearch)?

	▲	neilellis 5 hours ago \| parent [-]
		That I would like to see too, usearch is amazingly fast, 44m embeddings in < 100ms

▲

OfficialTurkey 4 hours ago | parent | prev | next [-]

I haven't been following the vector db space closely for a couple years now, but I find it strange that they didn't compare their performance to the newest generation serverless vector dbs: Pinecone Serverless, turbopuffer, Chroma (distributed, not the original single-node implementation). I understand that those are (mostly) hosted products so there's not a true apples-to-apples comparison with the same hardware, but surely the most interesting numbers are cost vs performance.

▲

cjonas 4 hours ago | parent | prev | next [-]

How does this compare to duckdbs vector capabilities (vss extension)?

▲

jgalt212 2 hours ago | parent [-]

Yes, nothing on that or sqlite-vec (both of which seem to be apples to apples comparisons).

https://zvec.org/en/docs/benchmarks/

	▲	mceachen an hour ago \| parent [-]
		I maintain a fork of sqlite-vec (because there hasn't been activity on the main repo for more than a year): sqlite-vec is great for smaller dimensionality or smaller cardinality datasets, but know that it's brute-force, and query latency scales exactly linearly. You only avoid full table scans if you add filterable columns to your vec0 table and include them in your WHERE clause. There's no probabilistic lookup algorithm in sqlite-vec.

▲

_pdp_ 5 hours ago | parent | prev | next [-]

I thought you need memory for these things and CPU is not the bottleneck?

	▲	binarymax 5 hours ago \| parent [-]
		I haven’t looked at this repo, but new techniques taking advantage of nvme and io_uring make on disk performance really good without needing to keep everything in RAM.

▲

skybrian 6 hours ago | parent | prev [-]

Are these sort of similarity searches useful for classifying text?

	▲	CuriouslyC 5 hours ago \| parent \| next [-]
		Embeddings are good at partitioning document stores at a coarse grained level, and they can be very useful for documents where there's a lot of keyword overlap and the semantic differentiation is distributed. They're definitely not a good primary recall mechanism, and they often don't even fully pull weight for their cost in hybrid setups, so it's worth doing evals for your specific use case.
	▲	neilellis 5 hours ago \| parent \| prev \| next [-]
		Yes, also for semantic indexes, I use one for person/role/org matches. So that CEO == chief executive ~= managing director good when you have grey data and multiple look up data sources that use different terms.
	▲	esafak 5 hours ago \| parent \| prev \| next [-]
		You could assign the cluster based on what the k nearest neighbors are, if there is a clear majority. The quality will depend on the suitability of your embeddings.
	▲	OutOfHere 6 hours ago \| parent \| prev [-]
		It altogether depends on the quality and suitability of the provided embedding vector that you provide. Even with a long embedding vector using a recent model, my estimation is that the classification will be better than random but not too accurate. You would typically do better by asking a large model directly for a classification. The good thing is that it is often easy to create a small human labeled dataset and estimate the error confusion matrix via each approach.