Just brute force your embeddings

emschwartz 17 hours ago | parent | next [-]

This works especially well if your embedding model was trained to perform well with quantized embeddings. Binary + hamming distance = incredibly fast.

This post is from 2024 but I wrote about using this technique in https://emschwartz.me/binary-vector-embeddings-are-so-cool/

▲

softwaredoug 17 hours ago | parent [-]

Hamming w/xor+popcount is the only thing I can make numpy do faster than float32 dot products :)

int8s, float16s are all fairly slow. I suppose it’s because BLAS does float32/64 very fast.

▲

emschwartz 16 hours ago | parent [-]

Yeah that makes sense. I took the optimizations of my hamming distance library to a bit of an insane level. I wrote this about the first round https://emschwartz.me/unnecessary-optimization-in-rust-hammi... The second round brought the best result down to 0.8 nanoseconds per comparison on x86 with SIMD (for a batch of 1000).

Separately, I’m a huge fan of your writing about search! It’s been very helpful while I’ve been building https://scour.ing.

	▲	softwaredoug 16 hours ago \| parent [-]
		Thank you! <3

▲

aitchnyu 8 hours ago | parent | prev [-]

Umm, is pgvector relevant to this usecase?