| ▲ | Arch485 2 hours ago | |
Re: Max dimensions: You can truncate your input vector specifically for the index, and then when querying, you can query the index via a truncated vector and perform an additional full-length comparison as well. For example: (taken from a project I'm currently working on) ``` SELECT * FROM report_embeddings a INNER JOIN report_embeddings b
subvector(b.embedding, 1, 768)::vector(768)
WHERE a.id = $2
```In this example, I truncate the embedding to 768 dimensions for the index and the lookup, (see the INNER JOIN) but do an additional full vector comparison as well. (in the WHERE clause) This performs a fast but less accurate lookup into the index, and then only does the non-indexed full vector check for the matched results. | ||
| ▲ | sroussey 2 hours ago | parent [-] | |
That’s only going to work on an MRL embedding. | ||