Remix.run Logo
torutofu a day ago

Curious how the ranking holds up on messy product text versus something like ParadeDB.

dukepiki 18 hours ago | parent [-]

BM25 is pretty similar everywhere: if a word in your query appears a lot in a document, the score for that document (for that one query) goes up, especially if that word isn't in very many other documents.

There are a couple of tuning parameters, and the notion of what a "word" is varies based on the tokenizer and stemming you use.

So it won't be exactly the same, but it's likely to be close.