You could use BM25S [0] instead of rank-bm25 for a nice speedup.

Also, there are tradeoffs associated with using BM25 instead of embedding similarity. You're essentially trading semantic understanding for computational speed and keyword matching.

[0] https://github.com/xhluca/bm25s