embeddings are easy to fine tune. Try modern bert.