I've found good results from summarizing my documents using a large context model then embedding those summaries using a standard embedding model (e.g. e5)

This way I can tune what aspects of the doc I want to focus retrieval on, it's easier to determine when there are any data quality issues that need to be fixed, and the summaries have turned out to be useful for other use cases in the company.

Agreed. Esp if you gonna call an API, you can call something cheaper than this embeddings model, like 4o-mini, summarize, then use a small embeddings model fine-tuned for your needs locally.

I was critical about these guys before (not about their quality of work but rather about building a business around embeddings). This work though seems interesting and I might even give it a try, esp if they provide a fine-tuning API (is that on the roadmap?)