I was more skeptical of openrouter a year ago. Not hard to implement multiple models, but after getting burned multiple times (eg Google in particular) I see openrouter as serving a few key functions: (1) aggregating demand across neoclouds for performance etc (2) providing a great developer experience (3) easily switching models and normalizing the idiosyncratic nature of each. There are so many (good) model providers today this is more compelling now with 10+ solid model companies than a year ago when I thought it might be just 3.

So I have shifted load away from calling some models to openrouter.

This is really interesting, because I'm actively building an open source SaaS router product. We're starting in the FAL area rather than the OpenRouter area (much higher margin), but they eventually intersect.

My thinking is businesses will want the ability to fire their provider and bring things on-prem if they want (but that they'd rather stick with a good provider as long as they're happy). Not so much that they'll do it, more that they'll want the option.

Flexibility isn't super important now as most routers are fungible, but as we move into an era dominated by open source models and fine tunes, that becomes a major impediment to switching. So building this now seems especially important. Especially if we guarantee data and model exports to customers.

A customer with fine tunes of open source models won't find it simple and easy to migrate. But we can build this escape hatch now before that story has a chance to evolve.

And of course there's a lot of add-ons that can be sprinkled on top later: handling customer data/outputs (videos, thumbnails, etc.), spinning up domains for products (eg. Three.js video game LLM outputs), etc. Vercel/Itch.io as a service, Vimeo as a service, media catalogue search as a service, etc. It'd be nice if this was all open source and portable.

Is anyone here interested in a closed beta of a Rust-based "Open" OpenRouter, Fal, (and eventually RunPod and Featherless)? We host it for you, but the entire stack is OSS and you can bring it on-prem if you want. I'd be happy to send invites. My email is "echelon at gmail".

We also do BYOK forwarding, so it's super simple to switch without even paying us. We even support router keys for double routing (OpenRouter and FAL keys in addition to OpenAI, Google, etc.)

Might need to rethink our Stripe payment processing. They have our customer book.