I would recommend tracking this data over time. I work on Cloudflare's KV cache for Kimi K2.6, and while there are periods where our cache rate is low, we are frequently in the 80-90% range. OpenRouter shows us at 87.3% at the time of this post. We observe cache rates change quite a bit from hour to hour.

Agents push the full conversation history into context every turn

Why?

Maybe this is a dumb question, but why wouldn't an agent "keep the conversation going", like I do when interacting with an LLM through a web page? (I understand how it's impractical for long-running tasks where the agent has to wait days for the next input, but assume that's not the majority of use cases)

Thank you so much for this! I've been working on exactly this problem this week (which OpenRouter providers have the highest cache rate on average) because cache cost is sometimes half your cost: I'd much rather use a provider with more input caching with a more expensive/better LLM. Your results and lists seem more comprehensive than what I've done so far. Very helpful!

> You should expect that any inputs and outputs are going into someone's training database.

True enough, in theory; but what exactly are you imagining would be a useful-enough signal in the OpenRouter request+response stream, that any company would want their data as training material?

Even a single OpenRouter-API-key-identified subscriber's traffic, may consist of an mixture of traffic from multiple different sessions, under potentially multiple different end-users. (Where, if the subscriber is doing security correctly, then their OpenRouter key lives on a gateway rather than in a frontend app; and so the only IP address / UA / etc OpenRouter sees is that of the gateway itself.)

And the traffic stream may also invoke multiple models, and provide multiple different system prompts for those models; which, while marked in the traffic (i.e. conveyed as part of each request), makes the resulting data much less useful in aggregate, than if it were all training data for one model with one system prompt.

Plus, there are no RLHF signals in OpenRouter data. Even if OpenRouter wanted to build a general model-neutral framework for collecting RLHF-type data, it can't force subscriber apps to do the UI-level stuff necessary to collect it (i.e. the things ChatGPT/Claude do, with "thumbs-down" buttons, A/B tested responses, etc.) Analysis would have to rely on pure transcript-level user sentiment extraction.

It’s interesting all the focus on opt-out from training. Sometimes I worry there is an intentional focus on that so people don’t think about the other ways the company might be profiting off our data. Like I pay for Anthropic and they don’t train on that but are they selling my “anonymized” usage data in some other way?

> Clearly anyone who can pay should be using paid models with privacy protections

Clearly, anyone who needs privacy should be using models with privacy protections. Some people build open source and the models will get the code anyway.

I recommend nvidia nim for completely free dev access for young people.

If you believe there will be lots of LLM providers in the future, then OpenRouter could be a DoorDash play.

Established restaurants didn't need DoorDash because they were already on everyone's speed dial. But new or small restaurants couldn't afford to advertise or maintain a team of delivery people. DoorDash created a two-sided marketplace that made it a lot easier for new entrants to bootstrap. Today even the established restaurants have to pay them their tithe because hungry people have learned to start with the DoorDash app. A bit of a prisoner's dilemma.

If OpenRouter plays its cards right and gets very lucky, a large number of people will configure their hungry LLM clients to start with OpenRouter, and then LLM providers will have to join the marketplace or else miss out on all those customers.

AWS does not provide nearly as many different models as OpenRouter. Perhaps they have an incentive to not do that, move slower as a big company or more legal risks to consider. If AI model outputs becomes commoditized then having one place where you can switch effortlessly from one to the next based on price might just justify OpenRouter. It could become a commodity marketplace/exchange.

They never claimed it was technically hard. Brand recognition is their forte. They found out there is a need, developped a product around it.

functionally they operate as a marketplace for cloud providers. I feel like there is value there, especially as API costs rise and companies explore cost-saving/efficiency. IMO, this is a particularly attractive value prop in the SMB space, where it is common to interoperate between multiple SaaS/software stacks.

Yah I don’t think they have a long term play without a pivot

The top model / prices are changing all the time though. Lately I've been auditioning 4-5 models before a big ingest and I wouldn't be able to do that easily without OR.

I tend to agree, but there's also a lot of tax to build and maintain the different provider abstractions that OpenRouter eliminates.

Everything has a cost of some sort. It's just who you're going to pay and what the currency is.

And what do you do when Fireworks is down? If you stuck with Openrouter, when Fireworks is down it would auto route you to Friendli.

What if Fireworks stops offering your preferred model?

There are enough services that don't want the model provider to know who they are.

Huh, so they do.

Anthropic: https://support.claude.com/en/articles/8977456-how-do-i-pay-... - you can pre-pay and get a hard cutoff.

OpenAI: https://community.openai.com/t/how-to-set-billing-limits-and... - last time I looked OpenAI had a soft but not hard limit, I guess they fixed that last year.

I remember bugging them both about this last year, I need to update my mental model!

I tried Alibaba Cloud. They have no caps. This was the reason to cancel my account there.

Deepseek has a prepaid model. (Pretty impressive, what fits into 10 Dollar)

Based on experience, Google Cloud. No idea if that translates to Gemini usage billing.

Microsoft

Google Vertex

There is a scheme to send gifts with a compromised anthropic key even if the limit is reached.

Would you recommend Kagi Ultimate over OpenRouter? I'm already a customer of Kagi and would rather give them my money, but only if I'm not really compromising.

I'll be honest, I hadn't clocked that Cloudflare and Vercel were offering equivalent products.

Looks like Vercel even have their own leaderboard: https://vercel.com/ai-gateway/leaderboards/models

Surprising that they have Opus 4.8 and 4.6 listed on the leaderboard but not Opus 4.7.

I didn't know about these options either. I am using Cline: Cloudflare isn't an option but Vercel is. My spending is pretty low overall now that I'm using local models much more but good to know that there are cheaper alternatives to try or at least suggest to others.

Other features I've just noticed: - configurable prompt injection protection using OWASP regex (https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_In...) - configurable PIM protection for outbound prompts - input/output logging - "JSON healing" to auto-correct minor hallucinations

Lots of other stuff too. The business model seems pretty simple and the value-add features don't look particularly expensive or difficult to copy.

Separation, I don't want to have my domains blocked the day AI bill go Brrr

I don't think they do. They proxy to a bunch of open-weight model hosts, but I've not seen that they host them themselves.

They don't list themselves on https://openrouter.ai/providers

middle man? Model providers/hyperscalers -> OpenRouter -> consumers?

coffee farmers -> middle man -> you

Credit is prepaid at a 5% surcharge.

Commission on API calls extracted from you when you charge your account.