Remix.run Logo
Roark66 2 hours ago

Openrouter is not ideal, because you don't know who they send your traffic to and there are rumours of vendors cheating by providing quantized models.

I'm very happy with Deepinfra. Less model coverage, but good prices and quite fast.

However I have to caution you about one thing.

No one will give you as many input tokens for so little money as Claude Max x5 (maybe x20 too, I use x5).

I tend to use 1.1B to 1.4B a week about 0.8-1B cached. Even with cache were talking thousands of $ in API prices a week. Hundreds if we're talking cheap cloud like Deepinfra.

However, local AI well setup is actually a good alternative for this if Claude Max was unavailable.

For example my system a ryzen 7950x 192GB ram, 5x rtx3090 plus an rtx5060 ti 16gb. (3 rtx3090 cards via usb4 egpu dock). Let's me run Qwen3.8-Flash-Next with 3slots (no rtx5060 used) at 55tok/s decode dropping to 50 at the end of a 260k context, 1200tok/s refill dropping to 950 at the end of context.

With RAM and ssd cashing and 80% cache were talking on the order of 4B a week could be ingested by this setup (roughly) if it was running 24/7. I found 6 interactive cloud code sessions are fairly pleasant with this 3 user setup.

If I include the rtx5060 in the mix I can bump to 5 users, but it slows down by about 15% (note the speeds are give are for one active user, multiple users at once see maybe 70% of tgat per user so aggregate is much higher in multi user setup).

So in theory I should be able to run 10 cloud code sessions. Although I'm testing CC alternative now (pi with own plugins) because this model, while multimodal has only 260k context 30k of which CC eats on the getgo.

Many people say local AI makes no sense financially. But in the event you process huge inputs that are often cached it does make sense.

Ciantic 2 hours ago | parent | next [-]

> Openrouter is not ideal, because you don't know who they send your traffic to and there are rumours of vendors cheating by providing quantized models.

OpenRouter allows to make an API key locked to certain provider, I do that myself.

bix6 2 hours ago | parent | prev [-]

I haven’t heard of Deepinfra so I’ll check them out!

I think your local setup would be a bit much for me capability / price wise. But maybe I can find a scaled down version. I don’t need insane tok/s. Oftentimes I just let things run and come back later.