> 150k TPM limit on public endpoint means that it's likely unusable for many coding tasks.

I don't understand. How does that make it unusable? Is the limit shared by an entire team at once?

150,000 tokens per minute is a lot. You could start hitting that with a lot of concurrent requests in your session, but even throttled to 150k TPM it's still going to be faster than anything else you find.

I think the 128K context limit is the real ceiling. These models aren't amazing at long context, but once you account for a short input prompt, the input files, and headroom for a compaction summary, there isn't a lot left for the problem.

▲

wild_egg 3 hours ago | parent | next [-]

It's a limit on input tokens. So that's 3 50k requests per minute. At Cerebras speeds, that's about 5 seconds of usage per minute.

I was very excited last year for their coding plan but seeing a burst of requests pulse and then sitting there watching the cooldown reset is really not a great time.

Even though each individual request was fast, the sessions were only maybe 10% faster on wall clock time since there was so much waiting time.

▲

amelius 3 hours ago | parent [-]

Can't you do something with multiple accounts?

▲

sandworm101 3 hours ago | parent [-]

Or just buy a 5060. This will run on most any 16gb card. Slower for sure but far cheaper than another subscription.

	▲	embedding-shape an hour ago \| parent [-]
		Or buy a raspberry pi with a SSD, about the same difference, if you're giving up on the 1500 tokens/s anyways.

▲

gerdesj 3 hours ago | parent | prev | next [-]

128k context is not a limit of the model, that's a limit of implementation:

"Context Length: 262,144 natively and extensible up to 1,000,000 tokens."

https://huggingface.co/Qwen/Qwen3.8-27B

▲

datadrivenangel 3 hours ago | parent | prev | next [-]

150k tokens per minute at 1.5k tokens per second means you can have like 3 users concurrently and that's not a lot.

▲

conception 4 hours ago | parent | prev | next [-]

150k by account. At 1.5k a second you hit it very quickly.

▲

devy 3 hours ago | parent [-]

Exactly, it burns the tokens 3000x faster, which means the budget ($$$$$$) runs out so faster it will stop super quick, not able to perform long-duration work. At 27B parameter size, the intelligence is not able to accomplish work within a short amount time. Consequently, it become not usable.

▲

a012 33 minutes ago | parent | next [-]

Unusable is too stretch IMO, you can still use it in tiny tasks that’ll respond almost instantly

▲

gerdesj 3 hours ago | parent | prev [-]

I (we) run Qwen3.8-27B-FP8 on a DGX Spark box - that's roughly £4000 of hardware.

I did benchmark it in various ways and it runs quite well but it is a quantised jobbie and 1.5k t/s is also rather faster than anything I can possibly hope to achieve.

To run that model at those sorts of speeds is going to need some serious investment and you are going to have to pay for it.

	▲	jacquesm 30 minutes ago \| parent [-]
		How fast is it?

▲

2 hours ago | parent | prev [-]

[deleted]