| ▲ | collin 6 hours ago |
| This was my experience a year ago on some other model they could run super fast. Routine coding tasks would hit the per-minute token limits. Just the math there... 150k TPM... and 15k TPS means... you can run for 10 seconds every minute? The basic math boggles the mind. |
|
| ▲ | baegi 6 hours ago | parent [-] |
| Not sure how the rate limiting works, but it's 1.5k TPS, not 15k, so you could run it for 100s/min, which seems good enough to me |
| |
| ▲ | nostrebored 6 hours ago | parent | next [-] | | iirc input (uncached) goes towards the limit as well | | | |
| ▲ | fc417fc802 5 hours ago | parent | prev [-] | | It seems you forgot to account for the fact that cerebras uses a baker's minute which is 144 seconds instead of 60. (Seriously though what's the supposed issue here?) | | |
| ▲ | RussianCow 5 hours ago | parent [-] | | The issue is that all input (including context) counts towards that limit. So 10 requests with 50k of context will blow through the limit, even if little to no output was generated, which is incredibly easy to do with agentic workloads. |
|
|
