| ▲ | d2p 6 hours ago | |||||||
> There is a limit of 450,000 tokens per minute. I hit this limit in about 90 seconds I'm confused. If it's 1500t/s, isn't that only 90k per minute? How do you hit a 450k/minute limit? | ||||||||
| ▲ | gpugreg 6 hours ago | parent [-] | |||||||
Cached tokens count towards the limit as well. For example, if your context window is 50,000 tokens, it takes 9 requests to reach that limit without generating a single token. | ||||||||
| ||||||||