| ▲ | gpugreg 6 hours ago | |
Cached tokens count towards the limit as well. For example, if your context window is 50,000 tokens, it takes 9 requests to reach that limit without generating a single token. | ||
| ▲ | perching_aix 3 hours ago | parent [-] | |
then it's basically useless lol, wtf, this has to be a defect | ||