| ▲ | captainmuon an hour ago | |
It's one thing that the APIs don't return the reasoning tokens in a readable fashion. But then why do I have to send them the whole conversation every time? If they don't allow me to see parts of the conversation then they should have the burden of storing and transmitting it, and shouldn't charge more and more the longer the conversation gets (unless opaque token caching kicks in) like some LLM Shlemil the painter. | ||
| ▲ | charcircuit 22 minutes ago | parent [-] | |
>and shouldn't charge more and more the longer the conversation gets The cost of generating token N is O(N) with KV cache so it's unrealistic to expect to not be charged more the longer the conversation is if you are looking for the minimum price. | ||