| ▲ | tedsanders an hour ago | |
I work at OpenAI and can confirm that's correct: reasoning tokens are discarded after each new user turn (though not after each message or tool call). Our docs show a diagram here: https://developers.openai.com/api/docs/guides/reasoning > Input and output tokens from each step are carried over, while reasoning tokens are discarded. Keeping reasoning tokens around is better for caching and for remembering past insights, so you might reasonably wonder why we designed it this way. The main benefit of dropping reasoning tokens is that you can fit a lot more work inside the model's context window before you're forced into a slow and lossy compaction step. This was a larger consideration with our earlier reasoning models that had shorter context windows (~200k), longer thinking times (up to ~100k per message), and poor compaction. However, now that we've shipped longer context windows, we've trained our models think much more efficiently, and we've made compaction way better than it used to be, the balance of factors is changing. Tune in Thursday! | ||
| ▲ | throwuxiytayq an hour ago | parent [-] | |
dude what the hell | ||