| ▲ | bitwize an hour ago | |
It's copying a summary of what it needs to know to produce a set of tokens to solve the problem at hand to the front of its context window, so that it doesn't slide off the back and the model doesn't lose track of why it walked into the kitchen in the first place once it gets there. | ||