| ▲ | chrisweekly 5 hours ago | |
"causing" -> "costing", right? | ||
| ▲ | vorticalbox an hour ago | parent [-] | |
In this case yeah. If it’s not reading the cache then it has to compute all the context window again and not just the newest tokens. | ||
| ▲ | chrisweekly 5 hours ago | |
"causing" -> "costing", right? | ||
| ▲ | vorticalbox an hour ago | parent [-] | |
In this case yeah. If it’s not reading the cache then it has to compute all the context window again and not just the newest tokens. | ||