|
| ▲ | s0ck_r4w 4 hours ago | parent | next [-] |
| Make the cache TTL more adaptive. Have more tiers than just 5m and 1h. Long-running subagents become increasingly ubiquitous. There's no reason for Anthropic to not do a better job for this use case. |
|
| ▲ | randomblock1 an hour ago | parent | prev | next [-] |
| Default to 1h. Allow setting it to longer or shorter, granularly. Add /pause to mark it for eviction, for a token refund. |
|
| ▲ | addaon 3 hours ago | parent | prev [-] |
| There are a lot of workflows where cache is likely to be consumed exactly once (e.g. anything chat-like where a single thread is extended; if a prefix cache hit is found, and a set of new longer prefixes is inserted in the cache, it is unlikely that the original shorter prefix will be consumed in later turns). I could see allowing each session key a small number (1?) of 24-hour cache entries, where inserting a new cache entry (perhaps the maximum-length prefix, perhaps an API-tagged specified-length prefix) consumes that slot and either boots the rest of the items from cache, or demotes them to 5 minutes or something. Basically: workflow awareness, not uniform handling. |