Remix.run Logo
cortesoft 5 hours ago

How should they fix it?

s0ck_r4w 4 hours ago | parent | next [-]

Make the cache TTL more adaptive. Have more tiers than just 5m and 1h. Long-running subagents become increasingly ubiquitous. There's no reason for Anthropic to not do a better job for this use case.

randomblock1 an hour ago | parent | prev | next [-]

Default to 1h. Allow setting it to longer or shorter, granularly. Add /pause to mark it for eviction, for a token refund.

addaon 3 hours ago | parent | prev [-]

There are a lot of workflows where cache is likely to be consumed exactly once (e.g. anything chat-like where a single thread is extended; if a prefix cache hit is found, and a set of new longer prefixes is inserted in the cache, it is unlikely that the original shorter prefix will be consumed in later turns). I could see allowing each session key a small number (1?) of 24-hour cache entries, where inserting a new cache entry (perhaps the maximum-length prefix, perhaps an API-tagged specified-length prefix) consumes that slot and either boots the rest of the items from cache, or demotes them to 5 minutes or something. Basically: workflow awareness, not uniform handling.