Can you expand on this? I can't imagine this is holding up an entire machine or materially impacting capacity. Isn't it common to have a tiered cache and to ultimately evict (or move to another tier) if there's an active inference request and no available capacity elsewhere?

▲

Lalabadie 3 hours ago | parent [-]

Yes, however if a tool like this commoditizes a way to stay "higher" in the cache hierarchy to avoid getting dropped, then for the same total capacity, Anthropic have to become stricter about retention.

	▲	supern0va 3 hours ago \| parent [-]
		It's unfortunate that there isn't some better way to signal that there's a high chance that a turn is coming (ie, due to active sub-agents). If CC could send a ping saying "I don't need a turn, but please keep this cached at some tier because there will be a turn soon." that would probably help with cache efficiency. Instead, it seems like there's just the shotgun one-hour solution.