| ▲ | alukin 5 hours ago |
| Feels like will be shut down real quick |
|
| ▲ | gogobio 5 hours ago | parent | next [-] |
| Yes, this wastes cycle. They dump your cache and deallocate the VMs so that others can use it. This will result in tokens wasted and downtime for others. |
| |
| ▲ | supern0va 3 hours ago | parent [-] | | Can you expand on this? I can't imagine this is holding up an entire machine or materially impacting capacity. Isn't it common to have a tiered cache and to ultimately evict (or move to another tier) if there's an active inference request and no available capacity elsewhere? | | |
| ▲ | Lalabadie 3 hours ago | parent [-] | | Yes, however if a tool like this commoditizes a way to stay "higher" in the cache hierarchy to avoid getting dropped, then for the same total capacity, Anthropic have to become stricter about retention. | | |
| ▲ | supern0va 3 hours ago | parent [-] | | It's unfortunate that there isn't some better way to signal that there's a high chance that a turn is coming (ie, due to active sub-agents). If CC could send a ping saying "I don't need a turn, but please keep this cached at some tier because there will be a turn soon." that would probably help with cache efficiency. Instead, it seems like there's just the shotgun one-hour solution. |
|
|
|
|
| ▲ | l1n 5 hours ago | parent | prev | next [-] |
| Why? Seems fine if people want to burn more cache reads. |
| |
| ▲ | 5 hours ago | parent | next [-] | | [deleted] | |
| ▲ | agluszak 5 hours ago | parent | prev [-] | | Because it makes Anthropic earn less money | | |
| ▲ | munk-a 5 hours ago | parent | next [-] | | > Because it makes Anthropic burn more money Fixed that for you. | |
| ▲ | cadamsdotcom 5 hours ago | parent | prev [-] | | "Earn"? Cache duration is arbitrary. What it actually does (if used en masse) is decrease the amount of oversubscription their infra can handle.. | | |
| ▲ | janderson215 3 hours ago | parent [-] | | Earn is synonymous with profit, and this hurts their profits. Introducing latency can benefit much more people and actually make usage more efficient. Reducing latency can sieze up a functioning system. Lately, I’ve been thinking about how this related to fractional banking. If you were to eliminate fractional banking introduced in the US by Hamilton, you would destroy a lot of current prosperity. |
|
|
|
|
| ▲ | Cyberdog 3 hours ago | parent | prev [-] |
| Agreed, the Thermos company's lawyers will probably fire off a terse letter once they find out. |
