| ▲ | dakolli a day ago |
| You can sort by the cost cache cose, but you cannot by cache hit %. You have to click on the provider and see what their cache hit % is. A provider could have a super low cache cost, but a 50% cache hit percentage, making the cheap price of cache read's meaningless. |
|
| ▲ | andai a day ago | parent [-] |
| Wait, what does that number mean? I thought it always uses the cache price when the prefix matches. |
| |
| ▲ | dakolli a day ago | parent [-] | | When the prefix matches a request sent to the same Providor. The thing is the TTL is different for each provider, some cache for 5 minutes some cache for 1hr. Its ideal to only use one provider per agent session / and per model with the best cache hit % if you care about costs. | | |
| ▲ | andai 16 hours ago | parent [-] | | But before 5m the hit rate is 100%, and after it's 0%? Why is there a probability? Is there some stochastic process that takes place during those 5 minutes that determines whether or not you get the discount? | | |
|
|
