Much of the complexity of caching comes from trying to work well all workloads. If the workload is known, I think in many cases a specialized simpler cache can outperform some of these libraries.

▲

maypok86 2 days ago | parent [-]

What exactly do you mean by a "more specialized simple cache"? Just a map, mutex and LRU/LFU/ARC as eviction policies?

1. Even using sync.RWMutex and specialized policies won't really help you outperform a well-implemented BP-Wrapper in terms of latency/throughput.

2. I've never seen cases where W-TinyLFU loses more than 2-3% hit rate compared to simpler eviction policies. But most simple policies are vulnerable to attacks and can drop your hit rate by dozens of percentage points under workload variations. Even ignoring adversarial workloads, you'd still need to guess which specific policy gives you those extra few percentage points. I question the very premise of this approach.

3. When it comes to loading and refreshing, writing a correct implementation is non-trivial. After implementing it, I'm not sure the cache could still be called "simple". And at the very least, refreshing can reduce end-to-end latency by orders of magnitude.

▲

jasonthorsness 2 days ago | parent [-]

You're correct on all points. I should not have used the word "outperform" and should have said a simple cache could be sufficient. If for example you know you have more than enough memory to cache all items you receive in 60 seconds and items strictly expire after 60 seconds, then a sync.RWMutex with optional lock striping is going to work just fine. You don't need to reach for one of these libraries in that case (and I have seen developers do that, and at that point the risk becomes misconfiguration/misuse of a complex library).

	▲	maypok86 2 days ago \| parent [-]
		Yeah, I basically agree with that.