Remix.run Logo
▲ porker 4 hours ago

Congratulations on the project!

I haven't run into problems with Dragonfly yet, was it only OOM you encountered with it? Garnet I had GC pauses causing problems, so I dropped that.

▲murtazayusuf 3 hours ago | parent [-]

OOM was a big one, because when we had large dataset for example 96G on a 126G dragonfly pod, when it was time for a snapshot, it created a copy within its memory before storing it to the disk, this was during dragonfly 1.35, not sure if they fixed it with 2.0. GC pauses was a second problem we faced, we noticed it from the dips in grafana charts for the read throughputs and it was on a schedule. Do let me know if the issues you faced are solved with kividb, we also have an operator for it as well https://github.com/kividbio/kividb-operator

▲murtazayusuf an hour ago | parent [-]

a correction here, it wasn't the GC pauses within dragonfly itself, there were periodic dips in the throughput graphs overtime, that looked similar to GC pauses, but can't be the case as dragonfly does manual memory management. For the memory spikes around snapshots though, I certainly noticed them.