Remix.run Logo
yupyupyups 3 days ago

I've heard Ceph is expensive to run. But maybe that's not true?

keeperofdakeys 3 days ago | parent | next [-]

Ceph overheads aren't that large for a small cluster, but they grow as you add more hosts, drives, and more storage. Probably the main gotcha is that you're (ideally) writing your data three times on different machines, which is going to lead to a large overhead compared with local storage.

Most resource requirements for Ceph assume you're going for a decently sized cluster, not something homelab sized.

jauntywundrkind 3 days ago | parent | prev | next [-]

I'm only just wading in, after years of intent. I don't feel like Ceph is particularly demanding. It does want a decent amount of ram. 1GB each for monitor, manager, and metadata, up to 16GB total for larger clusters, according to docs. But then each disk's OSD defaults to 4gb, which can add up fast!! And some users can use more. 10Gbe is recommended and more is better here but that seems not unique to ceph: syncing storage will want bandwidth. https://docs.ceph.com/en/octopus/start/hardware-recommendati...

westurner 3 days ago | parent | next [-]

This from 2023 says: https://www.redhat.com/en/blog/ceph-cluster-single-machine :

> All you need is a machine, virtual or physical, with two CPU cores, 4GB RAM, and at least two or three disks (plus one disk for the operating system).

xyzzy123 3 days ago | parent | prev [-]

For me it was the ram for the OSDs, 1GB per 1TB but ideally more for SSDs...

master_crab 3 days ago | parent | prev [-]

It’s going to do a good job saturating your lan maintaining quorum on the data.