This looks nice. What would be good on-prem S3 companion for this? I know if minio but I think there was some recent drama about it (I don't know specifics, just a feeling)

▲

redblueflame 5 days ago | parent | next [-]

If you want to have replication built in, you can give https://garagehq.deuxfleurs.fr/ a try

	▲	mdaniel 5 days ago \| parent \| next [-]
		Just make sure the AGPLv3 is compatible with your policies https://git.deuxfleurs.fr/Deuxfleurs/garage/src/tag/v2.0.0/L...
	▲	muhehe 5 days ago \| parent \| prev [-]
		Thanks, that looks good. Do you have some real experience with this in production?

▲

Jedd 5 days ago | parent | prev | next [-]

Do you actually need S3-alike?

I run distribution [0] / registry [1] as a docker (via Nomad) job, and it just uses a shared nfs mount in my cluster - ie. EXT4 FS underneath.

Currently has 12GB and 52 repositories, and is very performant. It's just a file server.

Unless your size requirements are huge, the complexity of having your registry rely on a self-hosted S3-alike sounds like a lot of pain for future-muhehe to contend with if / when the S3-alike hiccups.

[0] https://github.com/distribution/distribution

[1] https://github.com/Joxit/docker-registry-ui

▲

muhehe 4 days ago | parent [-]

For image registry...probably not that much. But it is quite common for other software, often those that target containers/k8s/... I already have few of those and that would be quite handy to have some good local s3.

	▲	Jedd 4 days ago \| parent [-]
		Concur - as noted in a sibling comment, I'm using both on Nomad/docker 5-node clusters, but started with minio then ran up garage-dev to compare (I just haven't migrated data over from minio yet). I use mine primarily for long-term stores for Grafana's Mimir and Loki products. Anyway, minio & garage are both lightweight, and happily run in docker, so if you've got a small cluster at home it would take maybe an hour or two to install them both from scratch.

▲

phireal 5 days ago | parent | prev | next [-]

Minio used to be de facto here, but they did a bait and switch recently and removed the UI from the free version. Garage is probably closest to best in class for open source on prem.

	▲	Jedd 5 days ago \| parent \| next [-]
		Does garage-dev have a GUI? I thought it did not. I'm using it in my lab, along with an older instance of minio, and both are excellent choices I think. (I'm running both as HA jobs within a Hashicorp Nomad cluster, which complicates / eases various things.) I had a vague memory of minio losing some favour a while back because they switched their underlying storage from basic object : normal file mapping to something more complex. A lot of home users, and I guess developers, liked the ability to navigate their S3-esque buckets via normal linux file system tooling.
	▲	muhehe 5 days ago \| parent \| prev [-]
		But isn't it still available separately? https://github.com/minio/object-browser?tab=readme-ov-file

▲

arccy 5 days ago | parent | prev | next [-]

https://github.com/seaweedfs/seaweedfs much simpler than minio

	▲	mdaniel 5 days ago \| parent [-]
		https://github.com/seaweedfs/seaweedfs#compared-to-other-fil... is handy, and I like that they support "sane" metadata stores and don't just do something dumb like make me run etcd

▲

ticklyjunk 4 days ago | parent | prev | next [-]

We are using deepspace storage for this. We can get/put objects into the cloud as one of our target volumes. It works as an auto archive writing anything in the fs that is over 90 to a compressed object and leaving behind a stub. You can point Harbor (or other tools like Mimir/Loki ) to a DeepSpace endpoint which looks like a standard S3 target. Then add policies and the files get moved, replicated, versioned to tape, cloud, disk array in the background. The users just interact with the file system as usual and admins have a UI with a catalog which shows where everything actually is.

▲

stephenlf 5 days ago | parent | prev [-]

The article mentions Pure Flashblade. Looks like dedicated hardware. https://www.purestorage.com/products/unstructured-data-stora...

▲

muhehe 5 days ago | parent [-]

Yes, but I'd rather some self hosted software solution

	▲	opless 5 days ago \| parent [-]
		Not S3, but I use longhorn to store persistent data on my clusters. Sometimes you'll need admin skills, but only if you spread your cluster out over availability zones or poor connections