Remix.run Logo
wvh 8 hours ago

Apart from Minio, we tried Garage and Ceph. I think there's definitely a need for something that interfaces using S3 API but is just a simple file system underneath, for local, testing and small scale deployments. Not sure that exists? Of course a lot of stuff is being bolted onto S3 and it's not as simple as it initially claimed to be.

hobofan 7 hours ago | parent | next [-]

SeaweedFS's new `weed mini` command[0] does a great job at that. Previously our most flakey tests in CI were due to MinIO sometimes not starting up properly, but with `weed mini` that was completely resolved.

[0]: https://github.com/seaweedfs/seaweedfs/wiki/Quick-Start-with...

egorfine 7 hours ago | parent | prev | next [-]

> for local, testing and small scale deployments

Yes I'm looking for exactly that and unfortunately haven't found a solution.

Tried garage, but they require running a proxy for CORS, which makes signed browser uploads a practical impossibility for the development machine. I had no idea that such a simple popular scenario is in fact too exotic.

PunchyHamster 7 hours ago | parent | prev | next [-]

Minio started like that but they migrated away from it. It's just hard to keep it up once you start implementing advanced S3 features (versioning/legal hold, metadata etc.) and storage features (replication/erasure coding)

seddonm1 7 hours ago | parent | prev | next [-]

What about s3 stored in SQLite? https://github.com/seddonm1/s3ite

This was written to store many thousands of images for machine learning

magicalhippo 7 hours ago | parent | prev | next [-]

From what I can gather, S3Proxy[1] can do this, but relies on a Java library that's no longer maintained[2], so not really much better.

I too think it would be great with a simple project that can serve S3 from filesystem, for local deployments that doesn't need balls to the walls performance.

[1]: https://github.com/gaul/s3proxy

[2]: https://jclouds.apache.org/

status_quo69 3 hours ago | parent | prev | next [-]

I've been looking into rclone which can serve s3 in a basic way https://rclone.org/commands/rclone_serve_s3/

amluto 3 hours ago | parent | prev | next [-]

Try versitygw.

I’m considering it for a production deployment, too. There’s much to be said for a system that doesn’t lock your data in to its custom storage format.

memset 7 hours ago | parent | prev [-]

For testing, consider https://github.com/localstack/localstack

9dev 7 hours ago | parent [-]

WAY too much. I just need a tiny service that translates common S3 ops into filesystem ops and back.