I wouldn’t say it’s “clear”. If you want good answers to your Internet blog begs, it’s probably good to actually state your use case. “I just want S3” means different things to different people.

I just spent some time with the s3 protocol and I agree completely. What should have been able to leverage the simplifying assumptions turned into another hodgepodge. It’s not like nfs is a real shining example of simplicity either. I’ve never worked with p9, but potentially that aside I think we really failed to come up with a decent distributed file model,

> It strikes me as a classic case of "we need all the interested people to pull in one project, not each start their own".

And every few weeks in the cooking subreddit we get a new person talking about a new soup they made. Just think if we put all 1000 of those cooks in one kitchen with one pot, we'd end up with the best soup in the world.

Anyway, we already have "the one" project everyone can coalesce on, we have CephFS. If all the redditors actually hopped into one project, it would end up as an even more complex difficult to manage mess I believe.

I'm pretty sure I set up most of what "Simple S3" using with Apache2 and WebDAV at least fifteen years ago.

Every month there's a post of "I just want a simple S3 server" and every single one of them has a different definition of "simple". The moment any project overlaps between the use cases of two "simple S3" projects, they're no longer "simple" enough.

That's probably why hosted S3-like services will exist even if writing "simple" S3 servers is so easy. Everyone has a different opinion of what basic S3 usage is like and only the large providers/startups with business licensing can afford to set up a system that supports all of them.

I think it's like NES emulators. It's not that anyone needs one more. It's just that they're fun to make.

Or maybe the underlying philosophy is different enough to warrant its own implementation. For example, the Filestash implementation (which I made) listed in the article is stateless and acts as a gateway that proxies everything to an underlying storage. We don't own the storage, you do, via any of the available connectors (SFTP, FTP, WebDAV, Azure, SMB, IPFS, Sharepoint, Dropbox, ...). You generate S3 keys bound to a specific backend and path, and everything gets proxied through. That's fundamentally different to not fit in the mold of other alternatives that mostly assume they own your storage and as a result can not be made stateless by design. That approach has pro and cons on each side

Diverse competition is the best way to identify a winning formula, which can then be perfected by a fewer number of players.

S3 with tree-shaking. i.e. specify the features you need, out comes an executable for that subset of S3 features you desire.

Or like lodash custom builds.

https://lodash.com/custom-builds

NFS in practice is too different from S3 to make this work.

I’ve been at a couple companies where somebody tried putting an S3 interface in front of an NFS cluster. In practice, the semantics of S3 and NFS are different enough that I’ve had to then deal with software failures. Software designed to work with S3 is designed to work with S3 semantics and S3 performance. Hook it up to an S3 API on what is otherwise an NFS server and you can get problems.

“You can get replication with RAID” is technically true, but it’s just not good enough in most NFS systems. S3 style replication keeps files available in spite of multiple node failures.

The problems I’m talking about arise because when you use an S3-compatible API on your NFS system, it’s often true that you’re rolling the dice with three different vendors—you have the storage appliance vendor, you have the vendor for the software talking to S3, and you have Amazon who wrote the S3 client libraries. It’s kind of a nightmare of compatibility problems in my experience. Amazon changes how the S3 client library works, the change wasn’t tested against the storage vendor’s implementation, and boom, things stop working. But your first call is to the application vendor, and they are completely unfamiliar with your storage appliance. :-(

It's probably the most important storage API in the industry. Implementing it gives you on-prem storage, AWS S3 (the Hoover Dam of Internet storage megaprojects, arguably the most reliable store of any kind available to any normal programmer), and a whole ecosystem of S3-compatible options with different features and price points.

It's a little like asking why you'd use SQL.

I want my application servers to be stateless and I've got state to keep that looks a lot more like files than database rows.

And I want things like backup, replication, scaling, etc. to be generic.

I wrote a git library implementation that uses S3 to store repositories for example.

Listen to this: 7... Minute... Abs. You walk into a video store and you see 8 minute abs and 7 minutes abs. Which one are you gonna buy?

Then why nobody forked minio?

Tried setting up rustfs today. It was easier that garagehq and it even comes with UI.

I think only "complex" thing in garage is the layout which only matters if you're doing distributed mode.

Has anyone that has set up microceph determined the overhead of the required multiple OSDs? The docs make it sound scary, but it's not clear if that's because people run it on a Pi with an sdcard for block storage or because someone once ran 18TB of OSDs in production that then fell over.

They want to run it locally.

Potential of startup for hosted object storage? I think Tigris (https://www.tigrisdata.com/docs/) will work pretty well with Render.

I think the post author is mainly addressing self-hostable and/or open-source options here - otherwise I'd expect a whole host of other commercial storage providers to have been mentioned!

Render has Object Storage in alpha: https://render.com/object-storage

By itself rsync.net doesn't support S3. The one I wrote (Filestash) lets you use rsync.net as a downstream storage and proxies it through the S3 protocol.

People write applications that work with the S3 API but may want to host their own storage for a variety of reasons. Personally I make use of S3-compatible services for pre-signed url access to data on disks I own. The distributed aspect is only one reason why someone might want an S3-like service.

While not obvious from the article, it appears that they want something S3 like, but isn’t from Amazon, and possibly want to self host it. The article could be much more clear about the goals