Remix.run Logo
rockwotj 7 hours ago

author here! Glad you liked the post :)

> Motivation for combining Chorus and SlateDB for NFS

I work on an AI agent (tasklet) and we give every agent a linux machine. Having durable storage that is cheap, fast and multi-tenant is really important for our product. NFS is a great protocol (if complicated), and object storage is just the cheapest. But making it fast and reliable is key.

> other use cases

Any use case for SlateDB that you are willing to pay more for less latency but keep disaggregated storage without another system.

> GCP specific

Actually AWS and Azure zonal storage also support append operations, so I think the approach could be extended to all three major clouds. I don’t have a need for that ATM

> pricing

Probably worth a whole separate blog TBH. It would be cheaper than Kafka but more expensive than just using the built in WAL for SlateDB or OSWALD

Imustaskforhelp 6 hours ago | parent [-]

> I work on an AI agent (tasklet) and we give every agent a linux machine. Having durable storage that is cheap, fast and multi-tenant is really important for our product. NFS is a great protocol (if complicated), and object storage is just the cheapest. But making it fast and reliable is key.

Wouldn't 9fs/plan9fs be more suitable for this use case from my limited understanding though?

> Any use case for SlateDB that you are willing to pay more for less latency but keep disaggregated storage without another system.

I feel like IMO one of its benefits could be something like Figma/Google Docs alternative (Ironic considering that it would be running on google specific product but as you've mentioned in next point a potential support for AWS/Azure could also happen). I also found some use case within a live-streaming alternative in terms of how a live stream could add support for viewing things say a few seconds back. The current way seems to be much more messier than a simple s3. Though I feel like in any of these cases, pricing might be a factor.

I also find the use case that you shared really interesting as well. I wish you luck to see how that pans out and am looking forward for reading more technical pieces by you so keep publishing more and I am curious for more details as well.

There are so many interesting ideas when we combine s3 and databases. TigrisFS is taking an opposite approach to this by running filesystem on top of s3 on top of technically foundationDB whereas your approach/SlateDB essentially runs a DB on S3. It's fun to imagine both of these being used together and how both of these are approaching at somewhat of the same problem from completely different/opposite ways.

rockwotj 6 hours ago | parent [-]

> Wouldn't 9fs/plan9fs be more suitable for this use case from my limited understanding though?

Can you elaborate on why? NFS is really a great protocol with a lot of tricks to reduce round trips