| ▲ | necubi 5 hours ago | |
Definitely agree that every data system that doesn't need <100ms latency is moving to object storage. > I do wonder if we will see an expansion of the s3 api to support more of these use cases This is actually an area where I think we have a big leg up on folks building on top of S3. My team (which built K2) sits next to the R2 team, and we have the opportunity to co-evolve the products in mutually beneficial ways. | ||
| ▲ | sensodine 44 minutes ago | parent [-] | |
> every data system that doesn't need <100ms latency is moving to object storage I think the opportunity extends below 100ms too, particularly given the existence of faster object storage tiers like S3 express or more recently GCS rapid bucket (both only offering single-zone durability, so still need to do quorum writes to get region-level durability as with standard tiers). One of the tensions of course is how long to linger before flushing to object storage - you have to trade off directly between latency and cost of your API ops for PUTs. When building the serverless offering of s2.dev (which is in a similar space, full disclosure!), we designed around stateful backend processes capable of constantly flushing multi-tenant objects (i.e., containing records from many streams), allowing streams to offer low ack latencies (~50ms p99 from same region) without blowing up the unit economics. Congrats on the launch btw! | ||