It looks bizarre to see that Azure Storage support was included. That's because there's no need to use fancy SlateDB+LSM tree code to simulate a block device on Azure Storage, it already natively provides "page blobs" that can be mounted as standard disks and used for boot disks, databases, or whatever. These are used as the backing storage for all Azure virtual machine disks and can also be used directly via the HTTP API. For example, SQL Server can store its database files on page blobs, without having to attach them as disks and formatting them as NTFS volumes.

See: https://learn.microsoft.com/en-us/azure/storage/blobs/storag...

▲

Eikon 3 days ago | parent [-]

What's so bizarre about it?

Are we prevented from implementing features when the platform itself provides adjacent tech?

Page blobs are 2x+ more expensive ($0.045/GB vs $0.02/GB) and would require Azure-specific code. We don't dream of maintaining separate implementations for each cloud provider.

The approach works identically on S3, GCS, Azure, and local storage. If anything, ZeroFS helps you distance from vendor lock-in, especially if you start to run "usually managed" components on top, say databases.

Speaking of Azure's "native" solutions - we benchmarked ZeroFS against Azure Files. ZeroFS is 35-41x faster for most operations and 38% cheaper. If Azure Files performs that poorly, I don't hold much hope for page blobs either: https://www.zerofs.net/zerofs-vs-azure-files

▲

jiggawatts 3 days ago | parent [-]

Fair points.

Note that Azure Files is a significantly different offering than block storage, it is essentially a Windows file server cluster and uses the SMB v3.1.1 protocol. It's slow, but not that slow.

It looks like you benchmarked against the HDD version of it, which few people use. Everyone who cares about its performance uses Azure Files Premium, which is the pure-SSD version of the service. They recently updated it to have flexible performance that can be cheaper than the older HDD storage: https://techcommunity.microsoft.com/blog/azurestorageblog/lo...

You're also mounting it from Linux, which is just... weird. Nobody (for some values of nobody) does this. The sole purpose for this service is to provide backwards-compatible file shares for Windows servers and desktops.

Benchmarking cloud services can be tricky because they have odd quirks such as IOPS-per-GB ratios, so nearly empty storage can be much slower than you'd expect!

▲

Eikon 3 days ago | parent [-]

> You're also mounting it from Linux, which is just... weird. Nobody (for some values of nobody) does this. The sole purpose for this service is to provide backwards-compatible file shares for Windows servers and desktops.

That's not what Azure's marketing material says:

Serverless file shares

"Store your files on a distributed file system built from the ground up to be highly available and durable."

"Take advantage of fully managed file shares mounted concurrently by cloud or on-premises deployments of Windows, Linux, and macOS."

"Use integration with Azure Kubernetes Service (AKS) for easy cloud file storage and management of your data using NFS or SMB file shares."

Azure explicitly markets this for Linux and Kubernetes. If it performs terribly on Linux, that's Azure's problem, not mine.

https://azure.microsoft.com/en-us/products/storage/files

> Benchmarking cloud services can be tricky because they have odd quirks such as IOPS-per-GB ratios, so nearly empty storage can be much slower than you'd expect!

These managed services charge premium prices - they should deliver premium performance. At those prices, "quirks" and "you need to fill it up first" aren't acceptable excuses.

▲

jiggawatts 2 days ago | parent [-]

I don't believe anything Microsoft's marketing department says either.

I've had similar issues with Amazon's equivalent too, where I tried to use EFS for a tiny website and I got something like 0.1 IOPS!

I do like the design of ZeroFS, and it irks me that competing solutions make local caching and buffering weirdly difficult. For example, Azure Premium SSD v2 disks no longer support local SSD caches! It's also annoyingly hard to combine the local cache disks you do get with remote disks using Windows Server, it insists on trying to detect the "role" of each disk instead of just letting admins specify that.

Using something like ZeroFS could be a nice trick for getting decent price/performance on public cloud platforms. My only concern is integrity and durability: what testing have you done to validate that it doesn't lose data even in corner-cases?

	▲	Eikon 2 days ago \| parent [-]
		> Using something like ZeroFS could be a nice trick for getting decent price/performance on public cloud platforms. My only concern is integrity and durability: what testing have you done to validate that it doesn't lose data even in corner-cases? There's always going to be uncertainty with software that young, but the CI is pretty expensive and we didn't yet receive corruption-related reports. https://github.com/Barre/ZeroFS/tree/main/.github/workflows