This is being worked on (they call it AnyRaid), the work is being sponsored by HexOS[1].

[1]: https://hexos.com/blog/introducing-zfs-anyraid-sponsored-by-...

Underprivision your disks, then you don't have to worry about those edge cases...

One lovely experience I had when trying to remove a failing disk from my array was that the `btrfs device remove` failed with an I/O error - because the device was failing.

I then had to manually delete the file with the I/O error (for which I had to resovle the inode number it barfed into dmesg) and try again - until the next I/O error.

(I'm still not sure if the disk was really failing. I did a full wipe afterwards and a full read to /dev/null and experienced no errors - might have just been the meta-data that was messed up)

Pre-2020, but I had a BTRFS filesystem with over 40% free space start failing on all writes, including deletes, with a "no space left on device error". Took the main storage array for our company offline for over a day while we struggled to figure out wtf was going on. Supposedly this is better now, but basically BTRFS marks blocks as data or metadata, and once marked a block will be reassigned (without a rebalance). Supposedly this is better now, but this was after it had been stable for a few years. After that and some smaller foot guns, I'll never willingly run BTRFS on a critical system.

Just a few days ago I've had a checksum mismatch on a RAID-1 setup, on the metadata in both devices, which is very confusing.

Over the last one or two years I've experienced twice a checksum mismatch on the file storing the memory of a VMWare Workstation virtual machine.

Both are very likely bugs in Btrfs, and it's very unlikely that have been caused by the user (me).

In the relatively far past (around 5 years ago), I've had the system (root being on Btrfs) turning unbootable for no obvious reason, a couple of times.

I still have a btrfs with a big problem: more disk space used than expected. The explanation was helpfully provided by btdu:

    Despite not being directly used, these blocks are kept (and cannot be reused) because another part of the extent they belong to is actually used by files.
    
    This can happen if a large file is written in one go, and then later one block is overwritten - btrfs may keep the old extent which still contains the old copy of the overwritten block.

Ah yes, the famous "holding it wrong".

Killing ZFS on Linux would basically make Linux unsuitable for lots of usecases. What would you use instead? Btrfs, which keeps having stupid data corruption issues? Bcachefs, which is not yet stable and now it's being struck out of the kernel? LVM2+thin provisioning, which will happily eat your data if your data overlap? I hope some industrial players will force the kernel to drop this nonsense.

Heck no native filesystem besides btrfs has compression, I'm saving HUNDREDS of GB with zstd compression on my machines with basically zero overhead

Btrfs claims to be stable. IMO, it's not.

It's generally fine if you stay on the happy path. It will work for 99% of people. But if you fall off that happy path, bad things might happen and nobody is surprised. In my personal experience, nobody associated with the project seems to trust a btrfs filesystem that fell off the happy path, and they strongly recommend you delete it and start from scratch. I was horrified to discover that they don't trust fsck to actually fix a btrfs filesystem into a canonical state.

BCacheFS had the massive advantage that it knew it was experimental and embraced it. It took measures to keep data integrity despite the chaos, generally seems to be a better design and has a more trustworthy fsck.

It's not that I'd trust BCacheFS, it's still not quite there (even ignoring project management issues). But my trust for Btrfs is just so much lower.

Still more stable than btrfs. btrfs is also dead slow

Yes, like destroying itself and losing all data.

Can you give an example because to me it always appeared as NIH copy-cat fs?