I’m running an 8-drive ZFS RAIDZ2 pool. I’m wondering if you know — are the free space recommendations around ZFS cargo or real?

Like I’m already giving up two full drives for redundancy (which saved my ass - I recently had two drives fail on me in quick succession — both SSDs from what looks like an identical batch) but then the advice is kinda saying I need to keep at least another drive worth of space free for the pool to perform well and not crap itself. That hurts with current prices for sure.

> Not a great time price-wise to be building a NAS

That under-states the matter. It is a terrible time, price-wise, to build a NAS.

I'd almost rather have no AI whatsoever and have storage 1/10 the price of pre-AI times.

(If there were a magical choice between having AI and significantly more expensive storage, and having no AI and some program to dump that investment money into getting and somehow leveraging significantly more available storage, that is.)

I've been thinking about building my own NAS as well. Mind sharing how much did you pay for those hard drives, and what motherboard did you choose?

I'm also planning a new home server build now and the prices are definitely relatively ass. So far I've spent 820€ on two 22 TB WD Elements HDDs, 375€ on 2x16 GB DDR5 kit, and 520€ on two 2 TB M.2 SSDs (cache). About 1700€ and I still have no server to show for it. Doesn't help that I've been procrastinating on picking the CPU and motherboard.

It also surprised me that the author said "4 Cores, Xeon Server CPU can be had for cheap".

But the specs also said ECC RAM and I don't think the N100 supports that.

Same, using NixOS as a NAS, though it kept growing and now I'm trying to share it with other people. I use BTRFS on MDADM though. It's ended up being an all-in-one home server and router now because I was tired of the noisy power hungry rack I had in my closet.

https://HomeFree.host

Is snooty a autocorrect for Synology or some other product?

That is kind of exactly how zfs works though. The guide isn't complete, sure, but "rebuilding" the array is just replace the disk and run a single zfs command.

ZFS makes this completely trivial except for the "beeps and lights a led" part

It's complicated, but https://jro.io/capacity/ actually walks you though how it works, towards the middle of the page.

``` Unlike traditional RAID5 and RAID6 implementations, ZFS supports partial-stripe writes. This has a number of important advantages but also presents some implications for space calculation that we'll need to consider. Supporting partial stripe writes means that in our 7wZ2 vdev example, we can support a write of 12 total sectors even though 12 is not an even multiple of our stripe width (7). 12 is evenly divisible by +1 (3 in this case), so we don't even need any padding. We would have a single full stripe of 7 sectors (2 parity sectors plus 5 data sectors) followed by a partial stripe with 2 parity sectors and 3 data sectors. This will be important because even though we can support partial stripe writes, every stripe (including those partial stripes) need a full set of p parity sectors. ```

> and while this made sense when drives were expensive

I don't have the answer to the latency question, but HDDs have shot up in $/TB over the last couple years too. They are once again kind of expensive.

The RAID5 write hole is not present in RAIDZ1 I believe.

I'm sure there's better options now but the HP ProLiant MicroServers (used).

They support ECC ram, 4 caddies, one extra PCIe slot, and to my knowledge you're not cpu limited for a zfs file server usecase.

Keep in mind though, all you need is linux* support, iDRAC, ECC if you're a snob, and drive bays ... and that's basically any free server.

In my extremely opinionated opinion I would only get used enterprise server gear, because a zfs file server will just work unless hardware fails. And a UPS.

*ZFS will be a more natural choice on FreeBSD. It's better documented, and will meet Linux 1:1 in hardware compatibility for this.

Not strictly a recommendation, but Terramaster is a good brand to look at if you want Synology-shaped hardware which can run TrueNAS or Proxmox or any flavour of Linux you want.

Along with various other devices (including a large Synology which I wouldn’t buy today), I run Proxmox on a small two bay+two nvme Terramaster. I have a bare bones Ubuntu LXC running Samba configured for Apple Time Machine, an VM running Scrypyed, and PBS for Proxmox backups. Nothing on it is critical so I don’t bother with any storage redundancy.

It's a horrible idea likely, but I have an ancient old Dell PowerEdge R510. Probably sucks way too much energy, but it does what it does and the price of SSDs have skyrocketed so I'm not touching it.

I've lost more data to "proven reliable systems" than to my homegrown hacked-up stuff (which may be because I trust the latter much less).

> I would rather use a proven reliable system that’s been used and tested by millions of other users

You're describing ZFS.

Run ZFS backed filestore on FreeBSD, have migrated it to/from Debian. At work and home, not petabyte scale but certainly multi hundred terrabyte. Over 15 years, on over 50 hosts/NAS/SAN instances, different hardware.

Run ZFS on Raspberry Pi, on home builds, on Intel, on AMD, on other ARM chipsets.

I think you're over-stating things. Debian is fine for this. I do think FreeBSD is a better platform for myself.

The code bases adhere (modulo ZFS version numbers) to a spec and you can safely migrate the pools between OS. I've done it multiple times both directions.

You can not do this with BTRFS and other Linux things, I consider this feature of (Open)ZFS a killer-context for me: It's OS portable. I wish Mac OSX hadn't walked out of the room when Oracle went legal.

If you run ZFS with an LTS kernel you're pretty much fine. Yes new Linux releases will break existing ZFS releases - but the LTS tree is in support for long enough that this is never an issue.

> Running ZFS on anything but Solaris/Illumos/FreeBSD is asinine.

> ZFS is a permanent second-class citizen on Linux

Linux is the primary target of OpenZFS [0] [1], and has been since 2020 [2]. It may not be supported by the Linux kernel developers, but it's supported by the ZFS developers, and that's all that really matters.

> I don't want to trust my data to some half-assed out-of-tree solution that may or may not break in a week.

Sure, it's an out-of-tree module, but that doesn't mean that it will randomly break all the time; it just means that you may occasionally need to wait for a new OpenZFS release before upgrading your kernel.

> FreeBSD ZFS support has matured and is outstanding.

Agreed, but Linux and FreeBSD both use the same ZFS [3], so I don't really see how the ZFS in FreeBSD can be better than the one in Linux. The tooling and install procedure is certainly better on FreeBSD, but the actual filesystem code is the same (and is probably slightly more robust on Linux since that's going to be where most of the testing occurs).

[0]: https://github.com/openzfs/zfs#supported-kernels-and-distrib...

[1]: https://github.com/openzfs/zfs/pull/8987

[2]: https://github.com/openzfs/zfs/releases/tag/zfs-2.0.0

[3]: https://docs.freebsd.org/en/books/handbook/zfs/

And yet TrueNAS is moving away from BSD towards … Debian!