> when he described ZFS as a rampant layering violation

I read some blog posts back in the day about why they did this and it sounded a lot like those layers were more historical accidents or something.

You can turn it around and say that ZFS is a full stack filesystem (or vertically integrated if you will) and it should be pretty obvious that a rethink on that level can have big advantages.