Remix.run Logo
ants_everywhere 7 hours ago

I'm not a filesystem person, but this sets off similar red flags to rolling your own encryption.

Isn't writing a robust file system something that routinely takes on the order of decades? E.g. reiserfs, bcachefs, btrfs.

Not to rain on anyone's parade. The project looks cool. But if you're writing an OS, embarking on a custom ZFS-inspired file system seems like the ultimate yak shaving expedition.

jillesvangurp 3 hours ago | parent | next [-]

Sometimes doing things because they are hard is a great reason to do them to see if the reasons those things are hard are still valid. Doing a filesystem in Rust potentially mitigates some of those things. Most existing filesystems have gone through a lengthy stabilization phase where using them meant exposing yourself to nasty data corruption bugs, obscure race issues, and other issues that, when you root cause them, have a lot to do with the kinds of things Rust explicitly addresses (memory safety, safe concurrency, etc.). So there's a great argument to just try to leverage those features to make things easier and try to build an awesome file system.

Worst case this doesn't work. Best case, this works amazingly well. I think there's some valid reason for optimism here give other hard things that Rust has been used for in the past few years.

Macha an hour ago | parent | prev | next [-]

You're talking about a project to write their own OS for ultimately the fun of it. It probably shouldn't be too surprising that attitude extends elsewhere.

The people who just think OSes would be better with more Rust in them, but aren't looking to reinvent from first principles are in the Rust for Linux universe.

And you know what, that's fine. Linux started out as a hobby project with similar origins before it became a big serious OS.

Galanwe 5 hours ago | parent | prev | next [-]

I see Redox as an incubator of new developments for a low level Rust ecosystem. It's not a production ready OS, its purpose is to spark new ideas, propose alternative implementations, try on new paths, etc. I see them implementing a ZFS variant as completely in-line with this objective.

There needs to be projects like that for any kind of innovation to happen.

scns 33 minutes ago | parent [-]

> I see Redox as an incubator of new developments for a low level Rust ecosystem. It's not a production ready OS

Why leave at that? One day it can be production ready.

smittywerben 5 hours ago | parent | prev | next [-]

I don't believe in the "never roll your own encryption" it's literally giving up. Does it make economic sense, or is it just for a hobby? That's more debatable. It's also like a foil of 'don't use regex to parse html' or whatever, where the thread gets closed for comments.

The filesystem is so deeply connected to the OS I bet there's a lot of horror around swapping those interfaces. On the contrary, I've never heard anything bad about DragonflyBSD's HAMMER. But it's basically assumed you're using DragonFlyBSD.

Would I keep a company's database on a new filesystem? No, nobody would know how to recover it from failed disk hardware.

This isn't really my area but a Rust OS using a ZFS-like filesystem seems like a lot of classic Linux maintainer triggers. What a funny little project this is. It's the first I've heard of Redox.

Edit: reminds me of The Tarpit chapter from the Mythical Man Month

> The fiercer the struggle, the more entangling the tar, and no beast is so strong or so skillful but that he ultimately sinks.

rmunn 4 hours ago | parent [-]

The "never create your own encryption" advice is specifically because crypto is full of subtle ways to get it wrong, which you will NOT catch on your own. It's a special case of "never use encryption that hasn't been poked at for years by hundreds of crypto specialists" — because any encryption you create yourself would fail that test.

Filesystems, as complex as they are, aren't full of traps like encryption is. Still plenty of subtle traps, don't get me wrong: you have to be prepared for all kinds of edge cases like the power failing at exactly the wrong moment, hardware going flaky and yet you have to somehow retrieve the data since it's probably the only copy of someone's TPS report, that sort of thing. But at least you don't have millions of highly-motivated people deliberately trying to break your filesystem, the way you would if you rolled your own encryption.

smittywerben 3 hours ago | parent | next [-]

That matches what I've heard, so I think you stated the trope perfectly. Your response is a good point about the actual difficulty. Perhaps I'm confused about what 'rolling your own encryption' means at an abstraction level. I just think it's weird that it comes up in an OS thread. Anyone who is serious about encryption is serious about the encryption hardware. At a higher level, WolfSSL limits the ciphers to a small, modern suite, which reduces the attack surface. Replacing OpenSSL is a fool's errand, I think; it's clearly the perfect implementation of OpenSSL, and it's a perfect security scapegoat. However, this is still about the x86 OS topic. Perhaps it's some TPM politics, similar to the decade-old stigma surrounding ZFS. Maybe I'm just questioning the limits of the x86 platform on any new operating system. Anyway, thanks for the response.

4 hours ago | parent | prev [-]
[deleted]
a-dub 5 hours ago | parent | prev | next [-]

i don't think it has to be all that robust yet as it mostly runs in vms (even though it may be!).

an internet community project to write an entire operating system from scratch using some newfangled programming language is literally the final boss of yak shaving. there is no reason to do it other than "it's fun" and of course writing a filesystem for it would be fun.

koverstreet 5 hours ago | parent [-]

Rust really is attractive to a filesystem developer. Over C, it brings generics for proper data structures, iterators (!), much better type safety, error handling - all the things Rust is good at are things you want.

For me, the things that would make it just perfect would be more ergonomic Cap'n Proto support (eliminate a ton of fiddly code for on disk data structures), and dependent types.

a-dub 5 hours ago | parent | next [-]

it remains an open question as to how reliable, performant and efficient a system built with these higher level constructs would compare to the highly optimized low level stuff you'd see in a mature linux filesystem project.

i suspect the linux stuff would be far more space and time efficient, but we won't know until projects like this mature more.

koverstreet 5 hours ago | parent | next [-]

Eh? That's not an open question at all anymore; Rust has a drastically lower defect rate than C and good Rust is every bit as fast as good C.

Now, the engineering effort required to rewrite or redo in Rust, that's a different story of course.

a-dub 4 hours ago | parent [-]

i'd be curious how many of the higher level features and libraries would be best avoided if attempting to match the performance and space efficiency of a filesystem implemented in purpose designed highly optimized c.

cyberax 4 hours ago | parent [-]

I'm rewriting some of my Arduino projects into Rust (using Embassy and embedded-hal).

It's _so_ _much_ _better_. I can use async, maps, iterators, typesafe deserialization, and so on. All while not using any dynamic allocations.

With full support from Cargo for repeatable builds. It's night and day compared to the regular Arduino landscape of random libraries that are written in bad pseudo-object-oriented C++.

IshKebab 4 hours ago | parent | prev [-]

Yeah it's only an open question if you have your eyes closed.

IshKebab 4 hours ago | parent | prev [-]

To be fair Cap'nProto's C++ API is hardly ergonomic.

koverstreet 4 hours ago | parent [-]

Doing it right needs lenses (from Swift)

5 hours ago | parent | prev | next [-]
[deleted]
madushan1000 3 hours ago | parent | prev | next [-]

Redox Os is a microkernel operating system, completely different from monolithic kernels like Linux or BSD. I doubt it'll be easy to get existing ZFS drivers working on it at all.

atoav an hour ago | parent | prev | next [-]

Normally I'd say the same, but it is a matter of what your goals are. Ifnyour goal is to try new things and learn about computers on the way, why the hell not write your own filesystem as well?

If your goal is to gain wide adoption fast, that is a bad idea.

MangoToupe 7 hours ago | parent | prev [-]

Isn't brtfs itself just a ZFS-inspired filesystem? If that can manage to find a foothold, why can't this?

koverstreet 7 hours ago | parent [-]

The only thing btrfs took from ZFS was the featureset - COW, data checksumming, snapshots, multi device. ZFS was a much more conservative design, btrfs is based on COW b-trees (with significant downsides) and if you can put it in any lineage it would be Reiserfs.