Remix.run Logo
taneq a day ago

While it’s important to make this explicit, at what point do we just assume a high-reliability UPS is table stakes?

Of course, if you need SIL2 type reliability then you need to assume any given hardware component can spontaneously combust and become a total loss, at which point the data loss caused by a power cut is a rounding error.

toast0 a day ago | parent | next [-]

> While it’s important to make this explicit, at what point do we just assume a high-reliability UPS is table stakes?

Several years after they become commercially available?

My experience with small UPSes is they tend to cook the batteries and you don't find out until they switch the load and the battery doesn't hold up.

Large facility UPSes tend to do better, but automatic transfer switches have a tendancy to fail ocassionally. If you're hosted in many locations, it's not unusual to have a couple ATS failures per decade.

All that said, unexpected power loss is certainly one reason that writes may be lost, but OSes crash too. Disk firmware can also crash, but if thst bricks the disk, writes in progress don't really matter. Sometimes cabling fails. Or you get a uncorrectable ECC error (which will typically cause an OS panic... unless you're running a very fancy OS, but if it's in dirty disk backed page, even a fancy OS wouldn't save you)

Plenty of applications don't need or want to pay the cost for full commit to disk, but calling something durable when it's not committed to disk is inaccurate.

And that's before we get into the whole thing where the OS and the disk like to return success when things haven't quite finished.

whilenot-dev a day ago | parent | prev | next [-]

What's got this to do with a UPS? Not having a UPS is an external threat on the reliability of the power grid.

Doing a hard shutdown or tripping over power cords seem much likelier local scenarios than any spontaneous combustion of hardware components.

bestouff 14 hours ago | parent | prev | next [-]

If it doesn't survive a power loss it may not survive a kernel crash.

zbentley 18 hours ago | parent | prev | next [-]

A UPS won’t save you from a kernel panic.

imtringued 21 hours ago | parent | prev [-]

Not sure how you managed to do it but you got it completely backwards.

If someone demands that the database should use fsync and only respond with success once the write finished, it is not some arbitrarily high reliability demand that needs to be implemented using reliable hardware. In fact, the entire point of implementing the power loss protection in software is so that you don't need perfectly reliable hardware. The power loss event turns into a downtime event which is often completely acceptable.

The requirement to have infallible hardware only emerged because the software refused to do its job. Infallible hardware is not a requirement decided by the user, it's a requirement decided by the developer of TurboKV to intentionally restrict his software to exclusively operate in a reliable hardware environment.

The fact that the user specified durability of the KV store during power loss does not make the user obsessed over hardware reliability, the software shifted the burden onto the hardware and forced the user to deal with this mess.

I don't know how exactly TurboKV works so let's talk about a hypothetical software instead.

Let's say the software cannot survive a power loss event and just corrupts the database. If the user wants to operate the software, he is forced by the software to operate it in an infallible environment where power loss can never occur. Based on how the software was designed, power loss is a catastrophic event. The SIL2 type reliability you're talking about only makes sense in contexts with catastrophic events.

So how it went is that the user made a reasonable demand with bounded reliability: "please survive power loss with durable writes" and the author says, sure just run the software on a SIL2 type reliability hardware environment.

It's not the user who blew up the hardware requirements.