[dead]

Is there any proof that "self hosted" is still working, wouldn't surprise me if at least part of it relies on centralised services.

In a sane world the CTOs choosing this would be held accountable. The massive global economic fragility which is created by the centralisation of society is a massive risk. Diversity is important, we see it time and time again how optimisation causes panic, from crowdstrike to microsoft. When your outsourced provider is "too big to fail", you have a problem that government needs to deal with.

My customers are small.

One of them has two servers on AWS behind a load balancer. That's the very least of resilience.

Another one has one server in a datacenter running their primary service on a single VM. That server had issues with RAM chips a couple of years ago that brought it down a few times. However they are not crazy. They made the choice to keep their data at home, bought reportedly good hardware, accepted there there might be consequences. They have other less beefy servers but they did not load balance their service on VMs on different servers.

I think that we can agree that this is no resilience. Is it by design as in "I want my service not to be reliable"? Of course not. Is it lack of quality in the design? Of course it is. But does it matter from the point of view of the users of the service?

While it's probably true that no one specifically seeks to remove resiliency for its own sake, basically the entirety of our corporate world has, over the past several decades, been relentlessly redesigning everything in ways that remove resiliency. Because resiliency is almost inherently antithetical to efficiency. At the most fundamental level, if you want to make sure you have a backup of your data, that means you need at least 2x the amount of storage. And, of course, if you want to make sure you can operate even if a critical employee gets sick/is hit by a bus/goes on vacation, you need to hire extra employees.

And that's anathema to companies nowadays.

You design to the needs. I have resilience in my home DNS so I have a 100% availability bounded on the home (if the home is burnt down then I'll lose DNS)

At work I designed DNS to survive the loss of a city, so again 100% uptime. The riskiest part is deploying changes, which is why the deployment doesn't run in parallel and confirms state before deployment.

Resilience isn't hard when you aren't trying to serve a billion customers. The problem is we are addicted to a "winner takes all" economy, which leads to fragility. If Tesco breaks, I can go to Sainsburys, or Aldi, or Lidl, or Iceland, or get in the car and go to Asda or Morrisons. That's diversity.

But in IT the goal is all those follow the same gartner reports and all use the "best supplier" for their backend infrastructure, and if that supplier breaks then they can't restock any supermarkets.

That's a terrible risk and needs to be stopped, not encouraged. But the individual incentives are for the CTO of each supermarket to choose what the consultants tell them. If they don't they are accountable for the choice of not using things like "industry standard".

But it's loading a 16MB JavaScript file much faster! Improvement (not in the direction I like, but still)!

Soon to be O364 :-(

Best of luck to those oncall.

It's the Great Rebootening. That's what they're trying apparently. Roll back and reboot and see if everything works.