Remix.run Logo
otterley 2 hours ago

If your AWS service is properly regionalized, that’s the minimum amount of cellular architecture required. Did your service ever fail in multiple regions simultaneously?

Cellular architecture within a region is the next level and is more difficult, but is achievable if you adhere to the same principles that prohibit inter-regional coupling:

https://docs.aws.amazon.com/wellarchitected/latest/reducing-...

https://docs.aws.amazon.com/wellarchitected/latest/reducing-...

HumanOstrich 2 hours ago | parent [-]

You didn't really put any thought into what I said. Thanks for the links.

otterley 2 hours ago | parent [-]

It wasn't worth thinking about. I'm not going to defend myself against arguments and absolute claims I didn't make. The key word here is mitigation, not perfection.

hedora an hour ago | parent [-]

> If your AWS service is properly regionalized, that’s the minimum amount of cellular architecture required

Amazon has had multi-region outages due to pushing bad configs, so it’s extremely difficult to believe whatever you are proposing solves that exact problem by relying on multi-regions.

Come to think of it, Cloudflare’s outage today is another good counterexample.

otterley 37 minutes ago | parent [-]

It has been a very, very long time since AWS had a simultaneous failure across multiple regions. Even customers impacted by the loss of Route 53 control plane functionality in last month’s us-east-1 were able to gracefully fail over to a backup region if they configured failover records in advance, had Application Recovery Controller set up, or fronted their APIs or websites with Global Accelerator.

Customers survive incidents on a daily basis by failing over across regions (even in the absence of an AWS regional failure, they can fail due to a bad deployment or other cause). The reason you don’t hear about it is because it works.