Remix.run Logo
▲ Google cloud outage takes down banks(neowin.net)
99 points by paweladamczuk 2 hours ago | 39 comments
▲nickcw an hour ago | parent | next [-]

From Google's status page https://status.cloud.google.com/incidents/Xs2XTHzAVC9YnafnPy...

    We are experiencing an issue with multiple services in
    europe-west2 beginning on Monday, 2026-10-19 23:08 PDT.
    
    Due to a power event we had suffered an outage within
    europe-west2-b zone. The power event is mitigated. Partial
    mitigation to infrastructure services will be observed within the
    next 30 minutes and the remaining services will be mitigated there
    after.
So some kind of power failure happened to bring down one zone. Unfortunate, but this kind of stuff happens. That is why all the cloud providers have multiple zones, and I'm amazed the big companies reported as down weren't using multiple zones.

(Also reporting the times in PDT is quite unhelpful for users of this zone who are likely based in the UK or EU)

▲bsuvc an hour ago | parent | next [-]

I'm not saying it's the case here, but sometimes the cloud providers themselves have inter-zone dependencies, or even inter-region dependencies that can cause outages in other zones and regions.

For example, AWS us-east-1 outages have caused global outages on multiple occasions, such as https://www.networkworld.com/article/4168878/aws-hit-by-us-e...

▲HelloUsername 4 minutes ago | parent | prev | next [-]

> https://status.cloud.google.com/incidents/Xs2XTHzAVC9YnafnPy...

https://news.ycombinator.com/item?id=50163858

▲robinson-wall an hour ago | parent | prev | next [-]

> I'm amazed the big companies reported as down weren't using multiple zones

Much as with backups - if you don't test them, they don't exist.

Everywhere I've worked has had multi-AZ infrastructure. I have never worked anywhere where the failure of an AZ has been simulated regularly, with second-order effects mapped out.

▲cavoirom 30 minutes ago | parent | prev [-]

I know some banks out-sourced most of its infrastructure to external operators, and some operator I worked with didn't seem competent.

▲nfrankel 2 minutes ago | parent | prev | next [-]

Migrate to the Cloud, they said. It will be more resilient and less expensive, they said.

▲gadders 2 hours ago | parent | prev | next [-]

>>Google cloud outage takes down HSBC, Revolut, Bank of Scotland, Llyods, Equifax, more

You will probably have issues logging into Llyods even when the cloud is back up.

▲venzaspa 32 minutes ago | parent | next [-]

It's crazy. My main account is with Starling and I have a Lloyds account for a saving an account and some free cinema tickets. I don't think I've ever had an issue with Starling, it's great - Lloyds fails to sign in about 50% of the time consistently.

▲cowlevel 9 minutes ago | parent [-]

Do you have an account with Llyods though?

▲michaellee8 37 minutes ago | parent | prev [-]

cannot agree more

▲bcvvvy an hour ago | parent | prev | next [-]

I bank with HSBC. Everything is up and working.

A lot of the information people are leveraging here comes from Downdetector which is mostly users reporting that stuff is down, not whether they are actually down. It's a really bad source.

▲m11a an hour ago | parent | next [-]

It's a pretty good source ime (at least, massive spikes tend to correlate well with issues). Traditional organisations aren't as transparent as SF tech companies about service downtime. Plus, if the service isn't working for its users, I wouldn't say it's actually up.

▲4ndrewl an hour ago | parent | prev [-]

Alternatively, and from the top of the HSBC homepage

"We would like to apologise for any inconvenience and continue to monitor the situation, which is also affecting other organisations."

▲gaiagraphia 3 minutes ago | parent | prev | next [-]

It's a little worrying that banking with an institution from one of the world's financial capitals is so vulnerable. I would've thought London would've been priding itself on security, stability, safety, etc. Confidence in their systems is their bread and butter

Afaik, even Russia and Ukraine aren't having such issues while dealing with data centres getting droned, lol.

It's really quite poor performance.

▲paweladamczuk 2 hours ago | parent | prev | next [-]

For me it was Revolut. No access to my main account, no withdrawals, no card payments since the morning. App doesn't work, there's no phone number to call, nobody to talk to. Dark side of neobanking, I guess.

▲Arbortheus 2 hours ago | parent [-]

Not all neobanks. I use Monzo, and they had an outage a while back. I was impressed that it barely affected their functionality.

They have a stand-in system that uses a separate codebase and separate cloud provider. They have a blog post about it here: https://monzo.com/blog/tolerating-full-cloud-outages-with-mo...

The stand-in system has a limited subset of features, i.e. no bells and whistles, but enough that you can get any actual banking done.

(Not an ad / I have no connection to Monzo, I just had some appreciation for a bank doing this properly as normally banking IT is a disaster. Plus the idea of keeping a separate codebase for almost an entire bank's functionality seems very invested.)

I have always got bad vibes from Revolut...

▲m11a an hour ago | parent | next [-]

Starling also has a disaster recovery system across cloud providers, although it is implemented differently to Monzo's (I used to work there on multi-cloud).

Multi-cloud is really tough, tbh. It's easier said than done, especially if you care about data consistency at all.

▲ndsipa_pomu an hour ago | parent | prev [-]

> I have always got bad vibes from Revolut...

I opened up a Revolut account a couple of years ago with the intention of switching my current account to it (i.e. wages going into it etc), but I got spooked after hearing that they weren't registered as a UK bank and thus subject to the various regulations and protections. I tried using Chase instead and have been happy with it so far.

▲misstuned an hour ago | parent | next [-]

I used to use Starling as my main and they were fine, until I started getting issues with random card declines due to "fraud prevention" and nobody to talk to. I switched to First Direct and like the peace of mind of having a human on the phone if needed (when I wanted to make a huge transfer for my house deposit it was nice to have!) and I still get perks like free foreign card transactions.

▲altmanaltman an hour ago | parent | prev [-]

https://www.revolut.com/blog/post/revolut-is-officially-a-ba...

They are reigistered now

▲gentlerain 2 hours ago | parent | prev | next [-]

Countries talking about AI sovereignty, when they forgot compute sovereignty

▲octocop 3 minutes ago | parent | next [-]

I think people have been fighting Googles monopolies for quite some time

▲Arbortheus 2 hours ago | parent | prev [-]

I think sovereignty might be the _cause_ of the banks' outages...

This is affecting companies that put everything into a single GCP region in the UK for their entire business - aka most likely British companies or government with a data sovereignty requirement.

▲gentlerain 2 hours ago | parent [-]

That's not sovereignty.

Sovereignty is when there is a UK company with multiple data centres in the country.

With that if one region goes down, another can carry on.

▲derektank 2 hours ago | parent | next [-]

Redundancy does not seem like an inherent component of sovereignty

▲pwagland 42 minutes ago | parent | next [-]

The UK, at least, is big enough that you can have multiple regions within the same country. There's a lot of distance between Glasgow and London, for example. Maybe not enough if both are targeted with nuclear warheads, but at that point, we have bigger fish to fry I suspect.

Most European countries have enough space within the country to do that, and, for most things, except banking typically, hosting within the EU is enough, it doesn't have to be within your own country.

▲pumplekin 29 minutes ago | parent [-]

Microsoft Azure has two entirely seperate regions.

A "full function" one in/around London. And a "tickbox" one for multi-region requirements near Cardiff.

The tickbox one has constant capacity constraints and a large number of Azure services aren't in it, but if you need to tick the "two regions and in country" requirement, you kinda/sorta can.

▲gentlerain an hour ago | parent | prev [-]

True on first glance.

But considering what these sovereignty rules aim to achive, and as someone mentioned, that the problem may be due to companies relying on a single region provided by a hypersclaler due to data sovereignity rules.

Then redundancy has to be considered by both the company and the rulers.

▲philipwhiuk 18 minutes ago | parent | prev [-]

That's not sovereignty, that's redundancy.

Sovereignty is just where stuff is, not how many place it is there.

▲misstuned an hour ago | parent | prev | next [-]

I couldn't unlock a shared city bike this morning on the Beryl system, probably the same outage.

▲diamondDrill 20 minutes ago | parent | prev | next [-]

how can cloud inf go down? that was the main selling point, it makes no sense

▲layer8 15 minutes ago | parent | next [-]

The main selling points were auto-scaling and not having to self-host. Or, on a higher level, outsourcing accountability, being able to point at someone else if something goes wrong, as exemplified by the present case.

▲dist-epoch 18 minutes ago | parent | prev [-]

It makes no sense only if you believe in magic and marketing

▲BlackRabbit1 an hour ago | parent | prev | next [-]

Most likely it knocked out the "Ing" bank in Germany.

▲mistic92 an hour ago | parent | prev | next [-]

Ah again London region :D

▲rvz an hour ago | parent | prev | next [-]

Since there are no humans working on outages here is the test for superintelligence:

Can someone ask Claude, Codex, Copilot, Antigravity, Devin to resolve this outage?

No Googling or help from humans this time.

▲dangoodmanUT 23 minutes ago | parent | prev [-]

At this point, the way I see it is:

- There's AWS

- There's Azure

- There's everything else

And I don't even use Azure, I just hear it's good (and they don't charge cross-AZ traffic)

▲esseph 6 minutes ago | parent | next [-]

[delayed]

▲jiggawatts 20 minutes ago | parent | prev [-]

Azure’s design is elegant and even — dare I say it — beautiful.

Its implementation is clumsy and ugly.

Its documentation is worse.

Its support… isn’t.