IRN, SEIM logs reporting a high incidence of telnet connections to port 25

Still ongoing, I don't think I remember such a long outage!

Yet another risk of 2FA that most people don't realize.

Bizzare. My self hosted email, which everyone told me was stupid to host myself has been sending and receiving just fine.

Of course I haven't received a single email because every important email is sent via Microsoft but I still feel superior because I'm not affected

Officially Microsoft 364

> Still ongoing, I haven't been able to access my personal outlook email for 12+ hours and as a result can't access my accounts that use email 2FA, including my virtual medical appointment!

I guess software is still not solved, I'll let Dario know

its so funny, that service outages might help people get away from american hyperscalers.

You mean a Microsoft 365 account?

Because "personal outlook" at https://outlook.live.com/mail has been working this whole time.

Because I know people here are devs but you guys dont know IT lmao

Crowdstrike crashed 8 million+ machines and broke many so bad they didn't fix on restart. Their stock is up 3x since then: https://en.wikipedia.org/wiki/2024_CrowdStrike-related_IT_ou... . I don't understand it but I also wouldn't bet on anything changing from this outage.

The reality for most companies, up until now at least, is that if they manage email servers themselves they will have more annualized downtime than if Microsoft does it. Of course if this outage lasts long enough (who knows what's going on over there), that will stop being true.

IMHO a bigger concern is that Russia hacked Microsoft in 2022 and China hacked Microsoft in 2023. Even that wasn't enough to scare companies off MS.

Basically, MS customers are locked in, the cost of migrating to anything else is huge, and there's a "no-one gets fired for buying MS" herd mentality, so MS has little to lose no matter how hard their product fails, just like Cloudstrike.

There's also the question of what alternative would really be better, or at least more reliable. My impression is that Google's infrastructure and engineering are genuinely better, but I'm recently ex-Google so take that with a grain of salt.

Maybe use a decentralized protocol, something like SMTP...

My small US gov org has been pushing more of our infra into centralized Microsoft services. The rationality seems to be that outages would be worse if we managed our own stuff. Nobody trusts each other and choosing Microsoft is an easy way to shift the blame outside the org. Nobody ever gets fired for choosing Microsoft and any issues they have must be justified.

It's ironic that the Internet was designed to accommodate nuclear war and we still decided to consolidate so much critical infrastructure onto monoliths.

It should, but knowing UK gov they'll instead pay Accenture and IBM £10bn for a backup system that'll take 15 years and never get delivered.

I mean why is outlook even centralised? If a company has 600,000 employees, why don’t they get their own infrastructure? Why is it all dependent on central Microslop infrastructure in America

It's been a while (~10y) since I administered windows servers, but I recall they were much harder to automate w.r.t. maintenance. So Microsoft created M365 to fill that niche and have organizations offload operational cost to them.

With a linux fleet can run via SSH/ansible and you can patch thousands of machines programmatically in minutes and control the deploy at a fine level. With Microsoft/Windows it often involves/d logging in to the one AD server where everything runs, back it up via 3rd party software, apply the patches and hope it doesn't break, and if it does, call MS support, because rolling back is really your last resort (I feel it's become better since 10y ago, but I don't have hands-on experience anymore).

Our company runs on-prem Exchange since 2019, and... well... it's a PITA to work with (Legacy Outlook/New Outlook), we downgraded from M365 and it was really a step down in terms of features and reliability. OTOH I guess it's keeping us afloat today when M365 is having a really bad day.

They’re paying a fortune every year.

You can run your own email easily and cheaply, but if you wan to run with a team big enough and experienced enough to recover quickly if there's a problem it gets expensive fast.

If everyone had their own servers, either they're centrally enough managed that they aren't really independent servers or the management overhead and chaos from version drift etc. creates a mess that overall lowers the reliability of the entire system. Sure, you won't have concentrated outages like this but your instance's overall uptime is likely to be lower.

Google Workspace isn't going to be any different.

Keeping your own server(s) up, including spam control, is work that takes time and costs money. Microsoft is probably still cheaper than that,[citation needed] and has more resources to keep everything running than even a medium-sized government. So it is understandable.

> Is it some sort of spying thing? Incompetence

It’s Microsoft we’re talking about so I feel like the latter is a given, and the former is a reasonable assumption.

Jokes aside, it does seem that somehow these hyperscaler cloud providers who tout the reliability of their multiple zones and global distribution sure do seem to always have some central dependency or single point of failure. AWS for example seems to have a global outage once every year or two at this point, or like us-east-1 problems always seem to affect other zones too. You’d think with their scale these clouds would be completely decentralized and redundant but apparently not.

> Is it some sort of spying thing? Incompetence?

LLM generated code and/or configs?

This take is bizarre and i’m really beginning to think that a scant few people are pushing it in bad faith in an attempt to spread rumours. It has no connection with reality. Extended downtime and faffing about with distributed systems issues all happen without it being caused by an attack.

A successful cybersecurity attack on a major email provider like Outlook or Gmail is an attack on the banks. Access to a user's email account gives attackers most or perhaps all they need to access their bank accounts.

Yeah it’s either a hack or someone slopcoded something that has broken the servers so badly that they can’t be restored.

https://isolveproblems.substack.com/p/how-microsoft-vaporize...

one of the all time tech gossip blog posts

Azure is, ironically, mostly unaffected, and Microsoft's issues long predate AI coding, by like half a century almost.

It is literally at the top of the front page on all major news services.

It shouldn't be news in the first place, why do we all use such centralized systems where everything breaks down because of one bottleneck is the better question.

Getting buried by the Jensen Huang downfall. MS is super lucky it overlapped

Coming back online will be tricky now for Outlook, as everyone has a day's worth of emails queued up waiting to be delivered.

If everyone uses Microsoft and it goes down for Microsoft, nobody blames you, and you're at no disadvantage compared to them.

Much of the productivity gain from this AI boom came from shuffling off the expensive emphasis on robustness and reliability, and this downtime is just one of the externalities of this increase in developer productivity.

As evidenced by Crowdstrike stock quadrupling since it took down half the internet in 2024, vanishingly few people actually care about downtime like this.

Meanwhile, there is decades-long familiarity with Microsoft's product offerings which makes for generational vendor lock in.

I think this is just a new normal. Genuinely speaking, what percentage of businesses will de-Microsoft in the next year over this? I don't think it would even be 2%.

If it goes wrong its bad for users.

As long as it works its good for them.

Because not centralizing it means you now have countless separate environments that you need to manage, and if you manage them centrally, congrats, you reintroduced a SPOF again.

You can reduce the blast radius of outages at the cost of increasing the frequency a lot.

Websites that don't support password (or passkey) login are infuriating. Sad to hear that Claude is one of them.

I am still broken with that, can't use outlook and claude code.

Claude is down.

Tried this. Took a while to load, but it did work.

An SMTP server is for sending / transferring mail. You’d need an IMAP/POP server that holds mail, setup accounts for everybody on your existing mail, provide access to all those users. It’s like asking how realistic is it to build outlook.com to cover the outage.

At any scale that unrealistic

Email based 2FA is the devil.

The problem is that making a business out of "less" is hard. Especially when dealing with the hellscape that is "enterprise sales". Both the marketing people and the customer love piling on additional features!

Two decades ago I was at a startup that moved from a Linux email solution to a Microsoft one. Why? For the CEO's blackberry and calendar integration.

There are loads of FOSS email solutions and providers. There are fewer "email+calendar" providers, and there are even fewer "email+calendar+desktop logon SSO" providers.

Running a "simple stable core service" at this sort of scale is far, far harder and more complicated than you understand. Even responding to ICMP ping requests at AWS/GCP/Azure scale requires custom solutions. I've seen very many near-misses in high profile global services due to the complexity of running at this scale that have nothing to do with the causes you're assuming here.

We need decentralisation coupled with a reliance on shared protocols, like we used to have before platforms took over everything and anti-trust legislation became meaningless.

Too bad for msft 2026 wasn't a leap year

In the hands of god!

Damn. I should stop advocating for running important stuff on-prem. We should move all the things to azure!

They saw GitHub’s 93% uptime and decided to knock a 9 off theirs

Microsoft OneDrive rename to OneBasket incoming

cool, more skiing for you!