Remix.run Logo
dcrazy 16 minutes ago

> We installed as much hardware as available power allowed in our existing data centers while accelerating our migration to Azure.

And from the RCA [1]:

> The immediate cause of the failure was network saturation on load balancers in Central US due to a new peak in traffic.

[1]: https://www.githubstatus.com/incidents/zkxwbgr0cnmx

ivraatiems 11 minutes ago | parent [-]

"While accelerating our migration to Azure," meaning, they will only solve problems if it helps them also use Azure more.

It is unbelivable that aload of 2.8b commits was totally fine, and a load of 2.9b was a sitewide outage, unless they have no reporting or their tooling is completely incompetent. If things can fall apart so easily, throwing more capacity at the problem won't fix it.

dcrazy 10 minutes ago | parent [-]

You’re torturing your own logic to make Azure the villain here. And it also sounds like you lack experience with capacity exhaustion. Things fail slowly, then suddenly.

ivraatiems 9 minutes ago | parent [-]

No, I am calling Github disingenuous liars with poor motives, offering an unbelivable explanation. Not sure where the torture is, except in their statement.

If it was failing slowly, why didn't they fix it then? Why didn't they notice?

Could it be because they fired all the people who know how to monitor and identify such problems? That's what SREs are for.

dcrazy 8 minutes ago | parent [-]

Baseless accusation made from a position of zero information.

ivraatiems 5 minutes ago | parent [-]

Opinion based on stated facts.

Please share the information you have which contradicts the conclusions I have drawn from Github's statement.

(And we know they're liars. They report very few of the actual incidents they have; see for example https://mrshu.github.io/github-statuses/)

dcrazy a few seconds ago | parent [-]

I don’t owe you anything, much less a separately sourced counterargument. You openly admit that your opinion is not based not on GitHub’s proffered statements but on your self-admitted assumption that GitHub is actively lying in an attempt to cover up an Azure-related root cause.