I don't think commits per se puts pressure on the infrastructure.

More likely pulls and pushes, and, naturally, the ci minutes they identify as the main issue.

But CI only increased by a factor of 2 since last year. Did they really not foresee that happening? And how does that affect git and api operations.

▲

munk-a 6 hours ago | parent [-]

It really shouldn't. The technical summary they released[1] is a very interesting read from a software engineering perspective. It seems to be blindsided by the increased traffic and gives stats related to commits/PRs (which should be relatively cheap for github to process) without any insight into their web traffic or details on how much actions are costing them. If they were super transparent they'd release information about their request response time and resourcing to fulfill that.

Their current path to resolution is to migrate their codebase to a new language[2], continue to drop their inhouse ops for Azure resources and get off MySQL. Maybe one or two of those steps are legitimately a good idea - I don't have an inside scope - but technology migrations are always fraught with issues. It's quite possible these changes are just a result of them vibe-coding a mature codebase into a new language.

1. https://github.blog/news-insights/company-news/an-update-on-...

2. I'll grant that Ruby isn't the best language to use as scale but I think we're all old enough to realize that language choice is far less impactful on performance than code quality.

▲

hosh 5 hours ago | parent | next [-]

Azure’s core hypervisor orchestrator was half-baked at launch and it has never been fixed. This long read blog series explains a lot for me — for example, why the FedRamp certification program was never able to get a straight answer from Azure about how they handled secrets.

https://isolveproblems.substack.com/p/how-microsoft-vaporize...

https://www.kunalganglani.com/blog/microsoft-fedramp-failure...

▲

evanelias 5 hours ago | parent | prev | next [-]

> migrate their codebase to a new language[2], continue to drop their inhouse ops for Azure resources and get off MySQL

The recent blog post you're linking to mentioned moving data only for webhooks off MySQL, not all relational data used by the entire site; and moving "performance or scale sensitive code out of Ruby", again not the entire codebase.

Do you have an official source suggesting these migrations are more comprehensive than that?

▲

munk-a 5 hours ago | parent [-]

I do not know - this is the only source I'm aware of and the wording is vague enough that the above is just my interpretation of it. It could be highly targeted but the manner of wording indicates a strong preference that smells of a large migration.

▲

evanelias 5 hours ago | parent [-]

What part of the wording gives you that impression? On these topics, the post literally just says the following:

"bottlenecks that appeared faster than expected from moving webhooks to a different backend (out of MySQL)"

"Similarly, we accelerated parts of migrating performance or scale sensitive code out of Ruby monolith into Go" (in a paragraph specifically about "critical services like git and GitHub Actions")

Both of those sound highly targeted to me!

▲

munk-a 5 hours ago | parent [-]

> While we were already in progress of migrating out of our smaller custom data centers into public cloud, we started working on path to multi cloud. This longer-term measure is necessary to achieve the level of resilience, low latency, and flexibility that will be needed in the future.

That paragraph read, to me at least, that the initial targeted changes were just the tip of the iceberg and that much heavier lifting than initially budgeted were now in scope.

	▲	evanelias 4 hours ago \| parent [-]
		"smaller custom data centers into public cloud" is talking about their Azure migration, so "multi cloud" would almost certainly mean extending a presence into AWS and/or GCP (or maybe others like OCI). I'm sorry but I really don't see how you're drawing conclusions about this meaning a move off of Ruby and MySQL entirely. That's a huuuge logical leap away from what is written in this post, and you originally stated it in a way that indicated this was a fact.

▲

spockz 6 hours ago | parent | prev [-]

Re 2, I would generally agree and there is a lot that can be done with caching. However, since writing services in Rust and Golang, there is whole other tier in speed. Architecture matters, code quality also matters, but Golang and Rust help a lot in making very fast services.

	▲	munk-a 6 hours ago \| parent [-]
		Yeah I don't disagree. To clarify. Rust, Golang etc - they give you a very noticeable advantage when it comes to writing good performant software with the assumption that you're putting in the effort on the design side. But poorly written Rust is likely going to be indistinguishable from poorly written Ruby.