Updating Rails is one of the most painful processes ever. There is no real way to be sure it works. You just have to have 50,000 unit tests, cypress tests, and then hope and pray.

On any sufficiently large and old app, something will always break in a way that isn't caught by your tests or manual testing, and only shows up after it goes out live.

This has not been my experience.

I have taken applications from 2-to-3, 3-to-4, and so on, through 8.X.

If anything it has gotten better/easier over time. The most challenging upgrades were 2.X to 3.X (for reasons I can't recall), and then 6.X to 7.X (for an application that had issues adopting zeitwerk). In both of those situations, there was a lot of rote legwork, but once tests were passing, the application was working reliably. The other upgrades (3-to-4, 4-to-5, 5-to-6, 7-to-8) weren't happy-fun-rainbows-and-unicorns, but they weren't catastrophically complicated? Not even in fairly large codebases.

For each of these, a strong test suite was the best tool, which it sounds like you already know?

In my experience, the other impactful factors were:

(1) Reading the CHANGELOGs and knowing enough about Rails to know what they meant for the application.

(2) Using test fixtures (or fixtures with only some factories) for a quicker feedback loop for engineers.

(3) Having a true QA function in the company that isn't just engineers testing their own code.

That may have been true (I'm skeptical) back in the RoR 2.x -> 4.x times, but nowadays updating it is a breeze.

Honest question: what’s an example of a fully-featured web framework that makes upgrading a “large & old” application painless? In my experience, upgrading an underlying framework that a piece of complex software depends upon without breaking everything pretty much always requires time & good test coverage.