In my experience it is the constant shifting of goal posts due to execs chasing the next shiny thing, or demanding a feature that they saw somewhere, or heard from client (singular, not plural)

I've seen plenty of incredible engineers let go because of "performance issues" that were just poor project management and goal posts that moved so fast waymo should study them to improve their self-driving capabilities.

Shit rolls downhill and there's a lot more fuss when an engineer calls out risks, piss-poor planning, etc. than any actual introspection on why the risks weren't caught sooner or why the planning was piss-poor.

that's how the inner sanctum engineering in Apple worked, just like you proposed, at least from 15 years ago to within the last 10 years. i could have been in a very lucky time window to have had that luxury, but it had been an Apple mandate to not have deep hierarchies at least in engineering.

Astronomical salaries probably has something to do with it.

In today’s market it’s mostly because of the lack of other options to earn a livelihood

There's a high switching cost with substantial information asymmetry. Good places are hard to find in some absolute sense and it's hard to evaluate whether a new team is actually going to be good or not. And even if you do find a good team, there's no guarantee it'll last past the next reorg.

serious answer - you find a team with a good direct manager who handles all the upward interactions themselves, and then you basically work for that manager, rather than for the company.

Rent wont pay itself. Switching jobs has costs.