You are essentially describing an over-eager developer having to run decisions up the chain of command to senior devs / product owners / company execs. I assure you, every one of those roles is being trained on to balance their own wide breadth of concerns, and is unlikely to advise something like a live rewrite of a production app or an entire product pivot - unless it actually made sense from an economics/management perspective. Every step bounded by its role and responsibilities.

Turtles all the way up, and all the way down. AI getting this good at coding means it's a hop skip and a jump away from being this good at every white collar role.

Yes, but the entire point is, you can't have a reasonable chain of command or any kind of sensible decision process without human judgement that has engineering expertise, so no, engineers are not going to be entirely replaced.

The reason human judgement is necessary is not because it's "better". It's because the entire reason the business exists is to execute the human's judgement. There is no "right or wrong" way to build a startup if different paths are all profitable, but the human decides what they want to do.

Could Bun have survived and done well even if they didn't rewrite the codebase in Rust? Yes, it would've been fine. But the human decided they wanted Rust, that's a different path they chose with different tradeoffs. The AI has no skin in the game , no ownership, it doesn't belong to the AI. The AI only exists to serve the human. And if the human wants to make good decisions, inevitably they need engineering expertise (or an engineer) to weigh the decision.