weird bias by authority aside, do you really think that the daily decisions in this world today, and anytime soon, will be made based on the "let humanity flourish axiom"?

In daily decisions perhaps not, and certainly not when we were in the cave. But from time to time, we will face fundamental questions that need addressing clearly and decisively, and I believe it’s the case with AI. The more advances we make, the more frequent such questions will arise. That is also one of the main reasons why we don’t see any alien civilization. It’s getting harder and harder to keep the entropy low as we gain more knowledges and cognitive power.