Shouldn't we be trying to do something even better than that, where the superintelligent AIs help every human to live a meaningful and thriving life? Perhaps substantially maintaining human autonomy over the trajectory of the future, even while doing things like keeping the peace, preventing people from becoming victims of crime or random disasters or disease.

Completely unrealistic. We could already be working toward such goals without super intelligent AI. I don’t see any reason to assume that ASI created through an evolutionary process would have a different outcome.

If ASI ended up being align-able and not an evolutionary process, why would we assume folks who already are currently in power would change their behavior?