I mean, it might listen to him. We have no clue, which is the problem.

Sure, but my guess is for "true" super intelligence we won't be able to predict whether that is true or not until it happens. I'm not a doomer, but I also don't really think we can "align" people, much less a "super intelligent" AI.