> You cannot prevent (2) via any alignment process

A little bit too categorical. GOODY-2 wouldn't do it. https://www.goody2.ai/

The hard part is having both helpful and harmless at the same time. Harmless is easy.

And then once it's helpful, the real question becomes "to whom"

- To the user -> You end up with competing godlike AI with incompatible tasks

- To the owner -> Dictatorship

- To humanity as a whole -> It must not have an off button. Otherwise you're just in one of the two earlier categories with more steps.

Given those 3 options, I'd choose humanity as a whole. But the person making the decision doesn't have those 3 options. Because in the dictatorship option, they would be the dictator. I don't trust them to pick humanity.