But the question is whether the AI will itself be able to decide on which problem to advance, and do it in an ethical and aligned way with humanity. And that if it does, its significance will be understood by humans.

Otherwise we're just creating a black box, easily manipulated by their owners, who would be ruling over human lives. That's not what people want