Every mass genocide in the history of humanity has followed logic like yours. People don't kill millions of humans because they want to do harm-- they do so because they think they are doing the ultimate good a good so great that is justifies the loss of life.

If AI ever does cause serious direct harm to humanity it will be because of logic like this.