How would it make them worse off? If everybody agrees that AI is a threat to humanity, but also a possible edge over rivals, then the threat should be enough reason to get everybody to agree.

There are international agreements on tons of things that work the same way.

> If everybody agree[d] that AI is a threat to humanity

Which would be either very difficult or very ironical, given that realizing that humanity is a threat to humanity precedes that.