In a way, "someone doesn't think I should have rights and/or exist and/or thinks I am inferior because of who I am" is pretty much the definition of (some kind of) politics. All sides play this game, e.g. many extremists these days argue that the "intolerant" shouldn't have rights or even exist by definition, but then the political football becomes who gets labeled as "intolerant" to begin with.

(And maybe it's true that those on opposite sides cannot work together on good software, but that's easily addressed since all FLOSS licenses include the right to fork and merge changes.)