> The issue is removing people due to their political opinions.

No. The issue is who is doing that action. Illegal speech is not the same as rude speech. My boss can’t declare my opinions illegal. They can fire me.

> You are still free to associate with anyone freely, but there is an expectation that you behave like an adult and can withstand different opinions.

Nah. I can throw a tantrum, publicly decry you as an asshole and go no-contact with you. That’s legal, and part of my freedom of expression and association.

The government can’t confiscate my license over the tantrum.