"Freedom of speech" crowd curiously picked whose speech is important freedom thing and whose speech is actually anti-free-speech, because it is uncomfortable for the former free speech.

Trying to remove this or that group from civil life and push them into helpless silent situation is important free speech. Protesting this person is not free speech, it is infraction against free speech. Offending certain group is important free speech and they should grow a thicker skin rather then responding. Responding to that is infraction, because bigot has free speech right to never hear something that does not make them feel good.