This offends me, because I believe code is speech, and speech should not be restricted.

There are plenty of reasons speech should be restricted. One of them is hate speech.