The government banning your book from a school library is clearly a repercussion. That's what free speech has always been about, limit the ability of the government to enact repercussions.

What is "absolute" about this?

Do you want your own speech to be absolutely free of repercussions to you, be they government or not? Is that it? I really have trouble trying to put some sort of consistent framework in this, unless it's dividing the world into two classes of people: those who will not experience repercussions and those who will experience repercussions for their speech.