The point is that you're now defending a completely different position from "Free speech is absolute." Determining what should count as "an actual crime or harm", how it can be proven, and so on, is pretty much the entirety of the problem you were claiming to have solved.