Or just drop websocket connection to them.

Refreshing pages to establish connection again is semi easy for someone who made a honest mistake (and Could make them more cautious), but if someone wants to trigger web-socket disconnect all the time, it would become cumbersome.

Add something like fail2ban, and you're set enough against passing trollers, but not interesting enough to draw attention from people that like the challenge.