Back in the day - port knocking was a perfect fit for this eventuality.

Nowadays, wireguard would probably be a better choice.

(both of above of course assume one is to do a sensible thing and add "perma-bans" a bit lower in firewall rules, below "established" and "port-knock")