Yes, sorry should've said I never doubted you!

My comment more addressed to the clients, who can mitigate the issue with appropriate ssh_config.

More specifically, make sure that you aren't forwarding your ssh keys; otherwise the game can impersonate you on any server the keys work on.