Closed ports are not "locked doors", and open ports are not "unlocked doors"

That is a binary thought process with a lot of assumptions. You might introduce even more attack surface in pursuit of this "security" measure by installing additional software like fail2ban, for example. Close your ports, maybe assign a non-standard port to the popular ones (like SSH) to reduce log spam, and patch your server often. Anything more complicated than that is not worth it, IMO.