Wow. This never crossed my mind but of course that's so simple. There really needs to be a better solution.

There is. Simply do not install sudo and do not allow access to root at runtime. I am serious. There is absolutely nothing you cannot run unprivileged these days. Can even run sshd from a systemd user unit in your home folder, and even assign port 22 to it if needed with Linux Capabilities.