Even as a pet I think a proper interface for managing the server would be better and more secure than ssh.

Often those proper interfaces are wrappers around what you would run via SSH and add their own security holes, so I would argue against “more secure than SSH”.