You need a publicly routable address in the mix. You would need a way of knowing that address.

I have that same feeling with the self hosting I do. To alleviate the small amount of stress it would bring me I rent a VPS that’s public on the internet. I configure a persistent keep alive, on the client I run locally to keep a connection to the server open, no port forwarding needed.