> our friends can't access our server

You're almost there with your excellent lineup of self-hosted tech. Just throw in Headscale and some Tailscale clients and you'll be there. (Or any number of mesh VPN alternatives, like NetBird)