Same here, though you do still have to expose it to the internet unless you use a VPN. I'd prefer something with less of an attack service especially because bridges don't currently encrypt, but I host behind vpn now.

That's not a fair criticism. Of course you need to expose the server to the network clients are going to be connecting from. Whether that is Internet or just a LAN, that's orthogonal to the protocol.

I've also had an intra-company Matrix server running completely on a company internal LAN, with no Internet access, so there is no inherent need for it to be on The Internet.