You can use something like Immich Public Proxy to only expose the /share path of your server and keep the main /api path that has everything else behind VPN