Once it's networked you lose the "whitelist of systemd services" and it's then no different from any networked secret store.

No, this is a solved problem: https://spiffe.io/

You can do service attestation securely, even for networked services.