MASQUE[0] is the protocol for this. Cloudflare already uses masque instead of wireguard in their warp vpn.

[0]https://datatracker.ietf.org/wg/masque/about/

i was curious about this and did some digging around for an open source implementation. this is what i found: https://github.com/iselt/masque-vpn