Sounds overkill, how about giving the agent its own user?

It's really not overkill if you have good tools to work with. Hyper-V is quite capable of providing ephemeral workspaces on timescales measured in minutes. Especially with nested virtualization. One big machine with fast local disks can provide very short cold start times for a golden image stored on the same.

That's what I do in part because I went it to use the same system libraries etc. installed on my laptop, but I worry it will try to use privesc exploits...