You run the agent in a tightly controlled remote environment / VM designed for this use-case (at least the SSH/command piece).

Ona (https://ona.com) is a great choice.

(full disclosure: Ona co-founder here)