Asymmetric encryption? Both you (the human) and the agent publish public keys, the agent sign/encrypt the OTP request with you public key, you verify/decrypt using your private key, then do the same the other way to send the OTP (always encrypted though, given you’re sending a secret).

Something like that?

But that doesn't help for the agent receiving mail from arbitrary 3rd parties