This is the same old good way from database world. A db client never touches the disk, but sends requests to a server who owns it. Same in your product. Keys are separated from agents and if needed, agents must send request to a proxy who owns them. Old proven design. Good luck.