It absolutely is a problem with human assistants (though, of course, those are currently much smarter). But people can and have scammed assistants to steal money or personal details from their bosses. Phishing and social engineering are exactly forms of this same vulnerability. Of course, human assistants are smart enough to not get phished by, say, reading a book that happens to contain phrases that are similar to commands that their boss could give them, but that's just the current difference of intelligence and the hugely larger context windows humans still have compared to LLMs.