"Heavy metal" in general is a bad term, but especially when used as a proxy for toxin. There is no universal definition of heavy metal and there is no inherent connection to toxicity in any specific organism.

Then again, pretty much every metal is toxic at some relatively low body-mass concentration, even iron (which actually can and does kill people, especially when children eat adult iron supplements).

Even lovely unreactive gold does have compounds that are toxic.