I’ve talked to some local people who are convinced that panels slowly leach heavy metals into the surrounding ground.

They mainly argue against it on residential & farm land (Midwest USA) for this reason.

The argument I've seen against it for prime farm land in the UK is "Well we could use it to farm things" but that maybe lands less well in the US because of the enormous scales involved. "¿Por qué no los dos?" is the obvious retort in a huge region like that.

I've never seen "heavy metals" conspiracies, though I'm sure if I just wait I'll run into some because people sure do like making up reasons good things are bad...