To be fair, the people who think it believe that smart meters are how it will happen, so their existence would be all the evidence they actually need that it will.

I have no view either way (or a smart meter)

Smart meters themselves aren't the problem. It's meters that can control HVAC.