As long as the car is dirty, then contact with it can damage the top coat. This is a lot more true if you need to drag or scrape the magnet to remove it.