Unfortunately, yes. Dropping a magnet onto a car and pulling it off, especially if not recently cleaned, will damage the paint to some degree. Maybe not enough for an average person to notice, but you really shouldn’t do this to other people’s cars.

Some people will get snide about anyone who cares about their car’s paint, but as someone who once bought a car I had to save a long time for and spent a lot of time with car care products I would be very sad if I saw you drop a magnet on to it and then pull it off without a second thought. Please don’t.

Also, the paint on cars isn't just cosmetic. It's what keeps the metal from getting wet and then rusting.