That is part of why downvotes exist. They also exist for personal attacks, off topic tangents, posts that don't make sense, trolling, advertisements, AI generated content, and other such things we don't want to see here.
But "downvote for disagreement" is a legitimate use. I personally tried to tell someone that it wasn't, and I got corrected by dang.